JP2006340198A - Coding device and coding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To further improve the compression efficiency of image data. <P>SOLUTION: The image data is divided into at least one brightness plane for indicating the brightness and at least one color-difference plane for indicating the color difference (step S2). Thereafter, the variation in brightness existing in the brightness plane is eliminated (step S3), and then the image data is coded after eliminating the variation in brightness (step S7). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an encoding processing apparatus and an encoding processing method for encoding image data.

  In recent years, with the development of digital cameras, natural images can be easily converted into digital information. Recently, still and movie cameras that can shoot not only still images but also movies are also emerging.

  Since the camera consumes a large amount of memory, a highly efficient color image compression method is desired. Further, since the camera is premised on electronic zoom, it is necessary to provide a high-definition image even at the time of zooming. However, since JPEG and MPEG, which are conventional main codecs, are irreversible compression methods, they are not suitable for reproducing the high-definition images.

  In addition, it is desirable that the camera can extract any frame in the moving image as a still image even during moving image shooting. In that respect, MPEG, which is the video codec, is disqualified. In addition, with the recent improvement in performance of display devices, the quality of images provided by conventional broadcast standards such as NTSC and MPEG are becoming limited.

  In this situation, a new lossless and lossy compression method called JPEG2000 is drawing attention. The compression method is essentially a still image compression method, and basically employs the same compression algorithm (wavelet transform) for both lossless and lossy compression. However, the compression ratio of the reversible compression is not so high and is about 1/2. The compression method can also be applied to moving image compression. For example, a good moving image can be provided at a compression ratio of about ８ of irreversible compression.

  Patent Document 1 discloses a method for converting the scan direction, and Patent Document 2 discloses a method for selecting and employing an optimal compression method by dividing an image. Patent Document 3 discloses a method of using a histogram for image division, and Patent Document 4 discloses a method of compressing an image based on an attribute map / object. Patent Document 5 discloses a method for discriminating the type of image from the run length distribution, and Patent Document 6 discloses a method for rotating a color difference signal around the origin.

  Other still image reversible compression methods include GIF, PNG, JPEG-LS, CALIC, TMW, and the like.

Japanese Patent Laid-Open No. 8-126009 JP 2000-333019 A JP 2001-22931 A JP 2001-169120 A JP 2002-163658 A Japanese Patent No. 3334242

  If a color image reversible compression method and codec for providing a high-definition image are realized, an enormous memory is not required for recording the moving image or the like. However, the still and movie cameras described above are currently limited by the amount of memory and the memory cost, and a full-scale camera has not yet reached the stage of realization.

  Also in the NTSC standard, although one frame image can be formed from two field images by a known technique, since the imaging time is slightly different between the two field images, it moves at high speed. There is a case that the object to be blurred. Therefore, in the still and movie camera that extracts still images from moving images, the recording method is preferably a progressive scan method, not an interlace method adopted by the NTSC or the like. Inevitably, the compression method is based on a still image.

  Although the reversible compression efficiency of the still image is said to be about 1/3, it has not been proved, and the compression ratio greatly depends on the properties of the still image.

  Therefore, an object of the present invention is to further improve the compression efficiency of image data.

According to a first aspect of the encoding processing apparatus of the present invention, there is a separation unit that separates image data into at least one luminance plane indicating luminance and at least one color difference plane indicating color difference, and exists in the luminance plane. And an inclination correction unit that corrects the luminance difference, and an encoding processing unit that encodes the image data after the luminance difference correction processing by the inclination correction unit.
According to a second aspect of the encoding processing apparatus of the present invention, a separation unit that separates image data into at least one luminance plane indicating luminance and at least one color difference plane indicating color difference, the luminance plane, and the color difference Based on the correlation between the planes, a plane reduction unit that reduces at least one of the luminance plane and the color difference plane, and a coding process of the image data after the plane reduction process by the plane reduction unit And encoding processing means.
A first aspect of the encoding processing method of the present invention is an encoding processing method by an encoding processing device that performs encoding processing of image data, and includes at least one luminance plane indicating luminance and at least one color difference. A separation step for separating the image data into color difference planes, a slope correction step for correcting a brightness difference existing in the brightness plane, and a sign of the image data after a brightness difference correction process by the slope correction step And an encoding process step for performing an encoding process.
According to a second aspect of the encoding processing method of the present invention, there is provided an encoding processing method by an encoding processing device that performs encoding processing of image data, wherein the luminance plane indicates at least one luminance and at least one color difference. A separation step of separating image data into color difference planes, and a plane reduction step of reducing at least one of the luminance plane and the color difference plane based on a correlation between the luminance plane and the color difference plane And a coding processing step for performing coding processing of the image data after the plane reduction processing in the plane reduction step.
A program according to the present invention causes a computer to execute the first or second aspect of the encoding processing method.
The computer-readable recording medium of the present invention is characterized in that the program is recorded.

  According to the present invention, it is possible to further improve the compression efficiency of image data.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, an outline of an embodiment of the present invention will be described. In the embodiment described below, the compression efficiency is improved by the techniques shown in the following items.
(1) Division of still image into partial images (2) Separation into one luminance plane and two color difference planes (3) Inclination correction of luminance plane (4) Determination of color coordinates of color difference plane (5) Color difference Object detection using plane information, image segmentation (6) Determination of optimal prediction direction (7) Determination of encoding direction in consideration of optimal prediction direction (8) Luminance plane and color difference plane for reducing file information amount (9) Point defect exception handling

  (3) is mainly aimed at improving the compression ratio in the run mode. As is well known, most codecs provide a run mode and a prediction error mode (arithmetic coding mode). Of these images, the compression rate dramatically increases for images that can be used extensively.

  It can be considered that the natural image to be compressed is a subject object that is exposed to illumination light such as the sun or a lamp from the outside. When an existing natural image is analyzed, a large luminance difference is usually observed in the vertical direction, as shown in FIG. This luminance difference reaches an average of 35 gradations / 256 gradations (14%) in a certain VGA image, for example.

  This is because the reflectance of the object is substantially the same in each part, whereas the distribution of the illumination light is largely different in the vertical direction as opposed to it. That is, generally, the upper part of an image is bright and high in luminance, and the lower part is dark and low in luminance.

  In the embodiment of the present invention, these luminance differences are corrected before encoding. Specifically, encoding is performed after correction is performed mainly using a linear or low-order function in the vertical direction. The increase in the amount of information of the image compression file by these conversion operations is only the coefficient of the linear or low-order function, and is so small that it can be virtually ignored.

  There are roughly two methods for determining the color coordinates of the color difference plane in (4). The conventional two color difference planes are, for example, (BG, RG) and (U, V). However, in the case where a specific subject object is divided into most of the partial image after being divided into partial images, it may be assumed that most of the pixel values of the pixels constituting the image are occupied by a specific color. . In such a case, it is inefficient to use the predetermined color difference plane coordinates.

  Therefore, one method is to obtain the specific color from, for example, a histogram of pixel values, rotate two color difference planes for that color, and adjust the color difference coordinates of one plane to that color, The color difference coordinates of the other plane are converted so as to be orthogonal to the color.

Specifically, if the three primary colors of the color image are C1, C2 and C3, the linear quantities K1 and K2 consisting of a total of four terms including the constant C0 are used as the color coordinates of the new color difference plane.
K1 = α * C0 + β * C1 + γ * C2 + δ * C3 (Formula 1)
K2 = ε * C0 + ζ * C1 + η * C2 + θ * C3 (Expression 2)
Here, α to θ are linear combination coefficients. The constant term C0 is provided so that K1 and K2 do not become negative values. The increase in information amount and file size here is a numerical value of 64 bits: 8 bytes represented by the coefficient of 8 degrees of freedom.

  The other method is a method in which the color coordinates of the color difference plane are not (R / W, B / W) or (R / G, B / G) instead of the linear combination of the three primary colors. However, since this is a reversible compression technique, the value used for the color coordinate needs to be a correct amount such as a camera output. Even if the color coordinate is obtained from a bitmap image already expressed by an integer, the effect is invalid (since a rounding error occurs). It is only necessary that the analog output (light quantity or the like) is directly AD converted into R / W or the like, for example, the camera output.

As described above, if the color (reflectance R) of the subject object is constant regardless of the location, even if the amount of illumination light I0 hitting the subject changes (luminance change), the previous color coordinate K is the value. The place that doesn't change is a good place.
Ir = I0 * Rr (Formula 3)
Kr = Ir / (Ir + Ig + Ib) = Rr / (Rr + Rg + Rb) (Formula 4)
Here, I is the amount of reflected light, and the subscripts r, g, and b correspond to the three primary colors, red, green, and blue, respectively.

  On the other hand, when the luminance changes, the value of the chrominance plane changes in addition to the pixel value of the luminance plane, so the run mode that can be used when the same pixel value is continuous cannot be used, and the compression rate decreases. Was. In the embodiment of the present invention, since the dynamic range of the color difference plane is increased from, for example, 8 bits (256 gradations), for example (8 + 10 = 18 bits), a slightly extra intermediate file size is required. However, it does not pose a problem after compression because the compression efficiency is improved.

  The division into partial images in (5) is preferably not a uniform division as seen in the conventional codec but sufficiently considering the objects included in the color image. The following three methods can be considered as such a method.

The first is a method of using the information of the color difference plane to extract the object.
Actually, the object extraction means whether or not the previous run mode can be adopted, so that the same color can be regarded as the same object. This is possible by observing the pixel value of the color difference plane.

  Second, the object is arranged in the same partial image without straddling a plurality of partial images. For this purpose, the size of the partial image to be divided is set to an arbitrary size different between the color images. That is, the division of the image is determined in consideration of the size of the object included in the image.

  Thirdly, the division start point of the divided image is translated from the upper left of the color image to an arbitrary position and changed to start division. According to this, at least the object can be placed at the center of the partial image. The problem of the edge processing of the image can be avoided by cyclically continuing the top down and the left right. The increase in the information amount and the file size here is about the number of bits (several bits) representing the size of the image.

  The prediction method of (6) is a part that has been variously studied even with conventional codecs. However, the improvement efficiency is not necessarily high. In the embodiment of the present invention, the prediction value is obtained in at least four directions around the target pixel, preferably eight directions or more, the optimum prediction direction that minimizes the prediction error is obtained, and the direction is adopted as the direction of compression coding. I do. At that time, a long-distance correlation is obtained also using pixels separated by 5 pitches or more, and the predicted value is predicted. For the prediction, for example, linear prediction (LPC) is used.

  In the conventional lossless JPEG, the predicted value is calculated with reference to the values of the three pixels in the upper left direction separated by one pitch (that is, adjacent). In the JPEG-LS, the value of 4 pixels is also referred to. The CALIC refers to the value of a total of 7 pixels in the upper left direction separated by 2 pitches. In addition, the TMW uses the values of neighboring eight pixels that are one pitch away from the target pixel for the linear prediction. However, in the embodiment of the present invention, these are different in distance (number of pitches) and pixel arrangement.

  The reason why the long distance correlation is used in the embodiment of the present invention is that the correlation with the target pixel is recognized up to a relatively long distance (several pixels) as shown in FIG. The horizontal axis of the graph is the distance from the target pixel, and the vertical axis is the absolute value of the gradation difference between that pixel and the target pixel. Each line in the graph represents an average value of all pixels (640 * 480) of 13 types of VGA color images. For example, if a point where the graph is horizontal is regarded as a point where there is no correlation, in the example in the figure, there is a correlation up to 5 pixels or more.

  In general, in LPC prediction, the higher the order (that is, the more pixel information is used), the higher the prediction accuracy. The existence of this long-range correlation is considered to become more prominent in multi-pixel, high-definition images developed in the future.

  Conventionally, edge detection or the like has been performed by examining pixels at a relatively short distance, but this is inaccurate. As in the embodiment of the present invention, a long distance correlation should be examined, and a direction having a high correlation (that is, a small prediction error) and a high possibility of being an edge should be used for the prediction.

  In FIG. 11, the prediction directions are four directions, that is, 0 degrees (horizontal direction), 45 degrees, 90 degrees (vertical direction), and 135 degrees around the target pixel, but this number is arbitrary. In general, predicted values in opposite directions are close, and the optimal prediction direction showing the minimum prediction error is often the horizontal direction. However, since there are rare special images, the prediction direction should be selected from all directions (360 degrees).

Now, the necessity of (7) will be described using a conventional codec. In a conventional codec, the upper left pixel is mainly used as a prediction pixel. This is because the encoding direction was from the upper left to the lower right, and since the pixel value yi of each pixel has been determined in the upper left direction, the actual value y of the pixel of interest using the pixel value yi and the prediction error Δe. It is because it is possible to ask for.
y = f (yi) + Δe (Formula 5)

  On the other hand, since the pixel value at the lower right is not yet determined, it is impossible to calculate the actual value y even if it is used. As described above, in the embodiment of the present invention, after obtaining the optimal direction of prediction in advance, an encoding direction corresponding to the optimal direction is determined.

  Since the designation of the prediction direction used in the embodiment of the present invention is, for example, only 3 bits in the 8 directions, the increase in the information amount and the file size can be virtually ignored.

  In the case of (8), conventionally, encoding of the luminance plane and the color difference plane has been usually performed independently. However, in the natural image of the subject object + illumination light, there should be an information amount common to both the planes. If such information amount can be integrated between planes, the information amount of the entire compressed image can be reduced, and the compression ratio can be greatly improved.

  In the embodiment of the present invention, this is realized by the following two methods. The first is to reduce the number of planes. In a specific partial image of an image composed of substantially a single color, the premise of one luminance plane and two color difference planes is not always necessary. In that case, the luminance plane may be obtained by a calculation formula from one luminance plane + one color difference plane or two color difference planes. Of course, there may be rare cases where only one color difference plane is sufficient.

The specific method is to compare tables of pixel values between both planes before compression. If each pixel value can be expressed by a simple expression between both planes, one plane is not necessary. If the correlation between the two is higher than a predetermined threshold, it is better to obtain the prediction error using the correlation between the planes, rather than the known encoding method using the adjacent correlation and the prediction error as described above. A good case also occurs. For example, assuming that the pixel values at P1 and P2 are y1 and y2, the prediction error Δe is expressed as follows.
y2 = α * y1 + β + Δe (Expression 6)
Here, α and β are constants.

  The second is the position of the object and the method of dividing the image. Further, depending on the image, it may be possible to share the optimum prediction direction and the encoding direction in each of the divided images.

  Further, (9) considers defective pixels that inevitably exist in the image sensor. A multi-pixel imaging sensor has many point defects as in the display device. Since they are usually processed and corrected on a frame memory, for example, they are generally not conspicuous. However, the run mode that dramatically increases the compression ratio may be interrupted due to a point defect at most. In addition, when the number of pixels to be removed as point defects is small, such as when correlation between planes is used, the compression ratio is sufficiently improved.

  The increase in information amount and file size according to the embodiment of the present invention is about the number of point defects * (number of bits representing coordinates: several bits + number of gradation bits: 8 bits).

  In this way, even though it is currently called a mammoth codec, the improvement in the circuit scale and operation speed of the semiconductor integrated circuit is extremely remarkable (each 3 years 4 times, 3 years 8 times). There is a possibility of becoming it.

  Since still image and moving image compression requires firstly a compressed image file size and secondly a decoding time, a configuration that sacrifices others can be considered sufficiently. Since parallel computing technology also advances, it is necessary to use a codec that fully considers the parallel processing. The present invention is also adapted to the times in the above points.

  Next, embodiments of the present invention will be specifically described. FIG. 12 is a diagram illustrating a system configuration of an imaging apparatus applied to the embodiment of the present invention. In FIG. 12, 10 is a photographing lens, 12 is a shutter having a diaphragm function, 14 is an image sensor that converts an optical image into an electric signal, and 16 is an A / D converter that converts an analog signal output of the image sensor 14 into a digital signal. It is.

  A timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 17, and the D / A converter 26, and is controlled by the memory control circuit 22 and the system control circuit 50.

  Reference numeral 15 is a CDS (correlated double sampling) circuit, and 16 is a programmable gain amplifier (PGA).

  An image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 17 or the data from the memory control circuit 22.

  Further, the image processing circuit 20 performs predetermined calculation processing using the captured image data, and the system control circuit 50 controls the exposure control means 40 and the distance measurement control means 42 based on the obtained calculation result. TTL (through-the-lens) AF (autofocus) processing, AE (automatic exposure) processing, and EF (strobe pre-flash) processing are performed.

  Further, the image processing circuit 20 performs predetermined arithmetic processing using captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained arithmetic result.

  A memory control circuit 22 controls the A / D converter 17, the timing generation circuit 18, the image processing circuit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression circuit 32.

  The data of the A / D converter 17 is written into the image display memory 24 or the memory 30 via the image processing circuit 20 and the memory control circuit 22 or the data of the A / D converter 17 is directly passed through the memory control circuit 22. It is.

Reference numeral 24 denotes an image display memory, 26 denotes a D / A converter, 28 denotes an image display unit including a TFT LCD, and the image data for display written in the image display memory 24 passes through the D / A converter 26. Displayed by the image display unit 28. If the image data captured using the image display unit 28 is sequentially displayed, the electronic viewfinder function can be realized.

  Reference numeral 30 denotes a memory for storing captured still images and moving images, and has a sufficient storage capacity to store a predetermined number of still images and a predetermined time of moving images. This makes it possible to write a large amount of images to the memory 30 at high speed even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot. The memory 30 can also be used as a work area for the system control circuit 50.

  A compression / decompression circuit 32 compresses and decompresses image data, reads an image stored in the memory 30, performs compression processing or decompression processing, and writes the processed data to the memory 30.

  Reference numeral 40 denotes exposure control means for controlling the shutter 12 having a diaphragm function, and has a flash light control function in cooperation with the flash 48.

  Reference numeral 42 denotes a distance measuring control means for controlling the focusing of the photographing lens 10, reference numeral 44 denotes a zoom control means for controlling zooming of the photographing lens 10, and reference numeral 46 denotes a barrier control means for controlling the operation of the protection means 102 as a barrier.

  A strobe 48 has an AF auxiliary light projecting function and a strobe dimming function. The exposure control means 40 and the distance measurement control means 42 are controlled using the TTL method. Based on the calculation result obtained by calculating the captured image data by the image processing circuit 20, the system control circuit 50 performs the exposure control means 40 and the distance measurement. Control is performed on the control means 42.

  Reference numeral 50 denotes a system control circuit that controls the entire imaging apparatus 100, and 52 denotes a memory that stores constants, variables, programs, and the like for operation of the system control circuit 50.

  Reference numeral 54 denotes a display unit such as a liquid crystal display device or a speaker that displays an operation state or a message using characters, images, sounds, and the like in accordance with execution of a program in the system control circuit 50, and an operation unit of the imaging device 100. A single or a plurality of locations are provided in the vicinity where they can be easily seen, and are configured by a combination of, for example, an LCD, an LED, and a sounding element.

  Reference numeral 56 denotes an electrically erasable / recordable nonvolatile memory such as an EEPROM or a flash memory.

  Reference numerals 60, 62, 64, 70, and 72 are operation means for inputting various operation instructions of the system control circuit 50, and include one or a plurality of switches, dials, touch panels, pointing by line-of-sight detection, voice recognition devices, and the like. Composed of a combination.

  Reference numeral 60 denotes a power switch (main switch) that can switch and set the power-on and power-off modes of the image processing apparatus 100. In addition, the power on and power off settings of various accessory devices connected to the image processing apparatus 100 can be switched and set together.

  Reference numeral 62 denotes a shutter switch SW1, which is turned on during the operation of a shutter button (not shown), and performs AF (auto focus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, EF (strobe pre-flash) processing, and the like. Instruct to start operation.

  Reference numeral 64 denotes a shutter switch SW2, which is turned on when an operation of a shutter button (not shown) is completed, and an exposure process for writing a signal read from the image sensor 14 to the memory 30 via the A / D converter 17 and the memory control circuit 22. A series of recording processes such as a development process using an operation in the image processing circuit 20 and the memory control circuit 22, a recording process in which the image data is read out from the memory 30, compressed in the compression / decompression circuit 32, and the image data is written in the recording medium 200. Instructs the start of processing operation. An operation unit 70 includes various buttons and a touch panel.

  Reference numeral 72 denotes a mode dial switch which can switch and set various function modes such as an automatic shooting mode, a shooting mode, a panoramic shooting mode, a playback mode, a multi-screen playback / erase mode, and a PC connection mode.

  Reference numeral 80 denotes a power supply control means, which includes a battery detection circuit, a DC-DC converter, a switch circuit for switching a block to be energized, etc., and detects the presence / absence of a battery, the type of battery, the remaining battery level, In addition, the DC-DC converter is controlled based on an instruction from the system control circuit 50, and a necessary voltage is supplied to each unit including the recording medium for a necessary period.

  Reference numeral 82 denotes a connector, 84 denotes a connector, 86 denotes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, NiMH battery, or Li-ion battery, an AC adapter, or the like.

  Reference numeral 90 denotes an interface with a recording medium such as a memory card or hard disk, and reference numeral 92 denotes a connector for connecting to a recording medium such as a memory card or hard disk.

  Reference numeral 102 denotes protection means that is a barrier that prevents the imaging unit from being soiled or damaged by covering the imaging unit including the lens 10 of the imaging device 100.

  Reference numeral 104 denotes an optical viewfinder, which can take an image using only the optical viewfinder without using the electronic viewfinder function of the image display unit 28. In the optical viewfinder 104, some functions of the display unit 54, for example, focus display, camera shake warning display, strobe charge display, shutter speed display, aperture value display, exposure correction display, and the like are installed.

  Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 202 composed of a semiconductor memory, a magnetic disk, and the like, an interface 204 with the imaging device 100, and a connector 206 that connects to the imaging device 100. Although the recording medium 200 is described as a configuration built in the imaging apparatus 100 in the present embodiment, it is also within the scope of the present invention to be configured outside.

  The embodiment described below is mainly realized by the image processing circuit 20 and the compression / decompression circuit 32. First, a first embodiment of the present invention will be described. The main routine of the lossless compression method for color images in this embodiment is shown in FIG.

  In the present embodiment, as shown in FIG. 5, a VGA image (640 * 480 pixels) 51 having 8-bit RGB color information in each pixel is reversibly compressed, and the image has a size of 80 * 60 pixels. The image is divided into 64 partial images 52 (step S1).

Next, the luminance plane (Y) and the color difference planes (Cb, Cr) are separated based on the following formulas 7 to 9 (step S2).
Y = R + 2 * G + B (Formula 7)
Cb = BG (Formula 8)
Cr = R−G (Formula 9)
Here, R, G, and B are pixel values (RGB values) at each pixel.

Next, in the luminance plane Y of each partial image, the inclination correction of the pixel value is performed only in the y direction (step S3). First, a value Y (j) obtained by adding each pixel value Y (i, j) for each row (j) is obtained.
Y (j) = ΣY (i, j) (Equation 10)
The sum is taken over i.

It is approximated by a linear expression by the following expression 11.
Y (j) p = a * i + b (Formula 11)
The values of a and b are obtained by, for example, a known least square method. Here, Y (j) p is an approximate value of the previous Y (j).

The tilt correction is performed by subtracting Y (j) p from the previous pixel value Y (i, j).
ΔY (i, j) = Y (i, j) −Y (j) p (Equation 12)

  What is actually encoded is the amount (difference) represented by ΔY (i, j) in Equation 12. The previous two coefficients a and b (information amount of 16 bits) are recorded and stored for each partial image.

Next, coordinate conversion of the color difference plane is performed independently for each partial image (step S4). First, in each color difference plane (Cr or Cb), a histogram of pixel values Cr (i, j) and Cb (i, j) is taken for the pixels of the entire partial image. If the values indicating the maximum frequency in the histogram are Crfreq and Cbfreq, respectively, new coordinates shown in Equations 14 and 15 can be obtained by rotating the color coordinates (Cr, Cb) at the rotation angle θ determined in Equation 13. A histogram is further obtained using the color coordinates (K1, K2). The relationship between the color coordinates (Cr, Cb) and the color coordinates (K1, K2) is shown in FIG.
tan θ = Cbfreq / Crfreq (Equation 13)
K1 = Cr * cos θ + Cb * sin θ (Expression 14)
K2 = −Cr * sin θ + Cb * cos θ (Expression 15)

Then, the following quantities are calculated using the color difference planes K1 and K2.
H = −ΣPi * (logPi / log2) (Equation 16)
The sum is taken for i.
Pi = N (Kn (i)) / (80 * 60) (Expression 17)
Expression 16 is a well-known expression (entropy expression) indicating the compression limit. Here, Kn (i) is a frequency indicating the value i in the histogram of each color difference plane. 80 * 60 is the total number of pixels of the partial image. Therefore, logPi is a probability indicated by the frequency of the value i.

  When the sum H1 + H2 of the values H1 and H2 of Expression 16 obtained by the color difference planes K1 and K2 is smaller than a predetermined value (threshold value), the color difference coordinates K1 and K2 are adopted as encoded coordinates. When the sum exceeds the threshold value, the same investigation is also performed around the previous mode values Crfreq and Cbfreq or around the second peak (the next most frequent value). If the relationship is still not satisfied, the color difference coordinate indicating the smallest sum among the sums is adopted as the encoding coordinate.

  What this embodiment is doing in this step is a device that concentrates the color information on one coordinate (in this case, K1) to increase the coding efficiency.

  Next, difference information for encoding is obtained using a known predictive encoding method. For this purpose, it is necessary to obtain a predicted pixel value used for the prediction, but in this embodiment, the prediction is performed from eight directions outside the five pixels of the target pixel as shown in FIG. S5).

Assuming that the pixel of interest is P0, the pixel 1 pitch away from it is P1, and the pixel 5 pitches apart is P5, the prediction error ΔP is obtained by the following linear prediction method (LPC) by the following equations 18 and 19. It is done.
ΔP = P0−Pproj (Equation 18)
Pproj = -u * P1-v * P2-w * P3-x * P4-y * P5 (Equation 19)
Here, the coefficients of u, v, w, x, and y are optimized for the pixels of the entire partial image.

  The least square sum of the prediction error ΔP indicates a different value for each direction of the prediction. Of these, the direction showing the minimum least square sum is adopted as the optimum direction of prediction in the present embodiment.

Once the optimum prediction direction has been determined, the encoding direction of each pixel is next determined (step S6). As shown in FIG. 9, the encoding direction 94 is the same as the optimal direction 93 of the prediction.
In the present embodiment, parallel processing encoding is performed for each row of each partial image. At that time, the four pixels from the right starting pixel 91 cannot be encoded by the equation 19, but the equation 19 is interpreted and diverted assuming that the value of the nonexistent pixel is 0. Finally, encoding is performed by known arithmetic encoding (step S7).

  According to the present embodiment, since the inclination correction of the luminance plane is performed, the ratio of adopting the run mode is increased and the compression rate is improved. In addition, since the color difference coordinates are changed for each partial image, the specific color included in the partial image can be efficiently encoded. In addition, since the encoding is performed by selecting the optimum direction, the encoding efficiency is good.

  This embodiment can also be used for any color image other than the VGA color image. The number of divisions into the partial images and the size of the divided images are also arbitrary. In addition, the luminance / chrominance plane separation method represented by Expression 7 to Expression 9 used in the present embodiment may be other known methods. For example, the luminance Y may be Y = R + G + B.

  Further, the correction of the inclination of the luminance plane expressed by Expression 12 is not limited to a linear expression, but may be a higher-order correction expression. In that case, the amount of information required for image compression slightly increases by the increased coefficient. Of course, the luminance correction may be performed not only in the y direction but also in the x direction (horizontal direction).

In addition, the selection of the color coordinates is not limited to the coordinate rotation expressed by the equations 14 and 15, but may be of a more general line format expressed by the equations 1 and 2. In addition, since this embodiment is intended for lossless compression, it is desirable that the gradation values represented by the color difference coordinates K1 and K2 are integer values rather than real values (since a rounding error occurs in the calculation). Therefore, the selection of the color difference coordinates K1, K2 needs to be determined in consideration of them. Such coordinates include the following equations 20 and 21.
K1 = αCr + βCb (Equation 20)
K2 = αCr−βCb (Formula 21)
Here, α and β are integers.

  As is well known, each gradation value of K1 and K2 can be set to 0 or a positive integer by subtracting the minimum values K1min and K2min from the respective values. In this way, later encoding becomes easy.

  In addition, the selection criteria used for selecting the color difference coordinates K1 and K2 need not be the entropy type ahead, and for example, as in the conventional codec TMW, the code size is actually determined by knowing the code size. K1 and K2 for generating a simple code may be selected.

  Further, the direction of the prediction is not limited to eight directions, and may be four or more directions considering all directions. Also, the order may be any number including pixels with a pixel pitch of 3 or more in order to improve the prediction accuracy. The prediction is not limited to linear prediction, and any known prediction method may be used. The encoding method may be known, for example, Huffman encoding.

  Next, a second embodiment of the present invention will be described. FIG. 2 shows a main routine for reversible compression of a VGA color image in this embodiment. In this embodiment, an object is detected prior to image division.

  First, as described above, the luminance color difference plane is separated (step S11). Next, as shown in FIG. 10, the object 102 is detected by the color difference plane 101 (step 12).

  In this method, a histogram of pixel values is created for each color difference plane, and an area where the pixel values indicate the mode value ± Δ is recognized as the object 102. Here, the value of Δ is about 0 to 3 in the case of 256 gradations. The VGA image is divided in consideration of the detected object 102 (step S13).

  In the division method, first, the horizontal and vertical sizes Lx, Ly and the center of gravity (Gx, Gy) of the object are detected. Next, the pitch of the dividing line 103 is determined by comparing the size of the object with the size of the assumed divided partial image (usually 40 * 30 to 160 * 120). When the size of the object 102 is large, the object 102 exists across a plurality of partial images as shown in FIG.

  When the image is divided, the division start position is determined in consideration of the barycentric position (Gx, Gy) as shown in the figure. Dividing lines 103 that divide the image are drawn on the image 101 at equal intervals.

Next, as described above, the coordinate system of the color difference plane suitable for the color of the object 102 is determined (step S14). Next, in order to reduce the file size after compression (that is, to improve the compression rate), the sharing of compression information among the three planes is considered (step S15). The items and order to be considered are as follows.
(1) Reduction of the number of planes (2) Change of prediction method (3) Coordination of coordinate parameters of color difference coordinate system

Since the most efficient compression efficiency is the reduction of the number of planes, first, the possibility of this is examined. The method is when three planes are examined and the following relationship is established between all pixel values Pn (i, j) of each plane.
Pm (i, j) = A * Pn (i, j) + B (Expression 22)
Here, Pm (i, j) and Pn (i, j) are pixel values of any two planes, and A and B are constants.

  Of course, this rarely occurs in natural images, but for example in the case of drawings created with PowerPoint, etc., or when the total number of pixels in a divided image becomes small, the probability is so high that it cannot be ignored. . The presence / absence of the plane reduction is written in the header portion of the compressed file. The increase in the amount of information required is only a few bits.

  Next, the prediction method is changed in (2), but encoding in the normal case is performed using adjacent correlation of pixel values in each plane, but it is close to Equation 22 even if not as much as (1). When such a relationship is established between the two planes, the compression efficiency may be improved by using the correlation between the planes without using the adjacent correlation. That is, the above-described linear prediction is not used, and the previous Pm (i, j) -A * Pn (i, j) -B is used as the information to be encoded. Which one is used is determined by the previous entropy method. The estimation of the adjacent correlation is temporarily performed with the prediction direction obtained as the horizontal direction.

  The common use of coordinate parameters in (3) is an examination of whether there is a common color difference coordinate conversion coefficient obtained for each partial image. If there is something that can be shared, they are organized and written in the header of the compressed file. The degree of reduction in the amount of information due to the common use of the coefficients is the amount of information indicated by the coefficients (several bytes) * the number of common partial images.

  Next, as described above, the optimum prediction direction is detected (step S16) and the encoding direction is determined (step S17).

  When the correlation with other planes is used for the detection of the predicted value, a part of the step S16 is omitted. Finally, encoding is performed similarly (step S18).

  According to the present embodiment, since image division is performed in consideration of objects, the compression efficiency when compressing the continuous objects is improved. In addition, the effect spreads throughout. Further, since the information sharing between the planes is also studied, the compression efficiency is further improved.

  Further, the object extraction of this embodiment may be performed separately between the planes. In that case, the division into the partial images has a different position, size, and shape. The extraction may be performed using a luminance plane. Since the object extraction object of the present embodiment is to improve the compression efficiency, the extracted object is not necessarily the subject object. Further, the division into partial images does not necessarily have the same size and equal pitch. The information amount can be shared in addition to the above three items. In addition, the entropy method used for the sharing determination may be an actual compression result or a file size (code size).

  Next, a third embodiment of the present invention will be described. FIG. 3 shows a main routine of the lossless compression method for VGA images in the present embodiment. Steps S21 to S24 in FIG. 3 are the same as steps S11 to S14 in FIG.

  In the inter-plane sharing study in step S25, the correlation between any two planes is examined in the previous (1) reduction process of the number of planes. Specifically, the difference from the predicted value predicted by the previous equation (22), the sum of squares of the prediction error Pm (i, j) −A * Pn (i, j) −B is calculated. When the value is smaller than a predetermined value (threshold value), the encoding method is changed to a method using correlation between planes instead of a prediction method using adjacent correlation as described above. .

In step S26, similarly, a prediction value and a prediction error are obtained by LPC prediction using the long distance correlation from eight directions. At that time, for pixels with a large prediction error,
(1) Store the pixel coordinates.
(2) A large prediction error is stored in a header portion provided for each partial image.
(3) The prediction error is set to 0.
In this operation (point defect method), an increase in the number of expression bits due to the large prediction error is prevented. Steps S27 to S29 are the same as those in the second embodiment.

  According to the present embodiment, it is possible to reduce the amount of information after compression by treating a pixel showing a large prediction error as a point defect and separately recording the pixel value (prediction error). In the point defect method of this embodiment, for example, gradation steps existing at an object boundary or the like are separately recorded, and only a relatively flat and continuous object inside can be efficiently encoded. In addition, it is possible to remove the adverse effects of an image that includes discontinuous bright spots due to failure of pixel defect correction such as a digital camera.

  Next, a fourth embodiment of the present invention will be described. FIG. 4 shows a main routine for acquiring a still image of the digital camera in this embodiment. Step S31 is a still image capturing step using an image sensor such as a CCD.

The imaging sensor outputs luminance Y and color differences K1 and K2 expressed by Expressions 23 to 25 as outputs.
Y = W = R + G + B (Equation 23)
K1 = R / W (Equation 24)
K2 = B / W (Equation 25)

  The output is converted into a 256-gradation digital signal at the final stage of the image sensor. Step S32 is a step of transferring the captured analog signal to the frame memory and storing it as a digital signal.

  The luminance Y and color difference K1, K2 signals output from the image sensor are converted into digital signals with a maximum of 256 gradations by an AD converter or the like. The converted digital image information is stored in a frame memory existing in the digital camera (step S33).

In step S34, image division is performed as described above, and optimal color difference plane coordinates are selected for each partial image (step S35). The selection method is as shown in Expression 26 and Expression 27.
J1 = αK1 + βK2 (Equation 26)
J2 = αK1-βK2 (Formula 27)
Here, J1 and J2 are new color difference coordinate systems, and α and β are integers.

  Next, compression is performed using arithmetic codes (step S36), and the compressed image is recorded on a recording medium (step S37).

  According to the present embodiment, since the color difference plane is completely separated from the luminance plane, even if there is unevenness in the intensity distribution of illumination light hitting the subject object, the color difference plane has continuous pixel values. The efficiency of run mode and predictive coding is improved. For this reason, the file size of the output compressed image is also reduced.

  The color difference coordinates used in this embodiment are not limited to R / W and B / W, and may be R / G, B / G, or the like. The point of selection is that it is completely separated from the luminance plane. Further, the present embodiment can be easily applied not only to a digital camera but also to other imaging systems.

  As described above, according to the embodiment of the present invention, the encoding method according to the property of each partial image generated by dividing the entire image is adopted in each image encoding, so the compression efficiency of the entire image is improved. It becomes possible to make it.

  Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can also be achieved by reading and executing the program code stored in.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (basic system or operating system) running on the computer based on the instruction of the program code. Needless to say, a case where the functions of the above-described embodiment are realized by performing part or all of the actual processing and the processing is included.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion board or function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

It is a figure which shows the main routine of the reversible compression method of the color image in the 1st Embodiment of this invention. It is a figure which shows the main routine of the reversible compression method of the color image in the 2nd Embodiment of this invention. It is a figure which shows the main routine of the lossless compression method of the color image in the 3rd Embodiment of this invention. It is a figure which shows the main routine of the lossless compression method of the color image in the 4th Embodiment of this invention. It is a figure for demonstrating the division | segmentation method with respect to a VGA image. It is a figure for demonstrating the luminance difference of an up-down direction. It is a figure which shows the relationship between color coordinates (Cr, Cb) and new color coordinates (K1, K2). It is a figure for demonstrating the prediction direction for calculating | requiring a pixel prediction value. It is a figure which shows the relationship between an encoding direction and the prediction direction for calculating | requiring a pixel prediction value. It is a figure for demonstrating the relationship between an object and a divided image. It is a figure for demonstrating the correlation with an attention pixel and a surrounding pixel. It is a figure which shows the structure of the imaging device applicable to embodiment of this invention.

Explanation of symbols

10: Shooting lens 12: Shutter 14: Image sensor 15: CDS circuit 16: PGA
17: A / D converter 18: Timing generation circuit 20: Image processing circuit 22: Memory control circuit 24: Image display memory 26: D / A converter 28: Image display unit 30: Memory 32: Image compression / decompression circuit 40 : Exposure control means 42: Distance measurement control means 44: Zoom control means 46: Barrier control means 48: Strobe 50: System control circuit 52: Memory 54: Display unit 56: Non-volatile memory 60: Power switch (main switch)
62: Shutter switch SW1
64: Shutter switch SW2
70: Operation unit 72: Mode dial switch 80: Power supply control means 82, 84, 92, 206: Connector 86: Power supply means 90, 204: Interface 100: Imaging device 102: Protection means 104: Optical viewfinder 200: Recording medium 202: Recording section

Claims (15)

Separating means for separating image data into a luminance plane indicating at least one luminance and a color difference plane indicating at least one color difference;
Inclination correction means for correcting a luminance difference existing in the luminance plane;
An encoding processing apparatus comprising encoding processing means for performing encoding processing of the image data after luminance difference correction processing by the inclination correction means.   The encoding processing apparatus according to claim 1, wherein the inclination correction unit corrects a luminance difference in a vertical direction in the luminance plane. Coordinate conversion means for performing color coordinate conversion of the color difference plane based on a value indicating the maximum frequency in at least two of the color difference planes;
The encoding processing apparatus according to claim 1, wherein the encoding processing unit uses the color difference plane coordinate-converted by the coordinate conversion unit for encoding processing. A prediction error calculation means for obtaining a prediction error from each of a plurality of directions based on the target pixel;
Prediction direction determining means for determining a direction for predicting the pixel value of the pixel of interest based on the prediction error obtained for each of the plurality of directions;
The code according to any one of claims 1 to 3, wherein the prediction processing means determines a coding direction of cough image data based on the prediction direction determined by the prediction direction determination means. Processing equipment.   The encoding processing apparatus according to claim 4, wherein the prediction error calculation unit obtains a prediction error by using pixels up to at least 5 pitches away from the target pixel in each of the plurality of directions. Separating means for separating image data into a luminance plane indicating at least one luminance and a color difference plane indicating at least one color difference;
Plane reduction means for reducing at least one of the luminance plane and the chrominance plane based on the correlation between the luminance plane and the chrominance plane;
An encoding processing apparatus comprising: encoding processing means for performing encoding processing of the image data after plane reduction processing by the plane reduction means. Based on the correlation between the luminance plane and the color difference plane, further comprising a pixel value prediction means for predicting the pixel value of the target pixel using the correlation between the planes,
The encoding processing apparatus according to claim 6, wherein the encoding processing unit performs encoding processing of the image data using a pixel value predicted by the pixel value prediction unit. A dividing means for dividing the color difference plane into a plurality of parts;
Coordinate conversion means for performing color coordinate conversion of the color difference plane based on a value indicating the maximum frequency in the color difference plane;
8. The information sharing means for sharing information related to the color coordinate conversion process used in the color coordinate conversion process by the coordinate conversion means between the divided areas by the dividing means. Encoding processing apparatus.   9. The encoding processing apparatus according to claim 8, wherein the dividing unit determines the size of the divided area based on the size of an object existing in the image data.   10. The division unit according to claim 8 or 9, wherein the division unit determines a division start position based on a center of gravity position of the object, and determines a division region by translating a division line from the start position. Encoding processing device.   The code according to claim 4, further comprising prediction error value changing means for changing the prediction error to a predetermined value when the prediction error calculated by the prediction error calculating means is larger than a predetermined value. Processing equipment. An encoding processing method by an encoding processing device for encoding image data,
A separation step of separating image data into a luminance plane indicating at least one luminance and a color difference plane indicating at least one color difference;
An inclination correction step for correcting a luminance difference existing in the luminance plane;
An encoding processing step of performing encoding processing of the image data after the luminance difference correction processing in the inclination correction step. An encoding processing method by an encoding processing device for encoding image data,
A separation step of separating image data into a luminance plane indicating at least one luminance and a color difference plane indicating at least one color difference;
A plane reduction step of reducing at least one of the luminance plane and the chrominance plane based on the correlation between the luminance plane and the chrominance plane;
And a coding processing step of performing coding processing of the image data after the plane reduction processing in the plane reduction step.   A program for causing a computer to execute the encoding processing method according to claim 12 or 13.   The computer-readable recording medium which recorded the program of Claim 14.

Images