JP2011015347A - Apparatus and method for processing image, program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve coding efficiency using an efficient prediction technique for one signal while avoiding mounting of any excessive circuit or program by separating four signals of RGBG into three signals and one signal to use a color conversion system of three-signal input for the three signals and applying a coding system as it is without deformation.SOLUTION: Signals R, G1, B, G2 output from a CCD 301 are separated into three signals of R, G1, B and one signal of G2 and three signals are color-converted 302 into YUV signals, encoded and then stored in a storage device 306. In a predictive value/differential value calculation part 303, a predictive value G^2 of the signal G2 is calculated, a differential value D(G2-G^2) is calculated and D is subjected to run-length coding and stored in the storage device 306.

Description

  The present invention relates to an image processing apparatus, an image processing method, a program, and a recording medium that perform reversible color conversion on pixel data (RAW data) obtained from a single-plate color image sensor and perform reversible data compression / decompression. The present invention relates to a technique suitable for the technical field of color conversion and data compression / decompression in a digital camera using a color image sensor.

  Since the human visual system is a system that emphasizes the luminance signal rather than the color difference signal, and assigns a larger amount of information to the luminance signal component than the color difference signal during irreversible encoding, Encoding can be performed efficiently. There are Y, Cb, and Cr color coordinate systems and Y, I, and Q color coordinate systems as the color space of the luminance color difference.

A color space is an area in a vector space of three or more dimensions, and a color coordinate system is defined by some basis, for example, three primary independent three-dimensional vectors. Commonly used color coordinate systems are R (red), G (green), and B (blue), each defined by a center wavelength. Given a three-dimensional color coordinate system, another three-dimensional linear color coordinate system can be represented by a reversible (regular) 3 × 3 matrix.
For example, the Y, I, and Q color coordinate systems are defined by the following matrix using R, G, and B.

  Note that not all color spaces are linear. For example, in order to more accurately model the human visual system, there is also color conversion that rescals vectors nonlinearly (for example, logarithmically). Examples are CIE L * u * v and L * a * b.

  Various color coordinate systems have been defined for various reasons. For example, when displaying data on a monitor, it is convenient for most digital images to use a fixed range, for example, an 8-bit / coordinate R, G, B coordinate system. In applications requiring color decorrelation, such as compression, R, G, and B are not optimal, and other color coordinate systems such as Y, I, and Q described above are more suitable.

  For printing images, a subtractive color system such as CYM (cyan, yellow, magenta) is used, and a four-dimensional color space such as CMYK (cyan, yellow, magenta, black) is used depending on the application.

  By the way, data compression is an extremely useful tool for storing and transmitting large amounts of data, and the time required for image transmission such as facsimile transmission of a document is necessary for image reproduction by compressing the image by encoding. If the number of bits is reduced, it will be drastically shortened.

  Conventionally, there are various data compression methods, and the compression methods can be roughly classified into two types, that is, lossless encoding and lossy encoding. Since lossy coding includes coding that causes loss of information, there is no guarantee that the original data can be completely restored. The goal of lossy encoding is to make the difference from the original data inconspicuous. In lossless encoding, all information is stored and the data is compressed in such a way that it can be completely restored.

  In lossless encoding, input symbols or luminance data are converted into output codewords. The input includes image data, audio data, one-dimensional data (for example, data that changes over time), two-dimensional data (for example, data that changes in two spatial directions), or multidimensional / multispectral data. If compression is successful, the codeword is represented with fewer bits than the number of input symbols (or luminance data). The lossless encoding method includes a dictionary encoding method (for example, Lempel-Ziv encoding method), a run-length encoding method, an arithmetic encoding method, and an entropy encoding method.

In lossless image compression, compression is based on prediction or context and coding. The JBIG standard for facsimile compression and the DPCM for continuous tone images (differential pulse code modulation—an option of the JPEG standard) are examples of reversible compression for images. In lossy compression, input symbols or luminance data are quantized and then converted to output codewords. The purpose of quantization is to remove less important data while preserving appropriate feature values. Lossy compression systems often make use of transformations for energy concentration prior to quantization. Baseline JPEG is an example of a lossy encoding method for image data.
Traditionally, color coordinate system transformation has been used for lossy compression along with quantization. In fact, some color spaces are intentionally irreversible, such as CCIR 601-1 (YCbCr). In some reversible systems or reversible / irreversible systems, the reversibility and efficiency of the conversion is a major requirement, while in other reversible / irreversible systems, in addition to the efficiency of the reversible conversion, color non-correlation is also a factor. One requirement. For example, the 3 × 3 matrix is useful only for lossy compression because its entry is a non-integer, so when non-correlation is required, the error is increased during repeated compression and expansion. Further, it is not preferable to apply this 3 × 3 matrix to the lower bits. In other words, in order to obtain the required accuracy by applying this 3 × 3 matrix, and to be able to perform reverse processing thereafter and reproduce the lower bits, it is necessary to use extra bits. There is.

  When color space conversion is performed, problems relating to numerical accuracy arise. For example, when 8 bits are input, the required conversion space is generally 10 bits or 11 bits, and in order to obtain a stable color space, higher accuracy is required by internal calculation. If the process of converting an image from the RGB color space, compressing, decompressing and returning to the RGB color space is applied repeatedly with insufficient accuracy, errors accumulate and result in the original color and the final Color does not match. This is said to be the result of color drift or unstable color space.

  By the way, the single-chip CCD built in the digital camera outputs a plurality of series in which one red signal (R), one blue signal (B), and two green signals (G1 and G2) are combined (RAW data). ) The digital camera system interpolates the missing image signal during the CCD output to create a set of R, G, and B image signals (full color of 3 colors per pixel), and then JPEG encoding etc. The image data is compressed using the encoding technique and stored in the storage unit.

  However, the gradation (pixel range) that can be handled by JPEG encoding used in digital cameras and many software is still 8 bits, and the dynamic range (for example, 12 bits) of CCD output in recent years is There is a problem that it cannot be handled by the JPEG encoding used in. Also, in order to losslessly encode the CCD output (RAW data), the CCD output is interpolated for a general color conversion method for only three components, and a set of R, G, and B signals is obtained. If the obtaining method is adopted, it becomes impossible to reversibly code the RAW data.

  For this reason, various methods for color-converting the R, G1, B, and G2 signals of the CCD output as they are have been proposed (see, for example, Patent Documents 1 and 2).

  For example, Patent Document 1 discloses a method of converting R, G1, B, and G2 into a luminance signal Y, color difference signals Co and Cg, and other signals Dg. Although some conversion expressions are defined, for example, the following expressions are defined.

この式では、輝度信号Ｙと色差信号Ｃｇの計算にＲとＧ１とＢとＧ２のすべての値を用いている。

In this equation, all values of R, G1, B, and G2 are used for calculation of the luminance signal Y and the color difference signal Cg.

  Similarly to Patent Document 1, Patent Document 2 shows a method of converting luminance signal Y, color difference signals Cr and Cb, and other signals Cg from R, G1, B, and G2 to calculate luminance signal Y. Calculation is performed using all values of G1, B, and G2.

  The color conversion method from R, G1, B, and G2 shown in these Patent Documents 1 and 2 uses a calculation formula that uses the values of the first green signal G1 and the second green signal G2, We have proposed a reversible color conversion formula suitable for conversion.

  However, the color conversion formula described above requires input of four signals to calculate at least one value, and requires both G1 and G2 to calculate color difference values, so it is used in JPEG and JPEG XR. The equation for inputting the three signals cannot be used as it is.

  For this reason, it is difficult to apply a coding method that supports lossless conversion, such as JPEG2000 or JPEG XR, to RAW data coding, and a new method for coding this RAW data must be introduced. This is a significant negative for a digital camera system that requires a reduced circuit mounting area in terms of cost and design.

The present invention has been made in view of the above problems,
An object of the present invention is to divide 4 signals of RGBG into 3 signals and 1 signal, to use a commonly used color conversion method of 3 signal input for 3 signals, and to be used in a digital camera or the like. By applying the coding method as it is without changing the coding system, it is possible to avoid the mounting of extra circuits and programs, to reduce the mounting cost, and to use an efficient prediction method for one signal. Another object of the present invention is to provide an image processing apparatus, an image processing method, a program, and a recording medium that have improved encoding efficiency.

  The present invention is an image processing apparatus that sequentially processes four images of a red signal, a first green signal, a blue signal, and a second green signal with respect to an image captured by a single-layer CCD sensor having a Bayer arrangement. A color conversion means for converting the three signals of the signal, the first green signal and the blue signal into a luminance signal and a color difference signal; a first encoding means for compressing and encoding the luminance signal and the color difference signal; A calculation unit that calculates a predicted value of the green signal, calculates a difference signal between the calculated predicted value and the second green signal, and a second encoding that compresses and encodes the calculated difference signal The main feature is the provision of means.

  According to the present invention, by separating the RGBG 4 signals into 3 signals and 1 signal, it is possible to use a high-speed color conversion method with existing know-how for 3 signals, and efficient for 1 signal. Therefore, the encoding efficiency is improved. Further, when displaying on the viewfinder, a reduced image is formed only by processing of three signals, so that the amount of calculation is reduced and display is possible at high speed.

The structural example of a general encoding system is shown. The structural example of JPEG XR is shown. The structure of the Example of this invention is shown. The output (RAW data) from a single-plate CCD sensor in a Bayer array is shown. The processing flowchart of a prediction value / difference value calculation part and a run length encoding part is shown. The figure explaining the example of calculation of the predicted value of G2 is shown. The processing flowchart of a G2 signal decoding part and a run length decoding part is shown. The structural example of a digital camera is shown. The structural example of the digital camera when not using this invention is shown. An example of the configuration of a digital camera when JPEG XR is used as an encoder will be shown. An example of the configuration of a digital camera when the present invention is applied is shown.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a configuration example of a general encoding system. In FIG. 1, 101 is a color conversion unit, 102 is an image conversion unit (for example, DCT), 103 is a quantization unit, 104 is a coefficient prediction unit, 105 is a coefficient scanning unit, and 106 is an entropy coding unit.

  As in the configuration example shown in FIG. 1, color conversion processing is used for encoding, and an RGB color space obtained by an image pickup device such as a CCD in order to be applied to an encoding method using characteristics of a human visual system. Is converted into a luminance color difference color space. As the color conversion, conversion from RGB to YUV or YCrCb color space is performed.

  For example, in JPEG XR, as shown in FIG. 2, a reversible color conversion process 201 is first executed. In FIG. 2, 201 is a color conversion unit, and 202 is a JPEG XR image conversion unit. The image conversion unit 202 includes a first overlap filter 203, a first core transform 204, a second overlap filter 205, and a second core transform 206. Reference numeral 207 denotes a quantization unit, 208 denotes a DCAC prediction unit, 209 denotes a coefficient scanning unit, 210 denotes an entropy coding unit, and 211 denotes a bit stream generation unit.

  The present invention relates to color conversion, and when four signals of red, blue, green, and green, which are the output of a single-layer CCD with a Bayer array, are directly encoded by a general encoding method such as JPEG2000 or JPEG XR, The three-signal input / three-signal output color conversion, which is generally used, is processed using the three signals of red, blue, and green, and the remaining green signal (one signal) is used for the color conversion processing. The difference from the green signal or the predicted value is calculated and the difference from the predicted value is calculated, and the subsequent encoding process is performed. By using this general three-signal input / output color conversion, the reproduction (decoding) of only the three-signal data becomes a quarter size of the image obtained by upsampling the original RAW data. For example, if the digital camera image is half the maximum size and the horizontal size and the encoding method is scalable, such as JPEG2000 or JPEG XR, an extra circuit is added to the viewfinder of the digital camera. An image can be displayed without adding, and the system can be realized at low cost.

  In JPEG 2000, the basic configuration is the same, and wavelets are used for image conversion. In JPEG2000 at the time of reversibility, reversible color conversion (RCT) is used (see, for example, Patent Document 3), which is realized by the following equation.

  FIG. 3 shows the configuration of an embodiment of the present invention. Reference numeral 301 denotes a Bayer array single-plate CCD sensor, 302 a color conversion unit, 303 a predicted value / difference value calculation unit (hereinafter, calculation unit), 304 a JPEG XR encoding unit, 305 a run-length encoding unit, and 306 307 is a JPEG XR decoding unit, 308 is a reverse color conversion unit, 309 is a run length decoding unit, 310 is a G2 signal decoding unit, 311 is restored RAW data, 312 is a viewfinder display unit, and 313 An external display unit.

  FIG. 4 shows an output (RAW data) from a Bayer array single-plate CCD sensor. The color conversion unit 302 and the calculation unit 303 are sequentially set in the x direction (main scanning direction) and y direction (sub scanning direction) in units of four pixel signals of G, B / R, and G (the arrangement is not limited to this example). ) To process.

  When the G, B / R, and G signals are G1, B / R, and G2, the G1, B / R, and G2 signals are divided into two signals of R, G1, B, and G2, and a color conversion unit In 302, the existing reversible color conversion method (Equation 3) is used for the R, G1, and B signals, and one luminance signal Y and two color difference signals U and V are obtained. The output Y, U, and V signals from the color conversion unit 302 have a resolution that is ¼ that of the RAW data (the output Y, U, and V pixels (one pixel) are G, B / R, and G pixels). It is considered to be at the center of 4 pixels).

  The Y, U, and V signals are encoded by the JPEG XR encoding unit 304 and stored in the storage device 306.

  On the other hand, the calculation unit 303 calculates the predicted value G ^ 2 and the difference value D of the G2 signal. FIG. 5 shows a processing flowchart of the calculation unit and the run-length encoding unit, and FIG. 6 shows a diagram for explaining a calculation example of the predicted value of G2.

  The processing in the calculation unit will be described with reference to FIGS. In step 401, the predicted value G ^ 2 of the green signal G2 is calculated. In FIG. 6, in the raster scan order, the predicted value and difference value of each pixel of the green signals G20, G21, G22, G2, and G23 are calculated and encoded, and the pixel of the green signal G2 to be processed is now G2x. Suppose there is.

When G2x is a pixel value of a pixel (target pixel) to be predicted (processed) and G ^ 2x is a predicted value,
The difference value D is D = G2x−G ^ 2x
(Step 402). This difference value D is encoded (step 403). The above processing is performed for all pixels (one screen) of the G2 signal (step 404).

An example of calculating the predicted value G ^ 2x is shown below.
(1) Average prediction 1
A predicted value is simply obtained by using an average value of four G1 signals (G10 to G13) around G2x.

(2) Average prediction 2
A prediction value is obtained by using a mean square value of four G1 signals (G10 to G13) around G2x.

  Using the green signal near the prediction target as the predicted value, the average is used as the predicted value. In general, natural images tend to change smoothly. In addition, since the resolution of digital cameras in recent years is expected to change the signal values in the vicinity with similar properties, the average value in the vicinity is a good prediction. Expected to give value.

(3) Average prediction 3
The mean square value of four G1 signals around G2x, the mean square value of G20, G21, and G22 of the G2 signal components that have been encoded so far (the mean value on the upper side of G2x), and G23 (G2x A value obtained by adding a value obtained by adding a weight to the left side) is a predicted value.

  The following is a method of switching the calculation formula for the calculation of the predicted value depending on the condition of the green signal around the prediction target.

(4) Prediction by context 1
As in Equation 7, when one adjacent difference value of the G1 signal around G2x is smaller than ε and the other adjacent difference value is larger than ε, a smaller value of the difference value is set as a predicted value of G2x. Used as

(5) Prediction by context 2
As shown in Equation 8, when one adjacent difference value of the G1 signal around G2x is smaller than ε and the other adjacent difference value is larger than ε, the values in the direction in which the difference is smaller have the same value. And the G2 signal value in that direction is used as the predicted value of G2x.

  Note that the G1 signal (G10 signal) may be used as the predicted value of the G2x signal. That is, the difference signal D is D = G2x−G10. Due to the nature of natural images and the recent improvement in resolution of digital cameras, close signal values often have close values, so it is expected that simply using G1 will be an efficient prediction value.

  As described above, the predicted value of the G2x signal is calculated, and the difference value D between G2x and the predicted value is encoded. The difference value D from the predicted value increases the number of occurrences of the value 0 or a value close to the value 0, and when the run length encoding method in which a short code is assigned to the value close to the value 0 is used, the compression rate Can be expected.

  The difference value D calculated for one image is encoded by the run-length encoding unit 305 and stored in the storage device 306.

  Next, the decoding process will be described. The encoded data stored in the storage device 306 is decoded by the JPEG XR decoding unit 307, and then the YUV signal decoded by the inverse color conversion unit 308 is inversely converted into R, G1, and B signals ( Formula 3). The image data that has been inversely converted into R, G1, and B signals displays a reduced image on the viewfinder 312 when the image data is simply displayed. In the present invention, by using a general color conversion formula and ignoring the D signal, the reduced image of the compressed RAW data becomes exactly the same as the encoding method used in the system, so a thumbnail as a preview Image display can be realized at low cost.

  On the other hand, the D signal in the storage device 306 is decoded by the run-length decoding unit 309, and the G2 signal decoding unit 310 decodes the G2 signal based on the decoded D signal and the predicted value G ^ 2. The RAW data 311 is restored by the R, G1, and B signals subjected to inverse color conversion with the decoded G2 signal.

  FIG. 7 shows a processing flowchart of the G2 signal decoding unit and the run-length decoding unit.

  First, the difference value D is decoded (step 501), the predicted value G ^ 2x of G2x is calculated (step 502), and by adding the predicted value G ^ 2x and the difference value D (step 503), the original value is calculated. G2x of the value of is decoded. The above processing is performed for all pixels (one screen) of the G2 signal (step 504).

  Decoding can process the G1 signal component and the G2 signal component separately. Therefore, since the decoding of the G1 signal component can be performed before G2, the G10, G11, G12, and G13 signal values are decoded when reproducing the predicted value of G2x as shown in FIG. Further, when G2 is decoded in, for example, raster order, the values of the G20, G21, G22, and G23 signals are G2 signals decoded before G2x as shown in FIG. When reproducing the predicted values, the G2 signals are decoded.

When G ^ 2x is a predicted value and the decoded difference value is D, the decoded value G2x is
G2x = D + G ^ 2x.

A calculation example of the predicted value G ^ 2x is shown below (using the same calculation formula as used at the time of encoding).
(6) Average prediction 1
Simply using the average of four G1 signals around G2x

(7) Average prediction 2
Using the root mean square of four G1 signals around G2x

(8) Average prediction 3
The mean square values of the four G1 signals around G2x, and the mean square values of G20, G21, and G22 (the average value on the upper side of G2x) and G23 (G2x of G2x) of the G2 signal components that have been decoded so far The value on the left) is added by weighting

(9) Prediction by context 1
As shown in Equation 12, when one adjacent difference value of the G1 signal around G2x is smaller than ε and the other adjacent difference value is larger than ε, a small value of the difference value is set as a predicted value of G2x. Used as

(10) Prediction by context 2
As shown in Equation 13, when one adjacent difference value of the G1 signal around G2x is smaller than ε and the other adjacent difference value is larger than ε, the values in the direction in which the difference is smaller have the same value. Using the G2 signal value in that direction as the predicted value of G2x

  Note that the G10 signal may be used as the predicted value G ^ 2x of the G2x signal. That is, the G2x signal is G2x = D + G10.

  Although the method for obtaining the predicted value has been described above, the present invention decomposes the four signals of red, blue, first green, and second green, such as a Bayer array single-plate CCD output, into three signals and one signal, The decomposed red, blue, and first green signals are then processed using color conversion of three-signal input / three-signal output, and the remaining second green signal calculates its predicted value, and the difference from the predicted value is calculated. As a result, a 3-input 3-output encoding process is performed as a color signal that is currently widely used.

  Decomposing these 4 signals into 3 signals and 1 signal to reproduce (decode) only 3 signals of code data is equivalent to decoding of 3 signal input encoding, so the encoding method is scalable. For example, JPEG2000, JPEG XR, etc. can display an image displayed on the viewfinder of a digital camera without adding an extra circuit, and can realize a system at low cost. .

  FIG. 8 shows a configuration example of a digital camera. The image captured by the CCD 601 enters the interpolation processing unit 602, and each signal is interpolated, encoded by the image encoder 603, input to the storage device 604, and stored in a memory card or the like. When previewing the stored code data, after decoding using the decoder 605, the code data is reduced and displayed on the finder display unit 606.

  Decoding processing and reduction can be performed simultaneously. For example, in JPEG, a 1/64 size image can be obtained by extracting only the DC component. Further, it is possible to directly pass data from the encoder to the decoder at the time of finder display.

  Now, a configuration example of a digital camera when the present invention is not used is shown below. In the configuration shown in FIG. 9, a four-component (R, G1, G2, B) encoder 608 is newly added as RAW data encoding. In addition, a 4-component decoder 609 is newly required for displaying the RAW data, the hardware configuration is increased, and the mounting cost is also increased.

  FIG. 10 shows a configuration example of a digital camera when a 4-component encoder (JPEG XR) is used as a RAW data encoder. In JPEG XR, since the maximum value of the color signal component of an image is not three components, for example, when JPEG XR that supports one component, three components, and four components is used, it can be configured as shown in FIG. Although the number of encoders is smaller than that of the configuration example of FIG. 9 and it is easier to configure, two types of color conversion (703, 704) having the same number of gates are necessary as color conversion, and the mounting area increases. . Also, two kinds of color conversion (708, 709) are required on the decoding side for display, and the mounting area increases.

  FIG. 11 shows a configuration example of a digital camera when the present invention is applied. In this configuration example, JPEG XR is used as an encoder. In order to clarify the method, the color converter 703 and the inverse color converter 708 are separately described. Therefore, although the number of blocks is large, the configuration other than the prediction calculation and the difference block 712 is the same as the configuration example of FIG. (The color conversion and encoder are shared for full color and RAW data. However, the D signal is encoded by JPEG XR).

  In other words, the present invention can be configured by adding only the prediction calculation and difference block 712, and the prediction calculation and difference block itself can be mounted with a smaller area than the color converter by selecting an implementation method. . Further, no special block for displaying RAW data is required.

  As described above, RAW data encoding can be efficiently implemented by applying the present invention to a digital camera using a Bayer array single-plate CCD.

  According to the present invention, a storage medium in which a program code of software for realizing the functions of the above-described embodiments is recorded is supplied to a system or apparatus, and a computer (CPU or MPU) of the system or apparatus is stored in the storage medium. This is also achieved by reading and executing the code. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment. As a storage medium for supplying the program code, for example, a hard disk, an optical disk, a magneto-optical disk, a nonvolatile memory card, a ROM, or the like can be used. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program code. A case where part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included. Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. A case where the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing is included. Further, the program for realizing the functions and the like of the embodiments of the present invention may be provided from a server by communication via a network.

DESCRIPTION OF SYMBOLS 301 CCD sensor 302 Color conversion part 303 Predicted value / difference value calculation part 304 JPEG XR encoding part 305 Run length encoding part 306 Storage device 307 JPEG XR decoding part 308 Inverse color conversion part 309 Run length decoding part 310 G2 signal Decoding unit 311 Raw data 312 Viewfinder display unit 313 External display unit

JP 2006-121669 A JP 2006-067390 A JP-A-9-6952

Claims (12)

  An image processing apparatus that sequentially processes four images of a red signal, a first green signal, a blue signal, and a second green signal with respect to an image captured by a Bayer array single-plate CCD sensor, wherein the red signal and the first signal Color conversion means for converting three signals of green signal and blue signal into luminance signal and color difference signal, first encoding means for compressing and encoding the luminance signal and color difference signal, and prediction of the second green signal A calculating unit that calculates a value and calculates a difference signal between the calculated predicted value and the second green signal; and a second encoding unit that compresses and encodes the calculated difference signal. An image processing apparatus characterized by that.   The image processing apparatus according to claim 1, wherein the first green signal in the vicinity of the second green signal is used as the predicted value.   The image processing apparatus according to claim 1, wherein the first green signal in the vicinity of the second green signal and the second green signal that has already been encoded are used as the predicted value.   As the predicted value, a pair of first green signals selected in accordance with the magnitude of a difference value between the adjacent first green signals in the vicinity of the second green signal, or the pair of first green signals The image processing apparatus according to claim 1, wherein a second green signal that has already been encoded is used in the vicinity of.   An image processing device that sequentially processes four images of a red signal, a first green signal, a blue signal, and a second green signal with respect to an image captured by a Bayer array single-plate CCD sensor. And a first decoding means for decoding the color difference signal, an inverse color conversion means for inversely converting the decoded luminance signal and color difference signal into a red signal, a first green signal and a blue signal, Second decoding means for decoding the difference signal; third decoding means for decoding the second green signal based on the decoded difference signal and the predicted value of the second green signal; An image processing apparatus comprising:   The image processing apparatus according to claim 5, wherein the first green signal in the vicinity of the second green signal is used as the predicted value.   The image processing apparatus according to claim 5, wherein the first green signal and the second green signal that have already been decoded are used as the predicted value in the vicinity of the second green signal.   As the predicted value, a pair of first green signals selected in accordance with the magnitude of a difference value between the adjacent first green signals in the vicinity of the second green signal, or the pair of first green signals The image processing apparatus according to claim 5, wherein a second green signal that has already been decoded is used in the vicinity of.   An image processing method of sequentially processing four images of a red signal, a first green signal, a blue signal, and a second green signal for an image captured by a single-layer CCD sensor having a Bayer arrangement, wherein the red signal and the first signal A color conversion step for converting the three signals of the green signal and the blue signal into a luminance signal and a color difference signal, a first encoding step for compressing and encoding the luminance signal and the color difference signal, and prediction of the second green signal A calculation step of calculating a value, calculating a difference signal between the calculated predicted value and the second green signal, and a second encoding step of compressing and encoding the calculated difference signal. An image processing method characterized by the above.   An image processing method for sequentially processing, in units of four signals of a red signal, a first green signal, a blue signal, and a second green signal, on an image captured by a Bayer array single-plate CCD sensor, A first decoding step for decoding the color difference signal, a reverse color conversion step for reversely converting the decoded luminance signal and color difference signal into a red signal, a first green signal, and a blue signal, A second decoding step for decoding the difference signal; a third decoding step for decoding the second green signal based on the decoded difference signal and the predicted value of the second green signal; An image processing method comprising:   A program for causing a computer to implement the image processing method according to claim 9 or 10.   A computer-readable recording medium on which a program for causing a computer to realize the image processing method according to claim 9 or 10 is recorded.

Images