WO2011010431A1 - Dispositif de traitement d’images, procédé de traitement d’images et dispositif de capture d’images - Google Patents

Dispositif de traitement d’images, procédé de traitement d’images et dispositif de capture d’images Download PDF

Info

Publication number
WO2011010431A1
WO2011010431A1 PCT/JP2010/004388 JP2010004388W WO2011010431A1 WO 2011010431 A1 WO2011010431 A1 WO 2011010431A1 JP 2010004388 W JP2010004388 W JP 2010004388W WO 2011010431 A1 WO2011010431 A1 WO 2011010431A1
Authority
WO
WIPO (PCT)
Prior art keywords
color
data
raw data
compression
image
Prior art date
Application number
PCT/JP2010/004388
Other languages
English (en)
Japanese (ja)
Inventor
秦野敏信
久方和之
橋永寿彦
吉廣秀章
Original Assignee
パナソニック株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニック株式会社 filed Critical パナソニック株式会社
Publication of WO2011010431A1 publication Critical patent/WO2011010431A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46—Colour picture communication systems
    • H04N1/64—Systems for the transmission or the storage of the colour picture signal; Details therefor, e.g. coding or decoding means therefor

Definitions

  • the present invention relates to an image processing apparatus and an image processing method for compressing RAW data in a data format in which a plurality of types of color components constituting a color image are repeatedly arranged on a pixel array in accordance with a certain rule.
  • the present invention relates to a technique for improving the compression efficiency of color-specific plane data.
  • the present invention also relates to an image pickup apparatus in which the image processing apparatus as described above is mounted and an image sensor of a type that performs color separation and picks up images. Examples of the target image processing apparatus and imaging apparatus include a digital still camera, a digital video camera, an independent image scanner, and an image scanner incorporated in a copying machine.
  • “Plain data” refers to data having an array form that is two-dimensionally developed when data is developed on an image memory.
  • a CCD-type or MOS-type image sensor for obtaining a color image is a color filter array in which color filters of a plurality of types of color components are repeatedly arranged according to a certain rule corresponding to the two-dimensional array of pixels in the image sensor.
  • Color separation filter For example, in a Bayer array color separation filter in which RGB primary color filters are arranged in a checkered pattern corresponding to the pixels of the image sensor, filters for each color component of RGB along both the horizontal and vertical directions. Are arranged every other pixel (BGgR).
  • the information obtained from each pixel is only information of one color component. Therefore, in order to enhance the expressive power, each pixel is interpolated using the color information of the surrounding pixels, and information on a plurality of color components is obtained for each pixel. That is, information on all color components is obtained for all pixels. This is called synchronized color interpolation processing.
  • color image data of each color component having the same number of pixels as that of the image sensor is obtained.
  • signal processing such as white balance (WB) adjustment, gamma correction processing, enhancement processing for edge enhancement, etc. is performed and converted into component signals of a luminance signal (Y) and two types of color difference signals (Cr, Cb). Then, the data is compressed with a compression encoding algorithm such as JPEG (Joint Photographic Experts Group) to obtain a small file size and recorded on a recording medium.
  • WB white balance
  • Cr color difference signals
  • JPEG Joint Photographic Experts Group
  • raw (RAW) digital data (without image processing) immediately after A / D conversion of an imaging analog signal is performed for high-quality development processing and retouch processing performed after shooting.
  • RAW data is reversibly compressed and recorded on a recording medium.
  • Image reproduction (development) processing is performed by an external device such as a personal computer, thereby realizing high-quality printing and image editing that matches the user's purpose.
  • RAW data without image processing is used because if image data with image processing added is recorded and if it is decoded for development processing or retouch processing and subjected to further image processing, the image quality deteriorates. It is.
  • Patent Document 1 discloses a common compression processing step for two modes, a mode for recording after performing the synchronized color interpolation process and a mode for recording in the RAW data state without performing the synchronized color interpolation process. A method of compressing independently is described (see FIGS. 13A and 13B).
  • Patent Document 2 describes a method of compressing component data separated for each color component from RAW data for each component data without going through a synchronized color interpolation process (see FIGS. 14 and 15). ).
  • RAW data is divided into color-specific plane data with high correlation between adjacent pixels, and the plane data is compressed. Therefore, the RAW data is higher than the case where RAW data not separated by color is compressed as it is. It is said that compression efficiency can be obtained.
  • a plurality of color plane data obtained by dividing RAW data is individually compressed. That is, the same compression process is repeatedly performed in four times. It takes time to switch the plane data during this repetition. Specifically, switching from the compression process of the first plane data to the compression process of the second plane data, switching from the compression process of the second plane data to the compression process of the third plane data, There are three times of switching, such as switching from the compression processing of the plane data to the compression processing of the fourth plane data. In this way, if a plurality of pieces of plane data for each color are individually compressed sequentially before and after each other, it takes a lot of time in total. Therefore, the compression efficiency is low.
  • Patent Document 2 also describes that a plurality of compression processing steps that can be operated in parallel are used (see paragraph [0032]). However, in that case, a CPU (Central Processing ⁇ ⁇ ⁇ Unit) having a very high processing capability is required. In addition, when the hardware configuration is adopted, the circuit scale is significantly increased. If the compression efficiency is low, it is disadvantageous for high-speed processing in the compressed RAW data recording mode in response to the recent increase in pixels of image sensors.
  • a CPU Central Processing ⁇ ⁇ ⁇ Unit
  • an imaging apparatus having a continuous shooting function may interfere with high-speed continuous shooting in the compressed RAW data recording mode.
  • the possibility of occurrence of block noise is increased.
  • the present invention was created in view of such circumstances, and it is intended to increase the compression efficiency of a plurality of color-specific plane data, and to avoid increasing the circuit scale and CPU processing capacity.
  • the purpose is that. More desirably, the object is to suppress the occurrence of block noise.
  • the present invention solves the above problems by taking the following measures.
  • An image processing apparatus is a RAW data (image data immediately after A / D conversion, in which signal processing is performed) in which a plurality of types of color components constituting a color image are repeatedly arranged on a pixel array according to a certain rule.
  • a RAW data reconstructor configured as follows and a compression processor.
  • the RAW data reconstructor inputs RAW data that has not been subjected to signal processing after A / D conversion of the analog signal of the image, and processes it as follows. That is, the input RAW data is decomposed for each color component and reassembled to generate a plurality of color-specific plane data. For example, it is assumed that the RAW data is obtained by repeatedly arranging the first to fourth color components on the pixel array according to a certain rule.
  • the RAW data is decomposed into a first color component, a second color component, a third color component, and a fourth color component, and the first color-specific plane data in which only the first color component is collected and only the second color component are collected.
  • the plane data is data having an array form that is expanded two-dimensionally when expanded on the image memory.
  • the case where the two color-specific plane data are the same color as in the Bayer array (BGgR) is also included.
  • the RAW data reconstructor arranges the plurality of color-specific plane data and collects them as one file that is a compression processing unit.
  • one file which is a compression processing unit, has a plurality of arrangement areas divided for each color component.
  • the RAW data reconstructor distributes and arranges the plurality of color-specific plane data in a plurality of arrangement areas divided for each color component, and generates reconstructed RAW data.
  • the first color-specific plane data is arranged in the first arrangement area
  • the second color-specific plane data is arranged in the second arrangement area
  • the third arrangement area is in the first arrangement area.
  • 3 color-specific plane data is arranged
  • the fourth color-specific plane data is arranged in the fourth arrangement area to generate reconstructed RAW data collected as one file as a compression processing unit.
  • the generated reconstructed RAW data is passed to the compression processor.
  • the compression processor inputs the reconstructed RAW data of the compression processing unit generated by the RAW data reconstructor, and performs the compression process.
  • the relative positional relationship between the first to fourth arrangement areas is arbitrary.
  • the image processing apparatus of the present invention is an apparatus that compresses RAW data in a data format in which a plurality of types of color components constituting a color image are repeatedly arranged on a pixel array according to a certain rule. And The RAW data is input, the RAW data is decomposed and reassembled for each color component to generate a plurality of color plane data, and the plurality of color classification A RAW data reconstructor for generating reconstructed RAW data arranged as a single file as a compression processing unit by arranging plain data; A compression processor that inputs and compresses the reconstructed RAW data of the compression processing unit generated by the RAW data reconstructor; It is comprised as that provided.
  • the image processing method of the present invention is a method of compressing RAW data in a data format in which a plurality of types of color components constituting a color image are repeatedly arranged on a pixel array according to a certain rule,
  • the RAW data is input, the RAW data is separated for each color component and reassembled to generate a plurality of color-specific plane data, and the plurality of color-specific plane data are divided into a plurality of arrangement areas divided for each color component.
  • the compression processing target of the compression processor is “reconstructed RAW data in which a plurality of color plane data are arranged and collected as one file as a compression processing unit”.
  • the plane data for each color has a high correlation between adjacent pixels, and the compression process provides a higher compression efficiency than the case where raw data with different color components are adjacent is compressed as it is.
  • This is also recognized in the prior art as described above.
  • not only that, but also reconstructed RAW data configured by arranging a plurality of types of color-specific plane data on one file which is a unit of compression processing is targeted for compression processing. Therefore, a plurality of color-specific plane data on the reconstructed RAW data can be compressed at once, and higher compression efficiency can be obtained.
  • the compression processing of the plane data for each color component by color is realized by one compression processing in one file (reconstructed RAW data) that is a unit of compression processing, a plurality of processing is performed as in the case of the prior art. Compared with the case of repeatedly performing compression processing while sequentially switching the color-specific plane data individually, the compression efficiency is greatly improved.
  • An image pickup apparatus includes an image pickup unit that converts an optical image input by an image sensor that performs color separation and picks up an image into an analog electric signal, and further converts it into digital RAW data, and the image processing device described above. It is equipped with.
  • a RAW data reconstructor and compression processing is performed on reconstructed RAW data configured by arranging a plurality of types of color-specific plane data having different color components on one file as a compression processing unit.
  • FIG. 1 is a block diagram illustrating a schematic configuration of an image processing apparatus that compresses reconstructed RAW data generated from RAW data according to an embodiment of the present invention. It is an illustration figure of the RAW data which are related with embodiment of this invention and are compression object.
  • FIG. 4 is an illustrative diagram in which four color plane data are adjacently arranged in a two-dimensional direction according to an embodiment of the present invention.
  • FIG. 4 is an illustrative diagram in which four color-specific plane data are arranged at appropriate intervals according to an embodiment of the present invention.
  • FIG. 4 is an explanatory diagram (part 1) showing reconstructed RAW data reconstructed as image data for one frame by two-dimensionally arranging four color-specific plane data in a Bayer array according to an embodiment of the present invention. . It is explanatory drawing of the block noise generation
  • FIG. 6 is an explanatory diagram (part 2) showing reconstructed RAW data in which four-color plane data of a Bayer array is two-dimensionally arranged and reconstructed as image data for one frame according to the embodiment of the present invention. .
  • FIG. 6 is an explanatory diagram of block noise occurrence positions in image data when the compressed RAW data in FIG.
  • FIG. 5 is decompressed according to the embodiment of the present invention. It is a figure which shows the mode of the block noise corresponding to embodiment of this invention and corresponding to FIG. 5, FIG. It is a figure which shows the mode of the block noise corresponding to embodiment of this invention and corresponding to FIG. 3, FIG. FIG. 7 is an explanatory diagram showing a block noise occurrence position of FIG. 6 corresponding to FIG. 7A according to the embodiment of the present invention. FIG. 7 is an explanatory diagram showing a block noise occurrence position of FIG. 4 corresponding to FIG. 7B according to the embodiment of the present invention. It is explanatory drawing which concerns on embodiment of this invention and shows the other form of four independent arrangement
  • FIG. 2 is a block diagram illustrating a schematic configuration of an imaging apparatus including an image processing apparatus that compresses reconstructed RAW data generated from RAW data according to an embodiment of the present invention.
  • It is explanatory drawing which shows the example (Bayer arrangement) of the color filter arrangement
  • It is explanatory drawing which shows the example (honeycomb arrangement
  • the RAW data reconstructor when arranging the plurality of color plane data in the one file, includes a compression processing unit block in the compression processor.
  • the reference position of each color plane data is shifted by a predetermined number of pixels in both the vertical and horizontal directions with respect to the periodic boundary position.
  • the step of generating the reconstructed RAW data includes arranging the plurality of color-specific plane data in the one file. In doing so, there is an aspect in which the reference position of each color plane data is shifted by a predetermined number of pixels in both the vertical and horizontal directions with respect to the periodic boundary position of the compression processing unit block in the compression processor.
  • the RAW data reconstructor when arranging the plurality of color plane data in the one file, is a compression processing unit block in the compression processor.
  • the reference position of each color plane data is arranged so as to be shifted by a predetermined pixel in either one of the vertical direction and the horizontal direction.
  • the step of generating the reconstructed RAW data includes arranging the plurality of color-specific plane data in the one file.
  • the reference position of the plane data for each color is shifted by a predetermined number of pixels in either the vertical direction or the horizontal direction with respect to the periodic boundary position of the compression processing unit block in the compression processor.
  • the compression processing is compressed. If decompression processing is performed for reproduction in the case of irreversible compression with a high rate, the block noise generation position of each color corresponding to the boundary position of the compression processing unit block in the reconstructed RAW data for one frame obtained. Overlapping characteristic image quality degradation may occur, and block noise may increase.
  • each color plane data is shifted relative to the periodic boundary position of the compression processing unit block configured as in (2) and (3) above, the block of each color It is possible to suppress image quality degradation peculiar to where noise generation positions overlap and to improve image quality.
  • the RAW data reconstructor arranges fixed data in a non-image area between the arrangement areas of the plurality of color-specific plane data, and There is an aspect of being configured to generate reconstructed RAW data.
  • the step of generating the reconstructed RAW data is performed between the plurality of color-specific plane data arrangement regions.
  • fixed data is arranged in the image area and the reconstructed RAW data is generated.
  • the fixed data is pixel data having no luminance change.
  • the compression processor has a plurality of component input terminals, the reconstructed RAW data input from one of the component input terminals, and the other component There is an aspect in which fixed data input from an input terminal is compressed simultaneously.
  • the compression processing step inputs the reconstructed RAW data and the fixed data in different systems, and There is an aspect in which the reconstructed RAW data and the fixed data are simultaneously compressed.
  • the compression processor has a plurality of component input terminals, the reconstructed RAW data input from one of the component input terminals, and the other component There is an aspect in which another image data input from the input terminal is compressed simultaneously.
  • the compression processing step inputs the reconstructed RAW data and another image data in different systems. There is a mode in which the reconstructed RAW data and the other image data are simultaneously compressed.
  • the reconstructed RAW data and another image data are simultaneously compressed to reduce the size for display. It is possible to further increase the compression efficiency including another image such as an image. Also in this case, resources of a compression processor in a general form having component input terminals for luminance signals (Y) and color difference signals (Cr / Cb) such as JPEG can be effectively used.
  • the compression processor may be configured to irreversibly compress the reconstructed RAW data.
  • the compression processing step includes irreversible compression of the reconstructed RAW data. JPEG, MPEG (Moving Picture Experts Group), H.264.
  • the lossy compression such as H.264 is a compression process with a higher compression rate than the lossless compression. Therefore, compressed RAW data with a smaller file size can be obtained.
  • the image processing method of the present invention may be configured as a single application software, or may be incorporated as a part of an application such as image processing software or file management software.
  • the image processing program according to the image processing method of the present invention is not limited to being applied to a computer system such as a personal computer, and the operation of a central processing unit (CPU) incorporated in an information device such as a digital camera or a cellular phone. It can also be applied as a program.
  • a computer system such as a personal computer
  • CPU central processing unit
  • an information device such as a digital camera or a cellular phone. It can also be applied as a program.
  • FIG. 1 is a block diagram illustrating a schematic configuration of an image processing apparatus that generates reconstructed RAW data from RAW data acquired by an imaging unit having a color filter with a Bayer array and compresses the reconstructed RAW data.
  • reference numeral 1 denotes an image pickup unit (image sensor) having a color filter in which a plurality of types of color components constituting a color image are repeatedly arranged on a pixel array according to a certain rule.
  • the color filter has four types of color components B (blue component), G (first green component), g (second green component), and R (red component) having a Bayer array.
  • B blue component
  • G first green component
  • g second green component
  • R red component
  • Reference numeral 2 denotes RAW data in a data format based on a Bayer array having four color components B, G, g, and R acquired by the imaging unit 1.
  • RAW data 2 is decomposed into color components B, G, g, and R and reassembled two-dimensionally to generate four color-specific plane data 41, 42, 43, 44. Further, the four color-specific plane data 41, 42, 43, and 44 are arranged in four independent arrangement areas a1, a2, a3, and a4 divided for each color component to form one file as a compression processing unit. It is a RAW data reconstructor that generates combined reconstructed RAW data 4.
  • Reference numeral 5 denotes a compression processor that inputs the reconstructed RAW data 4 of the compression processing unit generated by the RAW data reconstructor 3 as luminance data and performs irreversible compression processing.
  • RAW data 2 output from the imaging unit 1 and input to the RAW data reconstructor 3 is image data immediately after A / D conversion of the imaging analog signal in the imaging unit 1, and is a synchronized color interpolation process and a gamma correction process.
  • the image data is not subjected to signal processing such as white balance adjustment.
  • the RAW data 2 is mosaic image data in which only one color information different for each pixel corresponding to the color filter array pattern is held.
  • the RAW data reconstructor 3 receives the RAW data 2 that has not been subjected to signal processing, decomposes the input RAW data 2 for each of the color components, and reassembles the four pieces of plane data 41 and 42 for each color. , 43, 44 are generated. That is, the RAW data 2 is obtained by repeatedly arranging the first to fourth color components on the pixel array in accordance with a certain rule.
  • the RAW data 2 is composed of the first color component (B) and the second color component (G ), First color-specific plane data 41 that is decomposed into the third color component (g) and the fourth color component (R) and collects only the first color component (B), and the second color component (G) Second color plane data 42 that collects only the third color plane data 43 that collects only the third color component (g), and a fourth color that collects only the fourth color component (R) Another plane data 44 is generated.
  • the four color components in the Bayer array are the first color component (B), the second color component (G), the third color component (g), and the fourth color component (R), and the second color component. Both (G) and the third color component (g) are green, but these two are treated as independent plane data by color.
  • the RAW data reconstructor 3 further arranges the four color-specific plane data 41, 42, 43, and 44 in a two-dimensional manner and collects them as one file that is a compression processing unit.
  • one file as a compression processing unit has four arrangement areas a1, a2, a3, and a4 divided for each color component.
  • the RAW data reconstructor 3 distributes and arranges the four color-specific plane data 41, 42, 43, and 44 in the four arrangement areas a1, a2, a3, and a4 divided for each color component.
  • RAW data 4 is generated. That is, the first color-specific plane data 41 is arranged in the first arrangement area a1 corresponding to the color component B (blue), and the second arrangement area a2 corresponding to the color component G (first green).
  • color-specific plane data 42 is arranged, the third color-specific plane data 43 is arranged in the third arrangement area a3 corresponding to the color component g (second green), and it corresponds to the color component R (red).
  • the fourth color-specific plane data 44 is arranged in the fourth arrangement area a4 to be generated, and the reconstructed RAW data 4 collected as one file as a compression processing unit is generated. Then, the generated reconstructed RAW data 4 is transferred to the compression processor 5.
  • the compression processor 5 receives the reconstructed RAW data 4 of the compression processing unit generated by the RAW data reconstructor 3 and performs compression processing.
  • the compression process may be either lossy compression (lossy encoding) or lossless compression (lossless encoding).
  • the relative positional relationship between the first to fourth arrangement regions a1 to a4 is arbitrary. Here, it is arranged two-dimensionally in two horizontal and vertical directions, but as described later, it may be arranged one-dimensionally along the horizontal direction, or one-dimensionally arranged along the vertical direction. It is good (refer FIG. 9).
  • the second color component (G) is adjacent to the right side of the first color component (B), and the first color component (B) is adjacent to the right side of the second color component (G).
  • the fourth color component (R) adjacent to the right side of the third color component (g)
  • the fourth color component (R) is adjacent to the lower side of the second color component (G), and the second color component (G) is adjacent to the lower side of the fourth color component (R).
  • the pixel values of these adjacent pixels are not so highly correlated. Therefore, when the RAW data 2 is compressed as it is, the compression efficiency is low (equivalent to the prior art).
  • the compression processing target of the compression processing unit 5 is “compressed by placing four color-specific plane data 41, 42, 43, 44.
  • the reconstructed RAW data 4 "is collected as a single unit file.
  • the color-specific plane data has a high correlation between adjacent pixels, and the compression process can obtain a higher compression efficiency than the case where the raw data 2 is compressed as it is.
  • the reconstructed RAW data 4 configured by arranging four color-specific plane data 41, 42, 43, and 44 having different color components on one file as a compression processing unit is compressed. Since it is an object of processing, higher compression efficiency can be obtained. Specifically, since the compression processing of the color-specific plane data 41, 42, 43, 44 of all the color components is realized by one compression processing in one file (reconstructed RAW data 4) that is a compression processing unit. Compared with the case where the four color-specific plane data 41, 42, 43, and 44 are individually switched sequentially and repeatedly compressed as in the case of the prior art, the compression efficiency is greatly improved.
  • the compression processor 5 is a standard compression control software asset created so that the compression processing for image data for one frame, which is mounted on a digital camera as standard, is completed in one process. It can also be used effectively. Alternatively, existing JPEG hardware processing can also be supported. That is, it is not necessary to use four compression processors operating in parallel to complete the compression processing of image data for one frame including the four color-specific plane data 41, 42, 43, 44 in one process. Since only one compression processor 5 is required, there is no need to increase the circuit scale. Alternatively, there is no need to particularly increase the processing capacity of the CPU.
  • FIG. 2A illustrates RAW data 2 to be compressed.
  • FIG. 2B four color-specific plane data 41, 42, 43, and 44 are adjacently arranged in a two-dimensional direction.
  • FIG. 2C shows four color-specific plane data 41, 42, 43, 44 arranged at appropriate intervals.
  • FIG. 2C corresponds instead of FIG. 2B.
  • fixed data indicated in gray
  • invariant luminance levels are arranged between the peripheral part and the peripheral part to form a non-image area.
  • M, n, x, and y are arbitrary (two or more) natural numbers.
  • the size of the RAW data 2 to be compressed is 2 m pixels in the horizontal direction and 2n pixels in the vertical direction.
  • the size of the reconstructed RAW data 4 that is one compressed file is larger than that of the RAW data 2. It is 2 (m + x) pixels in the horizontal direction and 2 (n + y) pixels in the vertical direction.
  • the four arrangement regions a1, a2, a3, and a4 divided for each color component have the same size, and are (m + x) pixels in the horizontal direction and (n + y) pixels in the vertical direction.
  • the sizes of the four color-specific plane data 41, 42, 43, and 44 are common and are m pixels in the horizontal direction and n pixels in the vertical direction.
  • FIG. 3 shows a reconstructed RAW obtained by two-dimensionally arranging four color-specific plane data 41, 42, 43, and 44 of the Bayer array and reconstructing it as image data for one frame by the RAW data reconstructor 3 of FIG. Data 4 is shown.
  • 41 is the first color-specific plane data obtained by collecting only the first color components (B) arranged at the upper left part of the image
  • 42 is the second color component (at the upper right part of the image).
  • G) is the second color-specific plane data that collects only G
  • 43 is the third color-specific plane data that collects only the third color component (g) arranged in the lower left part of the image
  • 44 is the image
  • the fourth color-specific plane data is a collection of only the fourth color component (R) arranged in the lower right part of FIG.
  • reference numeral 47 shown in gray is fixed data for filling between adjacent boundary portions of the four color-specific plane data 41, 42, 43, 44.
  • the fixed data 47 is pixel data that has no luminance signal component, only a luminance signal component, and the luminance signal component is constant.
  • the size of the compression processing unit block is 4 pixels ⁇ 4 pixels (equivalent to H.264) here for simplification, but in the case of JPEG, it is 8 pixels ⁇ 8 pixels.
  • the four color-specific plane data 41, 42, 43, and 44 are two-dimensionally arranged at a predetermined interval on the reconstructed RAW data 4 that is one compressed file.
  • the reason why the plane data for each color is arranged with a space therebetween is as follows.
  • the compression process is repeatedly performed on a block unit of 4 pixels ⁇ 4 pixels or 8 pixels ⁇ 8 pixels. If four color plane data are arranged adjacent to each other, some pixel data of the last block of the first color plane data 41 in a certain block row depending on the data size of the color plane data. And some pixel data of the first block of the second color plane data 42 may be included in one compression processing unit block, and the third color plane data 43 and the fourth color The same applies to the other plane data 44.
  • the color-specific plane data are arranged with an interval between them, and an appropriate number of fixed data 47 whose luminance signal component is not changed is arranged between the adjacent color-specific plane data. By doing so, it is possible to prevent deterioration of data after compression.
  • Block noise is likely to occur at the boundary positions 45 and 46 when the compressed RAW data is decompressed for photo retouch processing or the like.
  • Block noise is likely to occur in DCT with JPEG. It tends to occur as the compression rate increases. It stands out in areas where there is relatively little change in density value. This is shown in FIG. FIG. 4 is an explanatory diagram of block noise occurrence positions in the image data when the compressed RAW data of FIG. 3 is decompressed.
  • the relative positional relations are equivalent to each other.
  • the reference position of each arrangement area is the upper left corner of each arrangement area, and the reference position of each color plane data is the upper left corner of each color plane data.
  • the displacement vectors V4 with respect to the reference position are equivalent to each other and are exactly overlapped with each other by parallel movement along the horizontal direction or the vertical direction.
  • the size of the basic processing block is 4 pixels ⁇ 4 pixels.
  • Each of the four color-specific plane data 41, 42, 43, 44 includes therein a horizontal period boundary position 45 extending in the vertical direction and a vertical period boundary position 46 extending in the horizontal direction of the compression processing unit block. Appears repeatedly.
  • the B (blue) pixel data facing the boundary position 45 from the left side in the first color-specific plane data 41 is the second column, the sixth column, the tenth column, etc., and faces the boundary position 45 from the right side.
  • the B (blue) pixel data is in the third, seventh, eleventh columns, etc., and when these pixel data are expanded, the original 2m ⁇ 2n large size is obtained as shown in FIG. Expanded in the image data, the third column, the eleventh column, the nineteenth column, the fifth column, the thirteenth column, the twenty-first column, and the like.
  • B (blue) pixel data facing the boundary position 46 from the upper side in the first color-specific plane data 41 is the second line, the sixth line, the tenth line, and the like, and the boundary position 46 from the lower side.
  • the pixel data of B (blue) facing the first line is the third line, the seventh line, the eleventh line, etc., which are expanded in FIG. 4 and the third line, the eleventh line, the nineteenth line, etc. , Line 13 and line 21.
  • the G (first green) pixel data facing the boundary position 45 from the left side in the second color-specific plane data 42 is the second column, the sixth column, the tenth column, etc., as described above.
  • the G (first green) pixel data facing the boundary position 45 from the right side is the third column, the seventh column, the eleventh column, and the like.
  • the G (first green) pixel data facing the boundary position 46 from the upper side in the first color-specific plane data 41 is in the second row, the sixth row, the tenth row, etc. as described above.
  • the G (first green) pixel data facing the boundary position 46 from the lower side is the third row, the seventh row, the eleventh row, etc., as described above, and these are expanded in FIG.
  • the third, eleventh, and nineteenth lines are the fifth, thirteenth, and twenty-first lines.
  • the relationship between the g (second green) pixel data and the boundary positions 45 and 46 in the third color-specific plane data 43 is the same, and in FIG. , 11th column, 19th column, etc., 5th column, 13th column, 21st column etc., 4th row, 12th row, 20th row, etc., 6th row, 14th row, etc. Eyes, 22nd line, etc.
  • the relationship of the R (red) pixel data with respect to the boundary positions 45 and 46 in the fourth color-specific plane data 44 is the same.
  • the boundary positions at which block noise is likely to occur are equivalent to the four color-specific plane data 41, 42, 43, and 44, and are continuous in the vertical direction and the horizontal direction in the decompressed image data. Become. That is, in the original large-sized image data after the decompression process, pixels corresponding to the boundary positions regularly appear every fourth column and every fourth row. As a result, block noise tends to appear prominently, and there is a possibility that image quality will deteriorate (see FIG. 7B).
  • the problem of block noise generation described above is that, in order to increase the compression efficiency, a plurality of types of color-specific plane data having different color components are arranged in order on one file as a compression processing unit (displacement with respect to each reference position). This is because the reconstructed RAW data configured by arranging the vectors V1 to V4 (equivalent to each other) is the target of compression processing, and the compression processing for one frame of image data is completed in one process. .
  • the data compression here is irreversible compression with a high compression rate.
  • the relative positional relationship arranged at is not equivalent to each other. That is, the four relative positional relationships are shifted from each other. The shift is one pixel in the right direction and one pixel in the downward direction (when the compression processing unit block is 4 pixels ⁇ 4 pixels).
  • the displacement vector V4 with respect to the reference position is not equivalent to each other.
  • V2 V1 ⁇ 2
  • V3 V1 ⁇ 3
  • V4 V1 ⁇ 4.
  • the second color-specific plane data 42 is shifted by one pixel in the lower right direction with respect to the first color-specific plane data 41, and the third color-specific plane data 43 is different from the second color-specific plane data 42.
  • the fourth color-specific plane data 44 is shifted by one pixel in the lower right direction with respect to the third color-specific plane data 43.
  • FIG. 6 shows the original large size image data after the decompression process corresponding to FIG.
  • the B (blue) pixel data facing the boundary position 45 from the left side in the first color-specific plane data 41 is the third column, the seventh column, the eleventh column, etc., and faces the boundary position 45 from the right side.
  • the pixel data of B (blue) is the fourth column, the eighth column, the twelfth column, etc., and these are expanded in FIG. 6, and the fifth column, the thirteenth column, the twenty-first column, etc.
  • the columns are the 23rd column and the like.
  • B (blue) pixel data facing the boundary position 46 from the upper side in the first color-specific plane data 41 is the third line, the seventh line, the eleventh line, etc., and the boundary position 46 from the lower side.
  • the B (blue) pixel data facing the first line is the fourth line, the eighth line, the twelfth line, etc., which are expanded in FIG. 6, and the fifth line, the 13th line, the 21st line, etc. 15th and 23rd lines.
  • the G (first green) pixel data facing the boundary position 45 from the left side in the second color-specific plane data 42 is the second column, the sixth column, the tenth column, etc., and the boundary position from the right side.
  • the pixel data of G (first green) facing 45 is the third column, the seventh column, the eleventh column, etc., which are developed in FIG. 6, and the fourth column, the twelfth column, the twentieth column, etc. , 6th row, 14th row, 22nd row and so on.
  • the G (first green) pixel data facing the boundary position 46 from the upper side in the second color-specific plane data 42 is the second row, the sixth row, the tenth row, etc., from the lower side.
  • the pixel data of G (first green) facing the boundary position 46 is the third row, the seventh row, the eleventh row, etc., which are expanded in FIG.
  • the pixel data of g (second green) facing the boundary position 45 from the left side in the third color-specific plane data 43 is the first column, the fifth column, the ninth column, etc., and the boundary position from the right side
  • the pixel data of g (second green) facing 45 is the second column, the sixth column, the tenth column, etc., and these are expanded in FIG. 6, and the first column, the ninth column, the 17th column, etc.
  • the third, eleventh, and nineteenth columns are provided.
  • the pixel data of g (second green) facing the boundary position 46 from the upper side in the third color-specific plane data 43 is the first row, the fifth row, the ninth row, etc., from the lower side.
  • the pixel data of g (second green) facing the boundary position 46 is the second row, the sixth row, the tenth row, etc., which are expanded in FIG. 6, and the second row, the tenth row, and the 18th row.
  • the R (red) pixel data facing the boundary position 45 from the left side in the fourth color-specific plane data 44 is the fourth, eighth, and twelfth columns, and faces the boundary position 45 from the right side.
  • the pixel data of R (red) is the 1st column, the 5th column, the 9th column, etc., and these are expanded in FIG. 6, and the 8th column, the 16th column, the 24th column, etc. The columns are the 18th column and the like.
  • the R (red) pixel data facing the boundary position 46 from the upper side in the fourth color-specific plane data 44 is the fourth line, the eighth line, the twelfth line, etc., and the boundary position 46 from the lower side.
  • the R (red) pixel data facing the first line is the first line, the fifth line, the ninth line, etc., which are expanded in FIG. 6, and the second line, the eighth line, the 16th line, the 24th line, etc.
  • Block noise may occur in the reconstructed RAW data for one frame after decoding that has been subjected to decompression processing due to image quality deterioration in which the image becomes discontinuous at the periodic boundary positions of the compression processing unit blocks.
  • the 8 pixel ⁇ 8 pixel pattern as described above repeatedly appears along the two-dimensional direction.
  • the pixel group facing the boundary position is aligned in a straight line in both the horizontal direction and the vertical direction, whereas in FIG. 6, it faces the boundary position.
  • the pixel group is in a dispersed arrangement state both in the horizontal direction and in the vertical direction. That is, in the development of FIG. 4, positions where block noise is likely to occur are in a concentrated arrangement state, whereas in the development of FIG. 6, positions where block noise is likely to occur are in a distributed arrangement state.
  • FIG. 8B applies the block noise occurrence position of FIG. 4 corresponding to FIG. 7B to FIG. 7B
  • FIG. 8A applies the block noise occurrence position of FIG. 6 corresponding to FIG. 7A to FIG.
  • FIG. 8A and FIG. 8B are shown as a reference.
  • the background is black in order to make the block noise occurrence position (white portion) clear.
  • the block noise occurrence positions are concentrated in one straight line, and the degree of concentration is extremely high.
  • image quality degradation occurs in which the image becomes discontinuous at the periodic boundary positions of the compression processing unit blocks, and noticeable image quality degradation occurs.
  • the four color-specific plane data 41, 42, 43, 44 are sequentially shifted by one pixel in the horizontal direction and one pixel in the vertical direction.
  • the compression processing unit block size is JPEG of 8 pixels ⁇ 8 pixels
  • the four color-specific plane data 41, 42, 43, and 44 may be sequentially shifted by 2 pixels in the horizontal direction and 2 pixels in the vertical direction.
  • the size of the compression processing unit block is 4 pixels ⁇ 4 pixels, and the four color-specific plane data 41, 42, 43, 44 are sequentially shifted by 1 pixel in the horizontal direction and 1 pixel in the vertical direction. Yes.
  • the size of the compression processing unit block is 8 pixels ⁇ 8 pixels JPEG
  • the boundary positions 45 of the horizontal period extending in the vertical direction appear every 8 columns
  • the boundary positions 46 of the vertical period extending in the horizontal direction are 8 It will appear every other line.
  • the four color-specific plane data 41, 42, 43, and 44 may be sequentially shifted by two pixels in the horizontal direction and two pixels in the vertical direction.
  • the four color plane data may be sequentially shifted in the horizontal direction a / 4 pixel and the vertical direction a / 4 pixel.
  • the arrangement relationship of the four color-specific plane data 41, 42, 43, 44 in the image data for one frame is a two-dimensional array arranged in both the horizontal direction and the vertical direction as described above.
  • a vertical one-dimensional array arranged in one column in the vertical direction or a horizontal one-dimensional array arranged in one column in the horizontal direction may be used.
  • the order in which the four color-specific plane data 41, 42, 43, and 44 are arranged is not particularly limited, and the arrangement information can be arranged in an arbitrary place by using the history of the arrangement information as data additional information after data compression. it can.
  • [B, G, g, R] as shown in FIG. 1 [G, B, R, g] may be used, and [R, G , G, B].
  • FIG. 10 is a block diagram showing a schematic configuration of an imaging apparatus equipped with an image processing apparatus that compresses reconstructed RAW data generated from RAW data according to an embodiment of the present invention.
  • This image pickup apparatus 50 is an image pickup apparatus using an image sensor of a type that separates and picks up images, and can record images in the JPEG format and can record RAW data immediately after A / D conversion. It is configured to be.
  • 50 is an imaging device
  • 60 is a single-plate imaging unit
  • 70 is an image processing device.
  • the imaging device 50 includes an imaging unit 60 and an image processing device 70.
  • the imaging unit 60 includes an optical lens 61, an optical low-pass filter 62, a color filter 63, an imaging element 64, and an analog front end unit 65.
  • the image processing apparatus 70 includes a CPU (Central Processing Unit) 71, a ROM (Read Only Memory) 72, a RAM (Random Access Memory) 73, a preprocessing unit 74, a memory control unit 75, an image memory 76, an image signal processing unit 77, A compression / decompression processing unit (encoder / decoder) 78, a recording media interface unit 79, a display processing unit 80, and a monitor interface unit 81 are included.
  • 91 is an operation panel
  • 92 is a recording medium.
  • the image sensor 64 is an image sensor such as a CCD type or a CMOS type, and a large number of photodiodes (photosensitive pixels) are two-dimensionally arranged on the light receiving surface thereof.
  • the photodiode photoelectrically converts subject information that has passed through the optical lens 61 and the optical low-pass filter 62.
  • the optical low-pass filter 62 has an action of removing high frequency components higher than the sampling frequency depending on the pixel pitch of the image sensor 64, and prevents aliasing in the final image after image reproduction (signal processing). It is supposed to be.
  • Aliasing means that when a waveform with a frequency component exceeding 1/2 of the sampling frequency is forcibly sampled, a frequency component that should not exist originally appears and is drawn in black letters or lines on a white background. In an image with clear contrast, the characters and lines are crushed and a pattern different from the original image appears.
  • the color filter 63 has a predetermined color arrangement in which any color of R, G, and B exists at a position corresponding to one pixel of the image sensor 64, and the light that enters the photodiode that is the light receiving element. Color selection is performed.
  • FIGS. 11A to 11C show examples of primary color filter arrangements.
  • the light receiving elements are arranged in a square matrix at a constant pitch in the row direction and the column direction, respectively.
  • the centers of the geometric shapes of the light receiving elements are arranged so as to be shifted by 1/2 pitch in the row direction and the column direction.
  • the structure of the pixel array is periodically repeated in the horizontal direction and the vertical direction.
  • FIG. 11C shows frequency characteristics in the case of the Bayer array.
  • a subject image that passes through the optical lens 61, the optical low-pass filter 62, and the color filter 63 and is imaged on the light receiving surface of the image sensor 64 is converted into signal charges of an amount corresponding to the amount of incident light by each photodiode, and is not shown. Based on the pulse given from the driver circuit, it is sequentially read out as a voltage signal (image signal) corresponding to the signal charge.
  • the image sensor 64 has an electronic shutter function that controls the charge accumulation time (shutter speed) of each photodiode according to the timing of a shutter gate pulse (not shown). The operation (exposure, reading, etc.) of the image sensor 64 is controlled by the CPU 71.
  • the analog front end unit 65 performs processing such as analog gain adjustment and CDS (correlated double sampling) on the image signal output from the image sensor 64, and then converts the image signal into a digital signal by the built-in A / D conversion unit. It has a function to convert.
  • the analog front end unit 65 outputs the RAW data immediately after A / D conversion to the preprocessing unit 74 of the image processing apparatus 70.
  • the pre-processing unit 74 in the image processing apparatus 70 includes an auto calculation unit that performs calculations necessary for AF and AE control.
  • the preprocessing unit 74 receives the RAW data from the analog front end unit 65, performs the focus evaluation value calculation, the AE calculation, and the like in the auto calculation unit, and transmits the calculation result to the CPU 71.
  • the pre-processing unit 74 performs Bayer array RAW data having four color components B, G, g, and R digitized by the A / D conversion of the analog front end unit 65. Then, the black DC level that is the reference of the data is adjusted.
  • a memory control unit 75 in the image processing apparatus 70 includes a preprocessing unit 74, an image signal processing unit 77, a compression / decompression processing unit 78, a recording media interface unit 79, a display processing unit 80, and an image memory 76. It is configured to relay the exchange of signals.
  • the memory control unit 75 incorporates a RAW data reconstructor.
  • the RAW data reconstructor generates four pieces of color-specific plane data 41, 42, 43, and 44, rearranges them into image data for one frame, and writes them into the memory space of the image memory 76. It is configured.
  • the RAW data reconstructor in the memory control unit 75 generates four pieces of color-specific plane data 41 and 42 with respect to the periodic boundary position of the compression processing unit block in the compression / decompression processing unit 78 when generating the reconstructed RAW data. , 43, and 44 are configured to be shifted relative to each other.
  • a CPU 71 in the image processing device 70 is a control unit that performs overall control of the imaging device 50 in accordance with a predetermined program. Under the cooperation of the ROM 72 and the RAM 73, a preprocessing unit 74, an image signal processing unit 77, and a recording media interface unit. 79 and the operation panel 91 are controlled.
  • the ROM 72 stores programs executed by the CPU 71 and various data necessary for control.
  • the RAM 73 is used as a work area for the CPU 71.
  • the CPU 71 controls the operation of each circuit in the imaging device 50 based on an instruction signal from the operation panel 91.
  • the CPU 71 controls the image pickup unit 60 such as the image pickup device 64 according to various shooting conditions (exposure conditions, presence / absence of strobe light emission, shooting mode, etc.) in accordance with an instruction signal input from the operation panel 91, and automatic exposure (AE). ) Control, automatic focus adjustment (AF) control, auto white balance (AWB) control, lens drive control, image processing control, read / write control of the recording medium 92, and the like.
  • the CPU 71 performs automatic focus adjustment (AF) control when it detects half-pressing of the release switch on the operation panel 91, and starts exposure and reading control for capturing an image for recording when full-pressing of the release switch is detected. Further, the CPU 71 controls the recording medium interface unit 79 so as to record the captured image data on the recording medium 92 according to the recording mode. Further, the CPU 71 sends a command to a strobe control circuit (not shown) as necessary to control light emission of a flash light emitting tube (light emitting unit) such as a xenon tube.
  • AF automatic focus adjustment
  • the image signal processing unit 77 uses the image memory 76 as a work memory via the memory control unit 75, and performs synchronized color interpolation processing, white balance adjustment, gamma correction, luminance / color difference signal generation, contour enhancement, and electronic zoom function.
  • the compression / decompression processing unit 78 reads the image data from the image memory 76 via the memory control unit 75, performs compression processing according to a compression encoding algorithm corresponding to the designated compression format, and sends the compressed RAW data to the memory control unit 75.
  • the image data is stored in the image memory 73.
  • the compression / decompression processing unit 78 performs decompression processing on the compressed RAW data read from the recording medium 92, and stores the RAW data restored by the decompression in the image memory 73 via the memory control unit 75. It is supposed to be.
  • As an algorithm used for compression / decompression processing there are MPEG and the like in addition to JPEG.
  • the recording media interface unit 79 is configured to read image data from the image memory 76 via the memory control unit 75 and transfer it to the recording medium 92 for recording.
  • the image data is read from the recording medium 92 and transferred to the memory control unit 75 for decoding.
  • the operation panel 91 is configured to allow a user to input various instructions to the imaging apparatus 50.
  • a mode selection switch for selecting an operation mode of the imaging apparatus 50, a menu item selection operation (cursor moving operation), and the like.
  • the cross key for inputting instructions such as frame advance / rewind of playback images, execution keys for confirming (registering) selection items and executing operations, and deleting desired objects such as selection items, and canceling instructions
  • Various operation devices such as a cancel key, a power switch, a zoom switch, and a release switch are included.
  • the recording medium 92 for storing image data various recording media such as a magnetic disk, an optical disk, and a magneto-optical disk can be used in addition to a semiconductor memory represented by a memory card. Further, the recording medium (internal memory) built in the imaging device 50 is not limited to a removable medium.
  • the subject image that has passed through the optical lens 61, the optical low-pass filter 62, and the color filter 63 in the imaging unit 60 and is imaged on the light receiving surface of the imaging device 64 is converted into signal charges corresponding to the amount of incident light by each photodiode. Then, it is sequentially read out as a voltage signal (image signal) corresponding to the signal charge based on a pulse given from a driver circuit (not shown), and an image analog signal having a Bayer arrangement of four color components B, G, g, and R is analog. It is sent to the front end portion 65.
  • the image analog signal from the image sensor 64 is digitized by the A / D conversion unit in the analog front end unit 65, and the Bayer array image data is sent to the preprocessing unit 74.
  • the Bayer array image data input to the preprocessing unit 74 is sent to the memory control unit 75 as RAW data after the black DC level as a data reference is adjusted.
  • the memory control unit 75 generates four color-specific plane data 41, 42, 43, and 44 by data rearrangement writing control of the built-in RAW data reconstructor, and further generates image data for one frame. Arrange and write in the memory space of the image memory 76. This is reconstructed RAW data in units of compression processing.
  • the reference positions of the four color-specific plane data 41, 42, 43, and 44 are set relative to the periodic boundary positions of the compression processing unit blocks in the compression / decompression processing unit 78. I try to shift it. This is to prevent the occurrence of block noise occurrence positions of the respective colors from overlapping when the compressed RAW data is read from the recording medium 92 and displayed on the monitor after being decompressed.
  • the reconstructed RAW data written in the image memory 76 is input as one component data to the compression / decompression processing unit 78 via the memory control unit 75.
  • fixed data that does not change the luminance component is input to the other component data input terminals of the compression / decompression processing unit 78, and compression processing is performed.
  • the processed compressed RAW data is written into the image memory 76 via the memory control unit 75 again.
  • the CPU 71 adds a header file as an image file in the JPEG file format to the data written as one compressed RAW data in the image memory 76, and then format information of the compressed RAW data, the shooting time, and the like. Information useful for searching and recognizing images such as the situation is acquired, added to the image file header, and recorded on the recording medium 92 via the memory control unit 75 and the recording medium interface unit 79.
  • the compressed file is read from the recording medium 92. That is, when a reproduction instruction is given on the expansion operation panel 91, the CPU 71 controls the compression / expansion processing unit 78 and the recording media interface unit 79.
  • the compressed RAW data read from the recording medium 92 is written into the image memory 73 via the recording medium interface unit 79 and the memory control unit 75.
  • the compressed RAW data written in the image memory 73 is transferred to the compression / decompression processing unit 78 via the memory control unit 75.
  • the decompression processing unit in the compression / decompression processing unit 78 decompresses the compressed RAW data.
  • the reconstructed RAW data configured by arranging the four color-specific plane data 41, 42, 43, and 44 on one file that is a compression processing unit is the target of the compression processing.
  • the four color-specific plane data 41, 42, 43, 44 can be compressed at once, and the four color-specific plane data 41, 42, 43, 44 are sequentially switched as in the case of the prior art.
  • the compression efficiency is greatly improved as compared with the case where the compression process is repeatedly performed.
  • This significant improvement in compression efficiency is advantageous when the imaging device 50 has a continuous shooting function, and provides high-speed continuous recording in the compressed RAW data recording mode in response to the recent increase in the number of pixels in an image sensor. It is possible to clear the problem in the prior art that hinders copying (pauses during the continuous shooting operation).
  • compression processor 5 compression / decompression processing unit 78
  • a single compression processor is required, so that the circuit scale does not increase, or There is no need to specially increase the processing capacity of the CPU.
  • FIG. 2A it is possible to use a general compression processor having input terminals for luminance signal (Y) and color difference signal (Cr / Cb) as input terminals for component data. Can be used.
  • a general JPEG compression processor 5 compression processor, compression / decompression processor
  • the reconstructed RAW data generated by the RAW data reconstructor 4 (memory control unit 75) is input to the input terminal of the luminance signal (Y), and the fixed data having no luminance change is input to the color difference signals (Cr, Cb).
  • a general JPEG compression processor processor
  • a general JPEG compression processor processor
  • the compressed RAW data is recorded on the recording medium 92, there is no influence of the signal processing of the preprocessing unit 74 and the image signal processing unit 77, and the image quality can be kept high. .
  • the generated compressed RAW data has a small file size, and the utilization efficiency of the recording medium 92 is improved. Since the use efficiency of the recording medium 92 is high and the reconstructed RAW data in which the above-described color-specific plane data is arranged in one file as a compression processing unit is the compression processing target, the compression efficiency is greatly improved. When combined with this, high-speed continuous shooting becomes even more advantageous.
  • the RAW data reconstructor in the memory control unit 75 arranges the four color-specific plane data 41, 42, 43, 44 to generate reconstructed RAW data in units of compression processing, and performs compression processing in the compression / decompression processing unit 78.
  • the reference position of the color-specific plane data is shifted relative to the periodic boundary position of the unit block. Therefore, when the compressed RAW data is read from the recording medium 92, decompressed by the compression / decompression processing unit 78, and displayed on the monitor via the display processing unit 80 and the monitor interface unit 81, the block noise occurrence position of each color is displayed. Overlap can be greatly reduced. Therefore, it is possible to suppress the occurrence of block noise, and to reduce the peculiar image quality degradation in which the occurrence positions of the block noise of each color overlap at the boundary position of the compression processing unit block.
  • the RAW data rearranged and written as image data for one frame in the image memory 76 is input as one component data to the compression / decompression processing unit 78 via the memory control unit 75, and the other components
  • the input data is compressed as fixed data having no luminance change.
  • it is also possible to input other image data to other component inputs and execute the process. is there.
  • As another image data for example, there is small YCrCb data or small RGB data which is reduced and resized for display.
  • the reconstructed RAW data 4 for one frame is input to the input terminal of the luminance signal (Y) in the compression processor 5 and the input terminal of the color difference signal (Cr / Cb) is for display.
  • Small YCrCb data 6 of an image composed of Y, Cr, and Cb that has been reduced and resized, small RGB data 7 of an image composed of R, G, and B are input.
  • the compression processor 5 simultaneously compresses the reconstructed RAW data 4, small YCrCb data 6, small RGB data 7, and the like.
  • the compressed file is read from the recording medium 92.
  • the compression / decompression processing unit 75 displays small YCrCb6 or small RGB7 while performing image signal processing. It is also possible to directly transfer to the processing unit 80 and perform playback preview.
  • JPEG has been described as an example of a still image compression encoding algorithm.
  • JPEG2000 or JPEG XR that can handle the number of bits of input data of more than 8 bits up to 12 bits may be used.
  • image quality deterioration due to compression is small compared to JPEG and the like, and image quality deterioration is particularly small at a high compression rate.
  • lossless compression and lossy compression are possible with the same algorithm.
  • the code string deletion process (post quantization) of the encoded data has an advantage that the compression rate can be adjusted without performing recompression.
  • the reconstructed RAW data is read from the image memory 76 and compressed. If the reconstructed RAW data has not been created in advance, the reconstructed RAW data for one frame is created in the memory control read process. Either aspect is encompassed by the present invention.
  • the process of creating the reconstructed RAW data for one frame has been described as the rearrangement process when the RAW data is written to the image memory 76, but the writing process to the image memory 73 is the original Bayer array.
  • the image memory 73 may be read as it is, and may be simultaneously read when it is read out via the memory control unit 75 during the compression process. Any of these modes is included in the present invention.
  • RAW data is divided into four color plane data and rearranged into one frame of reconstructed RAW data for compression, it is possible to use either lossless compression or lossy compression.
  • lossless compression it is possible to efficiently compress and record the RAW data that is not affected by the signal processing inside the imaging apparatus in a single process.
  • the original RAW data can be completely reproduced by decoding and decompressing the recorded encoded data by the compression / decompression processing unit 78 or an external decoder.
  • an image sensor having a color separation filter has been described as an example.
  • the present invention can also be applied to a case where an image sensor of a type that performs similar color separation by means other than a color filter is used. Is natural.
  • the compressed RAW data recorded by the imaging device 50 can be reproduced (developed) by a dedicated image processing device or a personal computer in addition to the imaging device main body. Specifically, using the arrangement information of each color template added to the compressed RAW data, the RAW data rearranged in the plurality of color regions after expansion may be rearranged again into the original RAW data array, When the reproduction (development) process is performed via the memory control unit 75, the original arrangement may be read.
  • the arrangement structure of the color filter 63 is not limited to the example shown in FIG. 11, and various arrangement structures such as RGB stripes are possible.
  • the primary color filter is used.
  • the present invention is not limited to the primary color filter, and is a complementary color filter composed of yellow (Y), magenta (M), cyan (C), and green (G). Or any combination of primary and complementary colors or white (W) may be used.
  • a noise processing unit and an A / D conversion unit are mounted in the image pickup device 64 as means for realizing high-speed reading, and output directly as a digital signal from the image pickup device. There is also.
  • the present invention relates to an image pickup apparatus equipped with an image sensor of a type that performs color separation such as a digital still camera, a digital video camera, an independent image scanner, an image scanner incorporated in a copying machine, etc.
  • an image pickup apparatus equipped with an image sensor of a type that performs color separation such as a digital still camera, a digital video camera, an independent image scanner, an image scanner incorporated in a copying machine, etc.
  • reconstructed RAW data configured by arranging a plurality of types of color-specific plane data on a single file as a compression processing unit is targeted for compression processing. Since the reconstructed RAW data is acquired by one compression process, it is useful as a technique for greatly improving the compression efficiency without causing an increase in circuit scale or CPU processing capacity.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

L’invention concerne un dispositif de traitement d’images comprenant un reconstructeur de données brutes et un processeur de compression. Dans le reconstructeur de données brutes, les données brutes sur lesquelles le traitement de signaux n’est pas effectué sont reçues après une conversion A/N et décomposées sur une base par composante de couleur et réassemblées pour générer une pluralité d’ensembles de données plans qui sont ensuite agencés et collectés dans un seul et même fichier qui est une unité de traitement de compression. Ce ficher qui est une unité de traitement de compression présente une pluralité de zones d’agencement partitionnées sur une base par composante de couleur. Le reconstructeur de données brutes trie les ensembles de données plans agencés couleur par couleur par rapport aux zones d’agencement, et agence à nouveau le résultat pour générer les données brutes reconstruites, qui sont transférées au processeur de compression. Le processeur de compression effectue le traitement de compression sur les données brutes reconstruites.
PCT/JP2010/004388 2009-07-23 2010-07-05 Dispositif de traitement d’images, procédé de traitement d’images et dispositif de capture d’images WO2011010431A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2009171953A JP2011029809A (ja) 2009-07-23 2009-07-23 画像処理装置、画像処理方法および撮像装置
JP2009-171953 2009-07-23

Publications (1)

Publication Number Publication Date
WO2011010431A1 true WO2011010431A1 (fr) 2011-01-27

Family

ID=43498913

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2010/004388 WO2011010431A1 (fr) 2009-07-23 2010-07-05 Dispositif de traitement d’images, procédé de traitement d’images et dispositif de capture d’images

Country Status (2)

Country Link
JP (1) JP2011029809A (fr)
WO (1) WO2011010431A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015064403A1 (fr) * 2013-11-01 2015-05-07 ソニー株式会社 Dispositif et procédé de traitement d'image
WO2015064402A1 (fr) * 2013-11-01 2015-05-07 ソニー株式会社 Dispositif et procédé de traitement d'image
CN113705553A (zh) * 2020-05-20 2021-11-26 深圳清华大学研究院 视觉任务执行方法、装置、电子设备、存储介质及系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6142833B2 (ja) 2014-03-31 2017-06-07 ブラザー工業株式会社 電子機器

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005286415A (ja) * 2004-03-26 2005-10-13 Olympus Corp 画像圧縮方法及び画像圧縮装置
JP2006173931A (ja) * 2004-12-14 2006-06-29 Canon Inc 画像処理装置及びその制御方法、並びに、コンピュータプログラム及びコンピュータ可読記憶媒体

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005286415A (ja) * 2004-03-26 2005-10-13 Olympus Corp 画像圧縮方法及び画像圧縮装置
JP2006173931A (ja) * 2004-12-14 2006-06-29 Canon Inc 画像処理装置及びその制御方法、並びに、コンピュータプログラム及びコンピュータ可読記憶媒体

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015064403A1 (fr) * 2013-11-01 2015-05-07 ソニー株式会社 Dispositif et procédé de traitement d'image
WO2015064402A1 (fr) * 2013-11-01 2015-05-07 ソニー株式会社 Dispositif et procédé de traitement d'image
JPWO2015064402A1 (ja) * 2013-11-01 2017-03-09 ソニー株式会社 画像処理装置および方法
JPWO2015064403A1 (ja) * 2013-11-01 2017-03-09 ソニー株式会社 画像処理装置および方法
US10356442B2 (en) 2013-11-01 2019-07-16 Sony Corporation Image processing apparatus and method
US10397614B2 (en) 2013-11-01 2019-08-27 Sony Corporation Image processing apparatus and method
CN113705553A (zh) * 2020-05-20 2021-11-26 深圳清华大学研究院 视觉任务执行方法、装置、电子设备、存储介质及系统
CN113705553B (zh) * 2020-05-20 2024-01-26 深圳清华大学研究院 视觉任务执行方法、装置、电子设备、存储介质及系统

Also Published As

Publication number Publication date
JP2011029809A (ja) 2011-02-10

Similar Documents

Publication Publication Date Title
US9560256B2 (en) Image capture apparatus and image capture method in which an image is processed by a plurality of image processing devices
JP5845464B2 (ja) 画像処理装置及び画像処理方法並びにデジタルカメラ
JP5280448B2 (ja) 画像処理装置、画像処理方法、画像処理プログラム及び半導体集積回路
US8428120B2 (en) Method and apparatus of Bayer pattern direct video compression
JP5853166B2 (ja) 画像処理装置及び画像処理方法並びにデジタルカメラ
WO2019142821A1 (fr) Codeur, décodeur, procédé de codage, procédé de décodage, programme de codage et programme de décodage
KR20080012207A (ko) 화상 신호 처리 장치, 화상 신호 처리 방법
KR101046012B1 (ko) 동화상 처리 장치 및 동화상 처리 방법, 동화상 처리 프로그램을 기록한 컴퓨터로 판독가능한 기록매체
JP4905279B2 (ja) 撮像回路および撮像装置
JP4497945B2 (ja) 撮像装置
WO2011010431A1 (fr) Dispositif de traitement d’images, procédé de traitement d’images et dispositif de capture d’images
JP4257145B2 (ja) 画像圧縮装置及び画像処理システム
JP4436733B2 (ja) 撮像装置および再生装置
JP6152642B2 (ja) 動画像圧縮装置、動画像復号装置およびプログラム
JP5974691B2 (ja) 動画像圧縮装置、撮像装置およびプログラム
JP2018032909A (ja) 画像符号化装置及びその制御方法及び撮像装置及びプログラム
JP6890993B2 (ja) 画像符号化装置、画像復号化装置、その制御方法、および制御プログラム
JP2019022225A (ja) 画像圧縮装置およびプログラム
JP2021078008A (ja) 画像処理装置、画像処理方法及び撮像装置
JP7289642B2 (ja) 画像処理装置、画像処理装置の制御方法、及びプログラム
JP2017200199A (ja) 動画像圧縮装置、動画像復号装置およびプログラム
JP2014143488A (ja) 画像圧縮装置、画像復号装置およびプログラム
WO2020188875A1 (fr) Dispositif de codage, dispositif de décodage, procédé de codage, procédé de décodage, programme de codage et programme de décodage
JP5974692B2 (ja) 動画像復号装置およびプログラム
JP2019036992A (ja) 圧縮装置、復号装置およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10802053

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10802053

Country of ref document: EP

Kind code of ref document: A1