TWI353779B - - Google Patents

Download PDF

Info

Publication number
TWI353779B
TWI353779B TW97145601A TW97145601A TWI353779B TW I353779 B TWI353779 B TW I353779B TW 97145601 A TW97145601 A TW 97145601A TW 97145601 A TW97145601 A TW 97145601A TW I353779 B TWI353779 B TW I353779B
Authority
TW
Taiwan
Prior art keywords
color
image
pixel
unit
mosaic image
Prior art date
Application number
TW97145601A
Other languages
Chinese (zh)
Other versions
TW200924512A (en
Original Assignee
Acutelogic Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Acutelogic Corp filed Critical Acutelogic Corp
Publication of TW200924512A publication Critical patent/TW200924512A/en
Application granted granted Critical
Publication of TWI353779B publication Critical patent/TWI353779B/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/843—Demosaicing, e.g. interpolating colour pixel values
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/12—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with one sensor only
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2101/00—Still video cameras

Description

1353779 六、發明說明： 【發明所屬之技術領域】 本發明涉及通過電子的攝影單元拍攝到的被攝體像的 - 圖像處理裝置和圖像處理方法，尤其涉及經由單板式的具 , 有濾色器的攝影元件而輸出，並按各像素由僅具有單一顔 色的亮度資訊的馬賽克圖像生成在各像素具有多個顔色的 亮度資訊的彩色圖像並且實施圖像變形處理的攝影裝置。 • 【先前技術】 ’ 以往，已知在數位相機等攝影裝置中，通過鏡頭使被 攝體像成像在攝影元件上，並借助於該攝影元件對被攝體 像進行光電變換而生成圖像信號的圖像處理裝置和圖像處 理方法。 此外，還有如下的圖像處理裝置和圖像處理方法，即 ’作爲單板式的攝影元件，呈矩陣狀構成多個光電變換元 φ 件，並且，在其前方與光電變換元件相對應地具有R (紅 )G (綠）B (藍）的各濾色器，對經由該濾色器而輸出的 單一顔色的圖像信號施加信號處理而生成彩色圖像。 即，經由單板式的攝影元件而輸出的圖像是各像素僅 具有單一顔色的顔色資訊的顔色馬賽克圖像，爲了生成彩 色圖像，需要各像素具有紅色（R)、綠色（G)、藍色（ B)等多個顔色資訊。 因此，在使用了單板式攝影元件的圖像處理中，以各 像素僅具有R、G、B成分中的任意一種的顔色資訊的顔色 -3- 1353779 馬賽克圖像爲基礎’進彳了所謂的反馬賽克處理（也稱爲顔 色插値處理）。在此，反馬賽克處理是通過將在顔色馬賽 克圖像的各像素中缺少的其他顔色資訊使用其像素周邊的 其他像素的顔色資訊進行插値運算，來生成各像素分別具 有R、G、B成分的所有顔色資訊的彩色圖像的處理（所謂 的顔色插値處理）。 此外，在利用了單板式攝影元件的圖像處理中，往往 在如上述那樣由顔色馬賽克圖像進行了反馬賽克處理之後 ，進行校正圖像的晃動和歪斜等或根據需要縮放圖像的圖 像變形處理。 即，在數位相機等攝影裝置中，在拍攝上述被攝體時 ，數位相機違背用戶的意願而搖動造成的圖像混亂、所謂 的晃動和由攝影鏡頭引起的歪斜成爲問題。因此，已知通 過陀螺儀和圖像的數位信號處理來檢測照相機的搖動，通 過使鏡頭的光軸光學移動或使圖像信號電子移動（即校正 像素位置）來校正晃動和歪斜的技術。 此時，爲了獲得更高品質的畫質，不僅進行平行移動 ，還進行旋轉和牽拉（7才V )這樣的圖像變形和子像素 單位的位置校正。 例如，在如圖9 ( a )所示從圖像感測器的輸出圖像提 取進行了若干旋轉和縮放的變形後的圖像進行輸出時，通 常要求通過光柵掃描而從圖像感測器讀出，變形後的圖像 也通過光柵掃描而輸出到後級。在此，在輸出圖像變形後 的第一行時，需要將圖像感測器輸出圖像的預定區域（陰 -4 - 1356779 影區域）可參考地緩存。另外，在進行90度的旋轉和上下 翻轉等圖像變形時，該預定區域成爲1個畫面的量。並且 ，在安裝上’由於是使作爲寫入物件的幀和作爲讀出物件 • 的幀分開的雙暫存器的結構，所以需要2幀量的圖像暫存 器（例如，參照專利文獻1、2)。1353779 VI. Description of the Invention: [Technical Field] The present invention relates to an image processing apparatus and an image processing method of a subject image captured by an electronic photographing unit, and more particularly to a single-plate type device having a filter The photographic element of the color device is outputted, and a color image in which each pixel has a color information of luminance information of a plurality of colors is generated from a mosaic image having luminance information of only a single color for each pixel, and an image forming process is performed. • [Prior Art] In the past, it has been known that in a photographing apparatus such as a digital camera, an object image is imaged on a photographing element by a lens, and an image signal is photoelectrically converted by the photographing element to generate an image signal. Image processing device and image processing method. Further, there is an image processing apparatus and an image processing method which have a plurality of photoelectric conversion elements φ in a matrix form as a single-plate type imaging element, and have a photoelectric conversion element in front of the photoelectric conversion element. Each color filter of R (red) G (green) B (blue) applies signal processing to an image signal of a single color output through the color filter to generate a color image. That is, the image output via the single-plate type imaging element is a color mosaic image in which each pixel has only color information of a single color, and in order to generate a color image, each pixel needs to have red (R), green (G), and blue. Color (B) and other color information. Therefore, in the image processing using the single-plate type imaging element, the so-called "color" of the color information of each of the R, G, and B components of each pixel is based on the mosaic image -3- 1 779 779 Anti-mosaic processing (also known as color interpolation processing). Here, the anti-mosaic processing is performed by interpolating the color information of other pixels in the periphery of the pixel using other color information missing in each pixel of the color mosaic image, thereby generating each pixel having R, G, and B components, respectively. The processing of color images of all color information (so-called color interpolation processing). Further, in image processing using a single-plate type photographic element, after performing an anti-mosaic process on a color mosaic image as described above, an image of a corrected image is shaken, skewed, or the like, or an image is scaled as needed. Deformation processing. That is, in a photographing apparatus such as a digital camera, when the subject is photographed, the image chaos caused by the shaking of the digital camera against the user's intention, so-called shaking, and skew caused by the photographing lens become problems. Therefore, it is known to detect the shaking of the camera by digital signal processing of the gyro and the image, and to correct the sway and skew by optically moving the optical axis of the lens or electronically moving the image signal (i.e., correcting the pixel position). At this time, in order to obtain higher quality image quality, not only parallel movement but also image deformation such as rotation and pulling (7 V) and position correction of sub-pixel units are performed. For example, when extracting a deformed image that has been rotated and scaled from the output image of the image sensor as shown in FIG. 9(a), it is usually required to scan from the image sensor by raster scanning. After reading, the deformed image is also output to the subsequent stage by raster scanning. Here, in the first line after the image is deformed, it is necessary to cache the predetermined area (yin - 4 - 1356779 shadow area) of the image sensor output image. Further, when image deformation such as 90-degree rotation and up-and-down reversal is performed, the predetermined area becomes the amount of one screen. Further, in the installation, since the frame of the object to be written is separated from the frame as the object to be read, a two-frame image buffer is required (for example, refer to Patent Document 1). ,2).

V 詳細地說，如圖9(b)所示，以“從5M像素的單板式 彩色圖像感測器輸出H D動圖像”的攝影機爲例，在現有的 φ 反馬賽克處理中’爲了在拜爾排列的各像素與輸出圖像的 各像素相對應地生成彩色圖像，首先，由從圖像感測器輸 出的5Μ像素的顔色馬賽克圖像生成5Μ的RGB彩色圖像（ 是各像素具有多個顔色資訊的彩色圖像），接下來，對該 RGB彩色圖像進行抖動校正和圖像尺寸的變更這樣的圖像 變形，輸出2M的彩色圖像。 此時，需要將成爲圖像變形物件的5M的RGB彩色圖像 緩存’所以取RGB資料爲24bit/pix，當爲了動圖像處理而 φ 取爲雙暫存器時，緩衝存儲器所需的存儲容量爲240Mbit (2x5Mx24) ° 此外’當考慮60 fps的處理速度時，即使參考每個像 素各一次，該緩衝存儲器所需的頻帶就爲14.4Gbps ( 2 ( 寫入和讀出的量）x5Mx24x60)。此外，由於實際上有時 會參考多次’所以緩衝存儲器所需的頻帶將在此之上。 此外’作爲彩色圖像的壓縮技術，已知有對單板式彩 色圖像感測器的輸出不進行顔色插値處理而直接進行壓縮 以縮小硬體的技術（例如，參考專利文獻3 )。 -5- [專利文獻1]日本特開2007-228515號公報 [專利文獻2]日本特開2007_22 8 5 1 4號公報 [專利文獻3]日本特開2005-217896號公報 但是’在利用了單板式圖像感測器的圖像處理裝置中 ，以往一直對彩色圖像（3parm/pix )進行圖像變形，所以 暫存器需要大的容量和頻帶，有發生功耗增大和成本增加 的危險。此外，專利文獻3的技術公開了爲了謀求硬體的 小型化而對生成彩色圖像前的馬賽克圖像進行壓縮的技術 ’但並沒有公開使用該技術有效地進行圖像變形和生成彩 色圖像時的技術，包含圖像變形和顔色生成在內，尙有發 明的餘地。 【發明內容】 因此，本發明的目的在於，提供一種圖像處理裝置和 圖像處理方法，在由從具有濾色器的單板式攝影元件輸出 的顔色馬賽克圖像生成各像素具有多個顔色的亮度資訊的 彩色圖像並且進行圖像變形處理時，能夠節省暫存器的存 儲容量和頻帶，實現低功耗化和低成本化。 爲了達到上述目的而做出的技術方案1所述的發明是 一種圖像處理裝置，其由各像素具有單色的亮度資訊的顔 色馬賽克圖像在各像素生成多個顔色的亮度資訊，並且實 施預定的圖像變形而生成彩色圖像，其中，上述顔色馬賽 克圖像是由具有對不同的多個顔色光進行光電變換的像素 的單板式的攝影元件獲得的，上述圖像處理裝置包括：緩 -6 - 1353779 衝存儲器，其存儲上述顔色馬賽克圖像；圖像變形部，其 使從上述緩衝存儲器輸出的上述顔色馬賽克圖像變形；以 及反馬賽克部，其由被實施了上述圖像變形的顔色馬賽克 - 圖像生成上述彩色圖像。 . 按照技術方案1所述的圖像處理裝置，包括存儲顔色 馬賽克圖像的緩衝存儲器、使從緩衝存儲器輸出的顔色馬 賽克圖像變形的圖像變形部、以及由被實施了圖像變形的 I 顔色馬賽克圖像生成彩色圖像的反馬賽克部，因此，在由 從單板式攝影元件輸出的顔色馬賽克圖像生成各像素具有 多個顔色的亮度資訊的彩色圖像並且進行圖像變形處理時 ’與對彩色圖像進行圖像變形相比，能夠節省暫存器的存 儲容量和頻帶，實現低功耗化和低成本化。即，按照將彩 色圖像存儲在暫存器中的結構，每1個像素需要與RGB相 對應的3個變數，但通過像本發明這樣將顔色馬賽克圖像 存儲在緩衝存儲器中’能夠使每1個像素的變數爲1個，能 φ 夠提供節省緩衝存儲器所需的容量和頻帶而實現低功耗化 和低成本化並具有圖像變形功能的攝影裝置。 例如，如圖3所示’以“從5 Μ像素的單板式彩色圖像感 測器輸出H D動圖像”的攝影機爲例，此時，以顔色馬賽克 圖像緩存，對顔色馬賽克圖像進行圖像變形和反馬賽克處 理。此時’顔色馬賽克圖像爲l2bit/pix，當爲了動圖像處 理而取爲雙暫存器時，緩衝存儲器所需的存儲容量爲 120Mbit ( 2χ5Μχ12 )，與圖9所示的現有例子相比能夠使 存儲容量減半。 1353779 此外，技術方案1所述的圖像處理裝置，如技術方案2 所述的發明那樣，還包括壓縮部，其壓縮上述顔色馬賽克 圖像；以及拉伸部，其拉伸由上述壓縮部壓縮後的上述顔 色馬賽克圖像，上述圖像處理裝置構成爲將上述顔色馬賽 克圖像由上述壓縮部壓縮後存儲到上述緩衝存儲器中。由 此，能夠進一步削減緩衝存儲器的存儲容量。 此外，技術方案2所述的圖像處理裝置，如技術方案3 所述的發明那樣，還包括顔色平面分解部，該顔色平面分 解部將上述顔色馬賽克圖像分解成僅包含同一顔色光的亮 度資訊的多個顔色平面，上述壓縮部將上述各顔色平面分 割成多個塊，按該分割而成的各塊壓縮成預定的尺寸以下 。由此，能夠容易地將從單板式彩色圖像感測器輸出的顔 色馬賽克圖像與各顔色相對應地壓縮。 此外，技術方案2所述的圖像處理裝置，優選如技術 方案4所述的發明那樣，上述壓縮部進行與上述亮度的灰 度相對應的灰度壓縮。 此外，技術方案3所述的圖像處理裝置，優選如技術 方案5所述的發明那樣’上述拉伸部將由上述壓縮部壓縮 後的顔色馬賽克圖像按上述各塊拉伸。 此外’技術方案1所述的圖像處理裝置，優選如技術 方案6所述的發明那樣’還包括存儲上述顔色馬賽克圖像 的像素値的高速緩衝存儲器，上述圖像變形部能夠不經由 上述緩衝存儲器而從上述高速緩衝存儲器獲得上述像素値 。即’在反馬賽克處理中，在生成彩色圖像的〗個像素時 -8 - 1353779 ’參照顔色馬賽克圖像中的多個顔色的像素値，但在生成 彩色圖像的相鄰的像素時，多參照（訪問）顔色馬賽克圖 像的同一像素。因此，通過使之前剛剛參照的像素存儲在 高速緩衝存儲器中，能夠削減緩衝存儲器的讀出頻帶，能 夠進一步削減功耗。 並且’技術方案6所述的圖像處理裝置，優選如技術 方案7所述的發明那樣，上述高速緩衝存儲器存儲最近訪 問過的像素的像素値或上述塊內的像素値組。 接下來，技術方案8所述的發明是一種圖像處理方法 ’由各像素具有單色的亮度資訊的顔色馬賽克圖像在各像 素生成多個顔色的亮度資訊，並且實施預定的圖像變形而 生成彩色圖像’其中，上述顔色馬賽克圖像是由具有對不 同的多個顔色光進行光電變換的像素的單板式的攝影元件 獲得的’在上述圖像處理方法中使用存儲上述顔色馬賽克 圖像的緩衝存儲器，上述圖像處理方法包括：圖像變形步 驟，使從上述緩衝存儲器輸出的顔色馬賽克圖像變形；以 及反馬賽克步驟，由被實施了上述圖像變形的顔色馬賽克 圖像生成上述彩色圖像。 按照技術方案8所述的圖像處理方法，使用存儲顔色 馬賽克圖像的緩衝存儲器’包括使從緩衝存儲器輸出的顔 色馬賽克圖像變形的圖像變形步驟、以及由被實施了上述 圖像變形的顔色馬賽克圖像生成上述彩色圖像的反馬賽克 步驟，因此，與技術方案1所述的發明相同，與對彩色圖 像進行圖像變形相比’能夠節省暫存器的存儲容量和頻帶 -9 - 1353779 ，實現低功耗化和低成本化。V In detail, as shown in FIG. 9(b), a camera that outputs "HD moving image from a single-plate color image sensor of 5 M pixels" is taken as an example, in the existing φ anti-mosaic processing, in order to Each pixel of the Bayer array generates a color image corresponding to each pixel of the output image. First, a 5-inch RGB color image is generated by a color mosaic image of 5 pixels output from the image sensor (is each pixel) The color image having a plurality of color information is next, and the RGB color image is subjected to image distortion such as shake correction and image size change, and a 2M color image is output. At this time, it is necessary to cache the 5M RGB color image of the image deformed object. Therefore, the RGB data is 24 bits/pix. When φ is taken as the dual register for moving image processing, the storage required for the buffer memory is required. The capacity is 240Mbit (2x5Mx24) ° In addition, when considering the processing speed of 60 fps, the frequency band required for the buffer memory is 14.4 Gbps (2 (the amount of writing and reading) x5Mx24x60) even if each pixel is referenced once. . In addition, since it is sometimes referred to multiple times in practice, the frequency band required for the buffer memory will be above this. Further, as a compression technique of a color image, a technique of directly compressing the output of the single-plate color image sensor without performing color interpolation processing to reduce the hardware is known (for example, refer to Patent Document 3). [Patent Document 1] Japanese Laid-Open Patent Publication No. 2007-228515 [Patent Document 2] Japanese Laid-Open Patent Publication No. 2007-22 No. 2005-217896 In the image processing apparatus of the plate type image sensor, since the color image (3parm/pix) is image-deformed in the past, the register requires a large capacity and a frequency band, and there is a risk of an increase in power consumption and an increase in cost. . Further, the technique of Patent Document 3 discloses a technique of compressing a mosaic image before generating a color image in order to reduce the size of the hardware. However, it is not disclosed to effectively perform image deformation and generate a color image using the technique. The technology of the time, including image deformation and color generation, has no room for invention. SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an image processing apparatus and an image processing method for generating each pixel having a plurality of colors in a color mosaic image output from a single-plate type imaging element having a color filter. When the color image of the luminance information is processed and the image is deformed, the storage capacity and the frequency band of the temporary storage device can be saved, and the power consumption can be reduced and the cost can be reduced. The invention described in claim 1 for achieving the above object is an image processing apparatus that generates luminance information of a plurality of colors in each pixel from a color mosaic image in which each pixel has monochrome luminance information, and implements The predetermined image is deformed to generate a color image, wherein the color mosaic image is obtained by a single-plate type photographic element having pixels for photoelectrically converting different plurality of color lights, and the image processing apparatus includes: -6 - 1353779 a flush memory storing the color mosaic image; an image deforming portion that deforms the color mosaic image output from the buffer memory; and an anti-mosaic portion that is subjected to the image deformation described above Color Mosaic - The image produces the above color image. The image processing device according to claim 1, comprising a buffer memory that stores a color mosaic image, an image deforming portion that deforms a color mosaic image output from the buffer memory, and an I that is deformed by the image The color mosaic image generates an anti-mosaic portion of the color image, and therefore, when a color image in which each pixel has luminance information of a plurality of colors is generated from the color mosaic image output from the single-plate imaging element and image deformation processing is performed' Compared with image deformation of a color image, the storage capacity and frequency band of the scratchpad can be saved, and the power consumption and cost can be reduced. That is, according to the configuration in which the color image is stored in the temporary memory, three variables corresponding to RGB are required for each pixel, but by storing the color mosaic image in the buffer memory as in the present invention, it is possible to make each The number of variables per pixel is one, and it is possible to provide a photographing device having an image distortion function by providing a capacity and a frequency band required for saving the buffer memory, thereby achieving low power consumption and low cost. For example, as shown in FIG. 3, a camera that outputs a HD moving image from a single-panel color image sensor of 5 pixels is used as an example. At this time, the color mosaic image is buffered by the color mosaic image. Image distortion and anti-mosaic processing. At this time, the 'color mosaic image is l2bit/pix. When the double buffer is used for moving image processing, the storage capacity required for the buffer memory is 120 Mbit (2χ5Μχ12), compared with the conventional example shown in FIG. Can halve storage capacity. 1353. The image processing device according to claim 2, further comprising a compression unit that compresses the color mosaic image; and a stretching portion that is stretched by the compression unit In the subsequent color mosaic image, the image processing device is configured to compress the color mosaic image by the compression unit and store the color mosaic image in the buffer memory. Thereby, the storage capacity of the buffer memory can be further reduced. Further, the image processing device according to claim 2, further comprising a color plane decomposition unit that decomposes the color mosaic image into brightness including only the same color light, as in the invention described in claim 3 In the plurality of color planes of the information, the compression unit divides each of the color planes into a plurality of blocks, and compresses the divided blocks into a predetermined size or smaller. Thereby, the color mosaic image output from the single-plate color image sensor can be easily compressed corresponding to each color. Further, in the image processing device according to the second aspect of the invention, the compression unit preferably performs gradation compression corresponding to the gradation of the luminance. Further, in the image processing device according to the third aspect of the invention, the stretching unit preferably stretches the color mosaic image compressed by the compression unit by the respective blocks. Further, the image processing device according to the first aspect of the present invention, as set forth in the sixth aspect of the invention, further includes a cache memory for storing a pixel of the color mosaic image, wherein the image deforming unit can pass the buffer. The above pixel is obtained from the above-mentioned cache memory by the memory. That is, in the anti-mosaic processing, when a pixel of a color image is generated -8 - 1353779 'refers to a pixel 多个 of a plurality of colors in the color mosaic image, but when generating adjacent pixels of the color image, Multi-reference (access) the same pixel of the color mosaic image. Therefore, by storing the pixels just referenced in the cache memory, the read frequency band of the buffer memory can be reduced, and power consumption can be further reduced. Further, in the image processing device according to claim 6, preferably, in the invention described in claim 7, the cache memory stores a pixel 像素 of a pixel that has been recently accessed or a pixel group in the block. Next, the invention described in claim 8 is an image processing method 'a color mosaic image in which each pixel has monochrome luminance information generates luminance information of a plurality of colors in each pixel, and performs predetermined image deformation. Generating a color image 'where the color mosaic image is obtained by a single-plate type photographic element having pixels for photoelectrically converting different color lights,' used in the image processing method described above to store the color mosaic image described above a buffer memory, the image processing method comprising: an image deforming step of deforming a color mosaic image output from the buffer memory; and an anti-mosaic step of generating the color by a color mosaic image subjected to the image deformation image. According to the image processing method of the eighth aspect, the buffer memory 'which stores the color mosaic image' includes an image deforming step of deforming the color mosaic image output from the buffer memory, and the image deformed by the image described above. The color mosaic image generates the anti-mosaic step of the above-described color image, and therefore, as in the invention described in the first aspect, the storage capacity and the frequency band of the scratchpad can be saved as compared with the image deformation of the color image. - 1353779 for low power consumption and low cost.

此外，技術方案8所述的圖像處理方法’如技術方案9 所述的發明那樣，還包括壓縮步驟，壓縮上述顔色馬賽克 圖像；以及拉伸步驟，拉伸在上述壓縮步驟中壓縮後的上 述顔色馬賽克圖像，在上述圖像處理方法中’將上述顔色 馬賽克圖像在上述壓縮步驟中壓縮後存儲到上述緩衝存儲 器中。由此，與技術方案2所述的發明相同，能夠進一步 削減緩衝存儲器的存儲容量。 此外，技術方案9所述的圖像處理方法，如技術方案 10所述的發明那樣，還包括顔色平面分解步驟，將上述顔 色馬賽克圖像分解成僅包含同一顔色光的亮度資訊的多個 顔色平面，上述壓縮步驟將上述各顔色平面分割成多個塊 ，按該分割而成的各塊壓縮成預定的尺寸以下。由此，與 技術方案3所述的發明相同，能夠容易地將從單板式彩色Further, the image processing method according to claim 8 further includes a compression step of compressing the color mosaic image as in the invention described in claim 9, and a stretching step of stretching the compression in the compression step In the above color mosaic image, the color mosaic image is compressed in the compression step and stored in the buffer memory. As a result, in the same manner as the invention described in claim 2, the storage capacity of the buffer memory can be further reduced. Further, the image processing method according to claim 9 further includes a color plane decomposition step of decomposing the color mosaic image into a plurality of colors including only luminance information of the same color light, as in the invention described in claim 10. In the plane, the compression step divides each of the color planes into a plurality of blocks, and the divided blocks are compressed to a predetermined size or smaller. Thus, as in the invention described in claim 3, it is possible to easily convert from a single-plate color

圖像感測器輸出的顔色馬賽克圖像與各顔色相對應地壓縮 〇 此外，技術方案9所述的圖像處理方法，優選如技術 方案1〗所述的發明那樣，在上述壓縮步驟中，進行與上述 亮度的灰度相對應的灰度壓縮。 此外，技術方案1 0所述的圖像處理方法，優選如技術 方案12所述的發明那樣，在上述拉伸步驟中，將在上述壓 縮步驟中壓縮後的顔色馬賽克圖像按上述各塊拉伸。 此外，技術方案8所述的圖像處理方法，如技術方案 1 3所述的發明那樣，使用存儲上述顔色馬賽克圖像的像素 -10 - 1353779The color mosaic image output by the image sensor is compressed corresponding to each color. Further, the image processing method according to claim 9 is preferably as in the invention described in the first aspect, in the above-described compression step, Grayscale compression corresponding to the gradation of the above luminance is performed. Further, in the image processing method according to claim 10, preferably, in the stretching step, the color mosaic image compressed in the compressing step is pulled in the respective blocks in the stretching step. Stretch. Further, in the image processing method according to the eighth aspect of the invention, as in the invention described in the first aspect, the pixel storing the color mosaic image is used -10 - 1353779

値的高速緩衝存儲器，在上述圖像變形步驟中，能夠不經 由上述緩衝存儲器而從上述高速緩衝存儲器獲得上述像素 値。由此，與技術方案6所述的發明相同，在反馬賽克處 理中，在生成彩色圖像的1個像素時，參照顔色馬賽克圖 像中的多個顔色的像素値，但在生成彩色圖像的相鄰的像 素時’多參照（訪問）顔色馬賽克圖像的同一像素。因此 ’通過使之前剛剛參照的像素存儲在高速緩衝存儲器中， 能夠削減緩衝存儲器的讀出頻帶，能夠進一步削減功耗。 並且，技術方案13所述的圖像處理方法，優選如技術 方案14所述的發明那樣，上述高速緩衝存儲器存儲最近訪 問過的像素的像素値或上述塊內的像素値組。 本發明的圖像處理裝置和圖像處理方法包括存儲顔色 馬賽克圖像的緩衝存儲器、使從緩衝存儲器輸出的顔色馬 賽克圖像變形的圖像變形部、以及由被實施了圖像變形的 顔色馬賽克圖像生成彩色圖像的反馬賽克部，因此，在由 從單板式攝影元件輸出的顔色馬賽克圖像生成各像素具有 多個顔色的亮度資訊的彩色圖像並且進行圖像變形處理時 ，與對彩色圖像進行圖像變形相比，能夠節省暫存器的存 儲容量和頻帶，實現低功耗化和低成本化。即，按照將;彩 色圖像存儲在暫存器中的結構，每1個像素需要與RGB相 對應的3個變數’但通過像本發明這樣將顔色馬賽克圖像 存儲在緩衝存儲器中’能夠使每1個像素的變數爲1個，能 夠提供節省緩衝存儲器所需的容量和頻帶而實現低功耗化 和低成本化並具有圖像變形功能的攝影裝置。 -11 - 1353779 此外’本發明的圖像處理裝置和圖像處理方法包括壓 縮上述顔色馬賽克圖像的壓縮部、以及拉伸由壓縮部壓縮 後的上述顔色馬賽克圖像的拉伸部，使得將顔色馬賽克圖 像由壓縮部壓縮後存儲到上述緩衝存儲器中，由此，能夠 進一步削減緩衝存儲器的存儲容量。 此外’本發明的圖像處理裝置和圖像處理方法包括將 顔色馬賽克圖像分解成僅包含同一顔色光的亮度資訊的多 個顔色平面的顔色平面分解部，壓縮部將上述各顔色平面 分割成多個塊，按該分割而成的各塊壓縮成預定的尺寸以 下，由此，能夠容易地將從單板式彩色圖像感測器輸出的 顔色馬賽克圖像與各顔色相對應地壓縮。 此外，本發明的圖像處理裝置和圖像處理方法包括存 儲馬賽克圖像的像素値的高速緩衝存儲器，圖像變形部能 夠不經由緩衝存儲器而從高速緩衝存儲器獲得上述像素値 ’由此，能夠削減緩衝存儲器的讀出頻帶，能夠進一步削 減功耗。 此外，本發明的圖像處理裝置和圖像處理方法優選的 是，壓縮部進行與上述亮度的灰度相對應的灰度壓縮，優 選的是，拉伸部按各塊拉伸由壓縮部壓縮後的顔色馬賽克 圖像’更加優選的是，高速緩衝存儲器存儲最近訪問過的 像素的像素値或上述塊內的像素値組。 【實施方式】 (第一實施方式） -12- 1353779 接下來，基於附圖說明本發明的圖像處理裝置和圖像 處理方法的第一實施方式。 圖1是表示應用了本發明的圖像處理裝置和圖像處理 方法的、第一實施方式的攝影裝置1的結構的框圖，圖2是 該第一實施方式中的顔色平面分解部和顔色生成部的功能 說明圖，（a)是表示從攝影單元2輸出的拜爾排列的顔色 馬賽克圖像的圖，（b) 、（c) 、（d) 、（e)分別是表 示由顔色平面分解部生成的R平面、Gr平面、Gb平面、B 平面的配置的圖，（f)是插値計算採樣座標的像素値時 的說明圖。 此外，圖3是該第一實施方式中的、由顔色馬賽克圖 像進行圖像變形和生成彩色圖像的反馬賽克處理的說明圖 ’圖4是表示該第一實施方式的圖像處理裝置和圖像處理 方法中的彩色圖像生成的順序的流程圖。 攝影裝置1是例如手機等移動設備上搭載的照相機模 組，如圖1所示，由攝影單元2和圖像處理裝置1 00構成， 其中’上述攝影單元2將被攝體像P引導至攝影元件5而作 爲數位圖像信號（馬賽克狀的圖像信號）輸出，上述圖像 處理裝置100根據經由攝影單元2輸出的數位圖像信號，進 行攝影單元2相對於被攝體的晃動校正和預定的圖像變形 ’並且生成各像素具有多個顔色資訊的彩色圖像。另外， 本實施例的攝影裝置1如圖3所示，在圖像處理裝置1〇〇中 ’將從攝影單元2輸出的顔色馬賽克圖像存儲到後述的緩 衝存儲器2 1，訪問緩衝存儲器2 1而進行圖像變形和反馬賽 -13- 1353779 收 元接 單將 影 3 攝頭 ’ 鏡 示影 所攝 ffl ^ 如件 ’ 元 來影 。下攝 理接至 處 導 克 弓 像 體 攝 被 將 括 包 2 爲 換 轉 光 影 攝 的 到 電學量而輸出的攝影元件（CCD : Charge C0UPled Devices )5、將從攝影元件5輸出的類比圖像信號轉換爲數位圖像 信號C而輸出的AFE( Analog Front End) 6、以預定的周 期控制攝影元件5和AFE6的TG ( Timing Generator) 13、 以及檢測攝影裝置1相對於被攝體的晃動並輸出與其晃動 量相應的電信號的角速度感測器1 5等。 攝影元件5是單板式攝影元件，其構成爲多個光電變 換元件呈矩陣狀配置，在其前面與光電變換元件相對應地 具有由R (紅）G (綠）B (藍）三原色的拜爾（Bayer ) 排列構成的濾色器5a，將通過各顔色的濾色器部的單一顔 色的光量轉換爲類比電信號。另外，從攝影元件5以光栅 順序輸出類比電信號。 上述原色拜爾排列如圖2 ( a )所示，G色的濾色器以 格子圖案配置，G色濾色器與R色濾色器交替配置的列和配 置了 G濾色器和B濾色器的列交替構成。另外，在本實施例 中，在圖2(a)中，將與R並排配置於一個方向上的G色表 示爲Gr、將與B並排配置於一個方向上的G色表示爲Gb。 AFE6由對經由攝影元件5輸出的類比圖像信號除去雜 訊的相關雙採樣電路（CDS: Correlated Double Sampling )7、將由相關雙採樣電路7進行了相關雙採樣的圖像信號 放大的可變增益放大器（AGC: Automatic Gain Control) -14- 1353779 8、以及將經由可變增益放大器8輸入的類比圖像信號轉換 t 爲數位圖像信號的A/D轉換器9等構成，將從攝影元件5輸 出的圖像信號轉換爲數位圖像信號C而輸出到圖像處理裝 - 置 1 00。 . 此外，在攝影單元2中，也可以使用CMOS (In the above-described image warping step, the pixel memory can be obtained from the cache memory without using the buffer memory. Therefore, in the same manner as the invention described in claim 6, in the inverse mosaic processing, when one pixel of the color image is generated, the pixels 多个 of the plurality of colors in the color mosaic image are referred to, but the color image is generated. The adjacent pixels of the 'multiple reference (access) color mosaic image of the same pixel. Therefore, by storing the pixels just referenced in the cache memory, the read frequency band of the buffer memory can be reduced, and power consumption can be further reduced. Further, in the image processing method according to claim 13, it is preferable that the cache memory stores the pixel 最近 of the most recently accessed pixel or the pixel 値 group in the block, as in the invention described in claim 14. An image processing apparatus and an image processing method of the present invention include a buffer memory that stores a color mosaic image, an image deforming portion that deforms a color mosaic image output from the buffer memory, and a color mosaic that is deformed by the image The image generates an anti-mosaic portion of the color image, and therefore, when a color image in which each pixel has luminance information of a plurality of colors is generated from the color mosaic image output from the single-plate imaging element, and image deformation processing is performed, Compared with the image distortion of the color image, the storage capacity and the frequency band of the scratchpad can be saved, and the power consumption can be reduced and the cost can be reduced. That is, according to the configuration in which the color image is stored in the temporary memory, three variables 'corresponding to RGB' are required for each pixel, but the color mosaic image is stored in the buffer memory as in the present invention'. Since the number of variables per one pixel is one, it is possible to provide a photographing apparatus that can reduce the power consumption and cost of the buffer memory and realize the image distortion function while saving the capacity and frequency band required for the buffer memory. -11 - 1353779 Further, the image processing apparatus and the image processing method of the present invention include a compression unit that compresses the color mosaic image, and a stretching unit that stretches the color mosaic image compressed by the compression unit, so that The color mosaic image is compressed by the compression unit and stored in the buffer memory, whereby the storage capacity of the buffer memory can be further reduced. Further, the image processing apparatus and the image processing method of the present invention include a color plane decomposition unit that decomposes a color mosaic image into a plurality of color planes including luminance information of the same color light, and the compression unit divides the respective color planes into Each of the plurality of blocks is compressed to a predetermined size or less, whereby the color mosaic image output from the single-plate color image sensor can be easily compressed in accordance with each color. Further, the image processing apparatus and the image processing method of the present invention include a cache memory for storing a pixel of a mosaic image, and the image deforming unit can obtain the pixel 値 from the cache memory without using a buffer memory. By reducing the read frequency band of the buffer memory, power consumption can be further reduced. Further, in the image processing apparatus and the image processing method of the present invention, it is preferable that the compression unit performs gradation compression corresponding to the gradation of the luminance, and it is preferable that the stretching portion is compressed by the compression unit for each block stretching. The latter color mosaic image is more preferably that the cache stores the pixels of the recently accessed pixels or the group of pixels within the above blocks. [Embodiment] (First Embodiment) -12 - 1353779 Next, a first embodiment of an image processing apparatus and an image processing method of the present invention will be described based on the drawings. 1 is a block diagram showing a configuration of an image pickup apparatus 1 according to a first embodiment to which an image processing apparatus and an image processing method according to the present invention are applied, and FIG. 2 is a color plane decomposition section and color in the first embodiment. A function explanatory diagram of the generating unit, (a) is a view showing a color mosaic image of a Bayer array output from the photographing unit 2, and (b), (c), (d), and (e) respectively indicate a color plane. (F) is an explanatory diagram of the arrangement of the R plane, the Gr plane, the Gb plane, and the B plane generated by the decomposition unit, and (f) is a case where the pixel 値 of the sampling coordinates is calculated. 3 is an explanatory diagram of an image mosaic processing for performing image deformation and color image generation by a color mosaic image in the first embodiment. FIG. 4 is a view showing an image processing apparatus according to the first embodiment. A flowchart of the sequence of color image generation in the image processing method. The photographing device 1 is a camera module mounted on a mobile device such as a mobile phone. As shown in FIG. 1, the photographing device 1 is composed of a photographing unit 2 and an image processing device 100, wherein the photographing unit 2 guides the subject image P to photographing. The element 5 is output as a digital image signal (mosaic image signal), and the image processing apparatus 100 performs shake correction and reservation of the photographing unit 2 with respect to the subject based on the digital image signal outputted through the photographing unit 2. The image is deformed' and generates a color image in which each pixel has a plurality of color information. Further, as shown in FIG. 3, the photographing apparatus 1 of the present embodiment stores in the image processing apparatus 1 'the color mosaic image output from the photographing unit 2 to a buffer memory 2 1 to be described later, and accesses the buffer memory 2 1 . And the image deformation and anti-Marseille-13- 1353779 receiving the order will be the shadow of the 3 camera 'mirror filming ffl ^ as the piece 'yuan to shadow. The photographic element (CCD: Charge C0UPled Devices) outputted from the photographic element (CCD: Charge C0UPled Devices) 5, the analog image output from the photographic element 5 An AFE (Analog Front End) 6 that converts a signal into a digital image signal C, controls a TG (Timing Generator) 13 of the photographic element 5 and the AFE 6 at a predetermined cycle, and detects sway of the photographic apparatus 1 with respect to the subject. An angular velocity sensor 15 or the like that outputs an electric signal corresponding to the amount of shaking thereof. The photographic element 5 is a single-plate type photographic element, and is configured such that a plurality of photoelectric conversion elements are arranged in a matrix, and a front side of the photoelectric conversion element has Bayer having three primary colors of R (red) G (green) B (blue). (Bayer) The color filter 5a is arranged to convert the amount of light of a single color passing through the color filter portion of each color into an analog electric signal. Further, an analog electrical signal is output from the photographing element 5 in raster order. The above-mentioned primary color Bayer arrangement is as shown in Fig. 2(a), the G color filter is arranged in a lattice pattern, the G color filter and the R color filter are alternately arranged in the column, and the G color filter and the B filter are arranged. The columns of the color switches are alternately formed. Further, in the present embodiment, in Fig. 2(a), the G color which is arranged in one direction with R is indicated as Gr, and the G color which is arranged in parallel with B in one direction is denoted as Gb. The AFE 6 is a correlated double sampling circuit (CDS: Correlated Double Sampling) that removes noise from the analog image signal output via the imaging element 5, and a variable gain that amplifies the image signal subjected to correlated double sampling by the correlated double sampling circuit 7. An amplifier (AGC: Automatic Gain Control) - 14 - 1353779 8 and an A/D converter 9 that converts an analog image signal input via the variable gain amplifier 8 into a digital image signal, and the like. The output image signal is converted into a digital image signal C and output to the image processing apparatus 100. In addition, in the photography unit 2, CMOS can also be used (

Complementary Metal Oxide Semiconductor)感測器來代 替攝影元件5、相關雙採樣電路7、可變增益放大器8、以 φ 及A/D轉換器9等。因爲從攝影元件5輸出的各像素的信號 只具有單一顔色的資訊，所以從攝影單元2向圖像處理裝 置100輸出馬賽克狀的圖像信號。另外，剛剛被AD轉換後 的顔色馬賽克圖像也被稱爲RAW資料。 角速度感測器1 5例如由公知的振動陀螺儀構成，生成 與該攝影單元2的、向被攝體前後方向的晃動、左右方向 的晃動、上下方向的晃動、以前後方向爲旋轉軸的旋轉晃 動、以左右方向爲旋轉軸的旋轉晃動、以上下方向爲旋轉 φ 軸的旋轉晃動等相應的電信號（以下稱爲晃動信號）並輸 出到圖像處理裝置100。另外，也可以使用壓電型的三軸 加速度感測器或與彼此正交的3個軸方向相對應地使用多 個加速度感測器來代替角速度感測器1 5。另外，在檢測晃 動時，也可以在圖像處理裝置1 〇 〇中檢測在預定時間內拍 攝的多個圖像間的圖像信號的差値並根據這些差値檢測上 述晃動，來代替利用上述感測器檢測晃動。 接下來’圖像處理裝置100由保存（存儲）從攝影單 元2輸出的馬賽克圖像的緩衝存儲器21 ;生成對從緩衝存 -15- 1353779 儲器21輸出的顔色馬賽克圖像實施了預定的圖像變形的彩 色圖像的反馬賽克部28;爲了提高顔色圖像的視覺質量而 對從顔色生成部33輸出的彩色圖像信號進行公知的伽瑪校 正和色度校正、邊緣增強等的視覺校正部34;將經由視覺 校正部34輸出的彩色圖像用例如JPEG等方法壓縮的壓縮部 3 5 ;將經由壓縮部3 5輸出的彩色圖像記錄到例如閃速存儲 器等記錄介質中的記錄部36 ; CPU ( Central Processing Unit) 18;以及 ROM (Read Only Memory) 19等構成， CPU 18按照存儲於ROM 19中的控制用程式，控制該攝影 裝置1和圖像處理裝置100的各種處理。 另外，圖像處理裝置100包括：與攝影鏡頭3的鏡頭狀 態相對應地記錄有用於校正攝影單元2的歪曲像差的歪曲 像差係數的像差係數表3 8、根據從角速度感測器1 5輸出的 晃動信號計算相對於從攝影單元2輸入的顔色馬賽克圖像 的晃動的校正値的晃動檢測部40等。 緩衝存儲器21由DRAM等實現，與拜爾排列相對應地 ’由存儲R的像素信號的R平面存儲器22、存儲Gr的像素信 號的Gr平面存儲器23a、存儲Gb的像素信號的Gb平面存儲 器23b、存儲B的像素信號的B平面存儲器24構成，將R像 素、Gr像素、Gb像素、B像素分別以光柵順序按序保存， 根據來自CPU 18的指令，將這些像素信號（以下稱爲像素 値）輸出到反馬賽克部28中的採樣部31。本發明中的顏色 平面分解部，由緩衝存儲器21來表現其功能。 在像差係數表38中保存有表示由攝影鏡頭3引起的像 -16- 1353779 差的像差係數。該像差係數被輸出到反馬賽克部28中的座 標變換部30。 晃動檢測部40根據從角度感測器15輸出的晃動的電信 號，檢測攝影單元2相對於被攝體的晃動，將用於校正晃 動的晃動校正參數（z、dx、dy、Θ、0y、0x等）輸出到 反馬賽克部28中的座標變換部30。 在晃動校正參數（以下也稱爲晃動校正値）中，相對 於被攝體而言，z是與攝影單元2的前後方向的晃動相伴的 被攝體像的大小的校正値，dx是與左右方向或搖擺軸的晃 動相伴的被攝體像的左右位置的校正値、dy是與上下方向 或俯仰軸的晃動相伴的被攝體像的上下位置的校正値、Θ 是與以前後方向爲旋轉軸的晃動相伴的被攝體像的旋轉的 校正値、4 y是與上下方向或俯仰軸的晃動相伴的被攝體 像的上下方向的牽拉的校正値、0X是與左右方向或搖擺 軸的晃動相伴的被攝體像的左右方向的牽拉的校正値。另 外，晃動校正參數z也可以包含攝影裝置1的變焦倍率。 接下來，反馬賽克部28包括：統一各種像差校正和抖 動校正、圖像的縮放這樣的圖像變形，並對輸出的彩色圖 像的像素座標進行掃描的輸出像素掃描部29 ;根據表示從 輸出像素掃描部29輸出的像素座標的信號和從晃動檢測部 40與像差係數表38輸出的信號，計算與輸出的彩色圖像的 像素位置對應的顔色馬賽克圖像上的採樣座標的座標變換 部30 ;將由座標變換部30計算出的採樣座標附近的顔色馬 賽克圖像上的各顔色的像素的像素値，從緩衝存儲器2 1讀 -17- 1353779 顔色平面由通過採 的像素値的插値部 標的各顔色的像素 色資料的顔色生成 爲顔色馬賽克圖像 對應，所以通過該 對應關係。接下來 序掃描輸出的彩色 Ud，Vd )輸出到座 檢測部40輸出的晃 差係數，包含歪曲 圖像變形在內，計 vd )對應的顔色馬 中的圖像變形借助 相對應的座標系用 標系用xy表示。另 ί標，下標d表示在 入的採樣部31 ;按各R、Gr、Gb、B的 樣部3 1獲得的像素値插値生成採樣座標 32 ;以及合成由插値部32獲得的採樣座 値’生成各像素具有多個顔色成分的顔 部33等》 此外，當進行上述圖像變形時，因 和輸出彩色圖像的像素位置不是單純地 反馬賽克部28中的座標變換部30建立其 ’說明反馬賽克部28處理的細節。 在輸出像素掃描部29中，以光柵順 圖像的全部像素，並將各像素的座標（ 標變換部3 0。 接下來，座標變換部30根據從晃動 動校正參數和像差係數表38中保存的像 像差校正和抖動校正、數位變焦這樣的 算與輸出的彩色圖像的像素位置（Ud， 賽克圖像上的採樣座標。此外，本發明 於座標變換部30來表現其功能。 首先，作爲座標系，將與像素位置 uv表示，將便於在校正計算中使用的座 外’下標s表示在顔色馬賽克圖像上的JI 輸出彩色圖像上的座標。 另外，對彩色圖像雙方，引入歸一化後的xy座標系。 在xy座標系中，以圖像中心爲原點，使其圖像對角長爲2 1353779Complementary Metal Oxide Semiconductor) replaces the photographic element 5, the correlated double sampling circuit 7, the variable gain amplifier 8, the φ and A/D converter 9, and the like. Since the signal of each pixel output from the imaging element 5 has only a single color of information, a mosaic image signal is output from the imaging unit 2 to the image processing apparatus 100. In addition, the color mosaic image just after AD conversion is also called RAW data. The angular velocity sensor 15 is configured by, for example, a known vibrating gyroscope, and generates a wobble in the front-rear direction of the subject, a wobble in the left-right direction, a wobble in the vertical direction, and a rotation in the front-rear direction in the front-rear direction. The sway, the rotation of the rotation axis in the left-right direction, and the corresponding electric signal (hereinafter referred to as a sway signal) such as the sway of the rotation φ axis are output to the image processing apparatus 100. Alternatively, instead of the angular velocity sensor 15, a piezoelectric type triaxial acceleration sensor or a plurality of acceleration sensors corresponding to three axial directions orthogonal to each other may be used. Further, when detecting the sway, the image processing device 1 may detect the difference of the image signals between the plurality of images captured within the predetermined time period, and detect the sway based on the lags instead of using the above. The sensor detects shaking. Next, the image processing apparatus 100 is configured to save (store) the buffer memory 21 of the mosaic image output from the photographing unit 2; and generate a predetermined map for the color mosaic image output from the buffer memory-15-1353779 reservoir 21. The anti-mosaic portion 28 of the deformed color image performs well-known gamma correction, chromaticity correction, edge enhancement, etc. for the color image signal output from the color generating portion 33 in order to improve the visual quality of the color image. a portion 34 that compresses the color image output via the visual correction unit 34 by a method such as JPEG, and a color portion that is output via the compression unit 35 to a recording unit such as a flash memory. 36; CPU (Central Processing Unit) 18; and ROM (Read Only Memory) 19 and the like, and the CPU 18 controls various processes of the imaging device 1 and the image processing device 100 in accordance with a control program stored in the ROM 19. Further, the image processing apparatus 100 includes an aberration coefficient table 38 in which a distortion aberration coefficient for correcting the distortion of the photographing unit 2 is recorded in correspondence with the lens state of the photographing lens 3, according to the slave angular velocity sensor 1 The sway signal of the output 5 calculates the sway detecting unit 40 and the like for correcting the sway of the color mosaic image input from the photographing unit 2. The buffer memory 21 is realized by a DRAM or the like, and corresponds to the Bayer array 'the R plane memory 22 that stores the pixel signals of R, the Gr plane memory 23a that stores the pixel signals of Gr, the Gb plane memory 23b that stores the pixel signals of Gb, The B-plane memory 24 storing the pixel signals of the B is configured to store the R pixels, the Gr pixels, the Gb pixels, and the B pixels in raster order, and these pixel signals (hereinafter referred to as pixels 根据) are instructed according to an instruction from the CPU 18. It is output to the sampling section 31 in the inverse mosaic section 28. The color plane decomposition unit in the present invention expresses its function by the buffer memory 21. In the aberration coefficient table 38, an aberration coefficient indicating the difference of the image -16 - 1353779 caused by the photographic lens 3 is stored. This aberration coefficient is output to the coordinate transformation unit 30 in the inverse mosaic unit 28. The sway detecting unit 40 detects the sway of the imaging unit 2 with respect to the subject based on the swaying electrical signal output from the angle sensor 15, and sets the sway correction parameter (z, dx, dy, Θ, 0y, for correcting the sway). 0x or the like) is output to the coordinate conversion unit 30 in the inverse mosaic unit 28. In the sway correction parameter (hereinafter also referred to as sway correction 値), z is a correction of the size of the subject image accompanying the sway of the photographing unit 2 in the front-rear direction with respect to the subject, and dx is about The correction 値 and dy of the left and right positions of the subject image accompanying the sway of the direction or the sway axis are the correction of the vertical position of the subject image accompanying the sway of the vertical direction or the pitch axis, and the rotation is the rotation of the front and rear directions. The correction 値, 4 y of the rotation of the subject image accompanying the sway of the axis is the correction of the vertical direction of the subject image accompanying the sway of the vertical direction or the pitch axis, and 0X is the left-right direction or the sway axis. Correction of the pulling of the left and right direction of the subject image accompanying the shaking. Further, the shake correction parameter z may also include the zoom magnification of the photographing apparatus 1. Next, the anti-mosaic portion 28 includes an image pixel portion 29 that unifies image distortion such as various aberration correction and shake correction, image scaling, and scans pixel coordinates of the output color image; The signal of the pixel coordinates output from the pixel scanning unit 29 and the signal output from the shake detecting unit 40 and the aberration coefficient table 38 are used to calculate the coordinate transformation of the sampling coordinates on the color mosaic image corresponding to the pixel position of the output color image. The portion 30 of the pixel of each color on the color mosaic image in the vicinity of the sampling coordinates calculated by the coordinate conversion unit 30 is read from the buffer memory 2 1 - 17353779. The color plane is inserted through the pixel of the pixel. The color of the pixel color data of each color of the part mark is generated as the color mosaic image corresponding, so the correspondence relationship is passed. The color Ud, Vd of the sequence scan output is outputted to the sag coefficient outputted by the seat detecting unit 40, and the image distortion in the color horse corresponding to the vd) is corrected by the corresponding coordinate system. The standard is represented by xy. In addition, the subscript d indicates the sampling portion 31 that is in the input; the pixel interpolation obtained by the sample 31 of each of R, Gr, Gb, and B generates the sampling coordinates 32; and the sampling obtained by the interpolation portion 32 is synthesized. Further, when the image is deformed, the pixel position of the output color image is not simply established by the coordinate conversion unit 30 in the inverse mosaic unit 28. It 'describes the details of the processing by the anti-mosaic part 28. In the output pixel scanning unit 29, all the pixels of the image are rastered, and the coordinates of each pixel are used (the scale conversion unit 30. Next, the coordinate conversion unit 30 is based on the slave shake correction parameter and the aberration coefficient table 38. The pixel position (Ud, the sampling coordinates on the Sike image) of the color image of the calculation and output such as the aberration correction, the shake correction, and the digital zoom. The present invention expresses its function in the coordinate conversion unit 30. First, as a coordinate system, it will be represented with the pixel position uv, which will facilitate the coordinates of the JI output color image on the color mosaic image for the extra-seat subscript s used in the correction calculation. Both sides introduce the normalized xy coordinate system. In the xy coordinate system, the image center is taken as the origin, and the diagonal length of the image is 2 1353779.

即，如果使彩色圖像的尺寸爲192 Oxl 080像素’則輸 出的彩色圖像的像素位置（ud，vd )在xy座標系下的表現 爲（式1)。即，在由彩色圖像的尺寸爲1920x1080的等間 隔的像素排列構成時，算出V〇9202 +1080^)/2与1 1 00，賦予偏 移(offset ) (x > y) = (960/ 1 1 00，540/ 1 1 00 = (0.87 > 0.49)， 使得圖像中心（Ud，vd ) = ( 960，540 )爲原點（xd，yd ) = (0，〇)，得至!j (式 1 )。 【式1】 Μ /1100-0.87 <Vd /1100-0.49 式1 接下來，應用上述晃動校正的參數（z、Θ、dx、dy、 0 x、0 y )，利用（式2 )計算晃動校正後的座標（Xs，ys )。此外’在（式 2)中，設 xs = wxs/w、ys = wys/w。That is, if the size of the color image is 192 Oxl 080 pixels', the pixel position (ud, vd) of the output color image is expressed as (Formula 1) under the xy coordinate system. That is, when the pixels of the color image are arranged at equal intervals of 1920×1080, V〇9202 + 1080^)/2 and 1 1 00 are calculated, and the offset (x > y) = (960) is assigned. / 1 1 00,540/ 1 1 00 = (0.87 > 0.49), so that the image center (Ud, vd ) = ( 960, 540 ) is the origin (xd, yd ) = (0, 〇), !j (Formula 1) [Equation 1] Μ /1100-0.87 <Vd /1100-0.49 Equation 1 Next, apply the parameters (z, Θ, dx, dy, 0 x, 0 y ) of the above shake correction, The sway corrected coordinate (Xs, ys ) is calculated by (Formula 2). Further, in (Formula 2), xs = wxs/w, ys = wys/w.

【式2】 /WX.\ / ZCOS θ wy® = -ζ sin Θ \w / V φχ ζ sin θ ζ cos θΦν dx\ dy <Xrf、 晃動檢測部40也可以將含有該晃動校.正參數的（式2 )的矩陣本身輸出到座標變換部30，來代替將晃動校正參 數（z、dx、dy、Θ、</> y、0 X等）輸出到座標變換部30。 而且’座標變換部30直接應用該矩陣本身而算出晃動校正 後的座標（xs，ys )即可。 -19- 1353779 進一步，對晃動校正後的座標應用被保存在像差係數 表38中的歪曲像差校正的係數h、k2、Pl、p2，用運算式 Χ5〇 = χ' ( 1 +k,r,2 + k2r,4 ) + 2 p , x ,y, + P 2 ( r ’2 + 2 x ’2 ) 、ysG = y，( l+kyd + kz^4) +2p2x 汐' + Pi(f2 + 2yd)計算被保存在緩衝 存儲器21中的Gr平面和Gb平面中的採樣座標xsG、ysG。此 時，設 r ’2 ξ X ’2 + y ’2。 h、k2、Pl、p2是表示攝影鏡頭3的歪曲像差的係數， ki、k2表示放射線方向的歪斜、pi、p2表示切線方向的歪 斜。 進一步，應用被保存在像差係數表38中的顔色像差係 數kR、kB、dRx、dRy、dBx、dBy，考慮攝影鏡頭3的顔色像 差而通過（式3)、（式4)計算R平面、B平面中的採樣座 標（XSR，ySR ) ( XSB，ySB )。[Formula 2] /WX.\ / ZCOS θ wy® = -ζ sin Θ \w / V φχ ζ sin θ ζ cos θΦν dx\ dy < Xrf, the sway detecting unit 40 may also contain the swaying correction positive parameter The matrix of (Expression 2) itself is output to the coordinate conversion unit 30 instead of outputting the shake correction parameters (z, dx, dy, Θ, </> y, 0 X, etc.) to the coordinate conversion unit 30. Further, the coordinate conversion unit 30 directly applies the matrix itself to calculate the coordinates (xs, ys) after the shake correction. -19- 1353779 Further, the coordinates after the shake correction are applied to the coefficients h, k2, P1, and p2 of the distortion correction in the aberration coefficient table 38, and the expression Χ5〇= χ' (1 + k, r,2 + k2r,4 ) + 2 p , x ,y, + P 2 ( r '2 + 2 x '2 ) , ysG = y, ( l+kyd + kz^4) +2p2x 汐' + Pi( F2 + 2yd) The sampling coordinates xsG, ysG stored in the Gr plane and the Gb plane in the buffer memory 21 are calculated. At this time, r ’2 ξ X ’2 + y '2 is set. h, k2, P1, and p2 are coefficients indicating the distortion of the photographing lens 3, ki and k2 indicate the skew in the radial direction, and pi and p2 indicate the skew in the tangential direction. Further, the color aberration coefficients kR, kB, dRx, dRy, dBx, and dBy stored in the aberration coefficient table 38 are applied, and R is calculated by (Expression 3) and (Expression 4) in consideration of the color aberration of the photographic lens 3. Sampling coordinates (XSR, ySR) in the plane, plane B (XSB, ySB).

【式3】 / \ XeR ( \ kR 0 dRx 〈X»o、 y.R 0 kR dRy y.〇 V ) 〈 ) l 1 J 【式4】 / \ X.B / \ ks 0 dBx yaB 0 kB dBy y,a •i -式3 式4·[Equation 3] / \ XeR ( \ kR 0 dRx <X»o, yR 0 kR dRy y.〇V ) < ) l 1 J [Equation 4] / \ XB / \ ks 0 dBx yaB 0 kB dBy y, a •i - Equation 3 Equation 4·

在此，kR、kB是以Gr平面和Gb平面爲基準時的R、BHere, kR and kB are R and B based on the Gr plane and the Gb plane.

平面的倍率，dRx、dRy是以Gr平面和Gb平面爲基準時的R 平面的平行偏移量，dBx、dBy是以Gr平面和Gb平面爲基準 -20- 1353779 時的B平面的平行偏移量。 然後’如果設顔色馬賽克圖像的尺寸爲2560x1920像 素’對應的顔色馬賽克圖像的像素位置（us，vs)在xy座 標系下的表現爲（式5)、（式6)、（式7)。即，當顔 色馬賽克圖像由2560x 1 920的等間隔的像素排列構成時， 算出 /(25602 +19202)/2= 1 600，賦予偏移（u，v) = (1280,The magnification of the plane, dRx and dRy are the parallel offsets of the R plane based on the Gr plane and the Gb plane. The dBx and dBy are the parallel offsets of the B plane based on the Gr plane and the Gb plane -20- 1353779. the amount. Then, if the size of the color mosaic image is 2560x1920 pixels, the pixel position (us, vs) of the corresponding color mosaic image is expressed by (Expression 5), (Expression 6), (Expression 7) under the xy coordinate system. . That is, when the color mosaic image is composed of equally spaced pixels of 2560x 1 920, /(25602 +19202)/2= 1 600 is calculated, and the offset (u,v) = (1280,

960 )，以使圖像中心（us，vs ) = ( 1 280，960 )成爲原點 (xs ’ ys ) = ( 〇 ’ 0 ) ’得到（式 5 )、（式 6 )、（式 7 ) 【式5】. 式5 ’UsR) — (XsR X 1600 + 128Q、 、VsrJ —kysR X 1600 十 960; ^UsG^ ^XsG X 1600 + kVsg) LysG X1600 +960), so that the image center (us, vs) = (1 280, 960) becomes the origin (xs ' ys ) = ( 〇 ' 0 ) ' to get (Equation 5), (Equation 6), (Equation 7) [Formula 5]. Equation 5 'UsR) — (XsR X 1600 + 128Q, , VsrJ — kysR X 1600 Ten 960; ^UsG^ ^XsG X 1600 + kVsg) LysG X1600 +

1280^ 960 J • •式61280^ 960 J • • Equation 6

【式6】 【式7】 ’XsB X 1600 + 、ys巳 x 1600 + 1280960 · 式7 此外，像素存在於（u，v)的整數格子中，但由（式 5)、（式6)、（式7)算出的採樣座標的値不限於整數 ，是包含小數部分的實數。 接下來，如圖2所示，採樣部3 1由緩衝存儲器2 1存儲 -21 - 1353779 的顔色馬賽克圖像，按R像素組、G r像素組、G b像素組、 B像素組分別輸出位於由座標變換部30算出的採樣座標周 圍的像素的値。 詳細地說，從R像素組輸出位於（UsR，vsR )周圍的像 素的値，從Gr像素組輸出位於（UsG，VsC )周圍的像素的 値’從Gb像素組輸出位於（UsG，VsG )周圍的像素的値， 從B像素組輸出位於（UsB，vsB )周圍的像素的値。[Equation 6] [Equation 7] 'XsB X 1600 + , ys巳x 1600 + 1280960 · Equation 7 In addition, the pixel exists in the integer lattice of (u, v), but by (Equation 5), (Equation 6), (Expression 7) The calculated sampling coordinates are not limited to integers, and are real numbers including fractional parts. Next, as shown in FIG. 2, the sampling unit 31 stores a color mosaic image of -21353779 from the buffer memory 2 1 , and outputs the image in the R pixel group, the Gr pixel group, the G b pixel group, and the B pixel group. The pupil of the pixel around the sampling coordinates calculated by the coordinate conversion unit 30. In detail, the 値 of the pixel located around (UsR, vsR) is output from the R pixel group, and the 値' output from the Gr pixel group is located around the (UsG, VsG) The 像素 of the pixel, from the B pixel group output 位于 located in the pixel around (UsB, vsB).

而且’如上述那樣，採樣位置（Us，Vs )不限於整數 座標’所以讀入包圍（us，vs )的4個有値像素的値。 如圖2所示’ R平面、Gr平面' Gb平面、B平面中的每 個平面都呈縱橫的格點狀具有4個有値像素，因此，包圍 採樣座標401、402、403、404的4個有値像素，如果該採 樣座標usR爲（1〇〇_8，101.4)，則包圍該採樣座標的4個 像素（u，v)爲（100， 1〇〇) 、 （100， 102) 、 （102, 100) 、（102’ 102) ’從緩衝存儲器21讀取這4個像素的Further, as described above, the sampling position (Us, Vs) is not limited to the integer coordinate, so the 値 of the four defective pixels surrounding (us, vs) is read. As shown in FIG. 2, each of the 'R plane, the Gr plane', the Gb plane, and the B plane has four 値 pixels in a vertical and horizontal lattice shape, and therefore, 4 surrounding the sampling coordinates 401, 402, 403, and 404 There are 値 pixels. If the sampling coordinate usR is (1〇〇_8, 101.4), the four pixels (u, v) surrounding the sampling coordinates are (100, 1〇〇), (100, 102), (102, 100), (102' 102) 'Reading these 4 pixels from the buffer memory 21

値。 接下來，插値部32如圖2(f)所示，求出隔著採樣座 標相對的有値像素間的距離的比（在此，x方向是〇 . 4 : 〇 . 6 ’ y方向是0.7: 0.3) ’使用4個有値像素的像素値，通過 插値計算採樣位置（1 0 0 · 8，1 0 1 · 4 )中的R的像素値。 例如，對R平面’將4個有値像素的像素値用r( 10〇， 100) 、R(100’102) 、R(102，100) 、R(l〇2，102 )表不’將採樣位置（100.8，101.4)的R的像素値用R( 100.8，101.4)表示，則通過 R( 100.8，1〇1_4) =0.6* 0.3 -22- 1353779 1value. Next, as shown in FIG. 2(f), the insertion unit 32 obtains a ratio of the distance between the pixels facing each other across the sampling coordinates (here, the x direction is 〇. 4 : 〇. 6 ' y direction is 0.7: 0.3) 'Using 4 pixels with 値 pixels, the pixel R of R in the sampling position (1 0 0 · 8, 1 0 1 · 4) is calculated by interpolation. For example, for the R plane, the pixels of four pixels with 値 pixels are represented by r(10〇, 100), R(100'102), R(102,100), R(l〇2,102). The pixel of R at the sampling position (100.8, 101.4) is represented by R (100.8, 101.4), and then passes R (100.8, 1〇1_4) = 0.6* 0.3 -22- 1353779 1

氺 R(100， 100) +0.6 氺 0-7 氺 R( 100’ 102 ) +0.4 氺 0.3 氺 R t (102，100) +0.4* 0·7* R ( 102 ’ 102)，算出 R 平面上 的採樣座標401的像素値（100.8 ’ 101.4)，作爲R的採樣 値 Rsamp|e(UsR ， VSR)。 並且，由Gr像素組、Gb像素組、B像素組也同樣通過 插値算出採樣座標中的像素値，分別爲GrsamPle(UsG，VsG) 、Gbsampie(UsG，VSG) ' Bsample(usB ’ VSB) 0氺R(100, 100) +0.6 氺0-7 氺R( 100' 102 ) +0.4 氺0.3 氺R t (102,100) +0.4* 0·7* R ( 102 ' 102), calculated on the R plane The pixel 値 of the sampling coordinate 401 (100.8 ' 101.4) is used as the sampling RRsamp|e(UsR , VSR) of R. Furthermore, the pixel 中 in the sampling coordinates is also calculated by the interpolation of the Gr pixel group, the Gb pixel group, and the B pixel group, respectively, GrsamPle (UsG, VsG), Gbsampie (UsG, VSG) 'Bsample (usB ’ VSB) 0

接下來，顔色生成部33根據由插値部32獲得的各顔色 的採樣値 Rsample(UsR ， v s R) Grsamp|e(UsG ， V s 〇)Next, the color generating portion 33 is based on the samples 各 Rsample(UsR , v s R) Grsamp|e(UsG , V s 〇) of the respective colors obtained by the transplanting portion 32.

Gbsamp|e(UsG，VsG)、Bsample(UsB，vsB)，生成各像素的顔色 資 gJl ( R G B 成分）R(ud，vd)、G(ud· vd)、B(ucj. vd)。 此時’卓純地使 R(ud· vd}_Rsample(UsR，Vsr)、B(u(j. vd) = Bsample(UsB ’ V s b ) 、 G(ud v d ) = ( G Γ s a m p 1 e ( U s G ’ V s 〇 ) +Gbsamp|e (UsG, VsG) and Bsample (UsB, vsB) generate the color of each pixel gJl (R G B component) R (ud, vd), G (ud · vd), B (ucj. vd). At this point, 'purely make R(ud·vd}_Rsample(UsR, Vsr), B(u(j. vd) = Bsample(UsB ' V sb ) , G(ud vd ) = ( G Γ samp 1 e ( U s G ' V s 〇) +

Gbsample(usG，vsG))/2。 另外，顔色生成部3 3還進行僞色抑制。作爲僞色抑制 的一個例子，在由攝影元件5構成的拜爾排列的彩色圖像 感測器中，在尼奎斯特頻率附近的高頻處易於發生紅和藍 的僞色，因此，通過取Gr和Gb的差値能夠檢測這些斑紋， 能夠抑制僞色。 δ羊細地說’首先’ g十算局頻成分1&lt;^ = 〇1&gt;531„1)|6(115〇，'^〇)- G b s a m p 】e ( u s G，v s G )。 接下來，設 SumRB = RsampU(uSR，vsR) + Bsampie(usB， VsB)、DiffRB = Rsamp|e(USR ’ VsR)-Bsamp|e(USB，VSB) ’ 使用運 算式 DiffRB, = sign(DiffRB)Max(0，abs(DiffRB)-abs(K))， -23- 1353779Gbsample(usG, vsG))/2. Further, the color generation unit 33 also performs pseudo color suppression. As an example of the pseudo color suppression, in the Bayer array color image sensor constituted by the photographic element 5, red and blue pseudo colors are liable to occur at high frequencies near the Nyquist frequency, and therefore, The difference between Gr and Gb can detect these markings and suppress false colors. δ sheep said that 'first' g's calculation of the frequency component 1&lt;^ = 〇1&gt;531„1)|6(115〇,'^〇)- G bsamp 】e ( us G,vs G ). Let SumRB = RsampU(uSR, vsR) + Bsampie(usB, VsB), DiffRB = Rsamp|e(USR 'VsR)-Bsamp|e(USB,VSB) ' Use the expression DiffRB, = sign(DiffRB)Max( 0,abs(DiffRB)-abs(K)), -23- 1353779

F 從 abs(DiffRB)不跨過 0地減去 abs(K)，計算 DiffRB'。Sign 是使符號爲+1/0/-1的運算符，abs是求絕對値的運算符。 接下來，使用運算式 R(ud，vd) = (SumRB + DiffRB')/2、 G(Ud ， vd) = (Grsampie(usG ， vsG) + Gbsampie(usG ’ vsG))/2 、 B(ud，vd) = (SumRB-DiffRB J/2，再次生成彩色圖像中的各 像素的顔色成分。由此，能夠進行抑制了在高頻部發生的 紅和藍的僞色的顔色生成。F subtracts abs(K) from abs (DiffRB) without crossing 0, and calculates DiffRB'. Sign is an operator that makes the symbol +1/0/-1, and abs is an operator that finds absolute 値. Next, use the expression R(ud,vd) = (SumRB + DiffRB')/2, G(Ud , vd) = (Grsampie(usG , vsG) + Gbsampie(usG ' vsG))/2 , B(ud (vd) = (SumRB-DiffRB J/2, the color component of each pixel in the color image is generated again. Thereby, color generation of the false color of red and blue generated in the high-frequency portion can be suppressed.

接下來，由視覺校正部34進行色調曲線（伽瑪）校正 、色度增強、邊緣增強這樣的圖像校正，然後，由壓縮部 3 5將彩色圖像的數位圖像信號用JPEG ( Joint Photographic Experts Group)等的方法進行壓縮，由記錄部36將壓縮後 的數位圖像信號存儲到記錄介質中。Next, image correction such as tone curve (gamma) correction, chrominance enhancement, and edge enhancement is performed by the vision correcting section 34, and then the digital image signal of the color image is used by the compression section 35 as a JPEG (Joint Photographic The method such as Experts Group) compresses, and the recorded digital image signal is stored in the recording medium by the recording unit 36.

接下來，根據圖4，說明由經由攝影單元2輸入的顔色 馬賽克圖像（輸入圖像）進行晃動校正和歪曲像差等的圖 像變形並生成彩色圖像（輸出圖像）時的順序。該順序由 CPU18根據被保存在R0M19中的程式對各功能部賦予指令 信號來執行。另外，圖4中的S表示步驟。另外，本發明的 反馬賽克步驟由S1 80〜S2 10實現其功能，本發明的圖像變 形由S1 80實現其功能。 首先，該順序在由操作者對圖像處理裝置100輸入了 啓動信號時開始。 接著，在S110中，經由攝影單元2取得顔色馬賽克圖 像，然後移至S120。 接著，在S120中，將顔色馬賽克圖像讀入緩衝存儲器 2 i，與拜爾排列相對應，按R的像素信號、Gr的像素信號 -24- 1353779 、Gb的像素信號、B的像素信號分別存儲，然後，移至 S190。又，本發明之顏色平面分解步驟及顏色平面記憶步 驟，由S120實現其功能。 另一方面’在S130中，使用輸出像素掃描部29，掃描 輸出圖像（彩色圖像）而順序取得處理物件像素（Ud， )，然後，移至S 1 8 0。 另外，在S 1 4 0中’使用鏡頭狀態檢測部3 7檢測與焦點 距離和被攝體距離相對應的鏡頭狀態，然後，在S 1 5 0中， 從像差係數表3 8取得與鏡頭狀態相對應地保存的像差係數 和變焦倍率等，然後，移至S180。 另外，在S 1 6 0中，使用角速度感測器】5和晃動檢測部 4〇檢測攝影裝置1 Α的晃動量，然後，移至S170，在sl7〇中 取得用於晃動量校正的參數，然後，移至S180。 接著，在S180中’使用座標變換部30，利用在sl5〇中 取得的像差係數、在Sl7〇中取得的晃動校正參數等，對在 S130中取得的輸出圖像（彩色圖像）的處理物件中的像素 位置’計算施加了歪曲像差和抖動、變焦倍率等的圖像變 形處理的座標（us，vs) ’然後，移至S190。又，本發明 之座標變換步驟，由S180實現其功能。 接著，在S190中，使用採樣部31，從緩衝存儲器21取 得位於由座標變換部30算出的採樣座標周圍的、各r、Gr 、Gb’ B的像素値，然後，移至S200。 接著’在S200中’使用插値部32，按各R、Gr、Gb， B由位於採樣座標周圍的像素的像素値插値生成位於採樣 座標的像素値 RSample(UsR ’ VsR)、Grsampie(Us〇j，、 -25- 1353779Next, a procedure in which image correction such as shake correction, distortion, and the like is performed by a color mosaic image (input image) input through the photographing unit 2 to generate a color image (output image) will be described with reference to Fig. 4 . This sequence is executed by the CPU 18 by giving a command signal to each functional unit based on the program stored in the ROM 19. In addition, S in Fig. 4 indicates a step. Further, the anti-mosaic step of the present invention realizes its function by S1 80 to S2 10, and the image modification of the present invention realizes its function by S1 80. First, the sequence starts when an operator inputs a start signal to the image processing apparatus 100. Next, in S110, the color mosaic image is acquired via the photographing unit 2, and then the process moves to S120. Next, in S120, the color mosaic image is read into the buffer memory 2 i, corresponding to the Bayer arrangement, and the pixel signal of R, the pixel signal of Gr -24- 1353779, the pixel signal of Gb, and the pixel signal of B are respectively Store, then move to S190. Further, the color plane decomposition step and the color plane memory step of the present invention realize their functions by S120. On the other hand, in S130, the output pixel scanning unit 29 scans the output image (color image) to sequentially acquire the processed object pixel (Ud, ), and then shifts to S 1 800. Further, in S 1 40, the lens state detecting unit 37 detects the lens state corresponding to the focal length and the subject distance, and then, in S 1 50, obtains the lens from the aberration coefficient table 38. The state corresponds to the aberration coefficient and the zoom magnification, and the like, and then moves to S180. Further, in S 1 60, the angular velocity sensor 5 and the sway detecting unit 4 〇 are used to detect the amount of shaking of the photographing device 1 ,, and then, the process proceeds to S170, and the parameter for the shake amount correction is obtained in sl7〇. Then, move to S180. Next, in S180, 'the coordinate conversion unit 30 is used to process the output image (color image) acquired in S130 using the aberration coefficient acquired in sl5〇, the shake correction parameter acquired in Sl7〇, and the like. The pixel position in the object 'calculates the coordinates (us, vs) of the image warping process to which the distortion and the shake, the zoom magnification, and the like are applied, and then moves to S190. Further, the coordinate conversion step of the present invention is realized by S180. Next, in S190, the sampling unit 31 is used to obtain the pixel 各 of each of r, Gr, and Gb' B located around the sampling coordinates calculated by the coordinate conversion unit 30 from the buffer memory 21, and then proceeds to S200. Then, 'in S200', using the interpolation unit 32, each pixel R, Gr, Gb, B is generated by the pixel interpolation of the pixel located around the sampling coordinates, and the pixel 値RSample (UsR 'VsR) and Grsampie (UsR) located at the sampling coordinates are generated. 〇j,, -25- 1353779

Gbsampie(usG’ vsG)、Bsample(usB，vsB)，然後，移至 S2l〇。 又，本發明之插値步驟，由S200實現其功能。 接著’在S210中，使用顔色生成部33合成由採樣部31 算出的各顔色的採樣値，由此，按各處理物件像素生成多 個顔色的顔色資訊R(ud，Vd)、G(Ud，Vd)、B(Ud，vd)，然 後’移至S220。又，本發明之顏色生成步驟，由S210實現 其功能。Gbsampie (usG' vsG), Bsample (usB, vsB), then move to S2l〇. Further, the plugging step of the present invention realizes its function by S200. Next, in S210, the color generation unit 33 synthesizes the sample 各 of each color calculated by the sampling unit 31, thereby generating color information R(ud, Vd), G(Ud, for a plurality of colors for each processed object pixel. Vd), B(Ud, vd), then 'shift to S220. Further, in the color generating step of the present invention, its function is realized by S210.

接著’在S2 20中，判斷輸出圖像內有無下一掃描像素 ，在判斷爲沒有像素（No)時移至S230，在判斷爲有像素 (Yes)時重複S180〜S220，在S220中成爲沒有像素（No )時移至S 2 3 0。 接著，在S230中，使用視覺校正部34，對由顔色生成 部33生成的彩色圖像進行色調曲線（伽瑪）校正、色度增 強、邊緣增強這樣的圖像校正，然後，移至S 2 4 0。Next, in S2 20, it is determined whether or not there is a next scan pixel in the output image. When it is determined that there is no pixel (No), the process proceeds to S230. When it is determined that there is a pixel (Yes), S180 to S220 are repeated, and in S220, there is no The pixel (No) moves to S 2 3 0. Next, in S230, the color correction unit 34 performs image correction such as tone curve (gamma) correction, chromaticity enhancement, and edge enhancement on the color image generated by the color generation unit 33, and then moves to S 2 . 4 0.

接著’在S240中，使用壓縮部35，將經由視覺校正部 34輸出的彩色圖像的數位圖像信號用jPEG ( Joint Photogfaphic Experts Group)等方法進行壓縮，縮小記錄 時的圖像資料的尺寸，然後，移至S25〇。 接著’在S250中，使用記錄部36，將壓縮後的數位圖 像信號存儲在閃速存儲器等記錄介質中，然後，結束本圖 像處理程式。 如上所述’第一實施方式記載的圖像處理裝置100和 圖像處理方法通過用顔色馬賽克圖像進行在進行圖像變形 時所需的圖像資料的緩衝，能夠將緩衝存儲器21的容量抑 制在每1個像素1個變數，能夠節省緩衝存儲器2 1的存儲容 量和頻帶，以低成本和低功耗實現具有彩色馬賽克和圖像 -26- 1353779 變形的攝影裝置1。 (第二實施方式） 接下來’使用圖5〜圖8說明本發明的第二實施方式。 圖5是表示應用了本發明的圖像處理裝置和圖像處理方法 的第二實施方式的攝影裝置1A的結構的框圖，圖6是表示 該第二實施方式的圖像處理裝置和圖像處理方法中的彩色 圖像生成的順序的流程圖，圖7是表示圖6中的壓縮和高速 緩衝控制器的順序的細節的流程圖。 第二實施方式中的攝影裝置1 A基本上具有與第一實施 方式所示的攝影裝置1相同的結構，所以對其相同的結構 部分標記同一標號並省略詳細的說明，以下說明成爲特徵 的部分。 攝影裝置1A中的圖像處理裝置100A包括：壓縮從攝 影單元2輸出的反馬賽克圖像的壓縮部20;讀入被壓縮並 保存在緩衝存儲器21中的反馬賽克圖像進行拉伸的拉伸部 25;按照來自反馬賽克部28的請求從高速緩衝存儲器27或 拉伸部2 5將像素資料輸出到採樣部3 1的高速緩衝 控制器26 ;以及存儲由高速緩衝控制器26取得的最新的像 素組的高速緩衝存儲器27等。 首先，將拍攝到的顔色馬賽克圖像從攝影元件5以光 柵順序作爲與各像素的曝光量相應的類比信號被輸出到 A/D轉換器9。 接下來，A/D轉換器9將該類比信號轉換爲12bit的數 -27- 1353779 位信號（以下稱爲像素値），將該像素値輸出到壓縮部20。 接下來，壓縮部20將從A/D轉換器9順序輸入的像素値 組按各顔色像素以水平8個像素的塊單位蓄積，將被蓄積 的塊資料從12x8 = 96bit壓縮到64bit，並輸出到緩衝存儲器 2!。另外，本發明中的顔色平面分解部借助於壓縮部2 0實 現其功能。又，本發明之壓縮部之功能係由壓縮部20來實 現，本發明之拉伸部之功能係由拉伸部2 5來實現。Then, in S240, the digital image signal of the color image output via the visual correction unit 34 is compressed by a method such as jPEG (Joint Photogapical Experts Group) using the compression unit 35, and the size of the image data at the time of recording is reduced. Then, move to S25〇. Then, in S250, the compressed digital image signal is stored in a recording medium such as a flash memory using the recording unit 36, and then the image processing program is terminated. As described above, the image processing device 100 and the image processing method according to the first embodiment can suppress the capacity of the buffer memory 21 by buffering the image data required for image deformation with the color mosaic image. With one variable per pixel, the storage capacity and frequency band of the buffer memory 21 can be saved, and the photographing apparatus 1 having the color mosaic and the image -26-1353779 can be realized with low cost and low power consumption. (Second Embodiment) Next, a second embodiment of the present invention will be described using Figs. 5 to 8 . 5 is a block diagram showing a configuration of a photographing apparatus 1A of a second embodiment to which an image processing apparatus and an image processing method according to the present invention are applied, and FIG. 6 is a view showing an image processing apparatus and an image of the second embodiment. A flowchart of the sequence of color image generation in the processing method, and FIG. 7 is a flowchart showing the details of the sequence of the compression and cache controller in FIG. The imaging device 1A of the second embodiment has substantially the same configuration as that of the imaging device 1 of the first embodiment. Therefore, the same components are denoted by the same reference numerals, and detailed description is omitted. . The image processing apparatus 100A in the photographing apparatus 1A includes a compression section 20 that compresses the anti-mosaic image output from the photographing unit 2, and stretches the stretched image that is compressed and stored in the buffer memory 21 for stretching. a portion 25 that outputs pixel data from the cache memory 27 or the stretching unit 25 to the cache controller 26 of the sampling unit 31 in response to a request from the anti-mosaic unit 28; and stores the latest latest information obtained by the cache controller 26. A cache memory 27 of a pixel group or the like. First, the captured color mosaic image is output from the photographing element 5 in the raster order as an analog signal corresponding to the exposure amount of each pixel to the A/D converter 9. Next, the A/D converter 9 converts the analog signal into a 12-bit number -27 - 1353779 bit signal (hereinafter referred to as a pixel 値), and outputs the pixel 値 to the compression unit 20. Next, the compression unit 20 accumulates the pixel groups sequentially input from the A/D converter 9 in block units of horizontal pixels for each color pixel, and compresses the accumulated block data from 12x8 = 96 bits to 64 bits, and outputs it. To buffer memory 2!. Further, the color plane decomposition portion in the present invention realizes its function by means of the compression portion 20. Further, the function of the compression portion of the present invention is realized by the compression portion 20, and the function of the stretching portion of the present invention is realized by the stretching portion 25.

另一方面，反馬賽克部28以光柵順序生成彩色圖像。 此時，因爲進行圖像變形，所以經由高速緩衝控制器2 6對 顔色馬賽克圖像請求非光柵順序的採樣。 接下來，高速緩衝控制器2 6接受該採樣請求，如果請 求的像素的像素値已保存在高速緩衝存儲器27中，則不訪 問緩衝存儲器21而從高速緩衝存儲器27讀入該像素値，並 輸出到採樣部3 1。On the other hand, the anti-mosaic unit 28 generates a color image in raster order. At this time, since the image is deformed, the color mosaic image is requested to be sampled in a non-raster order via the cache controller 26. Next, the cache controller 26 accepts the sampling request, and if the pixel 请求 of the requested pixel has been stored in the cache memory 27, the buffer memory 21 is not accessed, the pixel 値 is read from the cache memory 27, and is output. Go to the sampling unit 31.

另外，如果請求的像素的像素値沒有保存在高速緩衝 存儲器27中，則高速緩衝控制器26對緩衝存儲器2 1訪問含 有該像素的塊資料。 接下來，緩衝存儲器2 1將由高速緩衝控制器2 6訪問 的塊資料輸出到拉伸部2 5。 接下來，拉伸部25拉伸從緩衝存儲器21輸入的塊資料 ’再現8個12bit的像素値。 接下來，高速緩衝控制器26從拉伸部25接收被再現的 8個12bit的像素値並輸出到反馬賽克部28的採樣部31，同 時’將該8個12bit的像素値保存到高速緩衝存儲器27中。 -28- 1353779 控 緩 部 緩 本 外 像 經 料 座 32 的 s B ) 個 入 差 該 賦 所 明 此外’如果高速緩衝存儲器2 7的空間已滿，則高速緩衝 制器26用LRU ( Least-Recently-Used )法等已知的高速 衝控制法進行更新。 高速緩衝存儲器27與高速緩衝控制器26和反馬賽克 28在同一半導體積體電路上作爲SRam被集成，作爲與 衝存儲器21相比小容量、高速的存儲器而安裝。另外， 實施例的高速緩衝存儲器2 7作爲讀高速緩衝器動作。另 ，高速緩衝存儲器27的行尺寸構成爲能夠保存1個塊的 素値組。 接下來，反馬賽克部28與第一實施方式相同，使用 由高速緩衝控制器26輸入的顔色馬賽克圖像的各像素資 ，在採樣部31中，取得位於由座標變換部30算出的採樣 標周圍的各R、Gr、Gb，B的像素値，然後，在插値部 中，按各R、Gr、Gb，B，由位於採樣座標周圍的像素 像素値插値生成位於採樣座標的像素値Rsample(usR，vsR)Further, if the pixel 値 of the requested pixel is not stored in the cache memory 27, the cache controller 26 accesses the buffer memory 2 1 to the block material containing the pixel. Next, the buffer memory 21 outputs the block material accessed by the cache controller 26 to the stretching portion 25. Next, the stretching unit 25 stretches the block data ’ input from the buffer memory 21 to reproduce eight 12-bit pixels 値. Next, the cache controller 26 receives the reproduced 12 12-bit pixels from the stretching unit 25 and outputs them to the sampling unit 31 of the inverse mosaic unit 28, while saving the eight 12-bit pixels to the cache memory. 27 in. -28- 1353779 The s B) of the control unit is passed through the s B) difference. In addition, if the space of the cache memory 27 is full, the cache unit 26 uses the LRU (Least- A known high-speed rush control method such as the Recently-Used method is updated. The cache memory 27, the cache controller 26, and the inverse mosaic 28 are integrated as SRam on the same semiconductor integrated circuit, and are mounted as a small-capacity, high-speed memory as compared with the flash memory 21. Further, the cache memory 27 of the embodiment operates as a read cache. Further, the line size of the cache memory 27 is configured to be a prime group capable of storing one block. Next, in the same manner as in the first embodiment, the inverse mosaic unit 28 uses the pixels of the color mosaic image input from the cache controller 26, and the sampling unit 31 acquires the sample marks located by the coordinate conversion unit 30. The pixels of each of R, Gr, Gb, and B are 値, and then, in the interpolated portion, for each of R, Gr, Gb, and B, pixel pixels located around the sampling coordinates are used to generate a pixel 位于Rsample located at the sampling coordinates. (usR, vsR)

Grsarnple(UsG，Vs(j)、Gbample(UsG ’ Vsg)、Bsampie(UsB ’ V ，然後，在顔色生成部33中，按各處理物件像素生成多 顔色的顔色資訊 R(ud，vd)、G(ud ’ vd)、B(ud，vd)。 接下來，根據圖6、圖7，說明由經由攝影單元2輸 的顔色馬賽克圖像（輸入圖像）進行晃動校正和歪曲像 等的圖像變形並生成彩色圖像（輸出圖像）時的順序。 順序由CPU 18根據保存在ROM 19中的程式對各功能部 予指令信號來執行。另外’在本流程圖中對與第一方式 示的流程圖相同的步驟賦予同一編號’並省略其詳細說 -29- 1353779 。本發明中的壓縮步驟由S300實現其功能，本發明的拉伸 步驟由S500實現其功能。 首先，該順序在由操作者對圖像處理裝置100A輸入了 啓動信號時開始。 接著’在S110中，經由攝影單元2取得顔色馬賽克圖 像’然後移至S300的壓縮步驟。Grsarnple (UsG, Vs(j), Gbample (UsG 'Vsg), Bsampie (UsB ' V , then, in the color generation unit 33, multi-color color information R(ud, vd), G is generated for each processed object pixel. (ud ' vd), B (ud, vd) Next, an image of the shake correction, the distortion image, and the like by the color mosaic image (input image) transmitted via the photographing unit 2 will be described with reference to FIGS. 6 and 7 . The order in which the color image (output image) is deformed and generated. The sequence is executed by the CPU 18 by giving a command signal to each function unit based on the program stored in the ROM 19. In addition, in the present flowchart, the first mode is shown. The same steps of the flowchart are given the same number 'and the detailed description thereof is omitted -29-353779. The compression step in the present invention realizes its function by S300, and the stretching step of the present invention realizes its function by S500. First, the order is The operator starts when the activation signal is input to the image processing apparatus 100A. Next, 'in S110, the color mosaic image is acquired via the photographing unit 2' and then moves to the compression step of S300.

接著’在S300中，如圖7(a)所示，首先，在S310的 塊分割步驟中將顔色馬賽克圖像資料按各塊分割，然後， 移至S320。此時’按拜爾排列的r、Gr、Gb、B的各像素 ’以垂直1個像素X水平8個像素爲塊單位，將上述8個12bit 像素値（x(n) ’ n=l…8 )作爲像素値組，進行接下來的 S 3 20〜S 3 5 0的處理。另外，本發明中的顔色平面分解步驟 由S3 10實現其功能。 接著’在S320中，求出像素値組內的x(n)的最小値 Min和範圍R(=最大値_最小値），然後，移至S330。Next, in S300, as shown in Fig. 7(a), first, the color mosaic image data is divided into blocks in the block dividing step of S310, and then the process proceeds to S320. At this time, 'each pixel of r, Gr, Gb, and B arranged by Bayer' is 8 blocks of vertical pixels 1 pixel horizontally, and the above 12 12-bit pixels 値(x(n) 'n=l... 8) As the pixel group, the next processing of S 3 20 to S 3 50 is performed. Further, the color plane decomposition step in the present invention realizes its function by S3 10. Next, in S320, the minimum 値 Min and the range R (=maximum 値_min 値) of x(n) in the pixel group are obtained, and then the process proceeds to S330.

接著，在S330中，使用在S320中求出的範圍R，用運 算式£ = 1以2(11)&lt;12 8/ 127)-6計算共用指數£，然後，移至 S 3 40。此時，如果e是非整數則將E進位元，如果e是負數 ’則使E爲0而作爲非負整數。 接著’在S 340中，使用運算式M(n)= ( x(n)-Min) /2e 計算尾數M(n)。此時，如果尾數M(n)是非整數，則將尾數 M(n)四捨五入成整數，然後，移至S350。 由此，Min可取I2bit、E可取〇〜7的整數値，所以可 用3bit表示’ Μ(η)可取〇〜63的整數値，所以可用6bit表示 -30- 1353779 。此外’在S330的求E的運算式中，由於R與（128/127) 相乘而涉及S3 40的尾數算出步驟中的四捨五入引起的溢出 。排列這些資料則爲63bit( 12 + 3 + 6x8 = 63)。 接著’在S350中，彙集這些資料63bit( 12 + 3 + 6x8 = 63 )而將8個像素組作爲壓縮資料，然後，移至圖6的S400。 由S310〜S35 0實現本發明的灰度壓縮。Next, in S330, using the range R obtained in S320, the common index £ is calculated by 2(11) &lt;12 8/127)-6 using the operation formula £ = 1, and then moved to S 3 40. At this time, E is a carry if e is a non-integer and E is 0 as a non-negative integer if e is a negative number. Next, in S 340, the mantissa M(n) is calculated using the expression M(n) = ( x(n) - Min) /2e. At this time, if the mantissa M(n) is a non-integer, the mantissa M(n) is rounded to an integer, and then moved to S350. Therefore, Min can take I2bit and E can take an integer 値 of 77, so 3 bits can be used to represent ' Μ(η) can take an integer 値 of 6363, so 6bit can be used to represent -30- 1353779. Further, in the calculation formula of E in S330, since R is multiplied by (128/127), it involves the overflow caused by the rounding in the mantissa calculation step of S3 40. The arrangement of these data is 63bit (12 + 3 + 6x8 = 63). Next, in S350, these pieces of data 63 bits (12 + 3 + 6x8 = 63) are collected and 8 pixel groups are used as compressed data, and then moved to S400 of Fig. 6 . The gradation compression of the present invention is implemented by S310 to S35 0.

接著’在S400中，將在S35 0中被壓縮的資料保存在緩 衝存儲器21中，然後，移至S500。在作爲緩衝存儲器21使 用通用DRAM的情況下，其大多數將8χ2π bit作爲訪問單位 ，所以追加Ibit，用64bit邊界（boundary)進行保存是有 效率的。 接著，在S 500中，使用拉伸部25從緩衝存儲器21讀入 63bit的壓縮資料，將該壓縮資料拉伸爲8個12bit像素値 x'(n)，然後，移至S600。此時，在S5〇0中，從緩衝存儲器 21讀入包含所希望的像素資料的63 (或64 ) bit的封裝（ encapsulate)後的資料，將12bit分解爲Min、將3bit分解 爲E、將8個6b it資料分解爲M(n)，由這些分解後的資料， 使x'(n) = M(n)x2E + Min，再現像素値組。 另外，在S350的封裝步驟中，也可以具有可逆壓縮部 的功能。在壓縮部20中，對63bit的資料嘗試可逆壓縮，如 果不能壓縮成63bit以下’則將可逆壓縮使用標記lbit設置 成false賦予，作爲64bit的資料保存在緩衝存儲器21中。 另外，在能壓縮成63 bit以下的情況下，將可逆壓縮使 用標記Ibit設置成true賦予’然後’將被壓縮的資料繼續 -31 - 1353779 保存在緩衝存儲器21中。由此，能夠削減緩衝存儲器21的 改寫量’能夠進一步實現節省功耗。另外，作爲可逆壓縮 可以使用行程長度壓縮和熵編碼。Next, in S400, the data compressed in S35 0 is stored in the buffer memory 21, and then moved to S500. When a general-purpose DRAM is used as the buffer memory 21, most of them use 8 χ 2 π bits as access units. Therefore, it is efficient to add Ibit and store them with a 64-bit boundary. Next, in S500, 63 bits of compressed data is read from the buffer memory 21 using the stretching unit 25, and the compressed data is stretched into eight 12-bit pixels 値 x'(n), and then moved to S600. At this time, in S5〇0, 63 (or 64) bits of the block containing the desired pixel data are read from the buffer memory 21, and 12 bits are decomposed into Min, 3 bits are decomposed into E, and The eight 6b it data are decomposed into M(n), and from these decomposed data, x'(n) = M(n)x2E + Min is reproduced, and the pixel group is reproduced. Further, in the packaging step of S350, it is also possible to have a function of a reversible compression unit. In the compression unit 20, 63-bit data is attempted to be reversibly compressed, and if it cannot be compressed to 63 bits or less, the reversible compression use flag lbit is set to false, and 64-bit data is stored in the buffer memory 21. Further, in the case where it can be compressed to 63 bits or less, the reversible compression use flag Ibit is set to true to give 'and then' the compressed data continues -31 - 1353779 to be stored in the buffer memory 21. Thereby, the amount of rewriting of the buffer memory 21 can be reduced, and power consumption can be further realized. In addition, run length compression and entropy coding can be used as the reversible compression.

另一方面’在S500的拉伸步驟中，從緩衝存儲器21讀 入6 4bit的壓縮資料，首先確認可逆壓縮使用標記，如果可 逆壓縮使用標記是false，則由接下來的63bit輸出用1 2bit 表示的8個像素値χ'(η)。另外，如果可逆壓縮使用標記是 true ’則將接下來的資料可逆拉伸之後輸出用i2bit表示的 8個像素値X 1 η )。 接著’ S600的高速緩衝控制器步驟如圖7 ( b )所示， 按照來自反馬賽克部2 8的訪問請求，在S 6 1 0中，判斷由反 馬賽克部28請求的像素資料是否存在於高速緩衝存儲器27 中，在判斷爲像素資料存在（yes )時移至S620，在判斷 爲像素資料不存在（No)時移至S611。On the other hand, in the stretching step of S500, the 64-bit compressed data is read from the buffer memory 21, and the reversible compression use flag is first confirmed. If the reversible compression use flag is false, the next 63-bit output is represented by 1 2 bits. The 8 pixels 値χ '(η). In addition, if the reversible compression use flag is true ', the next data is reversibly stretched and the 8 pixels 値X 1 η ) represented by i2bit are output. Then, as shown in FIG. 7(b), the cache controller step of the S600 determines whether or not the pixel data requested by the inverse mosaic unit 28 exists at the high speed in S6 1 0 in accordance with the access request from the anti-mosaic unit 28. In the buffer memory 27, when it is determined that the pixel data exists (yes), the process proceeds to S620, and when it is determined that the pixel data does not exist (No), the process proceeds to S611.

接著’在S611中，讀入包含請求像素的壓縮資料，然 後，移至S 6 1 2。 接著’在S6 12中，使用拉伸部25，拉伸壓縮資料而取 得包含所請求的像素値的像素値組，然後，在S 6 1 3中，將 被拉伸的像素値組保存在高速緩衝存儲器2 7中，同時，移 至 S 6 3 0 〇 另一方面，在S610中像素資料存在（yes)時，在 S620中’從高速緩衝存儲器27讀出該像素値，然後，移至 S63 0 〇 接著，在S630中，將由反馬賽克部28請求的像素値輸 -32- 1353779 出到反馬賽克部28，然後，移至圖6的SI 90。 接著，與第一實施方式相同，在S180〜S210中進行圖 像變形和反馬賽克處理，然後，在進行了 S230的視覺校正 步驟、S2 40的壓縮步驟之後，結束本處理。Next, in S611, the compressed material containing the request pixel is read, and then moved to S 6 1 2 . Next, in S6 12, the stretched portion 25 is used to stretch the compressed data to obtain a pixel group including the requested pixel ,, and then, in S 6 1 3, the stretched pixel group is stored at a high speed. In the buffer memory 27, at the same time, it moves to S 6 3 0. On the other hand, when the pixel data exists in S610, the pixel is read from the cache memory 27 in S620, and then moved to S63. Then, in S630, the pixel-received -32-1353779 requested by the anti-mosaic portion 28 is output to the anti-mosaic portion 28, and then moved to the SI 90 of Fig. 6. Next, similarly to the first embodiment, image distortion and demosaic processing are performed in S180 to S210, and then the processing is terminated after the visual correction step of S230 and the compression step of S2 40.

如以上那樣，第二實施方式記載的圖像處理裝置100A 和圖像處理方法通過使用壓縮部20來壓縮緩衝存儲器21中 保存的顔色馬賽克圖像資料，能夠削減所需的緩衝存儲器 21的存儲容量，並且，對bit深度不是8的倍數的顔色馬賽 克圖像也能通過適當地設定壓縮率而高效地利用通用的8 X 2n寬度的DRAM。 另外，在反馬賽克部28中，爲了生成彩色圖像的1個 像素而參照顔色馬賽克圖像中的多個像素的像素値，而在 生成彩色圖像的相鄰的像素的情況下，多參照（訪問）顔 色馬賽克圖像中的同一像素。與此相應，第二實施方式記 載的圖像處理裝置100A和圖像處理方法引入了高速緩衝存 儲器27，所以能夠削減緩衝存儲器2 1的讀出頻帶，能進一 步削減功耗。 (變形例） 以上，說明了本發明的一個實施例，但本發明並不限 於上述實施例，能夠取各種方式。 例如，爲了更有效地利用商速緩衝存儲器，也可以構 成輸出像素掃描部2 9中的掃描順序。即，在掃描彩色圖像 的全部像素時，進行區域性強的掃描。作爲這樣的掃描的 -33- 1353779 例子’可列舉出希爾伯特曲線的活用。例如，可以如圖8 所示’將輸出彩色圖像分割爲8x8的塊，在各塊內沿希爾 伯特曲線進行掃描，將各像素的座標輸出到座標變換部3〇 。另外，如果1個塊內的像素掃描結束，則按光柵順序掃 描下一塊。 另外，壓縮部20通過像Motion JPEG或MPEG那樣，利 用基於縱向8個像素的塊的壓縮方式，能夠不需要在光柵 輸入中所需的光珊塊轉換。 另外’在本實施例的攝影裝置1、1A中，還包括使攝 影鏡頭3在光軸方向滑動的滑動機構和檢測攝影鏡頭3的位 置的檢測部，在座標變換部30中，可以與攝影鏡頭3的鏡 頭狀態相對應地進行座標變換，以校正保存在攝影單元2 的像差係數表3 8中的像差係數。 【圖式簡單說明】As described above, the image processing device 100A and the image processing method according to the second embodiment can compress the color mosaic image data stored in the buffer memory 21 by using the compression unit 20, thereby reducing the storage capacity of the buffer memory 21 required. Further, a color mosaic image having a bit depth other than a multiple of 8 can also efficiently utilize a general-purpose 8×2n-wide DRAM by appropriately setting a compression ratio. Further, in the inverse mosaic unit 28, in order to generate one pixel of the color image, the pixel 多个 of the plurality of pixels in the color mosaic image is referred to, and in the case of generating adjacent pixels of the color image, the multi-reference is performed. (Visit) the same pixel in the color mosaic image. Accordingly, since the image processing device 100A and the image processing method described in the second embodiment introduce the cache memory 27, the read band of the buffer memory 21 can be reduced, and power consumption can be further reduced. (Modification) Although an embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various aspects can be adopted. For example, in order to utilize the commercial buffer memory more efficiently, the scanning order in the output pixel scanning section 29 can also be constructed. That is, when scanning all the pixels of the color image, a strong regional scan is performed. As an example of such a scanning -33-1353779', the use of the Hilbert curve can be cited. For example, the output color image may be divided into 8x8 blocks as shown in Fig. 8, and scanned along the Hilbert curve in each block, and the coordinates of each pixel are output to the coordinate conversion unit 3''. In addition, if the scanning of pixels in one block is completed, the next block is scanned in raster order. Further, the compression unit 20 utilizes a compression method based on blocks of eight pixels in the vertical direction like Motion JPEG or MPEG, so that it is not necessary to perform optical block conversion required for raster input. Further, the photographing apparatuses 1 and 1A of the present embodiment further include a slide mechanism that slides the photographing lens 3 in the optical axis direction and a detecting unit that detects the position of the photographing lens 3, and the coordinate conversion unit 30 can be used with the photographing lens. The lens state of 3 is coordinate-converted correspondingly to correct the aberration coefficient stored in the aberration coefficient table 38 of the photographing unit 2. [Simple description of the map]

圖1是表示應用了本發明的圖像處理裝置和圖像處理 方法的、第一實施方式的攝影裝置1的結構的框圖。 圖2是該第一實施方式中的顔色平面分解部和顔色生 成部的功能說明圖，（a)是表示從攝影單元2輸出的拜爾 排列的顔色馬賽克圖像的圖，（b ) 、 （ c ) 、 （ d ) 、 （ e )分別是表示由顔色平面分解部生成的R平面、Gr平面、 Gb平面、B平面的配置的圖、（f)是插値計算採樣座標的 像素値時的說明圖。 圖3是該第一實施方式中由顔色馬賽克圖像進行圖像 -34- 135.3779 變形和生成彩色圖像的反馬賽克處理的說明圖。 圖4是表示該第一實施方式的圖像處理裝置和圖像處 理方法中的彩色圖像生成的順序的流程圖。 圖5是表示應用了本發明的圖像處理裝置和圖像處理 方法的、第二實施方式的攝影裝置1A的結構的框圖。 圖6是表示該第二實施方式的圖像處理裝置和圖像處 理方法中的彩色圖像生成的順序的流程圖。 圖7是表示圖6中的壓縮和高速緩衝控制器的順序的細 節的流程圖。 圖8是表示變形例中的從高速緩衝存儲器按各塊輸出 像素値時的掃描例的圖。 圖9是現有的反馬賽克處理和圖像變形處理的說明圖 【主要元件符號說明】 1、1 A :攝影裝置 2 :攝影單元 3 :攝影鏡頭 5 :攝影兀件 5 a :拜爾排列的濾色器 6 : AFE ( Analog Front End) 7 :相關雙採樣電路 8 :可變增益放大器（AGC : Automatic Gain Control -35- 1353779 9 : A/D轉換器 13 : TG ( Timing Generator) 1 5 :角速度感測器 18 : CPU ( Central Processing Unit) 19 : ROM ( Read Only Memory) 20 :壓縮部Fig. 1 is a block diagram showing the configuration of an image pickup apparatus 1 according to a first embodiment to which an image processing apparatus and an image processing method of the present invention are applied. 2 is a functional explanatory diagram of a color plane decomposition unit and a color generation unit in the first embodiment, and (a) is a view showing a color mosaic image of a Bayer array output from the imaging unit 2, (b), ( c), (d), and (e) are diagrams showing the arrangement of the R plane, the Gr plane, the Gb plane, and the B plane generated by the color plane decomposition unit, and (f) is the case where the pixel of the sampling coordinates is calculated by interpolation. Illustrating. Fig. 3 is an explanatory diagram of an anti-mosaic process of deforming an image -34 - 135.3779 by a color mosaic image and generating a color image in the first embodiment. Fig. 4 is a flowchart showing the procedure of color image generation in the image processing device and the image processing method of the first embodiment. Fig. 5 is a block diagram showing a configuration of an image pickup apparatus 1A according to a second embodiment to which an image processing apparatus and an image processing method of the present invention are applied. Fig. 6 is a flowchart showing the procedure of color image generation in the image processing device and the image processing method according to the second embodiment. Figure 7 is a flow chart showing the details of the sequence of the compression and cache controller of Figure 6. Fig. 8 is a view showing an example of scanning when a pixel 输出 is output from the cache memory for each block in the modification. 9 is an explanatory diagram of a conventional anti-mosaic processing and image deformation processing. [Main component symbol description] 1. 1 A: Photographing device 2: Photographing unit 3: Photographic lens 5: Photographic element 5 a: Bayer array filter Colorimeter 6: AFE (Analog Front End) 7: Correlated Double Sampling Circuit 8: Variable Gain Amplifier (AGC: Automatic Gain Control -35- 1353779 9 : A/D Converter 13 : TG ( Timing Generator) 1 5 : Angular Velocity Sensor 18: CPU (Central Processing Unit) 19 : ROM (Read Only Memory) 20 : Compression

2 1 :緩衝存儲器 22 : R平面存儲器 23a : Gr平面存儲器 23b: Gb平面存儲器 24 : B平面存儲器 2 5 :拉伸部 26 :高速緩衝控制器 27:高速緩衝存儲器 2 8 :反馬賽克部2 1 : Buffer memory 22 : R plane memory 23a : Gr plane memory 23b: Gb plane memory 24 : B plane memory 2 5 : Stretching section 26 : Cache controller 27 : Cache 2 8 : Anti-mosaic part

29 :輸出像素掃描部 30 :座標轉換部 31 :採樣部 3 2 :插値部 33 :顔色生成部 3 4 :視覺校正部 35 :壓縮部 3 6 :記錄部 3 8 =像差係數表 -36- 1353779 4 Ο :晃動檢測部 100、10 0Α:圖像處理裝置29 : Output pixel scanning unit 30 : coordinate conversion unit 31 : sampling unit 3 2 : insertion unit 33 : color generation unit 3 4 : visual correction unit 35 : compression unit 3 6 : recording unit 3 8 = aberration coefficient table - 36 - 1353779 4 Ο : Sway detection unit 100, 100 Α: Image processing unit

-37--37-

Claims (1)

1353779 七、申請專利範圍： 1·一種圖像處理裝置，其由各像素具有單色的亮度資 訊的顔色馬賽克圖像在各像素生成多個顔色的亮度資訊， 並且實施預定的圖像變形而生成彩色圖像，其中，上述顔 色馬賽克圖像是由具有對不同的多個顔色光進行光電變換 的像素的單板式的攝影元件獲得的， 上述圖像處理裝置包括：1353779 VII. Patent application scope: 1. An image processing apparatus which generates luminance information of a plurality of colors in each pixel by a color mosaic image in which each pixel has monochrome luminance information, and performs predetermined image deformation to generate a color image, wherein the color mosaic image is obtained by a single-plate type photographic element having pixels for photoelectrically converting different color lights, the image processing apparatus comprising: 顏色平面分解部，其將上述顏色馬賽克圖像分解成僅 包含同一顏色光的亮度資訊的多個顏色平面， 緩衝記憶體，其按上述多個顏色平面的各個，記憶在 上述顏色平面分解部所分解後的顏色馬賽克圖像；以及 去馬賽克部，其由被記億在上述緩衝記億體的顏色馬 賽克圖像生成上述彩色圖像， 上述去馬賽克部包括：a color plane decomposition unit that decomposes the color mosaic image into a plurality of color planes including only luminance information of the same color light, and a buffer memory that is stored in the color plane decomposition unit in each of the plurality of color planes a color mosaic image after decomposing; and a demosaicing portion for generating the color image by a color mosaic image of the above-mentioned buffered body, the demosaicing portion comprising: 座標變換部，其按被記憶在上述緩衝記憶體的上述顏 色馬賽克圖像的多個顔色平面的各個，計算與被實施上述 圖像變形時的上述彩色圖像的像素位置對應的上述顏色馬 賽克圖像上的採樣座標； 插値部，其按被記億在上述緩衝記憶體的上述顏色馬 賽克圖像的多個顏色平面的各個，由上述顔色平面內所包 含的同一顏色光的像素値，插値生成上述顏色馬賽克圖像 上的採樣座標中的各顏色的像素値；以及 顏色生成部，合成由上述插値部所插値生成的採樣座 標中的各顏色的像素値，生成上述彩色圖像。 -38- 1353779 2.如申請專利範圍第1項之圖像處理裝置，其中，上 述圖像處理裝置還包括： 壓縮部’其壓縮上述顔色馬賽克圖像；以及 拉伸部’其拉伸由上述壓縮部壓縮後的上述顔色馬賽 克圖像， 上述壓縮部按上述多個顏色平面的各個分割成多個區 塊，按該分割而成的各區塊壓縮成預定的尺寸以下，a coordinate conversion unit that calculates the color mosaic map corresponding to a pixel position of the color image when the image is deformed, in accordance with each of a plurality of color planes of the color mosaic image stored in the buffer memory a sampling coordinate on the image; a plug portion that is inserted into each of a plurality of color planes of the color mosaic image of the buffer memory, and is inserted by a pixel of the same color light included in the color plane値 generating a pixel 各 of each color in the sampling coordinates on the color mosaic image; and a color generating unit that combines pixels 各 of each color in the sampling coordinates generated by the interpolation unit to generate the color image . The image processing device according to claim 1, wherein the image processing device further includes: a compressing portion that compresses the color mosaic image; and a stretching portion that is stretched by the above The color mosaic image compressed by the compression unit, the compression unit is divided into a plurality of blocks for each of the plurality of color planes, and each of the divided blocks is compressed to a predetermined size or smaller. 上述緩衝記億體記憶由上述壓縮部壓縮後的各區塊的 顏色馬賽克圖像， 上述拉伸部按被記憶在上述緩衝記憶體的各區塊，拉 伸被壓縮後的顏色馬賽克圖像， 上述去馬賽克部由上述拉伸部所拉伸的顏色馬賽克圖 像生成上述彩色圖像。 3.—種圖像處理方法，其由各像素具有單色的亮度資 訊的顏色馬賽克圖像在各像素生成多個顏色的亮度資訊， 並且實施預定的圖像變形而生成彩色圖像，其中，上述顏 色馬賽克圖像是由具有對不同的多個顔色光進行光電變換 的像素的單板式的攝影元件獲得的， 上述圖像處理方法使用： 顏色平面分解步驟，其將上述顏色馬賽克圖像分解成 僅包含同一顏色光的亮度資訊的多個顏色平面， 顏色平面記億步驟，其按多個顏色平面的各個’將在 上述顏色平面分解步驟所分解後的顏色馬賽克圖像記憶在 緩衝記憶體；以及 -39- 1353779 去馬賽克步驟，其由被記憶在上述緩衝記憶體的顏色 馬賽克圖像生成上述彩色圖像， 在上述去馬賽克步驟中使用： 座標變換步驟，其按被記億在上述緩衝記憶體的上述 顏色馬賽克圖像的多個顔色平面的各個，計算與被實施上 述圖像變形時的上述彩色圖像的像素位置對應的上述顏色 馬賽克圖像上的採樣座標；The buffered memory is a color mosaic image of each block compressed by the compression unit, and the stretched portion is stretched in each block of the buffer memory to stretch the compressed color mosaic image. The demosaicing portion generates the color image by a color mosaic image stretched by the stretching portion. 3. An image processing method for generating luminance information of a plurality of colors in each pixel by a color mosaic image in which each pixel has monochrome luminance information, and performing predetermined image deformation to generate a color image, wherein The color mosaic image is obtained by a single-plate type imaging element having pixels for photoelectrically converting different color lights, and the image processing method uses a color plane decomposition step of decomposing the color mosaic image into a plurality of color planes containing only brightness information of the same color light, the color plane is recorded in steps of a plurality of color planes, and the color mosaic images decomposed in the color plane decomposition step are memorized in the buffer memory; And a -39- 1353779 demosaicing step of generating the color image by a color mosaic image memorized in the buffer memory, used in the demosaicing step: a coordinate transformation step, which is recorded in the buffer memory Each of the multiple color planes of the above-mentioned color mosaic image is calculated and implemented Pixel position on the color image when said image deformation corresponding to the sample color coordinates in the mosaic image; 插値步驟，其按被記憶在上述緩衝記憶體的上述顏色 馬賽克圖像的多個顏色平面的各個，由上述顏色平面內所 包含的同一顏色光的像素値，插値生成上述顏色馬賽克圖 像上的採樣座標中的各顏色的像素値；以及 顏色生成步驟，合成在上述插値步驟所插値生成的採 樣座標中的各顏色的像素値，生成上述彩色圖像。 4.如申請專利範圍第3項之圖像處理方法，其中，上 述圖像處理方法使用：a plugging step of inserting the color mosaic image by the pixels of the same color light contained in the color plane, each of the plurality of color planes of the color mosaic image stored in the buffer memory a pixel 各 of each color in the upper sampling coordinates; and a color generating step of synthesizing the pixel 各 of each color in the sampling coordinates generated by the interpolation step, to generate the color image. 4. The image processing method of claim 3, wherein the image processing method uses: 壓縮步驟，其壓縮上述顔色馬賽克圖像；以及 拉伸步驟，其拉伸在上述壓縮步驟壓縮後的上述顔色 馬賽克圖像* 在上述壓縮步驟中，按上述多個顏色平面的各個分割 成多個區塊，按該分割而成的各區塊壓縮成預定的尺寸以 下， 在上述顔色平面記億步驟中，在上述緩衝記憶體記億 在上述壓縮步驟壓縮後的各區塊的顏色馬賽克圖像， 在上述拉伸步驟中，按被記億在上述緩衝記憶體的各 -40- 1353779 區塊，拉伸被壓縮後的顔色馬賽克圖像， 在上述去馬賽克步驟中，由在上述拉伸步驟所拉伸的 顏色馬賽克圖像生成上述彩色圖像。a compression step of compressing the color mosaic image; and a stretching step of stretching the color mosaic image after compression in the compressing step * in the compressing step, dividing each of the plurality of color planes into a plurality of a block, wherein each of the divided blocks is compressed to a predetermined size or less, and in the color plane counting step, a color mosaic image of each block in the buffer memory is compressed in the compression step And in the stretching step, stretching the compressed color mosaic image according to each of the -4033535 blocks of the buffer memory, in the above-mentioned demosaicing step, in the above-mentioned stretching step The stretched color mosaic image generates the above color image. -41 - 1353779 四、 指定代表圊： (一） 本案指定代表圖為：第（1)圊。 (二） 本代表圖之元件符號簡單說明： 1:攝影裝置，2:攝影單元，3:攝影鏡頭， 5 :攝影元件，5 a :拜爾排列的濾色器， 6 ： AFE ( Analog Front End) ，7 :相關雙採樣電路， 8 :可變增益放大器（aGC : Automatic Gain Control)， 9 : A/D轉換器，13 : TG ( Timing Generator)， 1 5 :角速度感測器， 18 ： CPU ( Central Processing Unit) &gt; 19: ROM (Read Only Memory) ，21 :緩衝存儲器， 22: R平面存儲器，23a: Gr平面存儲器， 23b: Gb平面存儲器，24: B平面存儲器， 28 :反馬賽克部，29 :輸出像素掃描部， 3 〇 :座標轉換部，3 1 :採樣部，3 2 ··插値部， 33:顔色生成部’ 34:視覺校正部，35:壓縮部， 3 6 :記錄部，3 8 :像差係數表，4 0 :晃動檢測部， 1〇〇 :圖像處理裝置 五、 本案若有化學式時，請揭示最能顯示發明特徵的化學 式：無-41 - 1353779 IV. Designated representative: (1) The representative representative of the case is: (1)圊. (2) A brief description of the component symbols of this representative diagram: 1: Photographic device, 2: Photographic unit, 3: Photographic lens, 5: Photographic component, 5 a: Bayer array of color filters, 6: AFE (Analog Front End ), 7: correlated double sampling circuit, 8: variable gain amplifier (aGC: Automatic Gain Control), 9 : A/D converter, 13 : TG ( Timing Generator), 1 5 : angular velocity sensor, 18 : CPU (Central Processing Unit) &gt; 19: ROM (Read Only Memory), 21: buffer memory, 22: R plane memory, 23a: Gr plane memory, 23b: Gb plane memory, 24: B plane memory, 28: anti-mosaic 29: output pixel scanning unit, 3 〇: coordinate conversion unit, 3 1 : sampling unit, 3 2 · interpolation unit, 33: color generation unit 34: visual correction unit, 35: compression unit, 3 6 : recording Department, 3 8 : aberration coefficient table, 40: sway detection unit, 1 〇〇: image processing device 5. If there is a chemical formula in this case, please disclose the chemical formula that best shows the characteristics of the invention: none
TW97145601A 2007-11-28 2008-11-26 Image processing device and image processing method TW200924512A (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2007307034A JP4504412B2 (en) 2007-11-28 2007-11-28 Image processing apparatus and image processing method

Publications (2)

Publication Number Publication Date
TW200924512A TW200924512A (en) 2009-06-01
TWI353779B true TWI353779B (en) 2011-12-01

Family

ID=40678355

Family Applications (1)

Application Number Title Priority Date Filing Date
TW97145601A TW200924512A (en) 2007-11-28 2008-11-26 Image processing device and image processing method

Country Status (5)

Country Link
JP (1) JP4504412B2 (en)
KR (1) KR101009108B1 (en)
CN (1) CN101448169B (en)
TW (1) TW200924512A (en)
WO (1) WO2009069453A1 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5569042B2 (en) 2010-03-02 2014-08-13 株式会社リコー Image processing apparatus, imaging apparatus, and image processing method
JP5976315B2 (en) * 2011-12-28 2016-08-23 株式会社ザクティ Imaging device
CN104296664A (en) * 2014-09-17 2015-01-21 宁波高新区晓圆科技有限公司 Method for improving detection precision in geometric dimension visual detection
CN105847770B (en) * 2015-01-15 2017-08-08 展讯通信（上海）有限公司 The caching method of view data in picture processing chip and the chip
WO2016185763A1 (en) * 2015-05-15 2016-11-24 ソニー株式会社 Light-source control device, light-source control method, and imaging system
KR102581210B1 (en) 2019-01-10 2023-09-22 에스케이하이닉스 주식회사 Method for processing image signal, image signal processor, and image sensor chip
CN113793249B (en) * 2021-08-09 2023-12-08 深圳曦华科技有限公司 Method for converting Pentille image into RGB image and related equipment

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5652621A (en) * 1996-02-23 1997-07-29 Eastman Kodak Company Adaptive color plane interpolation in single sensor color electronic camera
JP3503372B2 (en) * 1996-11-26 2004-03-02 ミノルタ株式会社 Pixel interpolation device and pixel interpolation method
JP2005217896A (en) * 2004-01-30 2005-08-11 Shoji Kawahito Image encoding device
JP4815807B2 (en) * 2004-05-31 2011-11-16 株式会社ニコン Image processing apparatus, image processing program, and electronic camera for detecting chromatic aberration of magnification from RAW data
JP4438635B2 (en) * 2005-01-28 2010-03-24 セイコーエプソン株式会社 Image processing method, digital camera, and image processing program
CN1780405A (en) * 2005-05-26 2006-05-31 智多微电子（上海）有限公司 Conversion and module from primitive Bayer interpolation to full color
JP4508132B2 (en) * 2006-02-27 2010-07-21 ソニー株式会社 Imaging device, imaging circuit, and imaging method

Also Published As

Publication number Publication date
JP2009130898A (en) 2009-06-11
KR101009108B1 (en) 2011-01-18
CN101448169A (en) 2009-06-03
KR20100083197A (en) 2010-07-21
JP4504412B2 (en) 2010-07-14
WO2009069453A1 (en) 2009-06-04
CN101448169B (en) 2011-10-05
TW200924512A (en) 2009-06-01

Similar Documents

Publication Publication Date Title
JP4661922B2 (en) Image processing apparatus, imaging apparatus, solid-state imaging device, image processing method, and program
TWI353779B (en)
KR100917088B1 (en) Image capturing apparatus and electronic zoom method
KR101299055B1 (en) Image processing device, image processing method, and image processing program
US8570334B2 (en) Image processing device capable of efficiently correcting image data and imaging apparatus capable of performing the same
TWI617196B (en) Image sensor, imaging device, mobile terminal and imaging method
JP2005252626A (en) Image pickup device and image processing method
US9832382B2 (en) Imaging apparatus and imaging method for outputting image based on motion
JP2006293009A (en) Automatic focusing device and method
JP2005244980A (en) Apparatus and method for processing image signal
JP2006157568A (en) Imaging apparatus and program
JP2017098791A (en) Imaging element and imaging device
US20090066804A1 (en) Data-processing circuit and video camera
JP2007274599A (en) Imaging apparatus
JP5683858B2 (en) Imaging device
JP5996418B2 (en) Imaging apparatus and imaging method
JP5820720B2 (en) Imaging device
JP5959194B2 (en) Imaging device
JP2006157432A (en) Three-dimensional photographic apparatus and photographic method of three-dimensional image
JP2008072501A (en) Imaging apparatus, imaging method, program, and recording medium
JP2007180893A (en) Image processor and imaging apparatus
JP2014121024A (en) Imaging apparatus
JP2005277618A (en) Photography taking apparatus and device and method for correcting shading
US8804014B2 (en) Imaging device for reducing color moire
JPWO2019142836A1 (en) Encoding device, decoding device, coding method, decoding method, coding program, and decoding program

Legal Events

Date Code Title Description
MM4A Annulment or lapse of patent due to non-payment of fees