201208344 六、發明說明： 【發明所屬之技術領域】 [0001] 本發明係有關數位影像處理，特別是關於一種三維（3D )影像的深度加強系統及方法。 【先前技術】 [0002] 當三維物體藉由照相機或攝影機而投影映射至二維影像 平面時，會失去許多的資訊，特別是三維深度資訊。三 維成像系統可藉由錄製三維視覺資訊或創造深度幻覺而 傳達三維資訊給觀者。雖然三維成像技術已有一世紀以 Ο 上的歷史，然而由於近來高解析度及低價格顯示器（例 如液晶顯示器）的發展，使得三維顯示器變得更為實際 及普遍。201208344 VI. Description of the Invention: [Technical Field of the Invention] [0001] The present invention relates to digital image processing, and more particularly to a depth enhancement system and method for three-dimensional (3D) images. [Prior Art] [0002] When a three-dimensional object is projected onto a two-dimensional image plane by a camera or a camera, a lot of information, especially three-dimensional depth information, is lost. The 3D imaging system delivers 3D information to viewers by recording 3D visual information or creating depth illusions. Although 3D imaging technology has a history of the past century, the recent development of high resolution and low price displays (such as liquid crystal displays) has made 3D displays more practical and ubiquitous.
[0003] 第一 A圖顯示傳統三維成像系統的方塊圖，其中，深度產 生器10根據二維輸入影像而產生深度資訊。該深度資訊 經由深度影像成像器（dep..th-image-bas:.ed render-ing，DIBR) 12的處理而產生左（L)影像14A及右（R 〇 ) 影像14B，其經顯示而讓觀者觀看。第一B圖顯示另一 種傳統三維成像系統的方塊圖，其中，三維記錄器（例 如藍光數位影音光碟）16或電視廣播信號記錄左影像18A 及右影像18B，其可使用第一d圖所例示方法加以編碼， 例如肩並肩編碼（Side-by-side)、上下編碼（t〇p-bottom、over-under或above-below)、線交錯編碼 (line-by-line 或 line interlace)或棋盤式編碼（ checkerboard)。第—C圖顯示又一種傳統三維成像系 統的方塊圖，其中，三維記錄器16或電視廣播信號記錄 099125674 表單編號A0101 第3頁/共17頁 0992045048-0 201208344 深度圖（depth map) 20以及左影像18A和右影像18B其 中之一，此一般稱為二維+深度型態。深度影像成像器（ DIBR) 12根據深度圖20及左影像18A以產生右影像18B， 其經顯示而讓觀者觀看。 [0004] 傳統三維成像系統具有一些缺點，例如有限的深度範圍 。因此，經常使得觀者無法分辨原本應具不同深度的不 同物件。 [0005] 鑑於傳統三維成像系統無法有效顯示三維影像或視訊， 因此亟需提出一種新穎的三維成像系統及方法，其具改 良的深度特性。 【發明内容】 [0006] 鑑於上述，本發明實施例的目的之一在於提出一種三維 影像的深度加強系統及方法，用以提高三維影像或視訊 的品質，使得觀者得以感知更多的深度層級。 [0007] 根據本發明實施例，深度產生器產生相關於一影像的至 少一深度圖。深度加強器藉由伸展該深度圖所相關的一 深度直方圖，以加強深度圖，其中該深度直方圖係為影 像像素的深度層級之分佈。 [0008] 根據本發明另一實施例，接收相關於一影像的至少一深 度圖，且分割深度圖使成為多個分割之深度圖。接著， 根據該些分割深度圖，以產生多個區域深度直方圖。最 後，改變影像之深度特徵，以分別加強該些分割之深度 圖。 【實施方式】 099125674 表單編號A0101 第4頁/共17頁 0992045048-0 201208344 . [0009] 第二圖顯示本發明實施例之三維影像的深度加強系統之 方塊圖。第三圖顯示本發明實施例之三維影像的深度加 強方法之流程圖。三維影像又稱為立體（stereoscopic )影像。 [0010] ο 於步驟31，接收可顯示於三維成像系統的左（L)影像 21A及右（R)影像21B。接著，以深度產生器22產生至 少一深度圖（步驟32)。例如，深度產生器22產生左深 度圖及右深度圖，其分別對應至原始左影像及右影像。 於另一例子中，深度產生器22產生單一深度圖。於深度 圖中，每一像素或區塊具有相應的深度值。例如，靠近 觀者物件的深度值比遠離觀者物件的深度值來得大。因 此，於深度圖影像中，靠近觀者物件的亮度會比遠離觀 者物件的亮度來得大。在另一實施例中，可由深度產生 器（例如第一 A圖之10)來提供，或者由三維成像系統之 三維記錄器（例如第一C圖之16)來提供。 [0011] 〇 接下來，於步驟33，以深度加強器23對深度產生器22所 產生的一或多個深度圖進行加強，以更改或改進影像的 深度特性。在本實施例中，對深度產生器22所產生的深 度圖進行深度直方圖（histogram)的伸展（stretching) 。 深度直方圖係為像素之深度層級的分佈 （其通常 繪製為圖表），其中，每一深度層級具有相應的像素計 數值。當直方圖被繪製為圖表時，則水平軸代表深度層 級，而垂直軸則代表相應的像素數目。第四A圖例示原始 深度圖的深度直方圖，而第四B圖顯示進行深度直方圖伸 展後的加強深度直方圖。於此例子中，原始深度圖所對 099125674 表單編號A0101 第5頁/共17頁 0992045048-0 201208344 應之影像，其像素的深度層級皆小於1〇〇;然而加強深 度圖所對應之加強影像，其像素的深度層級則伸展於整 個深度層級範圍。 [0012]在一實施例中，前述深度直方圖伸展（depth hist〇_ gram stretching)之執行係對每一深度層級施以一增 益。在另一實施例中，深度直方圖伸展之執行係對深度 直方圖的深度層級施以特徵曲線（例如伽瑪（Gamma)曲 線）之轉換或映射。第五圖顯示對深度直方圖之深度層 級施以S曲線轉換。如圖所示，3曲線的最大斜率通過最 广 大像素數目之深度層級。一般来說，深度直方圖伸展之 執行可對深度直方圖的每一深度層級施以加、減、乘或 除一數值’用以等化（equal ize )每一深度層級。 [0013] 接著，於步驟34，來自深度加強器23之加強深度圖被饋 至深度影像成像（DIBR)單元24，其根據加強深度圖及 原始左影像及右影像，用以產生（或合成）加強左（L， )影像25A及加強右（R’ ）影像25B。所產生的加強左影 像及加強右影像則可由三維成像系統所顯示。深度影像 u 成像（DIBR)單元24之實施可使用傳統技術，例如The first A diagram shows a block diagram of a conventional three-dimensional imaging system in which the depth generator 10 generates depth information based on a two-dimensional input image. The depth information is processed by a depth image imager (dep..th-image-bas:.ed render-ing, DIBR) 12 to generate a left (L) image 14A and a right (R 〇) image 14B, which are displayed. Let the viewer watch. Figure 1B shows a block diagram of another conventional three-dimensional imaging system in which a three-dimensional recorder (e.g., a Blu-ray digital video disc) 16 or a television broadcast signal records a left image 18A and a right image 18B, which can be exemplified using the first d-picture. The method is coded, such as Side-by-side, up-and-down coding (t〇p-bottom, over-under or above-below), line-by-line or line interlace or checkerboard Encoding (checkerboard). Figure C shows a block diagram of another conventional three-dimensional imaging system in which the three-dimensional recorder 16 or television broadcast signal record 099125674 Form No. A0101 Page 3 of 17 0992045048-0 201208344 Depth map 20 and left One of the image 18A and the right image 18B, which is generally referred to as a two-dimensional + depth type. A depth image imager (DIBR) 12 generates a right image 18B based on the depth map 20 and the left image 18A, which is displayed for viewing by the viewer. Conventional three-dimensional imaging systems have some drawbacks, such as a limited depth range. Therefore, it often makes it impossible for the viewer to distinguish between different objects that should have different depths. [0005] In view of the inability of conventional three-dimensional imaging systems to effectively display three-dimensional images or video, it is desirable to propose a novel three-dimensional imaging system and method that has improved depth characteristics. SUMMARY OF THE INVENTION [0006] In view of the above, one of the objectives of the embodiments of the present invention is to provide a depth enhancement system and method for three-dimensional images, which are used to improve the quality of three-dimensional images or video, so that viewers can perceive more depth levels. . In accordance with an embodiment of the invention, the depth generator produces at least one depth map associated with an image. The depth enhancer enhances the depth map by extending a depth histogram associated with the depth map, wherein the depth histogram is a distribution of depth levels of the image pixels. In accordance with another embodiment of the present invention, at least one depth map associated with an image is received, and the segmentation depth map is rendered into a plurality of segmented depth maps. Then, according to the segmentation depth maps, a plurality of region depth histograms are generated. Finally, the depth features of the image are changed to enhance the depth maps of the segments, respectively. [Embodiment] 099125674 Form No. A0101 Page 4 of 17 0992045048-0 201208344. [0009] The second figure shows a block diagram of a depth enhancement system for a three-dimensional image according to an embodiment of the present invention. The third figure shows a flow chart of a method for deepening a three-dimensional image according to an embodiment of the present invention. The three-dimensional image is also called a stereoscopic image. [0010] In step 31, a left (L) image 21A and a right (R) image 21B displayable in the three-dimensional imaging system are received. Next, at least one depth map is generated by depth generator 22 (step 32). For example, depth generator 22 produces a left depth map and a right depth map, which correspond to the original left and right images, respectively. In another example, depth generator 22 produces a single depth map. In the depth map, each pixel or block has a corresponding depth value. For example, the depth value near the viewer object is larger than the depth value away from the viewer object. Therefore, in the depth map image, the brightness of the object close to the viewer is greater than the brightness of the object away from the viewer. In another embodiment, it may be provided by a depth generator (e.g., 10 of the first A) or by a three dimensional recorder of the three dimensional imaging system (e.g., 16 of the first C). [0011] Next, in step 33, one or more depth maps generated by the depth generator 22 are enhanced by the depth enhancer 23 to modify or improve the depth characteristics of the image. In the present embodiment, a depth histogram stretching is performed on the depth map generated by the depth generator 22. The depth histogram is the distribution of the depth levels of the pixels (which are usually plotted as a graph), where each depth level has a corresponding pixel count value. When the histogram is drawn as a chart, the horizontal axis represents the depth level and the vertical axis represents the corresponding number of pixels. The fourth A diagram illustrates the depth histogram of the original depth map, and the fourth B diagram shows the enhanced depth histogram after the depth histogram is extended. In this example, the original depth map is 099125674 Form No. A0101 Page 5 of 17 0992045048-0 201208344 The image should have a depth level of less than 1〇〇; however, the enhanced image corresponding to the depth map is enhanced. The depth level of its pixels extends over the entire depth level. [0012] In one embodiment, the execution of the depth hist__gram stretching is applied to each depth level. In another embodiment, the execution of the depth histogram stretch applies a transformation or mapping of a feature curve (e. g., a gamma curve) to the depth level of the depth histogram. The fifth plot shows the S-curve transformation for the depth level of the depth histogram. As shown, the maximum slope of the 3 curve passes through the depth level of the largest number of pixels. In general, the execution of the depth histogram stretch can add, subtract, multiply, or divide a value for each depth level of the depth histogram to equalize each depth level. [0013] Next, in step 34, the enhanced depth map from the depth enhancer 23 is fed to a depth image imaging (DIBR) unit 24 for generating (or synthesizing) the enhanced depth map and the original left and right images. Enhance the left (L,) image 25A and the enhanced right (R') image 25B. The resulting enhanced left image and enhanced right image can be displayed by a three dimensional imaging system. The depth image u imaging (DIBR) unit 24 can be implemented using conventional techniques, such as
Christoph Fehn所揭露之” A 3D-TV Approach Using Depth-Image-Based Rendering (DIBR)” 。 [0014] 第六圖顯示本發明另一實施例之三維影像的深度加強方 法之流程圖。於步驟61，自深度產生器22 (第二圖）接 收深度圖。接著，於步驟62，決定接下來的深度加強究 竟是要全域地執行抑或區域地執行。 099125674 表單編號A0101 第6頁/共17頁 0992045048-0 201208344 [0015] 如果為全域地執行深度加強，則根據整個深度圖以產生 全域深度直方圖（步驟63A)。接著，於步驟64A，對全 域深度直方圖的深度層級進行加強，例如使用第五圖所 例示之S曲線轉換（或映射）。 [0016] 如果為區域地執行深度加強，則根據分割之深度圖以產 生多個區域深度直方圖（步驟63B)。第七圖例示根據分 割之深度圖以產生多個相應之區域深度直方圖。在此例 子中，整個深度圖被分割為2x2或4個分割之深度圖。接 0 著，於步驟64B，對各個區域深度直方圖的深度層級分別 進行加強，例如使用S曲線對區域深度直方圖進行轉換（ 或映射）。每一區域深度直方圖可施以不同的轉換，而 一些區域深度直方圖也可不進行轉換。 [0017] 以上所述僅為本發明之較佳實施例而已，並非用以限定 本發明之申請專利範圍；凡其它未脫離發明所揭示之精 神下所完成之等效改變或修飾，均應包含在下述之申請 專利範圍内。 ^ 【圖式簡單說明】 [0018] 第一 A圖顯示傳統三維成像系統的方塊圖。 第一 B圖顯示另一種傳統三維成像系統的方塊圖。 第一 C圖顯示又一種傳統三維成像系統的方塊圖。 第一D圖顯示三維記錄器或廣播信號的三維格式。 第二圖顯示本發明實施例之三維影像的深度加強系統之 方塊圖。 第三圖顯示本發明實施例之三維影像的深度加強方法之 流程圖。 099125674 表單編號A0101 第7頁/共17頁 0992045048-0 201208344 第四A圖例示原始深度圖的深度直方圖。 第四B圖顯示進行深度直方圖伸展後的加強深度直方圖。 第五圖顯示對深度直方圖之深度層級施以S曲線轉換。 第六圖顯示本發明另一實施例之三維影像的深度加強方 法之流程圖。 第七圖例示根據分割之深度圖以產生多個相應之區域深 度直方圖。 【主要元件符號說明】 [0019] 10 深度產生器 12 深度影像成 14A 左影像 14B 右影像 16 三維記錄器 18A 左影像 18B 右影像 18A， 左影像 18B， 右影像 21A 左影像 21B 右影像 22 深度產生器 23 深度加強器 24 深度影像成 25A 加強左影像 25B 加強右影像 31-34 步驟 61-64B 步驟 表單編號A0101 第8頁/共17頁 099125674 0992045048-0"A 3D-TV Approach Using Depth-Image-Based Rendering (DIBR)" by Christoph Fehn. [0014] FIG. 6 is a flow chart showing a method of depth enhancement of a three-dimensional image according to another embodiment of the present invention. At step 61, the depth map is received from depth generator 22 (second map). Next, in step 62, it is determined whether the subsequent depth enhancement is to be performed globally or regionally. 099125674 Form No. A0101 Page 6 of 17 0992045048-0 201208344 [0015] If depth enhancement is performed globally, a global depth histogram is generated from the entire depth map (step 63A). Next, in step 64A, the depth level of the global depth histogram is enhanced, for example using the S-curve transformation (or mapping) illustrated in the fifth figure. [0016] If depth enhancement is performed regionally, a plurality of region depth histograms are generated according to the segmented depth map (step 63B). The seventh diagram illustrates the depth map according to the split to generate a plurality of corresponding region depth histograms. In this example, the entire depth map is split into 2x2 or 4 segmented depth maps. In step 64B, the depth levels of the depth histograms of the respective regions are respectively enhanced, for example, the region depth histogram is converted (or mapped) using the S curve. Each region depth histogram can be cast differently, while some region depth histograms can also be converted. The above description is only the preferred embodiment of the present invention, and is not intended to limit the scope of the claims of the present invention; any equivalent changes or modifications which are not included in the spirit of the invention should be included. It is within the scope of the following patent application. ^ [Simple Description of the Drawings] [0018] The first A diagram shows a block diagram of a conventional three-dimensional imaging system. The first B diagram shows a block diagram of another conventional three-dimensional imaging system. The first C-picture shows a block diagram of yet another conventional three-dimensional imaging system. The first D-picture shows a three-dimensional format of a three-dimensional recorder or broadcast signal. The second figure shows a block diagram of a depth enhancement system for a three-dimensional image of an embodiment of the present invention. The third figure shows a flow chart of a depth enhancement method for a three-dimensional image according to an embodiment of the present invention. 099125674 Form No. A0101 Page 7 of 17 0992045048-0 201208344 Figure 4A illustrates the depth histogram of the original depth map. Figure 4B shows the enhanced depth histogram after the depth histogram is stretched. The fifth plot shows the S-curve transformation applied to the depth level of the depth histogram. Fig. 6 is a flow chart showing a method of deepening a three-dimensional image according to another embodiment of the present invention. The seventh diagram illustrates the depth map according to the segmentation to generate a plurality of corresponding region depth histograms. [Main component symbol description] [0019] 10 Depth generator 12 Depth image into 14A Left image 14B Right image 16 3D recorder 18A Left image 18B Right image 18A, Left image 18B, Right image 21A Left image 21B Right image 22 Depth generation 23 depth enhancer 24 depth image into 25A enhanced left image 25B enhanced right image 31-34 step 61-64B step form number A0101 page 8 / total 17 page 099125674 0992045048-0