201125358 六、發明說明： - 【發明所屬之技術領域】 本發明係關於多視角互動電視系統與方法技術，特別是 指利用多視角拍攝及視訊處理技術，提供多視角影片來源， 令使用者擁有更完整的主控權操作模式，可透過不同視角及 景深來欣賞同一場景。 【先前技術】 φ 在本案之前並無一完整系統可達成此服務，目前之數 位電視服務僅提供頻道轉換，並且由於播放之内容在影片 製作初期就已被決定，無法臨時更改，因此使用者在同一 播放頻道中僅能觀賞到固定内容，不具有選擇性，亦無法 參與互動。 在進行拍攝工作時’同一場景通常只利用數台攝影機 捕捉關鍵鏡頭，拍攝結束後再透過後製剪輯成一部影片， 鲁而只有導演一人有權決定哪些鏡頭才具有關鍵性，值得被 保留。故在同一場景中若同時存在多位角色，使用者通常 ”會看到某一角色的特寫畫面，或是包含多位角色的遠拍 晝面，並無法隨心所欲地選擇此時此刻最想觀賞的人物或 %豕，而只能被動地接受事先安排好的内容。 在轉播球賽中，當裁判之判決具有爭議性時，若在此 瞬間有數十個觀賞角度可以選擇，則除了裁判外，觀幕也 可切換到最適合自己的角度來判斷結果，相較於以往只能 201125358 接收固定資料書面，容鉬 —®多視角互動電視系統將大幅增加臨場 感與參與感。 綜上所述 傳’先之數位電視服務互動性較低，使用者 僅此選擇頻道’並無法鎖定特定頻道再更進一步地進行互 動Ρ使知用&選視訊（Video on 〇emand; v〇D)之服務方 式，目前能提供之互動模式仍非f有限。 由此可見，上述習用方式仍有諸多不足，實非一良善 之設計，而亟待加以改良。 本案發月人鑑於上述習用方式所衍生的各項缺點’乃 亟心加以改良創新’並經多年苦心孤詣潛心研究後，終於 成功研發完成本件多視角互動電視系統與方法技術。 【發明内容] 本發明之目的在於提供一種多視角互動電視系統與 方法技術’除了傳統的服務功能外，搭配多視角拍攝技術 提供多視角影片來源1 了可提供切換視角之功能，亦可 切換場景深度（Zoom In and Z00m 〇ut)，使用者可透過機 上盒（Set-Top Box，STB)及遙控器進行操作。 達成上述發明目的之多視角互動電視系統與方法，係 採用同時同步拍攝技術，將取得的數位影音檔案儲存於資 料庫中，每一筆檔案分別帶有時間及空間的資訊，因此當 使用者透過機上盒（STB)切換視角時，便可立即由_流媒 體伺服器（Streaming Server)提取相對應的多媒體資料。有 201125358 別於以往以群播（Multicast)的方式進行播放本發明採用 即時争流的模式傳送多媒體内容，並藉由streaming201125358 VI. Description of the invention: - Technical field of the invention The present invention relates to a multi-view interactive television system and method technology, in particular to the use of multi-view shooting and video processing technology to provide a multi-view video source, so that the user has more The complete master control mode allows you to enjoy the same scene through different perspectives and depth of field. [Prior Art] φ There is no complete system to achieve this service before the case. The current digital TV service only provides channel conversion, and since the content of the broadcast has been decided at the beginning of the film production, it cannot be temporarily changed, so the user is Only fixed content can be viewed in the same playback channel, which is not selective and cannot participate in interaction. When shooting, the same scene usually uses only a few cameras to capture key shots. After the shooting is finished, it is edited into a film through post-production. Only the director has the right to decide which shots are critical and worthy of being retained. Therefore, if there are multiple characters in the same scene at the same time, the user usually "sees a close-up picture of a certain character, or a telephoto picture containing multiple characters, and cannot select whatever he wants most at the moment." The character or %豕 can only passively accept the pre-arranged content. In the broadcast ball game, when the judgment of the referee is controversial, if there are dozens of viewing angles to choose at this moment, then in addition to the referee, the view The screen can also be switched to the angle that suits you best to judge the result. Compared with the previous report that only fixed data can be received in 201125358, the Mo---Multi-view interactive TV system will greatly increase the sense of presence and participation. First, the digital TV service is less interactive, and the user only selects the channel 'and can't lock the specific channel and then further interacts with the video on Videoemand; v〇D). At present, the interactive mode that can be provided is still limited. It can be seen that there are still many shortcomings in the above-mentioned methods of use, which is not a good design and needs to be improved. In view of the shortcomings derived from the above-mentioned conventional methods, Yueren has been improving and innovating, and after years of painstaking research, he finally succeeded in research and development of this multi-view interactive TV system and method technology. [Summary of the Invention] It provides a multi-view interactive TV system and method technology. In addition to the traditional service functions, it provides multi-view video source with multi-view camera technology. It can provide the function of switching the view angle, and can also switch the scene depth (Zoom In and Z00m 〇ut The user can operate through the Set-Top Box (STB) and the remote controller. The multi-view interactive television system and method for achieving the above object is to use the simultaneous synchronous shooting technology to store the acquired digital audio and video files. In the database, each file has time and space information, so when the user switches the view through the set-top box (STB), the corresponding multimedia can be extracted immediately by the Streaming Server. Information. 201125358 is different from the previous play in the multicast mode (Multicast) Next immediate contention mode using streaming multimedia content delivery, and by streaming
Server即時處理使用者的請求，而將多項控制模式順利地 應用於STB之±。相較於以往之視訊服務技術，多視角 互動電視系統與方法技術不僅在STB上可兼具以往之服 務技術，例如快轉及倒轉等，並可提供使用者更多元化的 才呆作模式，例如視角之切換、景深之切換、環場預覽 (VieW-SweePing)以及時間凍結（Time Frozen)等，而提供 服務的頻道數目並不會隨著攝影機數量而增加總共只需 要利用個頻道傳送多媒體内容，與傳統的服務模式相 同〇 視角之切換即為使用者可透過不同的角度來觀賞同 一場景，每一個瞬間的每一個視角都確實存在於資料庫 中，透過串流媒體伺服器以提供使用者可隨時取用。使用 者可透過STB之遙控器來轉換想觀賞的角度，在切換視 角的過程中，並可透過Rendering相關技術以達到更平順 的切換效果。景深之切換即為使用者可在同一觀賞角度 下，再更進一步地縮放觀賞的的場景深度，拉遠（Z〇〇m 〇ut) 夺可觀貝到較大的視野，拉近（z〇〇m In)時可觀賞到較多 的細卽。切換景深的功能亦可透過STB之遙控器來執行， 在切換的過程中，透過Interp〇lati〇n相關技術可達到更平 順的縮放效果。 201125358 環場預覽（View-Sweeping)可提供使用者透過STB之 遙控器快速預覽每一個可切換的視角及景深，在預覽的同 時，多媒體可為靜止或是維持播放狀態，播放狀態可為正 常播放、快轉或倒轉。時間凍結（Time Fr〇zen)係提供使用 者在同一時間點之靜止狀態下，使用者透過STB之遙控 器切換不同的觀賞角度及景深，或是進行環場預覽。 【實施方式】 請參閱圖一，為本發明提供多視角互動電視系統與方 法之運作方式示意圖。多視角視訊服務内容提供者丨以 拍攝物體為中心，同時架設多部攝影機成一環繞之拍攝環 境’且同時對準欲拍攝之物體’拍攝時攝影機需同時同步 進行取像，並將取得之畫面儲存後，交由多視角視訊内容 棟取暨服務管理平台2進行多視角視訊影片製作，由攝影 機取得之畫面進行影像壓縮處理，兩部攝影機畫面間隔的 每-角度晝面’利用虛擬拍攝角度合成(Rendering)技術來 產生視訊資料’㈣所有視訊資料適當分類儲存，以提供 用戶使用相關設備3進行多視角視訊畫面存取，詳細架構 圖如圖二所示。 凊參閱圖二，為本發明提供各 乃杈t、夕視角互動電視系統與方 法之架構示意圖。首先雹靼 而木5又數台攝影機組成一具有環場 拍攝環境之環繞攝影機楳 機Μ組11 ’母台攝影機需具備自動 校正及同步轉向之功能，自缸 Χ 可使攝影機自動將拍攝 201125358 中心對準至拍攝物體的正中央；同步轉向可在拍攝物體移 動時’將所有攝影機同時轉動鏡頭，重新調整拍攝中心令 拍攝物體位在晝面中央之位置。攝影機拍攝取得之畫面儲 存至數台取像伺服器1 2，再經多視角視訊處理器21壓縮 處理（Compression)及虛擬拍攝角度合成（Rendering)後， 存放於串流媒體伺服器22中。當用戶端利用硬體播放器 (STB)31點選多視角視訊時，點選的内容再透過串流媒體 伺服器22將影片内容傳送至用戶端。若用戶端使用二維 平面顯示器32 ’則可利用STB 31觀看晝面，並搭配環場 預覽以及時間凍結之功能切換到想觀賞的視角以及景 深；若用戶端使用三維立體顯示器’則不僅可利用Stb 3 i 觀i到不同的視角及景深’更可觀賞到立體影像。 請參閱圖三’為該多視角互動電視系統與方法中取像 伺服器及視訊處理運作方式示意圖。首先取像伺服器12 將由多台攝影機取得實際拍攝晝面’經分類储存後，將帶 有時間及空間資訊之檔案傳送至多視角視訊處理器， 此設備使用Rendering技術’利用現有之實際拍攝畫面產 生出虛擬拍攝角度之電腦模擬畫面。取像伺服器丨2所具 備之功此包含控制多部攝影機同時同步進行拍攝以及將 多視角影音檔案分類儲存。多視角視訊處理器21所具備 之功能包含壓縮多視角影音檔案及Rendering演算法。 凊參閱圖四，為該多視角互動電視系統與方法中用戶 201125358 端操作方式示意圖。㈣媒體飼服器22由多視角視訊處 ㈣2!接收實際拍攝晝面以及電腦模擬晝面，並根據用 戶端之操作以串流(Streaming)的方式提供相對應之影音 棺案用戶利用硬體播放器（STB) 31選擇欲觀賞之視角 及景深或是選擇環場預覽以及時間康結的模式，透過遙控 器錢端服務平台互動。硬體播放器以所具備之功能包 含提供使用者向左或向右切換視角及向前或向後切換景 深’並提供選擇環場預覽或時m模式，以及提供使用 者切換頻道、控制音量、快轉及倒轉等基本控制。硬體播 放器”可接收由串流媒體伺服器22發送之多視角視訊訊 號，並將内容料至二維或三維顯以，且可接收搖控器 控：訊號，將用戶的切換指令透過網路傳送至串流媒體飼 服益22，以實踐多視角互動電視服務系統。 本發明所提供之多視角互動電視系統與方法技術，具 有下列之優點： 〃 可提供更多的觀賞 1.多部攝影機同時同步拍攝技術 視角以及景深； 2. 提供播放令即時切換角度功能； 3. 提供環場預覽（View_Sweeping)功能； 4. 應用Rendering技術令轉換過程更平滑； 5. 利用現有之STB架構作為播放器，配合自行開發 之視訊伺服器’提供多視角互動電視系統與方法 201125358 6.易整合在未來的31)環境中。 上列詳細說明係針對本發明之一可行實施例之具體 說明’惟泫實施例並非用以限制本發明之專利範圍，凡未 脫離本發明技藝精神所為之等效實施或變更，均應包含於 本案之專利範圍中。 綜上所述’本案不但在技術思想上確屬創新，並能較 習知方法增進上述多項功效，應已充分符合新穎性及進步 性之法定發明專利要件，爰依法提出申請，懇請貴局核 准本件發明專利申請案，以勵發明，至感德便。 【圖式簡單說明】 請參閱有關本發明之詳細說明及其附圖，將可進—步 瞭解本發明之技術内容及其目的功效；有關附圖為： 圖一為本發明多視角互動電視系統與方法之運作方 式示意圖； 圖二為多視角互動電視系統與方法之系統架構圖· 圖三為多視角互動電視系統與方法中取像词 視訊處理運作方式示意圖；以及 圖四為多視角互動電視系統與方法中用卢 %禪作方 式示意圖。 【主要元件符號說明】 1 多視角視訊服務内容提供者 2 多視角視訊内容擷取暨服務管理平台 9 201125358 3 用戶及相關使用設備 11 環繞攝影機模組 12 取像伺服器 21 多視角視訊處理器 22 串流媒體伺服器 3 1 硬體播放器 32 電視或平面顯示器The Server processes the user's request in real time, and applies multiple control modes to the STB. Compared with the previous video service technology, the multi-view interactive TV system and method technology can not only combine the previous service technologies, such as fast turn and reverse, on the STB, but also provide users with more diversified mode of staying. For example, switching of angle of view, switching of depth of field, VieW-SweePing, and Time Frozen, etc., and the number of channels providing services does not increase with the number of cameras. The content is the same as the traditional service mode. The user can view the same scene from different angles. Each perspective of each moment does exist in the database and is provided through the streaming media server. Can be accessed at any time. The user can use the remote control of the STB to convert the angle you want to watch. In the process of switching the viewing angle, you can use the related technology to achieve smoother switching. The depth of field is switched so that the user can further zoom the depth of the scene under the same viewing angle, and zoom in (Z〇〇m 〇ut) to capture a large field of view, zoom in (z〇〇 When you look at m In), you can see more details. The function of switching the depth of field can also be performed by the remote control of the STB. In the process of switching, the Interp〇lati〇n related technology can achieve a smoother zooming effect. 201125358 View-Sweeping provides users with a quick preview of each switchable view and depth of field through the STB's remote control. While previewing, the multimedia can be still or maintain playback status, and the playback status can be normal. , fast forward or reverse. Time Fr〇zen provides the user with the remote control of the STB remote control to switch between different viewing angles and depth of field, or to perform a ring preview. [Embodiment] Please refer to FIG. 1 for a schematic diagram of the operation mode of the multi-view interactive television system and method. Multi-view video service content provider 丨 centering on the subject, while erecting multiple cameras into a surrounding shooting environment 'and simultaneously aiming at the object to be photographed', the camera needs to simultaneously acquire images and store the acquired images. After that, the multi-view video content acquisition and service management platform 2 is used for multi-view video production, and the image obtained by the camera is subjected to image compression processing, and each camera angle interval of each camera is synthesized using a virtual shooting angle ( Rendering) technology to generate video data '(4) All video data is properly classified and stored to provide users with multi-view video access using related equipment 3. The detailed architecture is shown in Figure 2. Referring to FIG. 2, a schematic diagram of the architecture of the interactive television system and method for each of the present inventions is provided. Firstly, the 5 木 5 and several cameras form a surround camera with a ring shooting environment. The 'home camera needs to have the function of automatic correction and synchronous steering. The self-cylinder allows the camera to automatically shoot the center of 201125358. Align to the center of the subject; Synchro Steering can 'turn all the cameras at the same time as the subject moves, re-adjust the center of the shot so that the subject is in the center of the face. The image captured by the camera is stored in a plurality of image capturing servers 12, and is subjected to compression processing (Compression) and virtual shooting angle synthesis by the multi-view video processor 21, and then stored in the streaming media server 22. When the client uses the hardware player (STB) 31 to select multi-view video, the selected content is transmitted to the client through the streaming media server 22. If the user uses the 2D flat panel display 32 ', you can use the STB 31 to view the face, and use the ring field preview and time freeze function to switch to the viewing angle and depth of field you want to watch; if the user uses the 3D display, you can not only use Stb 3 i view i to different perspectives and depth of field' to view stereoscopic images. Please refer to Figure 3 for a schematic diagram of the operation mode of the image capture server and video processing in the multi-view interactive television system and method. First, the image capture server 12 will acquire the actual image taken by multiple cameras. After being classified and stored, the file with time and space information will be transmitted to the multi-view video processor. This device uses Rendering technology to generate the existing actual image. A computer simulation of the virtual shooting angle. The image server 丨2 has the functions of controlling multiple cameras to simultaneously capture and multi-view video files. The multi-view video processor 21 has functions including a compressed multi-view video file and a rendering algorithm.凊 Refer to Figure 4, which is a schematic diagram of the operation mode of the user 201125358 in the multi-view interactive television system and method. (4) The media feeding device 22 receives the actual shooting surface and the computer simulation surface by the multi-view video station (4) 2! and provides corresponding audio and video files according to the operation of the user terminal, and the user uses the hardware to play. (STB) 31 Select the viewing angle and depth of field to be viewed or select the ring field preview and time Kangjie mode to interact through the remote control platform. The hardware player includes functions to provide the user to switch the viewing angle to the left or right and to switch the depth of field forward or backward' and to provide a selection of the ring preview or the m mode, and to provide the user to switch channels, control the volume, and quickly Basic control such as turning and reversing. The hardware player can receive the multi-view video signal sent by the streaming media server 22, and feed the content into two-dimensional or three-dimensional display, and can receive the remote control: the signal, and the user's switching instruction is transmitted through the network. The road is transmitted to the streaming media feed service 22 to practice the multi-view interactive television service system. The multi-view interactive television system and method technology provided by the invention has the following advantages: 〃 can provide more viewings 1. more The camera simultaneously synchronizes the technical viewing angle and depth of field; 2. Provides playback to instantly switch the angle function; 3. Provides the View_Sweeping function; 4. Applys the Rendering technology to make the conversion process smoother; 5. Use the existing STB architecture as the playback , in conjunction with a self-developed video server, provides a multi-view interactive television system and method 201125358 6. Easy integration in the future 31) environment. The above detailed description is specific to one of the possible embodiments of the present invention. The embodiments are not intended to limit the scope of the invention, and the equivalents and modifications may be made without departing from the spirit of the invention. It is included in the patent scope of this case. In summary, the case is not only innovative in terms of technical thinking, but also able to enhance the above-mentioned multiple functions more than the conventional methods, and should fully comply with the statutory invention patent requirements of novelty and progressiveness. Applying in accordance with the law, I urge you to approve the invention patent application, in order to invent the invention, to the sense of convenience. [Simplified description of the drawings] Please refer to the detailed description of the present invention and its drawings, which will further understand the present invention. The technical content and its purpose effect; the related drawings are: Figure 1 is a schematic diagram of the operation mode of the multi-view interactive television system and method of the present invention; Figure 2 is a system architecture diagram of the multi-view interactive television system and method; Schematic diagram of the video processing operation mode of the interactive video system and method; and Figure 4 is a schematic diagram of the multi-view interactive television system and method using the Lu% Zen method. [Main component symbol description] 1 Multi-view video service content provider 2 Multi-view video content capture and service management platform 9 201125358 3 Users and related equipment 11 Surround photography Imaging server module 12 over 21 video processor 22 Angle streaming media server 31 hardware player, television or flat panel displays 32
1010