TWI605712B - Interactive media systems - Google Patents
Interactive media systems Download PDFInfo
- Publication number
- TWI605712B TWI605712B TW101150114A TW101150114A TWI605712B TW I605712 B TWI605712 B TW I605712B TW 101150114 A TW101150114 A TW 101150114A TW 101150114 A TW101150114 A TW 101150114A TW I605712 B TWI605712 B TW I605712B
- Authority
- TW
- Taiwan
- Prior art keywords
- icon
- content
- face
- viewer
- video monitor
- Prior art date
Links
- 230000002452 interceptive effect Effects 0.000 title description 18
- 230000001815 facial effect Effects 0.000 claims description 38
- 230000014509 gene expression Effects 0.000 claims description 21
- 238000000034 method Methods 0.000 claims description 11
- 238000001514 detection method Methods 0.000 description 13
- 238000000605 extraction Methods 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 7
- 230000008921 facial expression Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 210000000887 face Anatomy 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241000086550 Dinosauria Species 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000005375 photometry Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000001429 visible spectrum Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/441—Acquiring end-user identification, e.g. using personal code sent by the remote control or by inserting a card
- H04N21/4415—Acquiring end-user identification, e.g. using personal code sent by the remote control or by inserting a card using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/4223—Cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
- H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/4508—Management of client data or end-user data
- H04N21/4532—Management of client data or end-user data involving end-user characteristics, e.g. viewer profile, preferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2101/00—Still video cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2209/00—Details of colour television systems
- H04N2209/04—Picture signal generators
- H04N2209/041—Picture signal generators using solid-state devices
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Biomedical Technology (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Marketing (AREA)
- Business, Economics & Management (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Description
本揭示案涉及互動式媒體，並且更具體而言涉及互動式媒體之指標以及其用途。 The present disclosure relates to interactive media and, more particularly, to indicators of interactive media and their uses.
傳統上，電視為基於電視列表或在各個頻道中「瀏覽」來選擇頻道/內容的媒體。然而，出現被設計成增強觀看者體驗的新的服務。舉例而言，列印的電視列表現在可由網際網路驅動之應用程式來代替。可呈現在每個頻道上播送之內容的視覺概覽。 Traditionally, televisions are media that select channels/content based on a list of televisions or "browsing" across channels. However, there are new services designed to enhance the viewer experience. For example, a printed TV list can now be replaced by an Internet-driven application. A visual overview of the content broadcast on each channel can be presented.
依據本發明之一實施例，係特地提出一種系統，其包含：一攝影機，該攝影機係組配成可捕獲一影像；一視訊監視器，該視訊監視器係組配成可顯示至少內容及圖標；以及一或多個儲存媒體，該一或多個儲存媒體係個別地或組合地具有儲存在其上之指令，該等指令在由一或多個處理器執行時，導致以下操作，包含：捕獲一影像；偵測該影像中之至少一面部；決定對應於該至少一面部之一 身份及表情；基於該對應表情來產生該至少一面部之一圖標；以及在該視訊監視器上顯示該圖標。 According to an embodiment of the present invention, a system is specifically provided, comprising: a camera, the camera is configured to capture an image; and a video monitor is configured to display at least content and icons. And one or more storage media having instructions stored thereon, individually or in combination, that when executed by one or more processors, result in the following operations, including: Capturing an image; detecting at least one face in the image; determining to correspond to one of the at least one face An identity and an expression; generating an icon of the at least one face based on the corresponding expression; and displaying the icon on the video monitor.
100、100A、100B、100C、100D、100E、100F、100n‧‧‧系統 100, 100A, 100B, 100C, 100D, 100E, 100F, 100n‧‧‧ systems
102‧‧‧攝影機 102‧‧‧ camera
104‧‧‧面部偵測/追蹤模組 104‧‧‧Face Detection/Tracking Module
106‧‧‧觀看者識別模組 106‧‧‧ Viewer Identification Module
108‧‧‧面部特徵提取模組 108‧‧‧Face feature extraction module
110‧‧‧圖標產生模組 110‧‧‧ icon generation module
112‧‧‧圖標增強模組 112‧‧‧ icon enhancement module
114‧‧‧外部裝置 114‧‧‧External devices
116‧‧‧圖標覆蓋模組 116‧‧‧ icon overlay module
118‧‧‧觀看者資料庫 118‧‧‧ Viewer Database
120‧‧‧內容資料庫 120‧‧‧Content database
122‧‧‧觀看者選擇模組 122‧‧‧ Viewer selection module
124‧‧‧內容管理模組 124‧‧‧Content Management Module
126、126'、126"‧‧‧視訊監視器 126, 126', 126" ‧ ‧ video monitor
128、128'‧‧‧內容 128, 128'‧‧‧ contents
130、130'、130"‧‧‧圖標 130, 130', 130" ‧ ‧ icons
132、132'、134、134’‧‧‧指標 132, 132', 134, 134’ ‧ ‧ indicators
136、136'‧‧‧選擇框 136, 136'‧‧‧Selection box
138‧‧‧遙控器 138‧‧‧Remote control
200~208、400~410、600~606‧‧‧操作 200~208, 400~410, 600~606‧‧‧ operations
500‧‧‧網路 500‧‧‧Network
隨著以下詳細說明進行，並且參考附圖，所請求標的之各種實施例的特徵及優勢變得顯而易知，在該等附圖中相同數字標示相同部分，其中：圖1例示根據本揭示案之各種實施例的實例系統；圖2A為實例圖標產生及顯示操作之流程圖；圖2B為實例觀看者識別操作之流程圖；圖3例示根據本揭示案之一實施例的互動式媒體系統之實例顯示影像；圖4為對應於圖3之媒體系統實施例的實例操作之流程圖。圖5例示根據本揭示案之另一實施例的互動式媒體系統之另一實例顯示影像；以及圖6為對應於圖5之媒體系統實施例的實例操作之流程圖。 The features and advantages of the various embodiments of the present invention will become more apparent from the detailed description of the claims. Example system of various embodiments of the present invention; FIG. 2A is a flowchart of an example icon generation and display operation; FIG. 2B is a flowchart of an example viewer identification operation; FIG. 3 illustrates an interactive media system according to an embodiment of the present disclosure An example shows an image; Figure 4 is a flow diagram of an example operation corresponding to the media system embodiment of Figure 3. 5 illustrates another example display image of an interactive media system in accordance with another embodiment of the present disclosure; and FIG. 6 is a flow diagram of example operations corresponding to the media system embodiment of FIG. 5.
雖然以下詳細說明參考例示性實施例來進行，但是其許多替代方案、修改及變化形式為熟習此項技術者顯而易知。 While the following detailed description is made with reference to the exemplary embodiments of the present invention, many alternatives, modifications and variations are apparent to those skilled in the art.
本揭示案總體上針對互動式媒體系統(及方法)。在一實施例中，提供互動式媒體系統，其被組配成捕 獲視訊監視器之觀看者的視訊影像。互動式媒體系統亦被組配成偵測影像中之觀看者面部並且識別觀看者。識別觀看者可包括將影像中偵測之面部特徵與觀看者設定檔之資料庫比較。互動式媒體系統可基於偵測之面部及特徵來產生圖標並且將圖標顯示於視訊監視器上。在一實施例中，圖標可為卡通或草圖。另外，圖標可與一或多個指標一起顯示於視訊監視器上。指標可識別例如與對應圖標相關之觀看者當前觀看之內容。 The present disclosure is generally directed to an interactive media system (and method). In an embodiment, an interactive media system is provided that is configured to capture Video image of the viewer of the video monitor. The interactive media system is also configured to detect the viewer's face in the image and identify the viewer. Identifying the viewer may include comparing the detected facial features in the image to a database of viewer profiles. The interactive media system can generate icons based on the detected faces and features and display the icons on the video monitor. In an embodiment, the icon can be a cartoon or a sketch. In addition, the icon can be displayed on the video monitor along with one or more indicators. The indicator can identify, for example, the content currently viewed by the viewer associated with the corresponding icon.
在一實施例中顯示在視訊監視器上之內容藉由選擇對應於本地觀看者之圖標來控制。在如上所述識別觀看者並且產生圖標之後，至少一圖標被選擇為主要觀看者。互動式媒體系統決定主要觀看者之內容偏好。舉例而言，互動式媒體系統可存取觀看者之資料庫以便決定偏好。然後，互動式媒體系統基於主要觀看者之偏好來選擇內容。選擇內容可包括將主要觀看者之偏好與可獲得內容之資料庫比較。然後，互動式媒體系統可將所選擇的內容顯示於視訊監視器上。 The content displayed on the video monitor in one embodiment is controlled by selecting an icon corresponding to the local viewer. After the viewer is identified and an icon is generated as described above, at least one icon is selected as the primary viewer. The interactive media system determines the content preferences of the primary viewers. For example, an interactive media system can access a viewer's database to determine preferences. The interactive media system then selects content based on the preferences of the primary viewer. Selecting content may include comparing a primary viewer's preferences to a database of available content. The interactive media system can then display the selected content on the video monitor.
在一實施例中顯示在視訊監視器上之內容藉由選擇對應於遠程觀看者之圖標來控制。本地觀看者可與多個遠程觀看者一起關聯至定義組中。組中之每個觀看者可加以識別，並且可產生並且顯示每個觀看者之圖標。此外，互動式媒體系統可決定組中之每個用戶收看之內容，並且可顯示組中之所有其他觀看者的圖標。可在每個圖標附近顯示指標，其識別與圖標相關之觀看者正在觀看之內容。 然後，互動式媒體系統可允許選擇圖標，導致與圖標相關之內容顯示於視訊監視器上。 The content displayed on the video monitor in one embodiment is controlled by selecting an icon corresponding to the remote viewer. Local viewers can be associated with a plurality of remote viewers into a defined group. Each viewer in the group can be identified and an icon for each viewer can be generated and displayed. In addition, the interactive media system can determine what each user in the group is watching and can display icons for all other viewers in the group. An indicator can be displayed near each icon that identifies the content that the viewer associated with the icon is viewing. The interactive media system can then allow selection of an icon, causing the content associated with the icon to be displayed on the video monitor.
圖1例示與本揭示案之各種實施例一致的系統100。系統100總體上被組配成偵測/追蹤視訊監視器之觀看者、識別觀看者、產生每個觀看者之圖標、將圖標顯示於視訊監視器上，並且將內容顯示於視訊監視器上。系統100包括攝影機102。攝影機102可為用於捕獲代表包括一或多個人之環境的數位影像之任何裝置，並且可具有用於對如本文描述之環境中之一或多個人進行面部分析的足夠解析度。舉例而言，攝影機102可包括靜物攝影機(例如，被組配成捕獲靜止照片之攝影機)或視訊攝影機(例如，被組配成在多個訊框中捕獲多個移動影像之攝影機)。攝影機102可被組配成藉由可見光譜之光或藉由電磁波譜之其他部分，不限於紅外光譜、紫外光譜等來操作。可將攝影機102併入系統100之另一個組件(例如，TV 126)中或可為被組配成經由有線或無線通訊與至少面部偵測模組104通訊之獨立組件。攝影機102可包括例如網路攝影機(其可與個人電腦及/或視訊監視器相關)、手持裝置攝影機(例如，行動電話攝影機、智慧型手機攝影機(例如，與iPhone®、基於Android®之手機，Blackberry®、基於Palm®之手機、基於Symbian®之手機等相關之攝影機))、膝上型電腦攝影機、平板電腦(例如，但是不限於iPad®、Galaxy Tab®及其類似物)等。 FIG. 1 illustrates a system 100 consistent with various embodiments of the present disclosure. System 100 is generally configured to detect/track viewers of the video monitor, identify viewers, generate icons for each viewer, display icons on the video monitor, and display the content on the video monitor. System 100 includes a camera 102. Camera 102 may be any device for capturing digital images representing an environment including one or more persons, and may have sufficient resolution for facial analysis of one or more persons in an environment as described herein. For example, camera 102 may include a still camera (eg, a camera that is configured to capture still photos) or a video camera (eg, a camera that is configured to capture multiple moving images in multiple frames). Camera 102 can be configured to operate by light in the visible spectrum or by other portions of the electromagnetic spectrum, not limited to infrared, ultraviolet, and the like. Camera 102 may be incorporated into another component of system 100 (e.g., TV 126) or may be a separate component that is configured to communicate with at least face detection module 104 via wired or wireless communication. Camera 102 may include, for example, a webcam (which may be associated with a personal computer and/or video monitor), a handheld camera (eg, a mobile phone camera, a smart phone camera (eg, with an iPhone®, an Android® based mobile phone, Blackberry®, Palm®-based phones, Symbian®-based phones and other related cameras), laptop cameras, tablets (such as, but not limited to, iPad®, Galaxy Tab®, and the like).
面部偵測/追蹤模組104被組配成識別攝影機102提供之影像中之面部及/或面部區域。舉例而言，面部偵測 /追蹤模組104可包括定製、專屬、已知及/或以後開發之面部識別碼(或指令集)、硬體及/或韌體，其總體上經過妥當定義並且可操作以便接收標準格式影像(例如，但是不限於RGB彩色影像)並且至少在一定程度上識別影像中之面部。面部偵測/追蹤模組104亦可被組配成經由一系列影像(例如，24訊框/秒下之視訊訊框)來追蹤偵測之面部。可由面部偵測/追蹤模組104使用之已知追蹤系統可包括粒子濾波、均值漂移、卡爾曼(Kalman)濾波等，其每一者可利用邊緣分析、平方差之和分析、特徵點分析、直方圖分析、皮膚色調分析等。觀看者識別模組106被組配成決定與面部相關之身份，並且可包括定製、專屬、已知及/或以後開發之面部特徵識別碼(或指令集)，其總體上經過妥當定義並且可操作以便自攝影機102接收標準格式影像(例如，但是不限於RGB彩色影像)並且至少在一定程度上識別影像中之一或多個面部特徵。此等已知面部特徵系統包括但不限於科羅拉多州州立大學(Colorado State University)之CSU面部識別評估系統(CSU Face Identification Evaluation System)。 The face detection/tracking module 104 is configured to identify facial and/or facial regions in the images provided by the camera 102. For example, face detection / Tracking module 104 may include custom, proprietary, known and/or later developed facial identification codes (or sets of instructions), hardware and/or firmware that are generally well defined and operable to receive standard formats An image (such as, but not limited to, an RGB color image) and at least to some extent identifies the face in the image. The face detection/tracking module 104 can also be configured to track the detected face via a series of images (eg, a frame of 24 frames per second). The known tracking system that can be used by the face detection/tracking module 104 can include particle filtering, mean shift, Kalman filtering, etc., each of which can utilize edge analysis, sum of squared differences, feature point analysis, Histogram analysis, skin tone analysis, etc. The viewer identification module 106 is configured to determine the identity associated with the face, and may include custom, proprietary, known, and/or later developed facial feature identifiers (or sets of instructions) that are generally well defined and It is operable to receive a standard format image (e.g., but not limited to, an RGB color image) from the camera 102 and to identify at least to some extent one or more facial features in the image. Such known facial features systems include, but are not limited to, the CSU Face Identification Evaluation System of Colorado State University.
觀看者識別模組106亦可包括定製、專屬、已知及/或以後開發之面部識別碼(或指令集)，者通常經過妥當定義並且可操作以便將面部模式與儲存於資料庫中之對應面部模式匹配。舉例而言，觀看者識別模組106可被組配成將偵測之面部模式與先前儲存於觀看者資料庫118中之面部模式比較。觀看者資料庫118可包含包括用戶之內容偏好的帳戶或記錄。另外，觀看者資料庫118可本地或遠程(例 如，經由網際網路)來存取，並且可與現有在線互動系統(例如，Facebook、MySpace、Google+、Linked In、Yahoo等)相關或可專屬地用於互動式媒體系統。觀看者識別模組106可利用幾何分析(其著眼於有區別的特徵)及/或光度分析(其為自影像中提取出值並且將值與模板比較以便消除偏差的統計方法)來對模式進行比較。一些面部識別技術包括但不限於藉由特徵臉(eigenfaces)(及其衍生技術)來進行之主要成分分析、藉由費歇爾臉(fisherface)(及其衍生技術)來進行之線性判別分析、彈性束圖匹配(及其衍生技術)、隱藏馬爾可夫(Markov)模型(及其衍生技術)及神經元激發動態連結匹配。 The viewer identification module 106 can also include custom, proprietary, known, and/or later developed facial identification codes (or sets of instructions) that are generally well defined and operable to store facial patterns in a database. Corresponding to face pattern matching. For example, the viewer identification module 106 can be configured to compare the detected facial pattern to a facial pattern previously stored in the viewer database 118. The viewer database 118 may contain an account or record that includes the user's content preferences. Additionally, the viewer database 118 can be local or remote (eg, For example, accessed via the Internet, and can be associated with existing online interactive systems (eg, Facebook, MySpace, Google+, LinkedIn, Yahoo, etc.) or can be used exclusively for interactive media systems. The viewer identification module 106 can utilize the geometric analysis (which focuses on differentiated features) and/or photometric analysis (which is a statistical method that extracts values from the image and compares the values to the template to eliminate bias). Comparison. Some facial recognition techniques include, but are not limited to, principal component analysis by eigenfaces (and its derivatives), linear discriminant analysis by Fisherface (and its derivatives), Elastic beam map matching (and its derivatives), hidden Markov models (and its derivatives), and neuron-stimulated dynamic link matching.
面部特徵提取模組108被組配成辨識由面部偵測 /追蹤模組104偵測之面部中之各種特徵(例如，表情)。在識別面部表情(例如，識別是否先前偵測之面部為快樂的、憂愁的、微笑的、皺眉的、驚訝的、激動的等)中，面部特徵提取模組108可進一步包括定製、專屬、已知及/或以後開發之面部表情偵測及/或識別碼(或指令集)，其總體上經過妥當定義並且可操作以便提取及/或識別面部之面部表情。舉例而言，面部特徵識別模組108可決定面部特徵(例如，眼睛、嘴、臉頰、牙齒等)之大小及/或位置並且將此等面部特徵與面部特徵資料庫比較，該資料庫包括具有對應面部特徵分類(例如，微笑的、皺眉的、激動的、憂愁的等)的多個樣本面部特徵。 The facial feature extraction module 108 is configured to recognize the face detection / Tracking module 104 detects various features (eg, emoticons) in the face. In identifying facial expressions (eg, identifying whether the previously detected face is happy, sorrowful, smiling, frowning, surprised, excited, etc.), facial feature extraction module 108 may further include customized, exclusive, Facial expression detection and/or identification codes (or sets of instructions) that are known and/or developed in the future are generally well defined and operable to extract and/or identify facial expressions of the face. For example, facial feature recognition module 108 can determine the size and/or location of facial features (eg, eyes, mouth, cheeks, teeth, etc.) and compare such facial features to a facial feature database, the database including A plurality of sample facial features corresponding to facial feature classification (eg, smiling, frowning, excited, sad, etc.).
圖標產生模組110被組配成將由面部偵測模組 104所偵測，並且由面部特徵提取模組108所分析之面部影像轉換成顯示於視訊監視器126上之圖標130。舉例而言，圖標產生模組110可進一步包括定製、專屬、已知及/或以後開發之影像處理碼(或指令集)，其總體上經過妥當定義並且可操作以便將攝影機102捕獲之即時影像轉換成其他格式。在一實施例中，圖標產生模組110可將面部影像轉換成卡通或草圖以便作為圖標130使用。如本文提及，卡通可定義為基於實際對象之奇特影像。舉例而言，卡通可誇張實際對象之一或多個特徵。在與實際對象比較時，一些卡通可包括例如有限清晰度及/或有限調色板顯現(例如，四色顯現、八色顯現等)。如本文提及，草圖可定義為逼真地類似於實際對象的粗糙影像。例如，草圖可包括實際對象在單一顏色(例如，在白色背景中之黑色)中之素描表現。舉例而言，由面部偵測模組104識別之面部影像可被修剪，並且可藉由影像處理來產生卡通/草圖樣圖標，該影像處理使用線圖提取、變形、基於實例之面部草圖產生伴以非參數取樣、面部表現及畫草圖之語法模型等。或者，由面部特徵提取模組108識別之面部特徵(例如，特徵及表情)可應用於預先存在之卡通圖標模型以便產生呈卡通形式之面部表現。與更逼真之圖形影像或2D/3D化身表現相比，使用卡通/草圖樣圖標之優勢為與2D/3D圖形模型建構相比，卡通/草圖更穩健並且更容易產生/更新。另外，對應於圖標之觀看者之真實身份可保持隱藏，使得觀看者可在公眾論壇中匿名地操作並且與先前未知的觀看者互動而不擔憂其實際身份為 人所知。由於在觀看視訊監視器126時，觀看者面部位置及表情不斷地變化，圖標130可為動態的以便代表觀看者最近位置及表情。在實務中，圖標130可被更新以便代表觀看者之當前表情，此更新的方式為即時(例如，如攝影機102所提供，逐個訊框地)、在一定時間間隔下，例如每十秒更新圖標130，或永不更新(例如，圖標130自首次由圖標產生模組110產生時起保持無變化)。更新圖標130之時間間隔可視各種因素而定，例如攝影機102之能力(例如，速度)、系統100中可獲得之圖形處理能力等。 The icon generation module 110 is configured to be used by the face detection module The detected facial image is converted into an icon 130 displayed on the video monitor 126 by the facial feature extraction module 108. For example, the icon generation module 110 can further include custom, proprietary, known, and/or later developed image processing codes (or sets of instructions) that are generally well defined and operable to capture the instants of the camera 102. Convert images to other formats. In an embodiment, the icon generation module 110 can convert the facial image into a cartoon or a sketch for use as the icon 130. As mentioned herein, a cartoon can be defined as a fancy image based on an actual object. For example, a cartoon can exaggerate one or more features of an actual object. Some cartoons may include, for example, limited definition and/or limited palette representation (eg, four-color representation, eight-color representation, etc.) when compared to actual objects. As mentioned herein, a sketch can be defined as a coarse image that is realistically similar to an actual object. For example, a sketch can include a sketch representation of an actual object in a single color (eg, black in a white background). For example, the facial image recognized by the face detection module 104 can be trimmed, and a cartoon/sketch icon can be generated by image processing, which uses line graph extraction, deformation, and instance-based facial sketch generation. Non-parametric sampling, facial expressions, and grammatical models for sketching. Alternatively, facial features (eg, features and expressions) identified by facial feature extraction module 108 may be applied to a pre-existing cartoon icon model to produce a facial representation in cartoon form. The advantage of using cartoon/sketch icons is that the cartoon/sketch is more robust and easier to generate/update than the 2D/3D graphics model construction compared to more realistic graphics or 2D/3D avatar performance. In addition, the true identity of the viewer corresponding to the icon can remain hidden so that the viewer can operate anonymously in the public forum and interact with previously unknown viewers without fear of their actual identity being Known by everyone. Since the viewer's face position and expression constantly change while viewing the video monitor 126, the icon 130 can be dynamic to represent the viewer's recent location and expression. In practice, the icon 130 can be updated to represent the current expression of the viewer, this update being instant (eg, as provided by the camera 102, frame by frame), updating the icon at certain time intervals, such as every ten seconds. 130, or never updated (eg, icon 130 remains unchanged from the time it was first generated by icon generation module 110). The time interval for updating icon 130 may depend on various factors, such as the capabilities of camera 102 (e.g., speed), the graphics processing capabilities available in system 100, and the like.
圖標增強模組112可被組配成改變圖標130之外觀。舉例而言，觀看者可能認為圖標產生模組110所產生之圖標130太逼真或不夠逼真。或者，觀看者可希望圖標130看似異想天開的或愚蠢的。在一實施例中，圖標130可由觀看者手動地改變。舉例而言，外部裝置114可為桌上型PC、膝上型PC、平板電腦、蜂巢式手持話機等。外部裝置114可經由本地有線或無線通訊、經由在系統100中本地託管(例如，使用系統100中之伺服器之IP位址)之網路服務或經由在網際網路上之其他位置處託管的網路服務來存取系統100。舉例而言，網路服務可基於儲存於觀看者資料庫118中之觀看者設定檔來提供對於圖標130之存取。網路服務可向觀看者提供允許觀看者觀看並且編輯圖標130之介面。然後，觀看者可根據觀看者之偏好來改變圖標130之各種態樣(例如，眼睛、鼻子、嘴、頭髮等)以便使其更薄、更厚、更誇張等。 The icon enhancement module 112 can be configured to change the appearance of the icon 130. For example, the viewer may think that the icon 130 generated by the icon generation module 110 is too realistic or not realistic enough. Alternatively, the viewer may wish the icon 130 to appear whimsical or stupid. In an embodiment, the icon 130 can be manually changed by the viewer. For example, the external device 114 can be a desktop PC, a laptop PC, a tablet, a cellular handset, or the like. The external device 114 can be hosted via local wired or wireless communication, via a network service hosted locally in the system 100 (e.g., using an IP address of a server in the system 100) or via a network hosted at other locations on the Internet. The road service accesses the system 100. For example, the web service can provide access to the icon 130 based on the viewer profile stored in the viewer repository 118. The web service can provide the viewer with an interface that allows the viewer to view and edit the icon 130. The viewer can then change various aspects of the icon 130 (eg, eyes, nose, mouth, hair, etc.) to make it thinner, thicker, more exaggerated, etc., depending on the viewer's preferences.
圖標覆蓋模組116可被組配成將圖標130顯示於視訊監視器126上之內容128之上。在一實施例中，圖標130可被組配成覆蓋內容128以使得觀看者可同時觀察兩者。圖標130可被配置在視訊監視器126之顯示器上之不同位置處，並且圖標130之位置可組配成不妨礙觀看內容128。當前收看視訊監視器126上之內容128的所有觀看者之圖標130可顯示於內容128之上。具體而言，可產生實體上存在並且收看視訊監視器126之所有觀看者的圖標130。收看其自己TVs的所感興趣的其他人(例如，朋友、親戚、業務同事等)之圖標130亦可顯示在視訊監視器126上之內容128之上。此可提醒觀看視訊監視器126上之內容128的觀看者所感興趣的其他人亦正在收看其自己的視訊顯示器126。在一實施例中，指標132及134亦可顯示於圖標130附近。指標132及134可與TV操作之特徵有關。舉例而言，指標132可識別對應於相鄰圖標130之觀看者正在觀看之頻道，並且指標134可識別所觀看之具體節目。結果，看到圖標130與指標132及134之觀看者可以知道另一個觀看者當前收看之頻道及節目。 The icon overlay module 116 can be configured to display the icon 130 over the content 128 on the video monitor 126. In an embodiment, the icon 130 can be grouped to cover the content 128 such that the viewer can view both at the same time. The icons 130 can be configured at different locations on the display of the video monitor 126, and the locations of the icons 130 can be grouped to not interfere with viewing the content 128. All viewer icons 130 currently viewing content 128 on video monitor 126 may be displayed on content 128. In particular, an icon 130 that exists on the entity and that views all viewers of the video monitor 126 can be generated. Icons 130 of other people (e.g., friends, relatives, business colleagues, etc.) who are interested in watching their own TVs may also be displayed on content 128 on video monitor 126. This may alert other people interested in viewing the content 128 on the video monitor 126 to watch their own video display 126 as well. In an embodiment, indicators 132 and 134 may also be displayed adjacent to icon 130. Indicators 132 and 134 may be related to features of TV operation. For example, the indicator 132 can identify a channel that is being viewed by a viewer corresponding to the adjacent icon 130, and the indicator 134 can identify the particular program being viewed. As a result, viewers who see icon 130 and indicators 132 and 134 can know the channel and program currently being viewed by another viewer.
觀看者選擇模組122可被組配成提供輸入至內容管理模組124以便控制視訊監視器126。觀看者選擇模組122可被組配成接收輸入(例如來自遙控器138)以便選擇顯示於視訊監視器126上之圖標130。舉例而言，觀看者可移動選擇框136以便選擇所顯示之圖標130。具體圖標130之選擇可導致觀看者選擇模組122接收來自觀看者資料庫122之觀看 者資訊，該資訊包括觀看者特徵及/或偏好。資訊可被提供至內容管理模組124，其可被組配成基於用戶設定檔自內容資料庫120選擇內容。內容資料庫120可包含關於可獲得內容之資訊，諸如但不限於網路及有線電視之當前直播排程、包括先前播出之網路及電纜節目之隨選節目、電影、遊戲等、可自網際網路下載的內容等。內容資料庫120亦可包括對應於可獲得內容之其他特徵資訊，如指示內容之年齡適合程度的分級等。 The viewer selection module 122 can be configured to provide input to the content management module 124 for controlling the video monitor 126. The viewer selection module 122 can be configured to receive input (e.g., from the remote control 138) to select an icon 130 that is displayed on the video monitor 126. For example, the viewer can move the selection box 136 to select the displayed icon 130. Selection of the particular icon 130 may cause the viewer selection module 122 to receive viewing from the viewer database 122 Information, which includes viewer characteristics and/or preferences. The information can be provided to a content management module 124 that can be configured to select content from the content repository 120 based on the user profile. The content repository 120 may contain information about available content, such as, but not limited to, current live schedules for the network and cable television, on-demand programming including previously broadcasted network and cable programming, movies, games, etc., available from Content downloaded from the Internet, etc. The content database 120 may also include other feature information corresponding to the available content, such as a rating indicating the age suitability of the content, and the like.
在系統100之操作期間，與由選擇框136標示的圖標130相關之觀看者可具有儲存於觀看者資料庫118中之設定檔，其指示觀看者為兒童(例如，低於一定年齡)。然後，觀看者選擇模組122可提供年齡資訊至內容管理模組124。內容管理模組124可被組配成存取內容資料庫120以便選擇適合於觀看者之年齡的內容128，並且同樣地限制不適合於觀看者之內容128。亦可基於在觀看者資料庫118中指示之觀看者偏好來播出某些類型之內容128(例如，卡通、新聞、直播體育、電影等)或某些主題之內容128(例如，恐龍、技術等)。將選擇框136移動至對應於另一個觀看者的圖標130可改變觀看者特徵/偏好，並且由此變更內容128。 During operation of system 100, a viewer associated with icon 130 indicated by selection box 136 may have a profile stored in viewer database 118 that indicates that the viewer is a child (eg, below a certain age). The viewer selection module 122 can then provide age information to the content management module 124. The content management module 124 can be configured to access the content repository 120 to select content 128 that is appropriate for the viewer's age, and likewise to limit content 128 that is not suitable for the viewer. Certain types of content 128 (eg, cartoons, news, live sports, movies, etc.) or content of certain topics 128 (eg, dinosaurs, technology) may also be broadcast based on viewer preferences indicated in the viewer database 118. Wait). Moving the selection box 136 to the icon 130 corresponding to another viewer may change the viewer characteristics/preferences and thereby alter the content 128.
在一實施例中內容管理模組122亦可被組配成基於觀看者組中之遠程觀看者來選擇內容128。舉例而言，除了他/她自己的圖標130以外，觀看視訊監視器126上之內容128的觀看者可看到對應於所感興趣的組其他觀看者(例如，朋友、親戚、業務同事等)的圖標130。觀看者組之成 員可儲存於觀看者資料庫118中之觀看者設定檔中。當觀看者藉由觀看者識別模組126來識別時，關於觀看者組之資訊可用於顯示對應於當前正在觀看其自己的視訊顯示器126(例如，在其自己的系統100中)的所有組成員的圖標130。指標132及134亦可相鄰於每個圖標130而顯示於內容128之上。指標132及134可通知觀看者每個組成員正在觀看之頻道及/或內容。在觀看圖標130與指標132及134後，觀看者可變得對當前正在由組成員中之一或多者觀看之內容感興趣。在一實施例中，觀看者可藉由啟用系統100中之跟隨功能來「跟隨」另一個觀看者正在收看之內容。跟隨功能可藉由基於碼之觸發(例如，顯示於內容128之上之選單、按鈕、選擇框等，其可使用遙控器128來選擇)或另一種類型之觸發(例如，遙控器138上之實體「跟隨」按鈕)來啟用。在一實施例中，跟隨功能可被組配成導致觀看者選擇模組122存取觀看者資料庫118以便獲得關於當前正在由對應於所選擇的圖標130之組成員觀看之內容的資訊。然後，此資訊可提供至內容管理模組124以便將內容128改變為相鄰於所選擇的圖標130的指標132及134所反映之內容。視系統100之實行方案而定，重複地觸發跟隨功能(例如，重複地按下遙控器138上之跟隨按鈕)可觸發不同操作。舉例而言，重複地按下跟隨按鈕可導致選擇框136自一個顯示圖標130移動下一個圖標，並且同樣地，根據當前選擇之圖標130來改變內容128。或者，重複地按下跟隨按鈕可導致內容128在其圖標130當前被選定之組成員之「最愛」 頻道或內容中遍歷。最愛頻道及/或節目可自觀看者資料庫118中之組成員之設定檔獲得。 In an embodiment, the content management module 122 can also be configured to select content 128 based on remote viewers in the viewer group. For example, in addition to his/her own icon 130, a viewer viewing content 128 on video monitor 126 may see other viewers (eg, friends, relatives, business colleagues, etc.) corresponding to the group of interest. Icon 130. Viewer group The member can be stored in the viewer profile in the viewer database 118. When the viewer is identified by the viewer identification module 126, information about the viewer group can be used to display all group members corresponding to the video display 126 (e.g., in its own system 100) that is currently viewing itself. Icon 130. Indicators 132 and 134 may also be displayed on content 128 adjacent to each icon 130. Indicators 132 and 134 can inform the viewer of the channel and/or content that each group member is watching. After viewing icon 130 and indicators 132 and 134, the viewer may become interested in content that is currently being viewed by one or more of the group members. In an embodiment, the viewer can "follow" the content that another viewer is watching by enabling the follow function in system 100. The follow function can be triggered by a code based trigger (eg, a menu displayed above content 128, a button, a selection box, etc., which can be selected using remote control 128) or another type of trigger (eg, on remote control 138) Entity "follow" button to enable. In an embodiment, the follow function can be configured to cause the viewer selection module 122 to access the viewer database 118 to obtain information about the content currently being viewed by the group members corresponding to the selected icon 130. This information can then be provided to the content management module 124 to change the content 128 to the content reflected by the indicators 132 and 134 adjacent to the selected icon 130. Depending on the implementation of system 100, repeatedly triggering the follow function (eg, repeatedly pressing the follow button on remote control 138) may trigger different operations. For example, repeatedly pressing the follow button may cause selection box 136 to move the next icon from one display icon 130, and as such, change content 128 based on currently selected icon 130. Alternatively, repeatedly pressing the follow button may result in the "favorite" of the content member 128 at the group member whose icon 130 is currently selected. Traversing in channels or content. Favorite channels and/or programs may be obtained from profiles of group members in the viewer database 118.
用於面部偵測及圖標產生之實例操作之流程圖在圖2A中例示。在操作200中可偵測影像中之至少一面部。舉例而言，系統100中之攝影機102可捕獲當前正在觀看視訊監視器126之觀看者的影像。然後，操作200中偵測之任何面部可在操作202中加以分析以便提取所偵測面部之特徵。舉例而言，面部特徵之提取可包括偵測可用於決定面部之身份及面部之表情(例如，快樂的、憂愁的、生氣的、驚訝的、厭煩的等)的特徵。在操作204中圖標可基於產生之面部特徵來產生，然後顯示於視訊監視器126上。舉例而言，可產生具有類似於所偵測的面部之特徵的卡通或草圖圖標，然後予以顯示。操作202及204可即時或基於一定時間間隔來持續循環以便將外觀圖標更新至與觀看者之當前表情類似。 A flowchart of an example operation for face detection and icon generation is illustrated in Figure 2A. At operation 200, at least one of the faces in the image can be detected. For example, camera 102 in system 100 can capture an image of a viewer currently viewing video monitor 126. Any of the faces detected in operation 200 can then be analyzed in operation 202 to extract features of the detected face. For example, the extraction of facial features can include detecting features that can be used to determine the identity of the face and facial expressions (eg, happy, sad, angry, astonished, annoying, etc.). The icon may be generated based on the generated facial features in operation 204 and then displayed on video monitor 126. For example, a cartoon or sketch icon having features similar to the detected face can be generated and then displayed. Operations 202 and 204 may continue to loop, either immediately or based on a certain time interval, to update the appearance icon to be similar to the current expression of the viewer.
用於面部偵測及觀看者識別之實例操作之流程圖在圖2B中例示。如圖2A，初始操作200及202包括偵測攝影機102捕獲之影像中之至少一面部，然後提取面部特徵。舉例而言，攝影機102可捕獲收看視訊監視器126之觀看者之影像。可偵測影像中之觀看者之面部，然後可提取可用。於識別面部之特徵。在操作206中影像中之任何觀看者之身份可基於在操作202中提取之特徵來決定。舉例而言，提取的面部特徵可與含有觀看者設定檔之觀看者資料庫118比較。觀看者設定檔可含有觀看者特徵(例如，年齡、性別、 偏好、興趣等)，其可在操作208中用於決定所識別的使用者之內容偏好。舉例而言，觀看者之年齡可指示適合/不適合於觀看者之內容，並且偏好及/或興趣可用於選擇合適內容範圍內之具體內容。 A flowchart of an example operation for face detection and viewer identification is illustrated in Figure 2B. 2A, initial operations 200 and 202 include detecting at least one of the images captured by camera 102 and then extracting facial features. For example, camera 102 can capture an image of a viewer viewing video monitor 126. The viewer's face in the image can be detected and then extracted for use. To identify the features of the face. The identity of any viewer in the image in operation 206 may be determined based on the features extracted in operation 202. For example, the extracted facial features can be compared to a viewer database 118 containing viewer profiles. Viewer profiles can contain viewer characteristics (eg, age, gender, Preferences, interests, etc.), which may be used in operation 208 to determine content preferences of the identified user. For example, the age of the viewer may indicate content that is suitable/not suitable for the viewer, and preferences and/or interests may be used to select specific content within the appropriate content range.
圖3例示根據本地觀看者內容控制實施例之實例實行方案。在圖3中視訊監視器126'正在顯示內容128'，其為卡通程式。四個圖標130'亦顯示於內容128'之上。圖標130'為卡通，其可代表當前正在收看視訊監視器126'之觀看者之面部及表情。選擇框136'指示當前選定一個圖標130'。所選擇的圖標130'似乎具有與小孩面部類似的面部。結果，內容128'(例如，TV程式及廣告)可基於適合於作為小孩之觀看者的內容來選擇，其當然包括卡通程式。 FIG. 3 illustrates an example implementation scheme in accordance with a local viewer content control embodiment. In Figure 3 video monitor 126' is displaying content 128', which is a cartoon program. Four icons 130' are also displayed above the content 128'. The icon 130' is a cartoon that can represent the face and expression of the viewer currently watching the video monitor 126'. Selection box 136' indicates that an icon 130' is currently selected. The selected icon 130' appears to have a face similar to that of a child's face. As a result, the content 128' (e.g., TV program and advertisement) can be selected based on content suitable for being a viewer of a child, which of course includes a cartoon program.
圖4例示對應於圖3展示之本地觀看者內容控制實施例的實例操作。在操作400及402中，如例如圖2A及2B中描述，可偵測當前正在收看視訊監視器126'之觀看者之面部、可偵測所偵測的面部之特徵、可識別與所偵測的特徵相關之觀看者並且可顯示每個所識別觀看者之圖標。在操作404中，一圖標可選定為視訊監視器126'之主要觀看者。觀看者之選擇可例如藉由將選擇框136'移動至一個所顯示的圖標130'之上而發生。在操作406中，可例如藉由存取含有所選擇的觀看者之設定檔的觀看者資料庫來決定主要觀看者之內容偏好。在操作408中可視觀看者偏好而定來選擇主要觀看者之內容。舉例而言，例如年齡、偏好及興趣之 資訊可用於自內容資料庫選擇合適內容。在操作410中所選擇的內容可顯示於視訊監視器126'上。在圖3之實例中主要觀看者為兒童，因此內容128'為兒童程式。 4 illustrates example operations corresponding to the local viewer content control embodiment shown in FIG. In operations 400 and 402, as described, for example, in FIGS. 2A and 2B, the viewer's face currently viewing the video monitor 126' can be detected, the detected facial features can be detected, identifiable and detected. The feature-related viewers can also display an icon for each identified viewer. In operation 404, an icon can be selected as the primary viewer of the video monitor 126'. The viewer's selection can occur, for example, by moving the selection box 136' over a displayed icon 130'. In operation 406, the content preferences of the primary viewer may be determined, for example, by accessing a viewer repository containing the profile of the selected viewer. The content of the primary viewer is selected in operation 408 depending on the viewer's preferences. For example, age, preference and interest Information can be used to select the right content from the content library. The content selected in operation 410 can be displayed on video monitor 126'. In the example of Figure 3, the primary viewer is a child, so the content 128' is a children's program.
圖5揭示根據一實施例之另一個實例實行方案。在圖5中，視訊監視器126"為經由網路500(例如，網際網路)耦接至其他系統100B至100n的系統100A之一部分。視訊監視器126"顯示內容128"，其為直播體育賽事。五個圖標130"亦顯示於視訊監視器126"上。圖標130"為使用系統100A至100n之觀看者之草圖(例如，基於彼等系統中之攝影機102之獲得之影像)。一個圖標130"可對應於收看視訊監視器126"之觀看者，而其他四個圖標可對應於系統100B至100n中之觀看者，其為觀看者組之成員(例如，朋友、親戚、業務同事等)。在所揭示的實例中，指標132'及134'相鄰於圖標130"而顯示。指標132'可為對應於與每個圖標130"相關之觀看者正在觀看之頻道的符號。指標134'可為自與每個圖標130"相關之觀看者正在觀看之內容獲取之影像或快照。在觀看圖標130"與指標120'及122'後，觀看者可注意到其他當前活動成員及其他組成員正在觀看之頻道/程式。若另一個組成員正在觀看之內容似乎為有趣的，則觀看者可選擇跟隨其他組成員來觀看所識別之內容或其他觀看者建議之其他內容。 FIG. 5 discloses another example implementation in accordance with an embodiment. In Figure 5, video monitor 126" is part of system 100A coupled to other systems 100B through 100n via network 500 (e.g., the Internet). Video monitor 126 "displays content 128", which is live sports The five icons 130" are also displayed on the video monitor 126. The icon 130" is a sketch of the viewers using the systems 100A through 100n (eg, based on images obtained by the cameras 102 in their systems). One icon 130" may correspond to a viewer viewing video monitor 126", while the other four icons may correspond to viewers in systems 100B through 100n, which are members of the viewer group (eg, friends, relatives, business colleagues) Wait). In the disclosed example, indicators 132' and 134' are displayed adjacent to icon 130". Indicator 132' may be a symbol corresponding to the channel that the viewer associated with each icon 130" is viewing. The indicator 134' may be an image or snapshot taken from the content being viewed by the viewer associated with each icon 130. After viewing the icon 130" and the indicators 120' and 122', the viewer may notice other currently active members and Channels/programs that other group members are watching. If the content being viewed by another group member seems interesting, the viewer may choose to follow other group members to view the identified content or other content suggested by other viewers.
圖6例示對應於圖5展示之基於組之內容控制實施例的實例操作之流程圖。在操作600中，本地觀看者及一 或多個遠程觀看者可關聯至組中。舉例而言，本地觀看者或遠程觀看者中之至少一者可在其使用者設定檔中定義組之成員。在操作602中，可識別組中之每個觀看者，可產生觀看者之圖標並且可本地顯示圖標，該等操作例如基於圖2A及2B描述之操作。另外，可決定組中之每個觀看者正在觀看之當前內容。在操作604中，對於每個本地觀看者，可顯示組中之所有遠程觀看者之圖標。本地及遠程觀看者圖標可進一步被顯示為具有相鄰於每個圖標的一或多個指標，該等指標對應於每個組成員當前正在觀看之內容。舉例而言，一個指標可代表使用者正在觀看之頻道，而另一個指標可代表實際內容。在操作606中，本地觀看者可選擇與遠程組成員相關之圖標，並且與遠程組成員相關之內容可顯示於本地觀看者之視訊監視器上。 6 illustrates a flow diagram of example operations corresponding to the group-based content control embodiment shown in FIG. In operation 600, the local viewer and one Or multiple remote viewers can be associated with the group. For example, at least one of a local viewer or a remote viewer may define a member of the group in their user profile. In operation 602, each viewer in the group can be identified, an icon of the viewer can be generated and the icons can be displayed locally, such operations based on the operations described in Figures 2A and 2B, for example. In addition, the current content that each viewer in the group is viewing can be determined. In operation 604, for each local viewer, an icon for all remote viewers in the group can be displayed. The local and remote viewer icons can be further displayed as having one or more metrics adjacent to each icon that correspond to what the group member is currently viewing. For example, one indicator can represent the channel that the user is watching, and another indicator can represent the actual content. In operation 606, the local viewer may select an icon associated with the remote group member and the content associated with the remote group member may be displayed on the local viewer's video monitor.
雖然圖2A、2B、4及6例示根據多種實施例之不同操作，但是應瞭解其他實施例並不需要圖2A、2B、4及6描繪之所有操作。事實上，本文完全預期在本揭示案之其他實施例中，圖2A、2B、4及6描繪之操作及/或本文描述之其他操作可以未在任何圖中具體展示之方式來組合，但是仍然與本揭示案完全一致。因此，針對未在一個圖中精確地展示之特徵及/或操作的申請專利範圍被視為在本揭示案之範圍及內容中。 While Figures 2A, 2B, 4, and 6 illustrate different operations in accordance with various embodiments, it should be understood that other embodiments do not require all of the operations depicted in Figures 2A, 2B, 4, and 6. In fact, it is fully contemplated herein that in other embodiments of the present disclosure, the operations depicted in Figures 2A, 2B, 4, and 6 and/or other operations described herein may be combined in a manner not specifically shown in any of the Figures, but still It is fully consistent with this disclosure. Therefore, the scope of the patent application is intended to be limited to the scope and the scope of the present disclosure.
如本文之任何實施例中所使用，「模組」一詞可指被組配成執行任何前述操作之軟體、韌體及/或電路。軟體可體現為記錄於非暫時性電腦可讀儲存媒體上之軟體 包、碼、指令、指令集及/或資料。韌體可體現為在記憶體裝置中硬編碼(例如，具有非依電性)之碼、指令或指令集及/或資料。如在本文之任何實施例中所使用，「電路」可例如單獨地或以任何組合形式包括硬連線的電路、可規劃電路例如包含一或多個個別指令處理核心之電腦處理器、狀態機器電路，及/或儲存可規劃電路所執行之指令的韌體。模組可共同或個別地體現為形成更大系統之一部分的電路，該系統例如積體電路(IC)、單晶片系統(SoC)、桌上型電腦、膝上型電腦、平板電腦、伺服器、智慧型手機等。 As used in any embodiment herein, the term "module" may refer to a software, firmware, and/or circuit that is configured to perform any of the foregoing operations. Software can be embodied as software recorded on non-transitory computer readable storage media Packages, codes, instructions, instruction sets and/or materials. The firmware may be embodied as a code, instruction or set of instructions and/or material that is hard coded (e.g., has non-electricality) in the memory device. As used in any embodiment herein, a "circuit" can comprise, for example, a hardwired circuit, a programmable circuit, such as a computer processor, state machine, including one or more individual instruction processing cores, alone or in any combination. A circuit, and/or a firmware that stores instructions that can be programmed by the circuit. Modules may be implemented collectively or individually as circuitry that forms part of a larger system, such as an integrated circuit (IC), a single-chip system (SoC), a desktop computer, a laptop, a tablet, a server , smart phones, etc.
本文描述之任何操作可實施於系統中，該系統包括一或多個儲存媒體，該一或多個儲存媒體個別地或組合地具有儲存在其上的指令，該等指令在由一或多個處理器執行時執行方法。在本文中，處理器可包括例如伺服器CPU、行動裝置CPU及/或其他可規劃電路。另外，意圖本文描述之操作可分佈於多個實體裝置，例如一個以上不同實體位置處之處理結構。儲存媒體可包括任何類型之有形媒體，例如，任何類型之碟片，包括硬碟、軟碟、光碟、光碟片唯讀記憶體(CD-ROMs)、可重寫光碟片(CD-RWs)及磁光碟、半導體裝置例如唯讀記憶體(ROMs)、隨機存取記憶體(RAMs)例如動態及靜態RAMs、可抹除可規劃唯讀記憶體(EPROMs)、電子可抹除可規劃唯讀記憶體(EEPROMs)、快閃記憶體、固態磁碟(SSDs)、磁性或光學卡，或適合於儲存電子指令之任何類型之媒體。其他實施例可實施為可規劃控制器執行之軟體模組。儲存媒體可為 非暫時性的。 Any of the operations described herein can be implemented in a system that includes one or more storage media, individually or in combination, having instructions stored thereon, the instructions being in one or more The method is executed when the processor executes. In this context, a processor may include, for example, a server CPU, a mobile device CPU, and/or other programmable circuitry. Additionally, it is intended that the operations described herein can be distributed across multiple physical devices, such as processing structures at one or more different physical locations. The storage medium may include any type of tangible media, such as any type of disc, including hard discs, floppy discs, compact discs, CD-ROMs, rewritable discs (CD-RWs), and Magneto-optical disks, semiconductor devices such as read-only memory (ROMs), random access memory (RAMs) such as dynamic and static RAMs, erasable programmable read-only memory (EPROMs), electronic erasable programmable read-only memory EEPROMs, flash memory, solid state disks (SSDs), magnetic or optical cards, or any type of media suitable for storing electronic instructions. Other embodiments may be implemented as a software module that can be programmed by the controller. Storage media can be Non-transitory.
因此，本揭示案提供用於提供互動式媒體之狀態圖標的方法及系統。系統可被組配成捕獲觀看者之影像、識別觀看者並且產生顯示於TV上之圖標。圖標可與對應於觀看者正在觀看之頻道及/或節目的指標相關。除了系統基於觀看者之身份來控制TV以外，所感興趣的其他人之圖標亦可顯示於TV上，使得觀看者可注意到，並且可能跟隨其他人正在觀看之內容。 Accordingly, the present disclosure provides methods and systems for providing status icons for interactive media. The system can be configured to capture an image of the viewer, identify the viewer, and generate an icon that is displayed on the TV. The icon may be associated with an indicator corresponding to the channel and/or program being viewed by the viewer. In addition to the system controlling the TV based on the identity of the viewer, icons of other people of interest may also be displayed on the TV so that the viewer may notice and may follow what other people are watching.
根據一個態樣，提供方法。方法可包括捕獲影像、偵測影像中之至少一面部、決定對應於至少一面部的身份及表情、基於對應表情來產生至少一面部之圖標，以及將圖標顯示於視訊監視器上。 According to one aspect, the method is provided. The method can include capturing an image, detecting at least one of the faces in the image, determining an identity and an expression corresponding to the at least one face, generating an icon of the at least one face based on the corresponding expression, and displaying the icon on the video monitor.
根據另一態樣，提供系統。系統可包括被組配成捕獲影像之攝影機、被組配成顯示至少內容及圖標的視訊監視器，及一或多個儲存媒體，該一或多個儲存媒體個別地或組合地具有儲存在其上的指令，該等指令在由一或多個處理器執行時導致以下操作，包括捕獲影像、偵測影像中之至少一面部、決定對應於至少一面部的身份及表情、基於對應表情來產生至少一面部之圖標，以及將圖標顯示於視訊監視器上。 According to another aspect, a system is provided. The system can include a camera that is configured to capture images, a video monitor that is configured to display at least content and icons, and one or more storage media, the one or more storage media having storage thereon, individually or in combination The instructions, when executed by one or more processors, result in the following operations, including capturing an image, detecting at least one face in the image, determining an identity and an expression corresponding to the at least one face, generating based on the corresponding expression At least one face icon and an icon displayed on the video monitor.
根據另一態樣，提供系統。系統可包括一或多個儲存媒體，該一或多個儲存媒體個別地或組合地具有儲存在其上的指令，該等指令在由一或多個處理器執行時產生以下操作，包括捕獲影像、偵測影像中之至少一面部、決 定對應於至少一面部的身份及表情、基於對應表情來產生至少一面部之圖標，並且將圖標顯示於視訊監視器上。 According to another aspect, a system is provided. The system can include one or more storage media, individually or in combination, having instructions stored thereon that, when executed by one or more processors, produce the following operations, including capturing images Detecting at least one face in the image, An icon corresponding to at least one face and an expression are generated, an icon of at least one face is generated based on the corresponding expression, and the icon is displayed on the video monitor.
本文使用之用語及表述作為描述性而非限制性用語來使用，並且在使用此等用語及表述中不意圖排除所展示及描述之特徵的任何均等物(或其一部分)，並且應認識到不同的修改在申請專利範圍之範圍內為可能的。因此，申請專利範圍意圖涵蓋所有此等均等物。 The words and expressions used herein are for the purpose of description and description, and are not intended to Modifications are possible within the scope of the patent application. Therefore, the scope of the patent application is intended to cover all such equivalents.
100‧‧‧系統 100‧‧‧ system
102‧‧‧攝影機 102‧‧‧ camera
104‧‧‧面部偵測/追蹤模組 104‧‧‧Face Detection/Tracking Module
106‧‧‧觀看者識別模組 106‧‧‧ Viewer Identification Module
108‧‧‧面部特徵提取模組 108‧‧‧Face feature extraction module
110‧‧‧圖標產生模組 110‧‧‧ icon generation module
112‧‧‧圖標增強模組 112‧‧‧ icon enhancement module
114‧‧‧外部裝置 114‧‧‧External devices
116‧‧‧圖標覆蓋模組 116‧‧‧ icon overlay module
118‧‧‧觀看者資料庫 118‧‧‧ Viewer Database
120‧‧‧內容資料庫 120‧‧‧Content database
122‧‧‧觀看者選擇模組 122‧‧‧ Viewer selection module
124‧‧‧內容管理模組 124‧‧‧Content Management Module
126‧‧‧視訊監視器 126‧‧‧Video Monitor
128‧‧‧內容 128‧‧‧Content
130‧‧‧圖標 130‧‧‧ icon
132、134‧‧‧指標 132, 134‧‧ indicators
136‧‧‧選擇框 136‧‧‧Selection box
138‧‧‧遙控器 138‧‧‧Remote control
Claims (15)
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2011/084970 WO2013097160A1 (en) | 2011-12-30 | 2011-12-30 | Interactive media systems |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TW201342892A TW201342892A (en) | 2013-10-16 |
| TWI605712B true TWI605712B (en) | 2017-11-11 |
Family
ID=48696235
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW101150114A TWI605712B (en) | 2011-12-30 | 2012-12-26 | Interactive media systems |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US20140223474A1 (en) |
| EP (1) | EP2798853A4 (en) |
| TW (1) | TWI605712B (en) |
| WO (1) | WO2013097160A1 (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103618918A (en) * | 2013-11-27 | 2014-03-05 | 青岛海信电器股份有限公司 | Method and device for controlling display of smart television |
| US10121060B2 (en) * | 2014-02-13 | 2018-11-06 | Oath Inc. | Automatic group formation and group detection through media recognition |
| CN106162303B (en) * | 2015-04-27 | 2019-07-09 | 北京智谷睿拓技术服务有限公司 | Information processing method, information processing unit and user equipment |
| CN106162370B (en) * | 2015-04-27 | 2019-08-02 | 北京智谷睿拓技术服务有限公司 | Information processing method, information processing unit and user equipment |
| CN105072477A (en) * | 2015-07-27 | 2015-11-18 | 天脉聚源（北京）科技有限公司 | Method and device for generating interactive feedback information of interactive TV system |
| US9565481B1 (en) | 2015-09-04 | 2017-02-07 | International Business Machines Corporation | Event pop-ups for video selection |
| CN105407313A (en) * | 2015-10-28 | 2016-03-16 | 掌赢信息科技（上海）有限公司 | Video calling method, equipment and system |
| CN105578110B (en) * | 2015-11-19 | 2019-03-19 | 掌赢信息科技（上海）有限公司 | A kind of video call method |
| US10599730B2 (en) * | 2016-03-25 | 2020-03-24 | International Business Machines Corporation | Guided search via content analytics and ontology |
| CN106210808B (en) * | 2016-08-08 | 2019-04-16 | 腾讯科技（深圳）有限公司 | Media information put-on method, terminal, server and system |
| CN106803909A (en) * | 2017-02-21 | 2017-06-06 | 腾讯科技（深圳）有限公司 | The generation method and terminal of a kind of video file |
| CN107992822B (en) * | 2017-11-30 | 2020-04-10 | Oppo广东移动通信有限公司 | Image processing method and apparatus, computer device, computer-readable storage medium |
| CN110769186A (en) * | 2019-10-28 | 2020-02-07 | 维沃移动通信有限公司 | Video call method, first electronic device and second electronic device |
| IT202100018971A1 (en) * | 2021-07-19 | 2023-01-19 | Gualtiero Dragotti | Method and system for marking elements |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7496623B2 (en) * | 2004-04-23 | 2009-02-24 | Yahoo! Inc. | System and method for enhanced messaging including a displayable status indicator |
| US8243118B2 (en) * | 2007-01-23 | 2012-08-14 | Euclid Discoveries, Llc | Systems and methods for providing personal video services |
| JP4539712B2 (en) * | 2007-12-03 | 2010-09-08 | ソニー株式会社 | Information processing terminal, information processing method, and program |
| US20090201297A1 (en) * | 2008-02-07 | 2009-08-13 | Johansson Carolina S M | Electronic device with animated character and method |
| US7953255B2 (en) * | 2008-05-01 | 2011-05-31 | At&T Intellectual Property I, L.P. | Avatars in social interactive television |
| US9246613B2 (en) * | 2008-05-20 | 2016-01-26 | Verizon Patent And Licensing Inc. | Method and apparatus for providing online social networking for television viewing |
| US9159151B2 (en) * | 2009-07-13 | 2015-10-13 | Microsoft Technology Licensing, Llc | Bringing a visual representation to life via learned input from the user |
| CN101998161A (en) * | 2009-08-14 | 2011-03-30 | Tcl集团股份有限公司 | Face recognition-based television program watching method |
| EP2383984B1 (en) * | 2010-04-27 | 2019-03-06 | LG Electronics Inc. | Image display apparatus and method for operating the same |
| CN101909085A (en) * | 2010-08-06 | 2010-12-08 | 四川长虹电器股份有限公司 | Television impression sharing method |
-
2011
- 2011-12-30 EP EP11878911.4A patent/EP2798853A4/en not_active Withdrawn
- 2011-12-30 WO PCT/CN2011/084970 patent/WO2013097160A1/en active Application Filing
- 2011-12-30 US US13/994,815 patent/US20140223474A1/en not_active Abandoned
-
2012
- 2012-12-26 TW TW101150114A patent/TWI605712B/en not_active IP Right Cessation
Also Published As
| Publication number | Publication date |
|---|---|
| US20140223474A1 (en) | 2014-08-07 |
| WO2013097160A1 (en) | 2013-07-04 |
| EP2798853A4 (en) | 2015-07-15 |
| TW201342892A (en) | 2013-10-16 |
| EP2798853A1 (en) | 2014-11-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI605712B (en) | Interactive media systems | |
| KR101894956B1 (en) | Server and method for image generation using real-time enhancement synthesis technology | |
| CN109257645B (en) | Video cover generation method and device | |
| KR102490438B1 (en) | Display apparatus and control method thereof | |
| KR101895846B1 (en) | Facilitating television based interaction with social networking tools | |
| US20170118298A1 (en) | Method, device, and computer-readable medium for pushing information | |
| CN111343512B (en) | Information acquisition method, display device and server | |
| US20180013977A1 (en) | Deep product placement | |
| US11900683B2 (en) | Setting ad breakpoints in a video within a messaging system | |
| KR102673676B1 (en) | Inserting advertisements into videos within messaging systems | |
| US11856255B2 (en) | Selecting ads for a video within a messaging system | |
| KR20160016574A (en) | Method and device for providing image | |
| CN104902318B (en) | Control method for playing back and terminal device | |
| CN107908324A (en) | Method for showing interface and device | |
| US11682210B1 (en) | Methods and device for video data analysis | |
| KR20240081641A (en) | Method and system for generating growth diary using character recognition | |
| CN114491151A (en) | Video cover generation method and device and storage medium | |
| CN114780180A (en) | Object data display method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MM4A | Annulment or lapse of patent due to non-payment of fees |