TW201025110A - Method and apparatus for generation, distribution and display of interactive video content - Google Patents

Abstract

Proposed is a model for generation and use of metadata for interactive video navigation and video content identification.

Description

201025110 IX. Description of the invention: [Technical field of invention] A method for initiating behavior based on the dual means 'detecting the characteristics of the spatiotemporal position in the video between the client and the server and calculating the associated signature And its equipment. " [Prior Art] Hyper-video (Hypervideo) or Hyper-linked image (Hyperlinked ❹ deo) is embedded between video and other hypermedia elements for embedded, user-selected anchors Anchors) and the generic name of the video content (vide〇c〇ntent) linked by navigation. Hyper-image links are similar to the concept of Hypertext, which is widely used in the World Wide Web to link information to other documents to obtain information. This concept can be traced to the end of the job, its development is very slow, but with the prevailing of the wide-ranging and the likes of the YGuTube-style service _ increase the variability of the positive shape of the singularity, can be downloaded or string The video distribution of the Internet in the form of streaming will replace traditional media distribution such as digital audio and video (DVD) or television (TV). /曰 As the amount of video content on the Internet increases and becomes readily available, the need for navigation is becoming more important, and the super-image link is recognized as a convenient answer that will reform the consumption of video. As the original super-file link to the reform of the text media. Finally, the most important value of super-image links is in commercial advertising. Until the end of the year, it is very difficult to design a sales video of business form. The traditional advertising method is to use TV, for example, video intro, but The online community thinks it is very unsuitable. Hyper-image links provide an alternative way to sell video, allowing the possibility of generating video clips, linking their objects to advertising or e-commerce locations or providing more information about specific products. This new model of advertising is less intrusive when users When you select an object on the video to make a selection, only the advertisement information according to the demand is displayed. Since the user requests the product information, the advertisement type is more targeted and more effective.

BRIEF SUMMARY OF THE INVENTION The present invention is directed to a method and apparatus for initiating behavior based on a dual means of calculating the space-time position (4) of a video towel at the client and the feeding end, and calculating the associated signature. In one aspect, the present invention discloses a method according to a video stream operation, the steps of which include at least a segment of the user's processing system that receives video information. The video branch includes video information representing a link frame, and the video message includes, in essence, the video message. ?, silk conversion continuous miscellaneous image of the request of the receiving behavior of the raw material 'behavior is the corresponding image of the image displayed in the video segment of the a ^ r empty position 'automatic side video stream of the zone adjacent to the space-time zone characteristics The step of automatically detecting the feature in the segment of the video stream is operated according to the video data used to convert the continuous frame image; obtaining a representation image of the domain _ domain according to the 贞 贞 ;; And the characterization of the image of the electronic county; and the implementation of the electronic county of the action of the action of the invention, the invention includes a continuous frame of video streaming operation 201025110 hit,, * method 'the steps of at least included in the first - calculation position towel Receiving a video camera' video stream includes video information representing a continuous frame, and the video information includes video data that is essentially only used to transform a continuous frame image; The representation image of the region in the stream includes: automatically displaying the feature in the video stream, and the step of automatically detecting the feature in the segment of the video stream is the root of the scene, the video data of the image To operate; according to the characteristics of the detected 7, the e-county of the non-dragon domain; and the remuneration behavior • each electronic signature. Further includes, from the second calculation position of the same or = Receiving the client's electronic signature; matching the customer j electronic signature silk levy image saki _ fixed signature; lang _ specific signing early behavior; and starting behavior. The invention proceeds to Yang, the invention _ 露根魏讯The method of streaming operation comprises the steps of: at least physically separating each user processing system and the feeding end processing system: receiving a video stream, the video stream comprising video information representing the frame, and the video information includes In essence, the use of ® is to rely on the _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Video of continuous frame image According to the characteristics of the side, the electronic signature of each area is calculated according to the characteristics of the side. The cap is sent to the user to handle the system. #电子签章—to the servo end _ system; The comparison between the specific electronic signature and the electronic signature calculated by the electronic signature-conquering end processing system calculated by the feeding end processing system; and matching, the feeding end processing system Startup behavior. 7 201025110 In the above aspect, the occurrence of the characterization image is based on the _ of the side to obtain an implicit object such as the _ domain or to obtain a video element like the region according to the feature. [Embodiment] Hyper-image link (Hypervideo) ) Video elements • Broadly speaking, in the case of hypertext, hypertext links are represented by linear continuous symbols ("words in i," (1〇, 14) Connected to other media content (metadata) (12, 16) (hyperlink, as shown in Figure j). In the above type to video, since the video is a three-dimensional spatio-temporal array, the pixels representing the visual information are constantly changing, so that the adjacent regions in the array correspond to the visually bearable objects, and the associated release Data (such as "object object" in 22 (1), 22 (2), and φ 22 (3) in "Fig. 2"), in general, the "object" in the video is not necessarily It is semantically meaningful—the same area of pixels does not necessarily have to be visually interpreted, so to avoid the semantic meaning of the semantics, we call the pixel area a video w elements (or explicit objects) such as The video element 20, which is a video element 20(1), 20(2), and 20(3), is described in U.S. Patent Application Serial No. 11/778,633, entitled "Method and Apparatus For Video Digest Generation, application 曰2007/7/16 for reference. 8 201025110 • The transfer of data in the super file link and the super image link towel representation is different in the super file mark language (the coffee bean to marf^p ^guage, =TML) It is to release the certificate • #_结(_相相)3〇 to its compatible towel, Jingzi•, _ object and to the object has the green (four), for example: child = $ 3 figure 'A behavior can open another file (such as «full release bait link 3 〇 (1)) or a media slot case (such as the release of the data link 3 〇 3 (2) or super reference link Ower-reference) Show), or more generally, on the client side to perform any operation. -, as shown in "Figure 4" - means that the possible means of transferring data, such as a list of behaviors, is associated with the corresponding video element. The list may contain an element ID (element ID), a keyword description, and an indication of whether it can be clicked. Flag and associated behavior. More generally, the interpretation data may contain a list of possible behaviors associated with the parent video element from which to select a particular behavior', e.g., using personal φ information from the user profile (userpr〇file). In the proposal of the super-image link representation, the interpretation data is stored and directed to the video map, and the hierarchical data structure is described in the following part. The video image is included in the video. The description of the spatio-temporal locations of the video elements, the actual syntax of which is based on the compactness and correctness of the representation request. For example, the easiest way to represent video elements (as shown in 40(1)-40(5) in Figure 4) is by specifying the corners of the spatio-temporal bounding boxes. ' Through the identifier 9 201025110, the video image part, the angular position is presented, as shown in Figure 4, the description of the tender image in the video element 40 t is like a border box box) 〇•下I彳' Paying attention to S '帛来卿·Xing image to identify the items that can be selected and the scales, describing the behavior associated with the selected items. According to the 'Fuji video connection silk scarf, these two assets are separated. Video feature^ ❿ The layer of granularity under the video element of the image has video features, the following are simply referred to as "featoes", and the features are common names in computer vision j, often used Describe the information vector associated with the temporal and spatial subset of the video, for example, the feature is the space-time edge (the 3D direction of the chat (4), the local direction of the dynamic field, the color distribution, etc. As shown in Figure 5, distinguishing the local features (local features) 50 and the overall feature (global feat) 54, local 5 please connect to the space-time position (in the highest-order particle shot 'single-pixel 52), the new feature % associated • the largest space-time unit (frame (ftame ), lens or scene 56), typically, the local feature provides a description of the object, the overall feature provides a context (for example, the apple object in the computer advertisement and the fruit object in the fruit lens) has a description object The same local features, v but the overall context is different. ~ t narration ‘Sen can say that the sub-frame-level structure and the overall feature refer to the sequence-level structure. Audio features 201025110 When referring to "video", although not necessarily required, it is best to consider that in most cases, the audio component (channel) is part of the video, and the audio data is also included. The important asset S' used to identify the part of the object of interest. Since audio data is - dimensional, it provides information in the order hierarchy: therefore audio features are considered as an overall feature. 'Signatures'

Because the video object "consisting of pixel groups, _ to non-type multi-local features, aggregation _ to the local feature % of the video element 4G ^ its context to single-vector, we get the so-called signature % here, sign early 58 is a description of the video element 40, indicating its characteristics and context, and will be further elaborated below. The audio feature can also be part of the signature 58. Video map The video map is a hierarchical data structure used to view the image. The most hierarchical layer of granularity contains a representation of the smallest #material unit (individual or pixel cell block) and associated local features. The lower-level of the granularity includes the video element and the associated signature. The lower-level of the representation includes the high_material list_such as the SCeneS and the associated overall features, the selected audio features, and the coffee (for example) Subtitles, etc.). The lowest level of Old Bay contains information about complete video, such as name, release year, information about actors and directors, and more. 11 201025110 Video image provides video-only identification code ("bar code" or "fingerprint"). Of particular importance is the temporal and spatial structure of the video element and its signature, another example of the same video, Even after some degree of editing or modification, the same video will have similar assignments of objects and similar fingerprints. The video image can represent a generalized "symbol, contiguous, where the symbol is a value vector. Implicit objects Video images may contain only incomplete information, which means that the data structure described above is partially missing or incomplete. The name of the hidden object is considered in the important special case. When there is no video element 4G, it may be missing or not generated. The video image only contains features (local features 5〇 and overall features 54), but in video There is no clear indication of the area corresponding to the object. In this case, by accumulating the local features 50' in the vicinity of the time and space of the video point, together with the overall characteristics of the description context, the signature % can still be generated, so, possibly, we can associate the signature to each pixel and The features gathered around it are calculated. Fig. 11 is a view showing the difference between the hidden object 42(b) and the external object 42(a). Hyperderament 〇c〇ntent generation and distribution The conventional super video link content distribution system is composed of two main 70 pieces as shown in "Picture 6": content provider 6 () and front end use Application (Super Image Link Client 64). 12 201025110 'The content provider 6 in the traditional super image link distribution model not only provides video but also provides miscellaneous information. The fourth release is part of Wei φ stream (video = 7). The disadvantage of this mode is that the content provider 60 must Is the super-image link of Mum, that is, all the _ _ must be properly processed and converted into a format that can be compared with the client 64, in the above example, the content provider 60 will stream video, The video image and the certificate data are as easy as the client 64. According to the present invention, the mode described herein is based on the content provider providing super-image link distribution as shown in Figure 7, according to which the video information and video information only contains video information from the content. The provider 7 stream is streamed to the hyper-image link client 74, using legacy content containing no transfer data or any additional information, so the content provider 7G is agnostic to the hyper-image link. For example, the content provider 70 is the current Y〇uTube, and the key idea is that the super image link client 74 and the data feed end 72 use the same processing for the same view to independently generate two video images. The possible situations are as follows: - The contents of both sides of the super video link client 74 and the transfer data port 72 are the same, and the video images are the same. - The content on both sides of the super image link client 74 and the data feed end 72 are similar but different because the edit video images are similar. " - The content of the super image link client 74 is a subset of the contents of the data feed server * 72 'Super View' like the link client 44 video image is similar to the video image portion of the interpretation data server 72. Then the data server 72 is depicted. The data is streamed and combined with the video from the content supply 13 201025110 70 and the video image generated by the super image connection client 74 to form a super video link content. Hyper video distribution Hyper image distribution In an embodiment, each video has a unique identification code (such as an example of Y〇uTube ') sent by the content* providing end 70 together with the video. The super-image linking client 74 sends the identification code to the full-release data server. 72 is used to capture the correct release data corresponding to the video, and the video content of the same video can be simultaneously streamed to the super-image link client 74 and the release data server 72, and both sides generate the video image instantaneously. 8 shows a different embodiment, according to the signature 58 in the super-image linking client 74 can identify the video content of the content provider 7 stream, pass The signature 58 to the certificate data server 72 can stream the corresponding interpretation data, thereby facilitating the processing of illegally copied content: even the duplicate is still correctly identified (even if it has been modified or transcoded) and the super image The link power month b is still used. For example, in the case of illegal video, this will force the φ user to view the advertisement. Metadata generation "Fig. 9" illustrates the process of interpreting the data server 72 to generate _ data in a typical offline mode, and the % of the production supply end obtains the image of the stencil. Each video element 40 in the video image is woven to link the interpretation data as in step 92. The annotation process is hierarchical and contains automatic, manual or automatic and artificial words: solution to the video message. The data stored in the archives is based on - some personal profiles (pers〇nal pr〇file). During the inspection period, the 201025110 release data is partially generated and added. For example, some hyperlinks are invalid (for example, if the user There is no interest in the car, and the video elements 1 and 3 in the "Figure 4" example will not be selected or directed to another favorite target). As another example, consider the personalization model generated by the interpretation of the data. In this mode, the interpretation data generated by the user profile can be incomplete and complete. For example, BMW car video elements will not be linked to

BMW's global network is connected to local dealers based on the location of the user. There is also a dictionary that can be used to generate relevant evidence, for example, through a majority of votes, for example, if the majority of people click on the car and finally to the James B〇nd website, the car is directly linked to James Bond. In the fully automated mode, the transfer data can be similarly judged between the video elements, and in this case, the element can be linked to another 7G element or in a fine or other view with the current video element. The closest (same feeling) element. Month, J mentions that the additional process performed along with the video image generation is 58 generation, which will be further described below. - For any video image, more than one interpretation data can be associated with the parent video 7L. For example, if _ see BMw _ child, this car dealer thief sets his own BMW series data, these = binding data Is the data collection - part, not _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ The content does not need to be a specific copy of the content, which is used to interpret the interpretation data generated by the corresponding signature of the video image of the data server, such as a scaled or edited version, such as movie content. When it is judged that the interpretation material is used to generate an interpretation material for the video image, at the content providing terminal 70, the original film will be used to generate an image, and the image data producer can use any other version such as a DVD or a movie. Hyper-image link client (jjyperyideo cHent) On the super-image link client 74, the video image is generated during the video view, and the same (or similar) video image can be read from the data feed end 72. From the storage or instant generation, the common use situation is that the user interface provides the space-time coordinates of the selected location in order to query the video image 'j to identify the video elements corresponding to the coordinates. The video element 4 = sign = 58 will be attacked from the transfer of the 服 服 端 , , , , , , , 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为 因为Release the data, and then the super image link client 74 is used to perform the requested behavior, as shown in Figure 10. - Unlike the video image, which is intended to be her #, the transfer data can be quickly viewed and connected to the video element 40 in the simplest case. If an implied object is used, the video generated by the hyper-image link client 74 contains a bribe of the video element (10), so the signature % is associated with the point-and-click location rather than the object. The super-image linking client 74 is used to generate the content of the video image without 201025110. If the video recording of the video image is generated, it can be scaled or edited. For example, on the interpretation data server 72, the original movie is used to generate a video image, and on the super-image connection client 74, any other version of the broadcast, DVD or movie can be imaged, in this case, other content. In the case of the case, the calculus of Naraya's 72 can separate the electric chapter and perform the match by inserting the advertising signature. In this case, some content (e.g., evaluation purpose) is deleted from the movie, and the scale data server 72 can match with the movie signature _ lower part. The user interface of the user interface super-image link client provides the following three main functions: t*, no video, provides the user to select the point of interest in the video and performs the association to the selected behavior, similar to the super file link, super The image link client user interface is similar to browsing the web. A graphical description of the main components of the user interface is shown in Figure 12. The pointing device performs the position of the point of interest in the video by means of the pointing device 121. The position of the point of interest in the video is allowed to be input and the desired behavior 1220 is not required to be activated. Talk about it (by seeing the network like ^1), such as clicking, can initiate behavior, for example, behavior such as video playback 1230 'other' behavior such as video coverage 1240, the behavior of which can be used to allow results to display 1250 . The space coordinates of the device pointed to by the pointing device, and the moment between the moments of the selection, 201025110, will constitute the space-time position provided by the pointing device. Specifically, 'We check the supervised client's target application to identify the following possible pointing devices: • PC pointing device (4) devices): Devices used in personal computer applications, such as airslides, trackballs, trackpads, etc. Etc., use • These devices can display the current space coordinates of the cursor on the screen. Move the mouse (or the trackball, trackpad) to move the cursor to the desired position, and press the button (bu_) to perform the click. , as shown in Figure 13a. Touch screen: Touch-type screen is a touch-sensitive display device that can be used to determine the position of interest by touching a point on the screen. The mobile device is very good (4), such as the gesture recognition (gesture __οη) shown in the "Figure": the hand-made behavior is used to judge the position of interest to enter the person's choice from the means of the gesture, such a point The use of devices in the field of television is very helpful, and the difference in hand recognition can be directed to objects, like interactive devices with buttons (such as TV remote control). Actions Regarding the choice of _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ : Link (Link): _Super touches the county, _ contains the super reference (hyp (four) reference) associated with the corresponding location of the video element, the 201025110 line behavior, the reference is usually a media object such as text, web, audio or video, once阙, the currently displayed video will be terminated or paused, and the '4 No, Body #$外' related content will display an example of the coverage without interrupting the video playback. If the location is selected, the description will be displayed. Related information about the elements of the video, in another case, the selected location will start playing the same or most similar video elements in the video. Navigation: - View the current video mode, according to φ 阙 position, Display the video from the location of the video component in some relevant way. If the green device is located in the device, the remote location), the behavior depends not only on the location but also on pressing the button. For example, in the currently displayed video pointing to an actor's face and pressing the "Forward" button on the pointing device, the video will quickly advance to the next scene with the selected actor; the same, when Pressing "BACKWARD" will cause the video to return to the previous scene with the selected actor. If the pointing device has a single click (touch screen), the behavior will be judged by making the appropriate selection. Type: Based on the user profile selection (U ser profile based action selecti0n): at the interpretation data server 72, more than one interpretation data is associated with each signature 5 8, different decision methods to determine the execution behavior - One of the suggested methods for the full release data set is to use the user profile to decide which action to take. The user profile element can be transmitted via the use end of the connection signature 58 or stored on the interpretation data server 72. For example, the 'user profile' may contain the following elements: - User entity location 201025110 Requested time - User age group - User property group - User language * Previously requesting a history of network cookies on a personal computer, for example BMW dealers in San Jose, Calif., have special sales and purchases of links to BMW video elements on Labor Day, but only from this region and prior to this incident, only those with San Jose user profiles The user receives the purchase behavior after selecting the BMW video element in the selected location, and other users set the age method to receive information or receive different behaviors. Interactive mode: Clicking the location will interact with the user and the client. The application example is e-commerce. Clicking on the location invokes the interactive interface to allow the user to purchase the video # element associated with the location. project. Video map generation (Video map generation> Hyper-image link client and certificate release data server main calculation core is based on the process of searching and mapping data to calculate the video image. The important request feature of the video image is a hierarchical structure, allowing Partial time-space comparison, assuming that the plot is to consider two versions of the same video that constitute the video image on the interpretation data server 72 and the hyper-image link client 74, since they may undergo some changes, the video may be different, including: - spatial transformation: parsing Degree and aspect ratio conversion, pixel clipping or overlay 20 201025110 cover, color conversion, noise. _ time conversion: frame rate conversion, insertion or removal of content. In the retrieval data retrieval phase, compare some corresponding video images, The video image structure and comparison process implicit in the above conversion must be constant or non-feeling.

'Difficult real_ is - in _ format t (4) if using MPEG compression) video streaming to the super-image link client, undergoing the decoded video string • Dedicated frame, in the form of a three-dimensional pixel of the pixel The data 'information data' is not only attached to the encoding parameters but also to each frame obtained from the decoded video stream. The image of the local feature 5G is generated in pixels and encoding parameters, the local feature 50 is associated with each spatiotemporal position in the three-dimensional array of pixels, the granularity is a single pixel, or preferably a large data unit, such as a small space-time block of pixels . We refer to such blocks as spatio-temporal basic data units. φ Many methods used to describe feature detection and representation in computer vision. For example, local features 5〇 include:

-Harris angle detector and its differences are described in c Hards " M

Stephens “A combined comer and edge • detector» proceedings of the 4th Alvey Vision Conference, 1988. _ Scale invariant feature transform (SIFT) is described in D. G. Lowe, Distinctive image features from sea 1 einvariant Keypoints,” International Journal of Computer 21 201025110

Vision, 2004. - obtaining the motion vector (M〇ti〇n vect〇rs) by decoding the video stream; - the direction of the spatiotemporal edge; - the distribution of the color; - the structure description; the decomposition factor of the pixel in some known dictionaries, for example Wavelets conversion, curvelet conversion, etc. _ A priori known for a particular object. The next step is to clamp the video element 4〇. Through the process of performing video segmentation, the video element can be explicitly found from the pixel value, that is, the segmentation process is performed, the calculated feature is calculated, or the previous step is calculated. The local feature 5G is the current video element 'the special vector is associated with the space-time basic poor unit, and the vector quantization or aggregation can determine the spatio-temporal region with similar features to the shame' using any existing vector technique. The preferred embodiment is Use the Max-Lloyd method. • The 'pixel 3D array is therefore divided into regions that undergo a selection process with the goal of deleting meaningless regions (eg, too small), merging similar regions, and selecting regions of pre-judged values, each selected region marking video elements 40 And its location and stored in a particular data structure, the data structure - allows the pixel array to be given a spatiotemporal location to determine the associated video element 40. A preferred embodiment is to describe the position of the video element such as a three-dimensional image containing the overall value. The value X at the position corresponding to the basic data unit means that the basic data unit is a video element belonging to the number x; another embodiment is a bounding box. The corner coordinates are used to describe the video element 4〇. 22 2〇l〇25li〇 The feature vectors that belong to each video element 40 in the data unit are aggregated, and the possibility of such aggregation is performed—the features that are not aggregated, such as the signature vector, are the average of the coordinate directions of the vectors. In addition, the description of the video element, 7 such as its duration or lang size can be added to the signature vector, and the signature is stored in the data structure and associated with the corresponding video element. The way to represent the signature 58 that forms part of the video is spatial transformation: the feeling is late, the false domain branching is very effective, and the signatures of the two video messages will be similar or identical each morning. The structure of the feature, the video element and the signature are the entirety of the video image. During operation of the hyper-image linking client 74, the video image instant generation #x is preferred to decode the video material under the video image generation algorithm operation (further embodiment is that the audio material is also associated with the video material) stored in the temporary buffer. (tempomiy buffer), which displays a few frames of delayed view, enough for a small number of frames to generate a video image, conventional hardware and software, for example, a general-purpose computer with an application software that can perform the functions described. 'Embedded Negative Processors with Application Software Encoding the Functional Knowledge' or ASICS with hard-coded programming or hard-coded programming and software applications can be used as super-image links Client. For example, 'data processors can be used in general-purpose* microprocessors, microcontrollers, digital signal processors (DSP's) and application-specific integrated circuit data processing. Application specific integrated circuit processors ° When the pointing device clicks on the location of the video, the coordinates of the pointing device (or cursor) 23 201025110 together with the point of immediate execution of the click to query the corresponding video element in the video image 40 ' The video element 40, if the request location exists, can take its signature 58 from the video image class (as requested by the (10) embodiment is generated) and to the interpretation data server 72. * On the full release negative load port 72, the same process is used to generate the video image, the signature 58 is compared with the signature database to select the closest one, and then the related information is transmitted to the super image. Linking the client 74 determines the behavior of the 0 generated. As for hidden objects, the video element generation phase is missing. On the contrary, the video image only contains the feature vector. When pointing to the location where the device clicks on the video, the space coordinates of the pointing device (or cursor) together with the point of instant execution of the click are used to query Features of the spatiotemporal neighborhood of the spatiotemporal location (e.g., a predefined size) 'These features are gathered to the signature delivery to the delivery data feed end 72. Video Image Generation Exemplary Algorithm The following is a special φ algorithm for video image generation in a preferred embodiment of the present invention: 1. Input a set of sub-sequence frames from a video stream, each frame representing a two-dimensional array of pixels And the set of frames represents a three-dimensional (space-time) array of pixels. If the input stream is compressed, the frame can be obtained by video decoding process = result. « 2. Divide the 3D pixel array into small parts, possibly covering the 3D pixel. Block (for example: 16xl6x3), numbered block is bijk, i, j are two space coordinates, k: 疋 time coordinate (frame number) Represents a block such as a three-dimensional array, and each item corresponds to one block. 24 201025110 3. In the present invention, for each pixel block, the M local feature is calculated. The features in the embodiment include: - the intensity of the average luminance pixel in the block; - the intensity of the average density pixel in the block; the edge_computation area The spatiotemporal gradient of the pixels in the block, indicating the local spatiotemporal direction and intensity; &

- If the kernel is compressed, the preferred additional input is the average motion vector of the block, and the motion vector is the parameter information currently attached to the compressed stream. Each of the above features of the block bijk represents vector strictness, where the melon is the number of features, concatenated all features as a single vector. = (fijk, ..., 0), the feature vector is represented by a three-dimensional array, each An item corresponds to a feature of the associated pixel block. 4. The feature vector is subjected to clustering by a vector quantization algorithm. The Max-Lloyd algorithm of the embodiment of the present invention is described in a. Gersho, RM Gray for vector quantization and single compression, and vector quantization is to divide the three-dimensional array of pixels into the same Or a region of similar feature vector, the typical representative of the region is the block direction image, and each item is the region number of the block to which it belongs. 5. The image of the space-time region undergoes a morphological process, deleting the pre-threshold space and the time-sized region, and merging the same number of disconnected regions. Embodiments of the present invention are preset quantities for a given area, and regions 0 can be merged or deleted under the method that the total number of remaining areas in the process is equal to the preset number. 6. In the block belonging to stage 5 that experiences the cluster generation area The feature vector generates a signature vector associated with each region, and the global feature can capture 25 201025110 context and the audio can be added to the signature vector. In an embodiment of the invention, the area signature 58 has the dimension of the feature vector and is generated by the coordinates of the feature vector of the region to which the average block belongs. In another embodiment, the aggregation is performed by calculating a histogram of the feature vectors, and the number of intervals of the histogram is preset. In another embodiment, the feature vector of the region to which the block belongs undergoes principal component analysis (PCA) to generate a vector of a preset size to allow the PCA to extract the composition of the largest element, such as κ.

Fukunagal 990 Introduction to Statistical Pattern

Recognition Description. Video map matching 5 The full release data feed end 72 is illustrated and described herein as a single computing location, although it should be understood that the I full release data server 72 can be dispersed, in a particular implementation. In the example, according to the size of the generated video image, video image generation occurs at one or more locations, and different parts are merged. Further, after the video image is initially generated, the generated video image is distributed to various other locations, and the like. The location can help determine whether the signature 58c received from the client 58 matches or matches the video image stored in the signature to interpret the data server location, and the video image generated at the interpretation feed 72 can be generated in the week. _ or other intervals are generated. In addition, the update is used to generate the function of generating a video image. Although it is necessary to understand the update, it is necessary to perform the marriage update on the Nai-Shou 72 and the Super-Video Link Client 74. 26 201025110 For the matching of signatures, it is ideal that if the video content of the interpreter data server 72 and the super-view image client 74 are the same, the corresponding video images will be the same, so the result is completely match. however,

The version of the video image of the interpreter port 72 and the super image link client 74 is caused by different examples of the same video (for example, the video image of the interpreter data server 72 is from 1) ^ version]^1^ (: Decide on ad insertion is = 24 frames per second to generate 'super image link client % video image 疋 from HDTV to determine ad insertion is to play 3 frames per second), and other reasons for the difference, in this way Under the New Zealand, the pre-match is similar. Further, the video of the super-image linking client 74 may be new, such as the corresponding video image does not exist on the interpretation data server 72. In this case, the close match will be used to retrieve the interpretation data. Performing a match between the signature 58c (such as a probe) issued by the super-image linking client 74 and the video image signature S8s on the feeding end 72, using a pre-determined metric such as a vector ratio For the county 58e and the pool, the simplest case, the vector uses the series of the ridiculous escaping vector, the squared direction difference between the vectors is _number, close to the _ match through the probe signature and the feeding end signature And choose the one with the smallest metric. Although the embodiment disclosed in the present invention is as above, the content of the saki is not intended to directly limit the beauty of the gamma. Any of the techniques of the present invention, the domain towel is often known for its wealth, and the patent protection scope of the present invention can be modified in the form and details of the invention without prejudice. , still must be attached to the scope: defined by the scope. ^ Correction 27 201025110 [Simple description of the diagram] The first picture shows the common network should be visualized in the visual concept of the super file connection concept. Figure 2 is a schematic diagram of the visualization of the super-image link concept. ‘ 帛 3 is a schematic diagram of an embodiment of a super file embedded in the HTML format in the HTML format. Fig. 4 is a schematic view showing an embodiment of a super-image link for separating and releasing data from a content according to the present invention. ,. Figure 5 is a schematic diagram of an embodiment of a data hierarchy of a video image of the present invention. ^ The figure is a system block diagram of the distribution of the conventional super image link. Figure 7 is a block diagram of the system for super-image link content distribution according to the present invention. Figure 8 is a block diagram of a system for super-image link distribution with content recognition according to the present invention. Figure 9 is a schematic diagram of the invention for interpreting the data generation stage of the data server. Figure 10 is a schematic diagram showing the stage of viewing the data server and client contents of the present invention. Μ Figure 11 is a schematic view showing an example of an external object and an implicit object of the present invention. * 帛12® is not intended for the user interface version of the Hyper Image Link Client of this invention. Figures 13a through 13c are schematic views of different super-image linking clients using different pointing devices. Figure 14 is a schematic representation of an embodiment of the invention with different types of behavior 28 201025110. [Main component symbol description]

10 Characters 12 Certificate Information 14 Characters 16 Interpretation Data 20(1) Video Elements 20(2) Video Elements 20(3) Video Elements 22(1) Interpretation Data 22(2) Interpretation Data 22(3) Interpretation Data 3 〇(1) Interpret data link 30(2) Interpret data link 40(1) §fL ·7〇素40(2) Video element 40(3) Video element 40(4) Video element 40(3) Video element 42(8) Explicit object 42(b Implicit object 50 Local feature 52 Single pixel 54 Overall feature 29 201025110 56 Frame 58 Signature 60 Content provider 64 Hyper-image link client ^ 70 Content provider • 72 Interpreter data server 74 Hyper-image link client 1210 Pointing Device ® 1220 Behavior 1230 Video Recorder 1240 Video Coverage 1250 Display Step 90 Video Image Generation Step 92 Interpret Data Generation Process ❿ 30

Claims (1)

201025110 X. Patent application scope: 1. A method according to video stream operation, which comprises at least the following steps: In a user processing system: receiving a segment of the video stream, the segment The video stream includes video information representing one of the sequential frames, and the video information includes essentially one video data for rendering the continuous frame image; receiving a behavior (action) requesting, the behavior is a spatial-temporal location corresponding to the image displayed in the segment of the stream; automatically detecting at least the adjacent space-time region of the segment of the video stream a feature, the step of automatically detecting the features in the segment of the video stream is performed according to the video data used to convert the continuous frame image; obtaining the features according to the detected features a representation map of different regions in the video stream; calculating an electronic signature based on the request and the characterization image; Use the electronic signature to initiate the performance of the action. 2. The method according to claim 1, wherein the request for the behavior is an information request, and the step of using the power 31 201025110 sub-signature further comprises the following steps: The electronic signature is sent to an external server; receiving a message in response to the transmitted electronic signature; and displaying the information. 3. The method according to claim 1, wherein the request for the behavior is an information request, and the step of using the electronic signature further comprises the following steps: performing a partial request Used to use the electronic signature to obtain a piece of information that is completely stored in the user processing system; receive the information that is responsive to the execution of the local request; and display the information. 4. The method according to claim 6, wherein the request for the act is a content action request, and wherein the step of causing the electronic signature comprises performing the booting in the video stream The step of the section of the towel to make an electronic signature is a step of the process. 5. The method according to claim 4, wherein the content behavior request is to request to skip the content corresponding to the electronic signature. 6. The method according to claim 4, wherein the content behavior is to request swap or overlay the content corresponding to the electronic signature. 7. The method according to claim 4, wherein the content behavior request is to request an electronic signature of the mutual 32 201025110, wherein the step of calculating the electronic signature is required An additional request. 8. The method according to claim 1, wherein the step of receiving the request comprises the step of receiving an input of a user controlled controller and receiving a user control device. At least one of the steps of the text message of the (user controlled device). 9. The method according to claim 1, wherein the video data operated in the detecting step is an uncompressed video material. 10. The method of operating according to video stream as described in claim 9 wherein the uncompressed video material is pixel data. 11. The method according to claim 9, wherein the step of obtaining the characterization image is to obtain at least one implicit object as the region according to the detected features. And the step of calculating the electronic signature is to calculate the electronic signature based on the hidden object in the representation image associated with the request. 12. The method according to claim 9, wherein the step of obtaining the characterization image is to obtain at least one video element of the same region according to the detected features, and The step of calculating the electronic signature is to calculate the electronic signature based on the video element in the characterization image associated with the request. 33 201025110 13. The method according to claim 9, wherein the video information further includes an auxiliary video parameter information separated from the video data, and the detecting step is also based on The auxiliary video parameter information is operated. 14. The method according to claim 13, wherein the video information further comprises an audio data' and wherein the step of detecting is performed according to the audio data. 15. The method according to claim 1, wherein the video information further includes an auxiliary video parameter information separated from the video data and the step of detecting the step is also based on the auxiliary Video parameter information to operate. 16. The method according to claim 1, wherein the video information further comprises an audio material, and wherein the step of detecting is also performed according to the audio data. 17. The method according to claim 1, wherein the step of obtaining the characterization image is based on the detected features to obtain at least one hidden object as the region, and wherein the calculating The step of the electronic signature is to calculate the electronic signature based on the hidden object in the representation image associated with the request. 18. The method according to claim 1, wherein the step of obtaining the characterization image is to obtain at least one video element as the region based on the detected features, and wherein the electronic component is calculated The step of signing is to calculate the electronic signature based on the video element of 34 201025110 in the characterization image associated with the request. 19. The method according to claim 1, wherein the video data operated in the detecting dream is a compressed video material. 20. The method according to claim 19, wherein the compressed video material is motion vector data. 21. The method according to claim 19, wherein the video information further comprises an auxiliary video parameter information separated from the video data, and wherein the detecting step is also performed; According to the auxiliary video parameter information to operate. 22. The method according to claim 21, wherein the video information further comprises an audio material, and wherein the detecting step is further performed according to the audio data. 23. The method according to claim 19, wherein the step of obtaining the characterization image is based on the detected features to obtain at least one hidden object as the region, and wherein the calculating The step of the electronic signature is to calculate the electronic signature based on the hidden object in the representation image associated with the request. 24. The method according to claim 19, wherein the step of obtaining the characterization image is to obtain at least one video element as the region according to the detected features, and wherein the The step of electronically signing is to calculate the electronic signature based on the video element in the characterization image associated with the request. 25. The method of video streaming according to the method of claim 19, 2010 25110, wherein the method of detecting the audio data, and the steps of detecting the data are also operated according to the audio data. . 26. The method according to claim 1, wherein the video string is received from one of a digital video disc (DVD) processed by the user processing system and one of the broadcast contents. The step of receiving the segment of the stream is received from an external source. 27. The method according to the video streaming operation described in claim 1, wherein the 5H user processing system comprises a micro processor, a microprocessor, a digital controller, and a digital signal processor. Processor, DSP) and one of application specific integrated circuit processors. 28. A method according to a video stream operation comprising a continuous frame, comprising at least the following steps: 〇 in a first computing location: receiving the video stream, the video stream comprising a representative continuous signal One of the sequential frames (vide〇information), the video information includes essentially one of the video frames (vide〇data) of the continuous frame image; and the video stream is obtained The representation area of the representation area includes: automatically detecting at least one feature 36 201025110 (features) in the video stream, and automatically detecting the features in the section of the video stream The step is to operate according to the video data of the continuous frame image; calculate at least one electronic signature (dectr〇nie signature) of different regions according to the detected features; and identify an action and associate To each of these electronic signatures. 29. The method according to claim 28, wherein the method according to the video stream operation including the continuous frame further comprises the following steps: from the second calculation position of the same or different position: #亥弟一§10: receiving a client electronic signature; matching the client electronic signature with the characterization image to obtain a particular signature; identifying the behavior associated with the particular signature; and initiating the behavior. 30. The method according to claim 29, wherein the step of identifying the behavior comprises the step of associating a user profile with the specific signature, The method is used to identify the behavior according to the user profile and the specific signature. 31. The method according to claim 29, wherein the step of identifying the behavior comprises the step of identifying the monetary amount and the specific signature of the 2010. 'Use to identify the behavior based on the purchase amount and the particular signature. 32. The method according to claim 29, wherein the specific signature in the matching step is a type of electronic signature with the client in the matching image. Recently it seems like an electronic signature. 33. The method according to claim 32, wherein the most approximate electronic signature in the characterization image is the same signature as the method according to the φ video stream operation comprising the continuous frame. %, as described in claim 28, in accordance with the method of video stream operation including a continuous frame, wherein the video data operated in the detecting step is an uncompressed video material. 35. The method according to claim 34, wherein the uncompressed video material is pixel data, according to a method for video stream operation comprising a continuous frame. </ RTI> 36. The method according to claim 34, wherein the step of obtaining the characterization image is obtained according to the features of the debt measurement as at least one of the regions. The implied object 'and the step in which the electronic signature is calculated is based on each of the implicit-containing objects to calculate the electronic signature. 37. The method according to claim 34, wherein the step of obtaining the characterization image is to obtain at least one video like the region according to the detected features. The element 'and the step in which the electronic signature is calculated is to calculate the electronic signature based on each of the video elements. 38. The method according to claim 34, wherein the video information further comprises an auxiliary video parameter information separated from the video data and detecting the method according to the method of video streaming comprising the continuous frame. The steps will also be based on the auxiliary video parameter information. 39. The method according to claim 38, wherein the video information further comprises an audio % data, and wherein the detecting step is further performed according to the audio data. . The method according to claim 28, wherein the video information further includes an auxiliary video parameter information separated from the video data, and the detected information therein. The step will also operate based on the auxiliary video parameter information. 41. The method according to claim 28, wherein the video information further comprises an audio data, and wherein the detecting step is further performed according to the audio data. . 42. The method according to claim 28, wherein the step of obtaining the characterization image is: obtaining at least one hidden region of the region according to the detected features. The inclusion-object, and the step in which the electronic signature is calculated, is based on each of the hidden objects to calculate the electronic signature. 43. The method according to claim 28, wherein the step of obtaining the characterization image is 39 201025110 obtaining at least one video element according to the _ frequency of the side. And the step of calculating the electronic signature is to calculate the electronic signature according to each of the video elements. 44. The method according to claim 28, wherein the video data operated in the detecting step is a compressed video material. 45. The method of applying the patent model _ 44 is based on a method of operating a video frame comprising a continuous frame, wherein the collapsed video material is motion vector data. 46. The method according to claim 44, wherein the video information further comprises an auxiliary video parameter information separated from the video poor material, and the detected information in the video streaming operation. The step will also operate based on the auxiliary video parameter information. 47. The method according to claim 28, wherein the method according to the video streaming operation comprising the continuous frame receives the digital video disc (DVD) and the broadcast content processed in the user processing system. The step of receiving the segment of the video stream is received from an external source. 48. A method according to video stream operation, comprising at least the following steps: each of a user processing system and a server processing system that are physically separate: receiving the video stream, the The video stream includes video information (video 201025110 information) representing a sequential frame, and the video information includes essentially one video material (vide〇data) for rendering a continuous frame image; The representation map of the representation area in the video stream includes: . automatically detecting at least one feature in the video stream, and automatically detecting the part in the video stream The steps of the feature are: operating the video data of the continuous frame image; and calculating at least one electronic signature of the different regions according to the detected features; and wherein: transmitting One of the electronic signatures is calculated in the user processing system to the server processing system; • _ electronic signature of which is - in the comparison The electronic signature calculated by the server processing system determines one of the electronic signatures of the specific electronic signature of the nose of the processing system as a match; and - according to the matching, the processing The system starts with an action. 49. A method according to video stream operation as described in claim 48, wherein the electronic signature obtained by each of the user processing systems and the server processing system is substantially the same. 41