TW201330600A - Video and image information embedded technology system - Google Patents

Abstract

People are interested in the information shown in a video or image content. However, current technology can not satisfy people's curiosity about some certain objects in videos/images. Through the invention presented by the applicant, the applicant can embed relevant information on certain objects in videos/image content by an interactive method; and therefore, when people move the mouse cursor on the objects that interest them, all relevant information and auxiliary functions will pop up to allow users clicking freely to get more related information.

Description

Video and video information embedded in technology systems

The present invention provides a video and video information embedding technology system, in particular, an interactive method for embedding information related to a specific object in a movie/image content, allowing the user to freely click to obtain more relevant information embedding. system.

In everyday life, people are always curious about the many relevant information that appears in the film or image content, but the existing technology does not allow people to further understand the information about specific objects in the film/image. For example, when watching a drama, people may want to know what brand of clothes, pants, shoes, and even accessories worn on the male and female protagonists? Where can I buy? In addition, when watching a cooking show, people may also want to know what special pots are used to demonstrate the dishes on the show. What are the seasonings and ingredients used by the chef? Where can I buy it?

Unfortunately, today's technology can't satisfy people's curiosity about specific objects in movies/images. Information about specific objects cannot be easily obtained by people, and can only be obtained through passive search, retrieval, and snooping. Through the present invention, information related to specific objects in the movie/image content is embedded therein, and clicks to external links are allowed. So when the user is watching the movie/image, just move the cursor to the specific object they are interested in, all the information about the object will appear and can be clicked for more information. Through this interactive technology, users can get more information from movies/images in a more convenient, simple and intuitive way.

In the past, all external information related to film/image content could not be obtained directly. Therefore, when users are interested in specific content and specific objects in some movies/images, they must obtain such information in an indirect, passive, and separate way (for example, a search engine).

At present, although some applications already have some relevant external information embedded in the image [1], they have some innate limitations: (a) these external information are embedded in advance, lack of flexibility; b) There are no applications that extend to the film.

As for the video, the existing embedded technology only has subtitle embedding or message embedding [2], and lacks interactivity with the user (for example, these external information cannot respond to the user's interest and special attention).

Therefore, the inventors of the present invention have incorporated the relevant information in view of the above-mentioned deficiencies, and have incorporated and considered various information related to specific objects in the film/image content through multi-party evaluation and consideration, and with years of experience accumulated in the industry. And allow clicks to external links. So when the user is watching the movie/image, just move the cursor to the specific object they are interested in, all the information about the object will appear and can be clicked for more information. Through this interactive technology, users can get more information from images/movies in a more convenient, simple and intuitive way.

The present invention provides a new and interactive way for a user to obtain or embed relevant external information about a particular object in a movie/image content. The invention can be used for applications such as advertisement implantation and interactive external information provision. In the present invention, we display all the embedded external information in the form of a label and name it "tag".

The invention mainly comprises two parts: a client (110) and a server end (120). The server side includes a client-server interaction interface module (121), a movie/image database (122), a label database (123), a video/image content analysis module (124), and a movie/picture. The image-external information relationship analysis module (125) and the external information retrieval engine (126). The client includes a client-server interworking interface module (111), a video/image content analysis module (112), a tag embedding engine (113), an original movie/image database (114), and a tag. The information database (115) and the user interface module (116).

On the server side, the client-server interworking interface module is used to provide interaction between the client and the server. It can complete file upload/download, user data verification, user login operation, and operation command transmission. And other operations, etc. The video/image data library stores the video/image data, and the tag database stores the created tag files that contain external information related to the film/image content. In general, each movie/image has a corresponding tag file in the tag library. The video/image content analysis module on the server side will accept the operation instructions from the client from the client-server interaction interface module. When the module is triggered by an operation command, it will split, track, and recognize the target selected by the user in the movie/image. In addition, the external information retrieval engine obtains information from the video/image content analysis module and accepts operational commands from the client from the client-server interaction interface module. When the module is triggered by the client's operating instructions, it will retrieve the target analysis results from the movie/image content analysis module and retrieve and target from the public search engine, tag database, or other database. Object related external information. In addition, the Video/Image-External Information Relationship Analysis Module accepts data from three other modules: external information retrieval engine, video/image content analysis module, and client-server interaction. Interface module. After the module receives the operation instruction from the client-server interaction interface module, it further receives the analysis result information about the target from the video/image content analysis module, and receives the information from the external information retrieval engine. The external search result information of the target; after receiving the information, the module will generate tag information associated with the target in the movie/image specified by the client instruction. The generated tag information will be passed to the client using the client-server interworking interface module. At the same time, the tag information will also be saved as a tag file and stored in the tag database on the server side.

In addition, on the client side, the client-server interaction interface module is also used to provide interaction between the client and the server, and can complete file upload/download, user information verification, user login operation, and operation instructions. Delivery, and other operational actions. The user interface module is used to provide interaction with the user or tag generator, to perform movie/image uploading, tag information addition, to create a movie/image embedded with external information, and to play a movie/picture embedded with external information. Like, get the target or target information that the user is interested in, and other operations. The original movie/image repository of the client stores the original movie/image archive. In addition, new movie/image files can be added to the original movie/image library from the server/image library on the server side, or by the user through the user interface module. The tag information library stores all available external information that may be used to generate tags. Similarly, new external information can be added to the tag information library from the tag database on the server side or by the user through the user interface module. The client's video/image content analysis module receives operational commands from the user or tag generator through the user interface module. When the module is triggered by an operation command, it will segment, track, and recognize the target selected by the user in the movie/image. In addition, the tag embedding engine receives information from the following three modules: a video/image content analysis module, a tag information database, and a user operation interface module. When the engine is triggered, it first obtains tag information from the tag generator from the tag information repository or directly through the user interface module. At the same time, the engine will also receive the analyzed target related information from the film/image content analysis module. From then on, the tag embedding engine will generate an object external information tag associated with the user-specified target. The generated tag information will be saved as a file and stored in the client's tag information database, or uploaded to the server's tag database.

In addition, the original movie/image and its corresponding tag information file form a movie/image unit that embeds external information. In general, each movie/image unit that embeds external information contains an original movie/image file and a tag file. However, not every movie/image unit embedded with external information can contain only one original movie/image file and one tag file, or an original movie/image file and multiple tag files. The video/image unit embedded with external information can be interactively displayed by the user interface module. The tag information file contains the location of the target in the movie/image, the size of the region, and all external information related to the target. During the movie/image playback, the player in the user interface module also analyzes the tag information file in real time according to the user's operation (for example, an example of a user operation situation is the position of the user cursor in the movie). When the user moves the cursor to a range of objects specified by the tag file that contain external information, the external information tag corresponding to the target is automatically popped up. Otherwise, no external information tags will pop up and the movie/image will play as usual.

In the following sections, we will describe in detail two specific embodiments (or two modes of operation) with respect to our inventive framework, namely the tag producer-user mode (Embodiment 1) and the user center mode (Embodiment 2).

In Embodiment 1 (tag producer-user mode), on the client side, the tag producers (these tag producers including advertisers, brand companies, etc.) first select additional uploads from the original film/image database or themselves. Select the appropriate movie/image in the movie/image. The tag producer will then select the appropriate target from the movie/image that they want to embed external information through the user interface module (examples of these selected objects include clothes, objects, etc.). Then, the user operation interface module will trigger the film/image content analysis module to automatically segment and track the selected target object. At the same time, the tag generator can obtain the target tag information by directly inputting the relevant external tag information of the target or by retrieving the appropriate tag from the tag information database. The user interface module then embeds the tag embedding engine to embed the tag information into the movie/image and generates a separate tag file. After the tag file is generated, the file and its corresponding movie/image file will be uploaded to the server through the client-server interaction interface module.

On the server side, the server receives video/image files and tag files from the client-server interworking interface module. At the same time, save them separately to the video/image database and tag database on the server side.

The movie/image viewer (ie the user) also watches the movie/image through another client. The movie/image viewer first selects the movie/image file they are interested in through the user interface module. The user interface module further obtains the video/image and its corresponding label file directly from the video/image database and the label database of the server through the client-server interaction interface module. Once these files are obtained, they are played on the client. During playback, when the user moves the cursor to the object of interest in the movie/image, the corresponding tab information will be triggered and popped from the tag file. In this way, all external information related to the target will be displayed in the pop-up tab for browsing, clicking and providing the user for further interaction.

In Embodiment 2 (User Center Mode), at the client, the movie/image viewer (ie, the user) first selects the movie/image of interest by the user interface module. At this point, the user interface module will retrieve the movie/image directly from the video/image library on the server side and play it on the client. When the user moves the cursor to or selects the target they are interested in, the video/image content analysis module on the server side will be triggered and the selected target will be automatically segmented, tracked, and identified. The output of the film/image content analysis module is the position of the target in the movie/image and the information after identifying the target. After this step, the identified target information will be entered into an external information retrieval engine that will retrieve the external information using a public search engine, a tag database, or an additional database. The output of the external information retrieval engine will be external information related to the target selected by the user. Finally, the external information of the retrieved target and the information of the identified target itself will be input into the film/image-external information relationship analysis module, which will analyze the external information and the information of the target itself. Relationships, and generate tags that are appropriate for the user's selected target. The resulting tag will pop up on the side of the movie where the user selected the target. At the same time, the generated tag information is also saved to the tag file and stored in the tag database of the server. It is to be noted that, unlike the first embodiment in which the tag generator of the first embodiment generates a tag, in the second embodiment, the tag information is dynamically generated in real time.

A block diagram of the framework of the present invention will be shown in FIG. A playback flowchart of the movie/image embedded with external information will be shown in Figure 2. The implementation flowcharts of Embodiment 1 and Embodiment 2 will be shown in Figures 3 and 4, respectively.

110. . . Client

120. . . Server side

121. . . Client-server interaction interface module

122. . . Video/image database

123. . . Tag database

124. . . Video/image content analysis module

125. . . Video/Image-External Information Relationship Analysis Module

126. . . External information retrieval engine

111. . . Client-server interaction interface module

112. . . Video/image content analysis module

113. . . Tag embedding engine

114. . . Original film/image database

115. . . Tag information database

116. . . User interface module

First Figure A block diagram of the framework of the present invention (horizontal line stripes represent tag generator-user mode, vertical line stripes represent user center mode, and cross stripes represent two modes simultaneously).

The second picture shows the playback flowchart of the movie/image embedded with external information.

Third Embodiment The flowchart of the implementation of Embodiment 1.

Fourth Figure Flowchart of the implementation of Embodiment 2.

110. . . Client

120. . . Server side

121. . . Client-server interaction interface module

122. . . Video/image database

123. . . Tag database

124. . . Video/image content analysis module

125. . . Video/Image-External Information Relationship Analysis Module

126. . . External information retrieval engine

111. . . Client-server interaction interface module

112. . . Video/image content analysis module

113. . . Tag embedding engine

114. . . Original film/image database

115. . . Tag information database

116. . . User interface module

Claims (10)

An interactive video/image information embedding technology system embedding external information in a movie/image content, the system comprising: a server end comprising: a client-server interoperation interface module for interacting with a client A video/image database for storing movie/image data; a tag library for storing external information about the target; and a video/image content analysis module for segmenting, tracking, and recognizing the movie/image Specific target in the object; an external information retrieval engine for retrieving external information from a public search engine, tag database, or additional database; a video/image-external information relationship analysis module for instant production of movies / the label of the specific target in the image; and a client includes: a client-server interaction interface module for interacting with the server; a user interface module for interacting with the user/tag generator An original video/image library for storing movie/image data; a tag information database for storing objects related to the object Information portion; a video / image content analysis module for segmenting, tracking, identification video / image specific object; a tag embedded engine, for generating video / image files corresponding to the label. An interactive video/image information embedding technology system as described in claim 1, wherein the system is operable in a plurality of modes, including: a tag producer-user mode (ie, embodiment 1), wherein the tag The producer generates a label file corresponding to the movie/image on the client in advance, and uploads the movie/image unit embedded in the external information to the server. The video/image viewer (ie the user) selects and views the movie/image on the server and performs an interactive external information display; a user center mode (ie embodiment 2), wherein the film/image viewer (ie, the user) directly selects the target on the movie/image, and the external information and tags related to the target are instantly generated by target recognition and instant information retrieval. The interactive video/image information embedded technology system described in the first and second applications of the application scope, wherein the system embeds the external information of the film/image unit: a film/image unit contains: (a) a film/ The image file, (b) the corresponding label file specifies the location of the target in the movie/image, the size of the area, and the external information related to the target; etc.; when the movie/image is played, the corresponding label file will Instant analysis based on the user's operational information. When the user moves the cursor to a range of objects specified by the tag file that contain external information, the external information tag corresponding to the target is automatically popped up. Otherwise, no external information tab will pop up, and the movie/image will be played as usual; the video/image unit embedded with external information can be played not only on the client through the user interface module, but also directly on the servo. The device end is played through the user operation interface module and the client-server interaction interface module interconnection; the pop-up information label established by the system can be closed by using a lock/unlock button (disable) ) / Enable the pop-up information tag function in the movie/image. The interactive video/image information embedding technology system described in the first and second applications of the application scope, wherein the external information retrieval engine can be combined with a text-based or image-based public search engine, a tag database, or additional materials. The libraries are linked to each other so that the analysis results for the specific target obtained in the film/image content analysis module can be used as keywords for retrieval into these databases or search engines to retrieve and target specific objects. Relevant external information. The interactive video/image information embedded technology system described in the first and second applications of the application scope, wherein the video/image content analysis module can be used simultaneously on the client and the server for the video / Automatic or manual segmentation, tracking, and recognition of specific targets in the image. For example, the interactive video/image information embedded technology system described in the first and second applications of the application scope, wherein the tag embedding engine or the film/image-external information relationship analysis module can be used on the client or the server. The engine or module inputs the target analysis information generated by the video/image content analysis module and the external information of the tag generated by the tag information database or the external information retrieval engine, and finally outputs a tag file. For example, the interactive video/image information embedded technology system described in items 1 and 2 of the application scope, wherein the video/image database and the tag database used by the client and the server are used to store the video respectively. / Image data and label files. The interactive video/image information embedded technology system described in the first, second and third aspects of the application, wherein the system has two interface modules, including: a user operation running on the client. Interface module for interacting with users or tag producers for video/image uploading, tag information addition, generating movies/images embedded with external information, playing movies/images embedded with external information, and obtaining users interested in Target or target information, and other operations; a client-server interaction interface module running on both the client and the server for interaction between the client and the server, including File upload/download, user information verification, user login operation, operation instruction delivery, and other operations. The interactive video/image information embedded technology system described in the first and second aspects of the application, wherein the architecture between the server and the client is: one or more simultaneous and multiple clients A server that interacts with interactive information embedding and video/image playback for multiple users; client devices include TVs, personal computers, laptops, mobile phones, smart phones, tablets, projectors, And any other video/image display device, etc.; server-side devices include workstations, servers, cloud computing platforms, and the like. The interactive video/image information embedded technology system described in the first, second and third aspects of the application, wherein the target of interest in the film/image: a target of particular interest to the user in the content of the video The object can be selected or popped in a variety of ways, including but not limited to: moving the mouse cursor to the target, clicking on the target, circle the target, etc.; the target that can be embedded can be any target in the movie/image , including but not limited to: characters, costumes, accessories, accessories, animals, various vehicles, cosmetics, cosmetics, facial effects, background music, plants, geographical landscapes, backgrounds, restaurants, locations, any objects, etc.; Targets that have been embedded with information tags will use some tokens or representations to indicate that the target can be clicked for more external information. These marks or representations include but are not limited to: bottom line words, small color blocks, small dots, watermarks, illuminated lines to circle the target, floating outlines, etc.; pop-up information tags associated with the target can be varied The mode is closed, including but not limited to: moving the mouse cursor away from the target, clicking the "close" button on the information tab, clicking anywhere else in the movie/image, and so on.