TWI566123B - Method, system and wearable devices for presenting multimedia interface - Google Patents

Description

Multimedia interface imaging method, system and wearable device

The present invention provides a multimedia interface imaging method and apparatus based on a user's focus or attention behavior, and determines whether the focus of attention or the behavior of interest is related to the content of the user's browsing to provide dynamically updated content or information. .

Online advertising is a common way for advertisers to advertise products, services or related information to potential customers and customers. Advertisers can implement advertising campaigns through online access devices of online providers such as Yahoo. Online providers provide services to connect advertisers with users who are accessing online resources such as search engines, news, online shopping and other multimedia content. In this way, the advertiser's advertisement can be inserted into the content (including text, image and/or audio-visual related material) issued by the online provider, and provided to the user for browsing to attract the user's attention.

After the browser viewed by the user downloads and stores a webpage content from the website provided by the online provider, the browser displays the content of the webpage to the user. According to the prior art, the browser collects the past browsing behavior or retrieval behavior of the user, and returns the behavior message to the website. When the user downloads a webpage content from the website, the website provides the user based on the behavior message. The webpage of the content of interest is given to the browser, so that the browser depends on the webpage to be downloaded The content is displayed to the user for browsing. The web page is accessed by the user on the device, such as a desktop computer, a portable computer (such as a laptop), and a handheld device (such as a tablet and a smart phone), or a media device (for example) TV) and so on. The content presented by the web page can be displayed in a series of selected items (for example, one item and then another item). In some cases, under certain events (such as after a certain period of time, scrolling a mouse, or clicking a blank button), the stream is updated with new content at the top or bottom of the page.

The conventional browser has no technical means, such as dynamically updating a part of the content of the webpage based on the user's immediate browsing behavior of the webpage content, in particular, updating the part of the webpage that the user has not viewed the webpage. Content to provide content that the user is interested in. These behaviors of interest, especially the implicit behavior of the user, are related to the content of interest in the webpage by operating an input interface (such as a mouse/keyboard/touch panel), but not reloading ( Reload) a web page or link to a new web page, and does not produce the effect of updating the content of the web page.

When the conventional communication device performs the function of the conventional browser, the processor of the conventional communication device does not "instantly" collect the signal of the input interface operated by the user's immediate browsing of the behavior of the web page, so that the conventional communication device The processor cannot determine the association between the user's attention behavior and the displayed content of the webpage, so that the conventional communication device cannot provide the content of the user who is interested in the instant browsing or the previously visited webpage. Part of the content. The attention behavior may relate to the user's eye movements, or signals generated by the user operating an input device (eg, a mouse, a keyboard, a trackpad).

Existing websites can only collect user browsing behavior or search behavior based on the past (Belonging to explicit behavior, such as website browsing records), to provide a web page containing content that the user is interested in to a conventional browser. The existing website has no technical means, such as receiving the dynamic update request from the conventional browser and providing the media to the browser to update part of the content being browsed, and the dynamic update request is in response to the user's immediate browsing of the content of the webpage. The behavior is produced.

Moreover, for example, a conventional wearable device, such as "Google Glasses", can display a message through its projection device, but the conventional wearable device has no technical means, such as generating the focus based on the user's focus. A message in which the focus of attention relates to the user's eye movements.

For the user's implicit behavior, such techniques are not provided in the current related art to collect these implicit behaviors, nor do they analyze the topics that the user may be interested in or interested in based on the implicit behavior, and may advertise the topics that may be of interest or The content is presented to the user for immediate viewing, that is, the browser is given a dynamic update capability closer to the user.

The object of the present invention is to dynamically update the multimedia content viewed by the current user according to the explicit/recessive behavior of the interaction between the user and the communication device, which incorporates the user's attention behavior as a factor of judgment, that is, A collection of more reference samples is used to correlate to content that the user may be interested in, and to update a portion of the multimedia content in an instant and dynamic manner. The user does not have to interrupt the content being viewed, but still can receive dynamic updates of the multimedia content of interest.

In order to achieve the above object, the present invention provides a multimedia interface imaging method, which is implemented in a communication device, which includes one or more processors, a storage device, An input interface, a communication module, and an imaging device, the method comprising: the processor downloading a multimedia content from a network by using the communication module; the storage device stores the multimedia content; and the imaging device displays the multimedia content a first portion; the processor generates an associated signal in response to an event of interest; and the processor parses the associated signal to update a second portion of the multimedia content.

The present invention further provides a multimedia interface imaging method, which is implemented in a communication device. The communication device includes one or more processors, a storage device, an input interface, a communication module, and an imaging device. The method includes: Downloading, by the communication module, a multimedia content from a network; the storage device stores the multimedia content; the imaging device displays a first portion of the multimedia content; wherein the processor collects at least one collected signal, Collecting a signal related to an event of interest; the processor is transmitted to a remote end by the communication module to obtain an updated media content; and the processor updates the updated media content to a second portion of the multimedia content .

The present invention further provides a multimedia interface imaging system, including one or more processors, a storage device, an input interface, a communication module, and an imaging device, wherein the processor is downloaded from a network by using the communication module A multimedia content is stored in the storage device, and the processor processes the first portion of the multimedia content to be displayed by the imaging device, wherein the processor parses an associated signal generated in response to an event of interest, thereby changing the storage A second portion of the multimedia content in the storage device.

The present invention further provides a multimedia interface imaging system, including one or more processors, a storage device, an input interface, a communication module, and an imaging device, wherein the processor is downloaded from a network by using the communication module a multimedia content stored in the storage device, the processor Processing the first portion of the multimedia content to be displayed by the imaging device, wherein the processor collects at least one collected signal associated with an event of interest; and the processor parses the signal with the multimedia content The first part of the association is obtained by the communication module requesting a remote server to obtain an updated media content, and the processor embeds the updated media content in a second part of the multimedia content stored in the storage device.

The present invention further provides a media providing method, implemented in one or more servers, the server being connected to a media database, wherein the server is in communication with a communication device, the method comprising: the server providing a multimedia content, the communication The device causing the communication device to display a first portion of the multimedia content; the server providing a setting message to the communication device to enable the communication device to respond to an associated signal generated by an event of interest based on the setting message; the server Receiving an update request from the communication device, the update request being generated by the communication device parsing the associated signal; and the server providing an updated media content of the media database to the communication device according to the update request, The communication device embeds the updated media content into the multimedia content displayed by the communication device or the displayed multimedia content.

The present invention further provides a media providing system, comprising: one or more servers and a media database, the server is connected to a media database, wherein the server communicates with a communication device, and the server provides a multimedia The communication device enables the communication device to display a first portion of the multimedia content, wherein the server provides a setting message to the communication device, so that the communication device can generate a response event according to the setting message. An associated signal; the server receives an update request from the communication device, the update request is generated by the communication device parsing the associated signal; and the server provides an updated medium of the media database according to the update request Content to the communication device, causing the communication device The updated media content is embedded in the multimedia content displayed by the communication device.

The present invention further provides a multimedia interface imaging method, which is implemented in a wearable device, which includes one or more processors, a storage device, an image capturing device, a sensing device, and an imaging device. The method includes: the image capturing device acquires an image data; and the storage device stores the image data; wherein the processor generates an associated signal according to an event of interest, wherein the attention event is associated with a portion of the image data; The device parses a portion of the image data according to the associated signal to generate a message; and the imaging device displays the message, wherein the message is part of the image data.

The present invention further provides a wearable device comprising one or more processors, a storage device, an image capturing device, a sensing device and an imaging device, wherein the processor is connected to the image capturing device to obtain An image data, and the storage device is connected to store the image data, wherein the processor generates an associated signal according to an event of interest, and identifies a portion of the image data according to the associated signal to generate a message, and the processor connects the image The device displays the message, wherein the event of interest is associated with a portion of the image material; and the message is part of the image material.

As described above, the method and system of the present invention utilizes at least one collected signal to correlate with a user to view an event of interest defined by the content of interest, thereby correspondingly generating a portion of the multimedia content or a portion of the image data. Updates, so that users can dynamically link to related content and messages that are of interest to users in the next or recently viewed content.

1‧‧‧Communication device

100‧‧‧ processor

102‧‧‧Storage device

104‧‧‧Communication module

106‧‧‧Display/projection device

202‧‧‧The last paragraph that has been viewed

204‧‧‧Parts of the multimedia project being watched

206‧‧‧The following paragraphs that have not been viewed yet

302-320‧‧‧Steps

402-416‧‧‧Steps

602‧‧‧ refrigerator

604‧‧‧chandelier

606‧‧‧ wine

The first figure is a block diagram showing the composition of a communication device; The second figure is a schematic diagram of the information flow displayed on the communication device; the third figure is a flow chart of the steps provided by the multimedia of a specific embodiment; the fourth figure is a flow chart of the steps provided by the multimedia of another specific embodiment; A wearable device that can be used to implement a particular embodiment of the present invention; and a sixth view is a schematic view of a user viewing a scene through an optical head mounted display.

The present invention will be described more fully hereinafter with reference to the accompanying drawings, in which FIG. However, the subject matter of the present disclosure may be embodied in a variety of different forms, and thus the subject matter of the claims is intended to be illustrative, and not limited to any exemplified embodiments set forth herein; Use. Similarly, it is intended to mean a reasonable and broad range of subject matter claimed or covered. It is important that, for example, the subject matter identified herein is a method, apparatus, component or system. Therefore, the following detailed description is not intended to limit the scope of the claims.

Throughout the specification and claims, the terms have the meanings indicated or implied in the context, and are beyond the meaning of the explicit description. Similarly, when the phrase "in a particular embodiment" is used herein, it is not necessary to refer to the same embodiment, and the term "in another embodiment" is not necessarily used in this context. Refers to different specific embodiments. For example, it is intended that the subject matter of the invention may be construed as a combination of the specific embodiments.

In general, terms may be understood at least in part from the use in context. For example, terms such as "and", "or", or "and/or" as used herein may include The various meanings are based, at least in part, on the context in which such terms are used. In general, if you use "or" to associate a list, such as A, B, or C, it means here as the inclusive meanings A, B, and C, and the specific meanings A, B, or C. In addition, the phrase "one or more" is used herein to mean any feature, structure, or characteristic of the singular form, or a combination of features, structures or characteristics of the plural. Similarly, terms such as "a" or "the" are used to mean the s In addition, the term "based on" is understood to mean not necessarily a set of specific factors, but rather other factors that are not necessarily described, which are based, at least in part, on the context.

In a specific embodiment, a multimedia providing system disclosed in the present disclosure may include at least one communication device, at least one server, and at least one multimedia server. The communication device can communicate with the server through a network. The server can also communicate with the multimedia server through the network or other network, and obtain multimedia and provide multimedia to communication devices from the multimedia server. In another embodiment, the multimedia server may include at least one advertisement server and/or at least one content server, and the advertisement server and/or the content server may in turn be connected to a media library in which the multimedia content is stored.

The communication device is a computer or other processing system that includes one or more processors and has the ability to communicate with any one or more servers. For example, each communication device can be considered a client that can be accessed by a user to a website hosted by the server. The client can be a web crawler, a web browser, a non-web enabled client, or any other suitable client form. Those skilled in the art will appreciate that each communication device has the ability to connect to any server to access the website hosted on the server.

The network is made by using a network of well-known network communication protocols. The line can be a wired network, a wireless network, or a combination of both. The network may be a wide area network (e.g., internet), a regional network, other network forms, or a combination of the above.

Referring to a specific embodiment shown in the first figure of the present invention, the communication device has one or more processors 100, one or more storage devices 102, a communication module 104, a display/projection device 106, and an input interface. The processor 100 can execute an operating system (such as a computer/mobile operating system, Windows, iOS, Linux, Android, or Windows Mobile), an application (such as a web browser), multimedia content, or other software/toughness stored in the storage device 102. Body to perform various possible tasks, such as browsing, searching, and playing various multimedia content. The communication module 104 is controlled by the processor to communicate with other devices or servers through the network, and to transmit information to or receive information from the network, and to access data in the remote device or server (for example, , access to web content, email, advertising content and multimedia content, etc.). Display/projection device 106 can be a 2D or 3D monochrome/color display screen, or can include an optical mirror imaging system that is configured for use in a portion of a web page or multimedia content. As shown in the second figure, a part of a stream of streaming media can be displayed on the display screen of a communication device, and the connected content is browsed through scrolling. The input interface includes a keyboard, a mouse, a touchpad, and/or other sensors that generate an input signal by sensing (eg, a CCD sensor). In some embodiments, the display/projection device 106 is integrated with the input interface to provide an input means for visual response. In other implementations, the communication device can further include one or more bus devices (eg, USB ports), one or more accelerometers, one or more gyroscopes, global positioning system (GPS), or other location identification Type of function, audio and video equipment.

For example, the communication device can be a desktop computer configured with an input device Or a portable device, wherein the portable device can be a mobile phone, a smart phone, a radio frequency (RF) device, an infrared (IR) device, a personal digital assistant (PDA), a handheld computer, a tablet, a laptop A computer, a wearable computer device or an integrated device incorporating the features of the aforementioned device features, and the like. The foregoing is intended to illustrate that the claimed subject matter is intended to encompass a wide range of possible features or capabilities.

The server is a computer or other processing system that includes one or more processors and has the ability to communicate with the communication device. Each server is configured to host a service provided by one or more service providers (eg, a service such as a website and/or an email address), and the service can be stored by a user of the communication device over the network take. In the process of the user accessing the service provided by the server through the communication device, the service provided by the server may be combined with an advertisement service (for example, background advertisement, sponsor advertisement, display advertisement, etc., in any form) And provided to the user of the communication device via the network. The server includes one or more storage devices, which can store related data according to the identity of the user or the communication device, for example, can collect and collect the user's browsing history, the user's personal profile and the like.

The server is configured with an executable software program, which can be used to provide various multimedia content or information (including web pages, pictures, other forms of files, output on the server, instant messaging (IM) conversation content, electronic Mail, ads, etc.) to respond to requests received by the server. In another embodiment, the software program can also be used to collect various information stored in the communication device and/or various events appearing on the communication device, and collect information and/or information collected from the communication device. The event is analyzed to produce an analysis result and/or to generate a specific request. In yet another embodiment, the particular request may vary depending on the type of information and/or event and may be requested in response to a particular server/device.

The server can include a training module that performs a training algorithm. The training algorithm may use the signal collected by the server or the stored data as an input parameter, and perform comprehensive judgment on the input parameter through the algorithm, and output a better output result. For example, the server may retrieve at least one collected signal from a communication device as an input parameter of the algorithm, and determine one or more generated by the user of the communication device by comprehensively determining the at least one collected signal. Focus on the event. The definition and relevance of the collected signals will be detailed in subsequent paragraphs. In another embodiment, the training algorithm can be executed on a communication device, for example, the algorithm can be written into an application (such as a browser) of the communication device, and run along with the application. In another embodiment, the training module is disposed in a communication device, and directly captures required input parameters, such as at least one collected signal, directly to the user of the communication device. One or more attention events.

The server is configured to provide a setting message to the communication device, wherein the setting message can be associated with an event of interest. The communication device can generate an associated signal triggered by the user of the communication device according to the setting message, for example, the setting message can be generated by a training module. In another embodiment, the setting message may also be generated by a communication device, so that the communication device can respond to the setting message according to an associated signal triggered by the user of the communication device. Specifically, the setting message can be used to initiate a collection/judgment procedure for a user's immediate sexual behavior. When a user of the communication device generates a specific collected signal by paying attention to the content of some multimedia, it is regarded as an attention event, and the attention event is collected by at least one signal (collected by the communication device or collected by the server) Storage) is defined by a portion of the association with the multimedia content. The event of interest may generate an associated signal generated based on the collected signal being associated with a portion of the multimedia content, and the associated signal is parsed to generate an update request. For example, the generated correlation signal can be parsed by the communication device and generated as an associated update request. Alternatively, the generated correlation signal may be transmitted to a remote end, such as a server, and the associated signal is parsed by the server to generate an associated update request.

When the user browses a multimedia content, such as a web page, one or more collection signals are available for a period of time, which may determine the number of events of interest that occurred during the period of time, thereby generating one or more update requests.

In various preferred embodiments, the collected signal may include a key input signal, a scrolling signal, a module signal, a page signal, and a network signal ( Network signal), identification signal, cursor signal, and click signal.

The button input signal can be triggered by a user of the communication device, for example, a combination key (Control+C) input by the user to perform a whitening or copying operation on the text content through the keyboard, or a combination of a keyboard and/or a mouse. Key to capture, zoom (Zoom-in / out) multimedia content. Or, if the content is not redirected to another link or web page, the content entered in the input field. For another example, the user adjusts the physical/virtual dialer button, and the like. The above examples can be used as an implementation of a concern event generated by the user of the communication device for multimedia content. However, it should be understood by those of ordinary skill in the art that the present invention is described by way of example only and not limitation.

The scrolling signal can be triggered by a user of the communication device. For example, the user scrolls through a multimedia content, such as a webpage, through a keyboard or a mouse wheel to selectively view a part of the webpage, such as the second image. Expressed. Further, when the scrolling stops after a predetermined time has not been scrolled again, the scrolling signal is triggered, which represents the user of the communication device. A focus event is generated for certain multimedia content, and thus stays watched. The above examples are all examples of an event of interest that can be generated by the user of the communication device for multimedia content. However, it should be understood by those of ordinary skill in the art that the present invention is described by way of example only and not limitation.

The module signal can be triggered by a user of the communication device, for example, the user expands the multimedia content through the input of the keyboard or the mouse (without redirecting to other links or web pages). For example, the news category can be expanded into multiple sub-options by clicking on it. For example, "Sports News" is expanded into "Golf", "Basketball" and "Soccer". This may also represent a user of the communication device generating an event of interest for certain multimedia content while taking further action. The above examples are all examples of an event of interest that can be generated by the user of the communication device for multimedia content. However, it should be understood by those of ordinary skill in the art that the present invention is described by way of example only and not limitation.

The page signal is triggered by a page theme that is viewed by a user of the associated communication device, and the page theme is independent of whether it is scrolled, zoomed, input, clicked, or the like. For example, if the user of the communication device is viewing a page about the theme of the shoe for a predetermined period of time, the user representing the communication device may generate an event of interest for the multimedia content presented by the topic page. The above examples are all examples of an event of interest that can be generated by the user of the communication device for multimedia content. However, it should be understood by those of ordinary skill in the art that the present invention is described by way of example only and not limitation.

The network signal is triggered by the network status of the associated communication device. The network status includes relevant IP information of the IP location of the communication device, the geographic location of the GPS, the time of the location, and/or the weather. For example, the communication device used by the user will be specific The festival transforms the state of the network, such as Christmas. This may also be of interest to the user to generate an event of interest. The above examples are all examples of an event of interest that can be generated by the user of the communication device for multimedia content. However, it should be understood by those of ordinary skill in the art that the present invention is described by way of example only and not limitation.

The identification signal can be triggered by the action of the user of the communication device. For example, a communication device has an eyeball recognition device and an image recognition function. When the user's eyes pay attention to certain multimedia content on the communication device for a predetermined period of time, the image recognition can recognize the information displayed by the multimedia content of interest. It means that the user may generate an event of interest for the content of the multimedia. For another example, a communication device has a sound collection device and a voice recognition function, and the same result can be associated with the user's attention event for certain topics from the results of the sound collection and voice recognition. The above examples are all examples of an event of interest that can be generated by the user of the communication device for multimedia content. However, it should be understood by those of ordinary skill in the art that the present invention is described by way of example only and not limitation.

The cursor signal may be triggered by a user operating a mouse cursor or a cursor on a touch screen. For example, when the user views the multimedia content, the cursor is moved to a multimedia object, such as a picture, and remains on the object. For a period of time, a user representing the communication device may generate an event of interest for the multimedia object. For another example, the user operates the mouse cursor or the cursor on the touch screen to make the cursor sway for a predetermined time in a multimedia object, which may also represent a user's attention event for a multimedia object. The above examples are all examples of an event of interest that can be generated by the user of the communication device for multimedia content. However, it should be understood by those of ordinary skill in the art that the present invention is described by way of example only and not limitation.

The click signal can be triggered by a user operating a mouse or a touchpad to select a multimedia object, such as a picture, whether the click redirects to another page or reloads new content, which represents the communication device. The user may generate an event of interest for the multimedia object. The above examples are all examples of an event of interest that can be generated by the user of the communication device for multimedia content. However, it should be understood by those of ordinary skill in the art that the present invention is described by way of example only and not limitation.

The types and examples of the collected signals listed above are merely exemplary of the embodiments, and are not limited to the above examples. It will be apparent to those skilled in the art that a user of a communication device may exhibit various aspects of dominant behavior and/or recessive behavior by focusing on particular multimedia content to define a collection signal.

The server retrieves the multimedia item corresponding to the update request from the multimedia server according to the update request, and transmits the multimedia item to the communication device, and implants the multimedia content displayed by the communication device, such as a webpage. For example, in the second figure, the multimedia item obtained according to the update request may be immediately implanted into a part of the webpage that the user is currently viewing, or replace part of the multimedia content 204 currently being viewed in the webpage (the first part of the multimedia content) ). In another embodiment, the content of the upper page 202 of the webpage that the user has browsed and/or the content of the lower section of the webpage 206 (the second part of the multimedia content) that the user has not browsed may also be implanted or replaced with new multimedia content. . In yet another embodiment, the second portion of the multimedia can be part of the first portion. The second part of the multimedia is a sort of text or advertisement or search keyword or a picture or video.

The content server can communicate with a content repository. The content repository stores data that the content provider uses to define content items, such as text, graphics, and multimedia content. The content server can access content items from the content repository or from anywhere on the network. Such access may include developing content items, editing content items, deleting content items, and the like.

The content server provides a program, application or software subroutine (front end) that forms a user interface between the user and the content server. In some embodiments, the interface may be one or more operable web pages, which provide a user or content provider to view and edit the content item through the communication device, and save the content item to the content database for subsequent use. Transfer to the communication device.

The content item may be formatted by the content server and transmitted to a webpage or a stream of content items and advertisement items and provided to a communication device. The formatted content item, the appearance, size, shape, text format, graphic format, and the like of the presented content conform to the format in the webpage or the information stream, so as to facilitate integration and presentation to the user of the communication device.

The advertisement server supplies an advertisement item to the communication device, and the provision of the advertisement item is associated with the implicit behavior of the user of the communication device, and the association will be described in detail in the following paragraphs. The advertisement server communicates with an advertisement database, which stores information provided by the advertiser for defining an advertisement item, including text data, graphic data, video data, and/or audio data. In another embodiment, the advertising item may further include one or more links to other network resources.

The ad server can provide advertising items to a particular communication device based on an update request from the server. The advertisement server may format the advertisement item and transmit it to a webpage or a stream of content items and advertisement items and be provided to a communication device. For example, when the server issues an update request, the ad server can provide one or more ad items. To the server, the request should be updated back and forth. The update request can be triggered by a user of the communication device. The advertisement server may select an appropriate advertisement item to be provided to the user according to the explicit information and/or the implicit information contained in the update request. The formatted advertisement item, the appearance, size, shape, text format, graphic format, etc. of the presentation are in conformity with the format of the webpage or the information stream, so as to facilitate integration and presentation to the user of the communication device. Advertising items can be categorized according to their own characteristics, such as meta data contained in the advertising project itself.

The ad server provides a program, application or software subroutine (front end) that forms a user interface between the user and the ad server. In some embodiments, the interface may be one or more operable web pages, which provide an advertiser with a communication device to view and edit content items, and store the content items in an advertisement database for subsequent transmission to the communication device. .

The content server and/or advertisement server can be a computer system, one or more servers, or any other computing device known in the art, including one or more processors. Or in another embodiment, the advertisement server is a computer program, instructions, and/or software code stored on a computer readable storage medium and processed and executed.

Referring to the third figure, a method flow for providing and imaging multimedia includes steps 302 to 320, which may be provided by a multimedia providing system including a communication device (as shown in the first figure), a server, and a multimedia server. Implemented. The relevant contents of the communication device, the server and the multimedia server are described in the foregoing paragraphs, and therefore will not be described here.

The method begins in step 302. A user of a communication device can send a multimedia content request to one or more remote servers via the network via the network, for example, requesting a webpage containing one of multimedia content. The server obtains from the multimedia server according to the multimedia request One or more multimedia items can be downloaded to the storage device of the communication device along with the web page, and the process proceeds to step 304. For example, the user can access the webpage of the online video playback service through a search webpage hosted by the server (for example, Yahoo!). The requested web page may be a web page filled with information streams containing at least two different types of items, in whole or in part, as shown in the second figure. Contains content items and advertising items. However, in another embodiment, other types of items may also be provided in a stream of information, and the item type or item category may be selected according to any convenient or useful criteria.

In step 306, the content of the information stream can be selectively displayed on the communication device. The content items and advertising items in the information stream can be appropriately formatted and arranged according to the display size defined by the communication device to conform to the viewing standard. The webpage and/or the first part, the second part or the third part of the multimedia page may be displayed by scrolling or two-dimensional movement of the webpage, and each part may include at least one multimedia object, such as a partial photo, text or Is an image/animated advertising project.

In step 308, the collection action of the communication device can be initiated by a setting message generated by the server or the communication device. The setting message may include a condition for the communication device to collect (for example, a periodic rule that causes the communication device to take a collection action every predetermined time, or an action that causes the communication device to continue collecting for a predetermined period of time, Or collect it in an irregular way; or relate to the preferences and preferences of a particular user). The communication device collects behaviors related to the input interface of the communication device according to the setting signal, for example, the collected signals mentioned above, including a key input signal, a scroll signal, a module signal, a page signal, a network signal, and an identification signal. , a cursor signal and/or a click signal, etc., which are triggered by a user through an input interface of the communication device. The collected signal may be one or more, which represents the One or more events of interest generated by the user of the communication device for the multimedia item that he or she viewed during a time or a predetermined time. The event of interest is defined by one or more collected signals and/or by the set signal via a comprehensive determination.

After obtaining the collected signal in step 308, depending on the capabilities imparted by the communication device and the server itself, the subsequent processing may determine the event of interest via the communication device (steps 310-314) or via a remotely located servo. The device determines the event of interest (steps 316-320).

In step 310, the communication device can generate an associated signal based on the attention event generated by the previous step. The associated signal is generated based on the association of the collected signal with a portion of the multimedia item displayed by the communication device. In other words, the associated signal includes related information of a part of the multimedia item, such as information about the producer of the multimedia object, the source, the text description, etc., which can be interpreted from the metadata attached to the multimedia object, and performed. Comprehensive judgment.

In step 312, the communication device can parse the multimedia related information included in the associated signal by its own processor, and comprehensively determine the related information through a specific algorithm to generate an update request. For example, if the processor parses out the generated event of interest as "sneakers", the corresponding update request may include a content item request and/or an advertisement item request related to "sneakers". The server obtains an updated media content from the multimedia server according to the update request, which includes one or more content items and/or advertisement items, the number of which may be determined by the content items and the space of the advertisement items placed in the current information stream. . The advertising item can be a combination of graphics, pictures, text, audio and video animation or other multimedia. The advertising project may also include one or more links to other web pages or other multimedia applications.

In step 314, the server pushes the updated media content to the communication device for the processor of the communication device to place the updated media content into a portion of the multimedia item in the stream being viewed (as shown in the second figure). The first portion of the multimedia content 204) or the portion not viewed/viewed (such as the second portion 202, 206 of the multimedia content shown in the second figure), or the multimedia item belonging to the non-interest event in the original information stream Replace with this updated media content. In another embodiment, the location where the updated multimedia is placed and/or the location where the multimedia item is replaced may be included in the portion of the original information stream that has not been viewed or the portion that has been viewed, as shown in the second figure. The upper segment 202 content and the lower segment content 206 (such as the second portion of the multimedia content shown in the second figure). In another embodiment, the information stream has an option or link that is presented in a multimedia manner, and may also be an object that is updated by the updated media content. For example, a webpage in which a communication device presents various types of news has a plurality of clickable icons. News classification options, (political, sports, international, social, etc.), when the communication device determines that its user's event of interest is sports, the news classification options can be updated by the updated media content to be presented by the multimedia item Various options form different combinations. For example, the news classification options are updated to sports related categories and related links such as "golf", "basketball" and "soccer". In short, the event of interest is to determine whether a signal generated by the input interface of the communication device is associated with a portion of the multimedia content, such as the first portion 204 of the multimedia content shown in the second figure. It should be noted that the above update action does not redirect or separate the information stream or web page that the user of the communication device is viewing to other pages.

Alternatively, step 308 can continue with step 316. In step 316, the collected signals can be uploaded by the communication device to a remotely located server, and the server performs analysis and comprehensive judgment on the collected signals. In another embodiment, the collected letter uploaded to the server The number can be classified according to the identity of the user or the communication device, that is, the event of interest of the specific user is recorded, so as to provide a reference for the server to perform the comprehensive judgment of the user association.

In step 318, based on the result of the integrated judgment of the collected signal, an updated media content is obtained from the multimedia server, which is then selectively placed in a stream of information or a portion of the web page that the user of the communication device is viewing and / or other parts of the stream, or replace / replace the original stream or multimedia items in the web page, but not redirect the original page to other pages.

Referring to the fourth figure, which is a method for obtaining updated media by using a collected signal in another embodiment, comprising steps 402 to 416, which may include a communication device (such as the architecture shown in the first figure), and a servo. Implemented by a multimedia delivery system for multimedia servers. The relevant contents of the communication device, the server and the multimedia server are described in the foregoing paragraphs, and therefore will not be described here. Steps 402 through 416 can be included in steps 302 through 320.

In step 402, the acquisition of the collected signal is triggered by interaction between a communication device and its user through the input interface. The collected signals include a key input signal, a scroll signal, a module signal, a page signal, a network signal, an identification signal, a cursor signal, and/or a click signal, as exemplified above. Those skilled in the art should be able to understand that different types of communication devices can generate different types of collected signals depending on the input interface set by the communication device itself. There are usually a plurality of collected signals, but it is not excluded that there may be only one limitation that may be limited by the input interface of the communication device itself.

In step 404, the obtained collected signal can be comprehensively judged by a training algorithm provided by a training module to determine an event of interest. This is because the number of collected signals obtained in step 402 is very large, but it does not necessarily reflect that the user may actually shut down. Note events, such as user-generated collection signals may be related to "politics", "sports", "science", but in fact users may be concerned about "sports", and it is conceivable that the final update will be Not as expected. Therefore, the use of machine training can pre-empt events that the user may not be interested in, in order to facilitate the results of the final dynamic update. The training algorithm may be a function whose output is associated with one or more events of interest, and further generates a setting message according to the outputted event of interest (go to step 406). The collected signal is one or more input variables or samples of the function that are used to assist in training to continually correct the output to optimize the accuracy of the training module. Those skilled in the art should be able to understand that in order to respond to the ability of their comprehensive judgment, the function type of the training algorithm can be adjusted by conventional means, or an appropriate number of samples can be selected to improve the judgment accuracy of the training module.

In step 406, the training module has determined an event of interest based on the collected signal, and the event of interest is used to generate a setting message containing a feature associated with the event of interest, indicating that the user is generating a certain What are the possible concerns about the situation in which these signals are collected. Due to the pre-sample training, the description contained in the setting message should be closer to the actual event of interest of the user.

In step 408, the setting message is provided to the communication device, and the communication device further collects one or more collected signals according to the setting message.

In step 410, the communication device controls the associated message obtained in the previous step to be associated with an associated signal, and parses the associated signal to generate an update request, as described or similar to step 312. The update request is received by the server for storage.

In step 412, the server responds to the update request and retrieves an appropriate updated media content from the multimedia server, the signal associated with the signal signal having a certain degree of phase turn off. The updated media content is provided to the communication device and presented to the user of the communication device in a manner that is dynamically updated as described by synchronization or similar step 314, including dynamic updates of the content item and dynamic updates of the advertising item, and Does not interrupt the content or page that the user is watching.

Alternatively, step 408 can continue with step 414. In step 414, the server receives and stores the collected signal from the communication device. Next, following step 416, at this time, the collected signal can follow the training result of the training module and further perform the comprehensive judgment to generate one or more attention events. The attention event generated in this step is used as a reference for the server to retrieve an updated media content from the multimedia server. In another embodiment, the attention event generated by this step can be simultaneously returned to step 406 to provide re-execution (not shown) since it can be regarded as a new training result. Finally, an updated media content is provided to the communication device for dynamic update.

The foregoing specific embodiments of the invention may be implemented in various examples. For example, a user is browsing a webpage (search webpage) containing a multimedia content item through a tablet. When one or more collected signals are triggered during the interaction of the user with the webpage, such as an input interface signal, a cursor signal, and a network signal, wherein the input interface signal is triggered because the user is on the page The field is typed into the keyword of interest (such as moon cake), and the cursor signal is triggered because the user's cursor stays or wanders for a period of time (such as the Mid-Autumn Festival) corresponding to the content on the touchpad, and the network signal The trigger is caused by the network status of the user's communication device (for example, the geographic location is Taipei), and according to the result of the comprehensive judgment, a part of the webpage (including the part that the user is viewing, browsing, and not browsing) The content item field can be updated to, for example, a photo or article of the moon cake; and the advertising item field can be updated to a subscription advertisement for the mooncake gift box or a bakery for Taipei, and the number of items updated can be set to retain the portion of the original web page. Inside Rong.

Of course, the foregoing specific embodiments of the present invention can also be implemented in a wearable device. For example, as shown in the fifth figure, it can be implemented in an optical head mounted display 5 (such as Google Glass) equipped with an eyeball recognition device, except for the components of the aforementioned communication device (such as a processor, a storage device, etc.) The image capturing device 502, an imaging device 504, and/or a sensing device (not shown) are included. The image capturing device 502 is configured to be used for capturing image data, and can be stored in the storage device, such as taking pictures and recording. The imaging device 504 is configured to guide and zoom the light source emitted by the projection device and focus on the user's retina for the user to view a picture. The processor is configured to generate an associated signal in response to an event of interest, wherein the event of interest is associated with a portion of the image data, and the processor further parses a portion of the image data according to the associated signal to generate a message ( A part of the image data is displayed and displayed through the imaging device. The sensing device can be configured as an eyeball detector for recognizing an eyeball, and can adopt an image recognition method generally used to interpret the angle or focus position of the image viewed by the user, and can further pass the matching calculation. The method (or other algorithm that achieves the same purpose) matches the user's viewing behavior to the display's picture to arrive at the content that the user is viewing or watching on the screen. Briefly, the event of interest is to determine that the signal generated by the sensing device is associated with a portion of the image data. The triggering device can be configured to trigger an identification signal associated with an associated signal; and the input interface at this time can correspond to an eyeball sensing device (such as a CCD sensor or a camera) of the optical head mounted display, that is, the sensing device Capture image data about the user's eyeball and store it for inclusion in the judgment factor.

As a specific example, a user can wear an optical head mounted display with an eye recognition device to capture a scene (ie, take a picture or video), or through an image package. Set to watch a picture. The capture of the scene can be associated with an identification signal (such as an eyeball signal) in conjunction with the behavior (focusing position, viewing angle) viewed by the user. Similarly, the viewing of the screen can be associated with an identification signal in conjunction with the behavior viewed by the user. For example, referring to the sixth figure, the user views the scene through the optical head-mounted display as a kitchen scene, which may include various facilities (such as the refrigerator 602, the chandelier 604) and the items (red wine 606), and the image recognition can be adapted. The identification signal identifies the associated facility or item and obtains relevant information, such as the model/label of the refrigerator 602, the origin information of the red wine 606, the selling price of the chandelier 604, etc., and thereby dynamically updates the optical headset. The content displayed by the imaging device of the display. In addition, the result of image recognition can be further compared with the remote database through the network to obtain information about the identified object.

The foregoing examples are only one of many examples that can be implemented in the present invention, and are not intended to limit the present invention. Those skilled in the art should be able to make good use of the combination of collected signals to determine various various events of interest of the user.

The foregoing detailed description is to be considered as illustrative and not restrict

402-416‧‧‧Steps

Claims (30)

A multimedia interface imaging method is implemented in a communication device, the communication device includes one or more processors, a storage device, an input interface, a communication module, and an imaging device, and the method includes: the processor The communication module downloads a multimedia content from a network; the storage device stores the multimedia content; the imaging device displays a first portion of the multimedia content; wherein the processor generates an associated signal in response to an event of interest; And the processor parses the associated signal to update a second portion of the multimedia content, wherein the second portion of the multimedia content is content that has been displayed or not displayed. A multimedia interface imaging method is implemented in a communication device, the communication device includes one or more processors, a storage device, an input interface, a communication module, and an imaging device, and the method includes: the processor The communication module downloads a multimedia content from a network; the storage device stores the multimedia content; the imaging device displays a first portion of the multimedia content; wherein the processor collects at least one collected signal, the collected signal is related An event of interest; the processor transmits the at least one collected signal to a remote end by the communication module to obtain an updated media content; The processor updates the second portion of the multimedia content by the updated media content, wherein the second portion of the multimedia content is content that has been displayed or not displayed. The multimedia interface imaging method as set forth in claim 1 or 2, wherein the event of interest is to determine whether a signal generated by the input interface is associated with the first portion of the multimedia content. The multimedia interface imaging method as set forth in claim 1, wherein the event of interest is defined by an association of at least one of the collected signals with the first portion of the multimedia content. The multimedia interface imaging method as set forth in claim 2 or 4, wherein the collected signal is selected from the group consisting of an input interface signal, a scrolling signal, a module signal, a page signal, a network signal, an eyeball signal, a cursor signal, and a click signal. One or more of these signals. The multimedia interface imaging method as set forth in claim 1, wherein the processor parses the associated signal to issue an update request to a remote end, so that the remote end provides an updated media content to the communication device. The multimedia interface imaging method as set forth in claim 2, wherein the processor embeds the updated media content in the displayed or undisplayed content of the multimedia content. A multimedia interface imaging system includes one or more processors, a storage device, an input interface, a communication module, and an imaging device, wherein the processor downloads a multimedia content storage from a network by using the communication module In the storage device, the processor processes the first portion of the multimedia content to be displayed by the imaging device, wherein the processor parses an associated signal generated in response to an event of interest, thereby changing the storage in the storage device And a second portion of the multimedia content, wherein the second portion of the multimedia content is content that has been displayed or not displayed. A multimedia interface imaging system includes one or more processors, a storage device, an input interface, a communication module, and an imaging device, wherein the processor downloads a multimedia content storage from a network by using the communication module In the storage device, the processor processes the first portion of the multimedia content displayed by the imaging device, wherein the processor collects at least one collected signal, the collected signal is associated with an event of interest; and the processor resolves Corresponding to the first part of the multimedia content, the communication module requests an update media content from a remote end, and the processor embeds the updated media content in the storage device A second portion of the multimedia content that is displayed or not displayed. The multimedia interface imaging system of claim 8 or 9, wherein the event of interest is determining whether a signal of the input interface is associated with the first portion of the multimedia content. The multimedia interface imaging system of claim 8, wherein the event of interest is defined by an association of at least one of the collected signals with the first portion of the multimedia content. The multimedia interface imaging system as set forth in claim 9 or 11, wherein the collected signal is selected from the group consisting of an input interface signal, a scrolling signal, a module signal, a page signal, a network signal, an eyeball signal, a cursor signal, and a click signal. One or more of these signals. The multimedia interface imaging system as set forth in claim 8 or 9, wherein the input interface is a camera or a mouse or a keyboard or a button or a touch pad or a touch panel, and the imaging device is A display or a projection device. The multimedia interface imaging system of claim 8, wherein the processor sends an update request to a remote end according to the associated signal to receive an update medium provided by the remote end. Body content. The multimedia interface imaging system of claim 9 or claim 14, wherein the processor embeds the updated media content in the second portion of the multimedia content. A media providing method is implemented in one or more servers, the server is connected to a media database, wherein the server is in communication with a communication device, the method comprises: the server providing a multimedia content, the communication device, The communication device displays a first portion of the multimedia content, wherein the server provides a setting message to the communication device, so that the communication device can respond to an associated signal generated by an attention event according to the setting message; the server Receiving an update request from the communication device, the update request being generated by the communication device parsing the associated signal; and the server providing an updated media content of the media database to the communication device according to the update request, The communication device embeds the updated media content into the multimedia content displayed by the communication device or the displayed or undisplayed multimedia content. The media providing method as set forth in claim 16, wherein the event of interest is to determine whether a signal of an input interface of the communication device is associated with the first portion of the multimedia content. The media providing method as set forth in claim 16 wherein the event of interest is defined by at least one collected signal associated with the first portion of the multimedia content. The media providing method as set forth in claim 16 , wherein the server comprises a training module, and the training module determines the event of interest based on at least one collected signal based on a training algorithm. The media providing method as set forth in claim 18 or 19, wherein the collected signal is selected from the group consisting of an input interface signal, a scrolling signal, a module signal, a page signal, a network signal, an eyeball signal, a cursor signal, and a click signal. One or more of these signals. A media providing system includes: one or more servers and a media database, the server is connected to a media database, wherein the server communicates with a communication device, and the server provides a multimedia content communication device, ??? enabling the communication device to display a first portion of the multimedia content, wherein the server provides a setting message to the communication device, so that the communication device can respond to an associated signal generated by an event of interest according to the setting message; The server receives an update request from the communication device, the update request is generated by the communication device parsing the associated signal; and the server provides an updated media content of the media database to the communication device according to the update request And causing the communication device to embed the updated media content into the multimedia content displayed, displayed or not displayed by the communication device. The media providing system as set forth in claim 21, wherein the event of interest is to determine whether a signal of an input interface of the communication device is associated with the first portion of the multimedia content. A media providing system as set forth in claim 21, wherein the event of interest is defined by at least one collected signal associated with the first portion of the multimedia content. The media providing system as set forth in claim 21, wherein the server comprises a training module, and the training module determines the event of interest based on at least one collected signal based on a training algorithm. The media providing system as set forth in claim 23 or 24, wherein the collected signal is selected from the group consisting of an input interface signal, a scrolling signal, a module signal, a page signal, a network signal, an eyeball signal, a cursor signal, and a click signal. One or more of these signals. A multimedia interface imaging method is implemented in a wearable device, the wearable device includes one or more processors, a storage device, an image capturing device, a sensing device, and an imaging device, and the method includes: capturing the image The device obtains an image data; and the storage device stores the image data; wherein the processor generates an associated signal according to an event of interest, wherein the attention event is associated with a portion of the image data; the processor is based on the associated signal Parsing a portion of the image data to generate a message; and the imaging device displays the message, wherein the message is part of the image data. The multimedia interface imaging method as set forth in claim 26, wherein the processor generates the associated signal if the event of interest is to determine that a signal of the sensing device is associated with a portion of the image data. The multimedia interface imaging method as set forth in claim 26, wherein the sensing device is an eyeball detector or an eyeball imaging device. A wearable device includes one or more processors, a storage device, an image capturing device, a sensing device, and an imaging device, wherein the processor is coupled to the image capturing device to obtain an image data, and Connecting the storage device to store the image data, wherein the processor generates an associated signal according to an event of interest, and identifies a portion of the image data according to the associated signal to generate a message, and the processor connects the The imaging device displays the message, wherein the event of interest is associated with a portion of the image material; and the message is part of the image material. The wearable device of claim 29, wherein the event of interest is determining whether a signal generated by the sensing device is associated with a portion of the image data.