TWI632811B - Film and television interactive system and method - Google Patents

Abstract

The invention discloses a video interactive system and method, which comprises a video interactive subsystem, a display device, a biometric capture device and a video interactive server. The video interactive subsystem is an executable video interactive program that can be connected to the video interactive server to play the video content and output to the display device. When playing a specific video clip or time point, the video interactive subsystem can control the biometric capture. The device automatically collects at least one biometric feature and response feedback message of the viewer to watch the video in the manner of not obscuring the viewer's viewing of the video content, and after the recognition and processing of the feedback message, generates a unified information and transmits the video and time integration. Back to the film and television interactive server, let the film and television business or the live master get the viewer behavior during the video playback and analyze the video playback effect.

Description

Film and television interactive system and method

The invention belongs to a film and television interactive system and method, and particularly relates to a method for automatically collecting at least one biometric feature of a viewer watching a movie when the specific movie clip is played or a specific time point is reached, so that the viewer can watch the video content without disturbing the viewer. And feedback information such as response, after processing the feedback message of the viewer, and integrating the video and time axis to generate unified information, the film and television interactive system and method for obtaining the viewer's behavior during the video playback and analyzing the effect of the video playback.

In the past, the TV interactive system used the facial expression to identify the auxiliary judgment and correct the test result, or to analyze the facial expression to determine the time of the pupil dip offset to generate feedback results to understand the advertising effect, and to use the person. Face expression recognition to obtain film viewer behavior and analysis of film and television effects. Furthermore, it is common to analyze the facial expression, dwell time, and pupil offset angle when a user views a specific block in a plurality of advertisement blocks to generate feedback results. In order to understand the effect of advertising, but still can not analyze the viewer's multiple biometric feedback messages according to the film and television timeline.

In view of the shortcomings derived from the above-mentioned conventional methods, the inventor of the present invention has improved and innovated, and after years of painstaking research, he finally succeeded in researching and developing the interactive system and method of this film.

In order to achieve the above object, the present invention provides a film and television interactive system and method, and the application of online video and live video is quite extensive, such as live fitness, promotion of fitness equipment or gym; there are also live teaching, cooking, painting, game live, etc. However, film and television dealers and Streamer can only assess which movies are more noticeable by the viewers' ratings, but they can't grasp which movie clips are more popular, and they can't get much video content. .

Therefore, the object of the present invention is to disclose a video interactive system and method for solving the conventional online video and live video, which only has a large range of statistics and predictions for viewers, but cannot know a certain piece of a particular movie or Whether the content of a video time point is subject to high attention.

Based on the above object, the present invention provides a video interactive system and method, which comprises a video interactive subsystem, a display device, a biometric capture device, and a video interactive server. The film and television interactive subsystem is an executable film and television interactive program. The interactive film and television interactive program can be connected to the video interactive server to play the video content and output to the display device. When playing a specific video clip or time point, the video interactive subsystem can The biometric feature capture device is controlled to automatically capture the feedback information of the at least one biometric feature and response of the viewer while watching the video content without disturbing the viewer, and after the recognition and processing of the feedback message, the video and time are merged. Integrate the information and send it back to the video interactive server, so that the film and television dealer or the live master can obtain the viewer behavior during the video playback and analyze the video playback effect.

A video interactive system, comprising a display device, which is an output end for playing video content; a biometric capture device, which can be a built-in sensor for a handheld device; a video interactive server for providing a pre-recorded Good shadow Depending on the content, providing live and live video content and providing viewers with feedback to obtain the viewer's behavior during film and television playback and analyzing the effects of video playback; and a video interactive subsystem that is connected to the display device and biometrics The device is captured and connected to the video interactive server.

The display device can be a display interface of a handheld device or a personal computer, or can be an external screen display device, wherein the biometric capture device can be a camera or a microphone, or can be operated by an external connection. A sensor that counts a sensor, a breath count sensor, or a wearable device.

The video interactive subsystem includes a playing module, which is to play the video content of the video interactive server and output to the display device; the time triggering module is to detect the time axis of the video playing, and when the set time is to be analyzed When the point is triggered, the capture module is triggered to obtain at least one biometric feature of the viewer; the capture module automatically collects at least one biometric feature and response of the viewer through the biometric capture device; the identification module is The feedback message is combined with the facial expression data and the sound data to perform echo cancellation and identification comparison, so as to obtain corresponding feedback results, and convert the sound decibel and heart rate values into normalized values; the integrated integration module is to be recognized. The feedback result is integrated with the film and television time point to integrate the data. According to the set timeline event, the viewer can watch the statistics of the feedback result of the film and television; and the transmission module returns the interactive result to the video interactive server for film and television. The business or the live master can obtain the viewer behavior during the video playback and analyze the video playback effect.

Among them, biometrics are feedbacks for facial expressions, environmental sounds, speech recognition, or heart rate counting, and responses to viewers watching movies and television.

A film and television interaction method, comprising: step one, storing the film and television content in a video interactive server; Step 2: Connect the film and television timeline to trigger the collection event; Step 3: Play the video content and the user; Step 4: Collect the feedback message in a way that does not disturb the viewer to watch the video content; Step 5: Analyze the feedback message and integrate the film and television time The entire information result and the integrated information.

Step 1 is placed on the video interactive server, which uploads the pre-recorded video content to the video interactive server for live broadcast, or broadcasts live video to provide live video content.

The triggering acquisition event of step 2 is to set the video content to be associated with at least one video timeline, and the video timeline can correspond to at least one acquisition event, as a subsequent time when the video playing time reaches a set time axis, triggering the biometric capturing device to perform the acquisition. The basis for the biometrics and behavior of the viewer. The movie timeline can be a specific movie clip, a specific playback time point or an interval time. The interval time can be an interval of repetition or a plurality of intervals. The collecting event may be to trigger at least one biometric capturing device.

In the third step, the viewer uses the video interactive subsystem to connect to the video interactive server, play the video content and output it to the display device. Its video interactive subsystem is the APP (Application) of the handheld device, the APP mode of the version of iOS, Android or Windows Phone, and the PC application mode which can be executed on the Windows, Mac or Linux operating system.

The step 4 of collecting the feedback message is that when the film and television playing time reaches the set time axis, the film and television interactive subsystem can trigger the control of the biological feature capturing device, so as not to disturb the viewer to watch the film and television content, and automatically collect the viewer to watch at least the current film and television. A biometric and response feedback message. Biological characteristics The pick-up device is a sensor that can be built into the hand-held device or can be operated in an external connection.

The fifth step is to analyze the feedback message after the film and television interactive subsystem collects the information, and then integrate it with the film and television time to process the information result, and finally send it back to the video interactive server. The feedback and integration information can be used by the film or television company or live broadcast. The live master gets the behavior of the viewer in the process of film and television playback, and analyzes the effect of video playback. The feedback message is a biometric feature that includes the facial expression of the viewer, the environmental sound capture, the biometric characteristics of the speech recognition or heart rate, and the reaction of the viewer.

The invention provides a film and television interaction system and method, and has the following advantages when compared with other conventional technologies:

1. When playing a specific movie clip or time point, the present invention can automatically collect at least one biometric and reaction feedback message of the viewer watching the movie without disturbing the viewer's viewing of the video content.

2. The invention analyzes and processes the feedback message and integrates with the film and television timeline to generate the integrated information result, which can help the film and television dealer or the live master to obtain the feedback information such as the viewer's behavior and reaction during the film and television playing process, thereby understanding and analyzing the effect of the video playing. .

3. The invention solves the problem that the online video and live video have only a large number of statistics and predictions of the viewers, and it is impossible to know whether the content of the specific video clip or the video time point is highly concerned.

110‧‧‧ display device

120‧‧‧Biometric extraction device

130‧‧‧Video Interactive Subsystem

131‧‧‧Playing module

132‧‧‧Time Trigger Module

133‧‧‧Capture module

134‧‧‧ Identification Module

135‧‧‧Integrated module

136‧‧‧Transmission module

140‧‧‧Video Interactive Server

S210~S250‧‧‧ Process

Please refer to the detailed description of the invention and its drawings, which will be The technical content of the present invention and its effect are understood in one step; the related drawings are: FIG. 1 is an architectural diagram of a video interactive system and method according to the present invention; and FIG. 2 is a flow chart of a video interactive system and method according to the present invention.

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The present invention is further described with reference to the accompanying drawings: FIG. 1 is an architectural diagram of a video interactive system and method, including a display device 110, which is an output end for playing video content; and a biometric capture device. 120 is a sensor that can be built in a handheld device; a video interactive server 140 is provided for providing pre-recorded video content, providing live video content and providing viewers with feedback information to obtain video playback. The viewer behavior in the process and the analysis of the video playback effect; and a video interaction subsystem 130 is connected to the display device, and the biometric capture device, and is also connected to the video interactive server.

The display device 110 can be a display interface of a handheld device or a personal computer, and can also be an external display device. The biometric capture device 120 can be a camera or a microphone, or can be connected by external connection. Heart rate counting sensor, breath count sensor or sensor for wearable devices.

The video interaction subsystem 130 includes a play module 131 for playing the video content of the video interactive server 140 and outputting it to the display device 110. The time trigger module 132 is a time axis for detecting the video play, and When the time point to be analyzed is set, the capture module 133 is triggered to obtain the viewer. At least one biometric feature; the capture module 133 automatically collects at least one biometric feature and response of the viewer through the biometric capture device 120; the recognition module 134 is to feed back the message with facial expression data and sound. The data is echo-cancelled, the identification is compared, the corresponding feedback result is obtained, and the sound decibel and heart rate values are converted into normalized values; the fusion integration module 135 is to integrate the recognized feedback result with the film and television time point. The entire data, according to the set timeline event, can obtain the statistics of the feedback result of the viewer watching the movie; and the transmission module 136 returns the interactive integration result to the video interactive server 140, so that the film and television dealer or the live master can obtain the follow-up The behavior of viewers during film and television playback and the analysis of film and television playback effects.

Among them, biometrics are feedbacks for facial expressions, environmental sounds, speech recognition, or heart rate counting, and responses to viewers watching movies and television.

Please refer to FIG. 1 , which is a flow chart of a video interactive system and method, which includes: Step 1: S210 stores the video content in the video interactive server; Step 2: S220 connects the video time axis to trigger the collection event; Step 3: S230 plays the video content and the user; in step 4, the S240 collects the feedback message in a manner that does not disturb the viewer to view the video content; in step 5, the S250 analyzes the feedback message and integrates the video time to integrate the information result and transmit the unified information.

It can be seen from the above steps that the first step is placed on the video interactive server, which is to upload the pre-recorded video content to the video interactive server for recording, or to broadcast the live content to provide the video content.

The triggering acquisition event of step 2 is to set the video content to be associated with at least one video timeline, and the video timeline can correspond to at least one collection event. In order to set the time axis for the subsequent movie playing time, the biometric capturing device is triggered to perform the basis for collecting the biometric characteristics and behavior of the viewer. The movie timeline can be a specific movie clip, a specific playback time point or an interval time. The interval time can be an interval of repetition or a plurality of intervals. The collecting event may be to trigger at least one biometric capturing device.

In the third step, the viewer uses the video interactive subsystem to connect to the video interactive server, play the video content and output it to the display device. Its video interactive subsystem is the APP (Application) of the handheld device, the APP mode of the version of iOS, Android or Windows Phone, and the PC application mode which can be executed on the Windows, Mac or Linux operating system.

The step 4 of collecting the feedback message is that when the film and television playing time reaches the set time axis, the film and television interactive subsystem can trigger the control of the biological feature capturing device, so as not to disturb the viewer to watch the film and television content, and automatically collect the viewer to watch at least the current film and television. A biometric and response feedback message. The biometric extraction device is a sensor that can be built in a handheld device or can be operated in an external connection mode.

The fifth step is to analyze the feedback message after the film and television interactive subsystem collects the information, and then integrate it with the film and television time to process the information result, and finally send it back to the video interactive server. The feedback and integration information can be used by the film or television company or live broadcast. The live master gets the behavior of the viewer in the process of film and television playback, and analyzes the effect of video playback. The feedback message is a biometric feature that includes the facial expression of the viewer, the environmental sound capture, the biometric characteristics of the speech recognition or heart rate, and the reaction of the viewer.

It can be seen from the above that the video content is stored in the video interactive server. The film and television company uploads the pre-recorded video content to the video interactive server for storage, or straight The broadcaster provides live video content in live broadcast mode; the connected video timeline triggers the collection event. The feature is that the film and television content is associated with at least one film and television timeline, and the film and television time axis corresponds to at least one acquisition event, and as a follow-up, when the film and television playing time reaches a set time axis, the biometric feature capturing device is triggered to perform the biometrics and behaviors of the collecting viewer. The basis. The movie timeline can be a specific movie clip, a specific playback time point or an interval time. The interval time may be a repetition interval or a plurality of interval times. The collecting event can trigger at least one biometric capturing device; watching the video content. The viewer uses the video interactive subsystem to connect to the video interactive server, play the video content and output it to the display device. The video interactive subsystem can be an APP (Application) of a handheld device, for example, an APP mode such as iOS, Android or Windows Phone, or a personal computer application mode executed on an operating system such as Windows, Mac or Linux. The display device may be a handheld device, such as a smart phone or a tablet computer, or a screen display device, such as a television, a video wall, a liquid crystal screen, a plasma TV, a projector, or an LED electronic signboard; The viewer collects feedback messages by watching the content of the film and television. During the playback of video content, when the video playback time reaches the set time axis, the video interactive subsystem can trigger the control of the biometric capture device, which is characterized in that the viewer is automatically disturbed by watching the video content without disturbing the viewer. At least one biometric and reaction feedback message. The biometric extraction device may be a sensor built into the handheld device, such as a camera or a microphone, or an externally connected sensor, such as a heart rate counter, a respiratory count, or a wearable device. The feedback message may include the biometrics of the viewer's facial expressions, environmental sounds, speech recognition or heart rate values, as well as the reaction of the viewers watching the video content; analyzing the feedback message and integrating the film and television time into a unified information result And transmit the unified information. The feature is that the video interactive subsystem collects the feedback message and analyzes it, then integrates it with the film and television time to process the information result, and finally sends it back to the video interactive server. The feedback and integration information can be obtained by the film or television broadcaster or the live broadcaster. The behavior of viewers during film and television playback, and the analysis of film and television playback effects.

The following embodiments can be simultaneously referred to FIG. 1. A video interactive system includes a display device 110, a biometric capture device 120, a video interaction subsystem 130, and a video interactive server 140. The display device 110 is an output terminal for playing video content. The display device can be the original display interface of the handheld device or the personal computer, or can be an external screen display device. The biometric feature extraction device 120 is mainly characterized in that the viewer is automatically captured to view at least one biometric feature and response information of the current movie and television in a manner that does not disturb the viewer to view the video content. The biometric capture device 120 can be a sensor built into the handheld device, such as a camera or a microphone, or a sensor that operates in an external connection, such as a heart rate counter, a respiratory count, or a wearable device. The feedback message may include the biometrics of the viewer's facial expressions, environmental sounds, speech recognition or heart rate values, and the reaction of the viewer.

The video interaction subsystem 130 can be implemented in a handheld device or a personal computer, and can be connected to the video interactive server 140 to play the video content and output to the display device 110 when playing a specific video clip or time point. The video interaction subsystem 130 can control the biometric capture device 120 to automatically capture the feedback information of the at least one biometric feature and response, and identify and process the feedback message, without disturbing the manner in which the viewer views the video content. After that, the integrated information integrated with the film and television time is generated and transmitted back to the video interactive server 140, so that the film and television dealer or the live master can obtain the viewer behavior during the video playing process and analyze the effect of the video playing.

The video interactive server 140, the main feature of the video interactive server is to provide pre-recorded video content, provide real-time live video content and provide viewers feedback information to obtain viewers' behavior during film and television playback and analyze video Playback effect.

The video interaction subsystem 130 further includes a play module 131, a time trigger module 132, a capture module 133, an identification module 134, a fusion integration module 135, and a transmission module 136. The playback module 131 plays a video interaction. The video and output of the server are output to the display device. The time triggering module 132 detects the time axis of the video playing. When the set time point to be analyzed is reached, the capturing module 133 is triggered to obtain at least one biometric feature of the viewer, for example, facial expression, sound, heart rate counting, etc. Wait, become a feedback message. The capture module 133 is configured to obtain at least one biometric feedback message of the viewer captured by the biometric device. The identification module 134 performs echo cancellation and recognition comparison between the feedback message and the facial expression data and the sound data to obtain a corresponding feedback result, and converts the sound decibel and heart rate values into normalized values. The 135 fusion integration module combines the recognized feedback result with the film and television time point to integrate the data, and compares the set timeline event to obtain the statistics of the feedback result of the viewer watching the movie. The transmission module 136 transmits the interactive integration result back to the video interactive server, so that the film and television dealer or the live master can obtain the viewer behavior during the video playback and analyze the video playback effect.

For example, the teaching film and television content has a lecturer telling a joke in a particular film and television clip. At the end of the joke, the time axis triggers the collection event, collects the viewer's facial expression and reacts to the feedback message, and analyzes the facial expression through the video interactive subsystem. After that, the statistical viewer's facial expressions are presented in a happy, calm, unpleasant proportion, and then the statistical results and the timeline are merged into a unified information, and then transmitted back to the video interactive server.

Taking a handheld device as an example, the video interactive subsystem takes a picture The interactive server plays the video content and outputs the video content to the display device. The viewer watches the video through the display device. During the playback process, if the time point is to be analyzed, the video interactive subsystem automatically triggers the biometric capture device to collect the biometric characteristics and responses of the viewer, for example, collecting the facial expression of the viewer or the ambient sound. The video interactive subsystem can extend the wearer's device to obtain the biometric characteristics and response of the viewer, for example, collecting the viewer's heart rate value with the wearable device. The video interactive subsystem processes the feedback message, converts it and integrates it with the film and television time to integrate the information. The video interactive subsystem transmits the integrated information to the video interactive server. The film and television supplier or the live master gets the viewer's behavior and analyzes the film and television effect.

Taking the personal computer as an example, the video interactive subsystem sequentially plays the video from the video interactive server, and outputs the video content to the display device. The viewer watches the video through the display device. During the playback process, if the time point is to be analyzed, the video interactive subsystem automatically triggers the biometric capture device to collect the biometric characteristics and responses of the viewer, for example, collecting the facial expression of the viewer or the ambient sound.

The video interactive subsystem processes the feedback message, converts it and integrates it with the film and television time to integrate the information.

The video interactive subsystem transmits the integrated information to the video interactive server.

The film and television supplier or the live master gets the viewer's behavior and analyzes the film and television effect.

When collecting the viewer feedback message, it is combined with the film and television time to integrate the information result. Video application, such as online learning or live broadcast, can play video content from the video interactive server, set the continuous interval or specific video clip to trigger the biometric device, and automatically collect the viewer's facial expression without disturbing the viewer's viewing of the video content. In the feedback message, after integration with the film and television time into a unified information, it can be used by film and television dealers or live masters to analyze the effectiveness of video playback. The facial expressions include anger, disgust, fear, happiness, sadness, surprise, and expressionlessness. Wait for the status.

The detailed description of the present invention is intended to be illustrative of a preferred embodiment of the invention, and is not intended to limit the scope of the invention. The patent scope of this case.

To sum up, this case is not only innovative in terms of technical thinking, but also has many of the above-mentioned functions that are not in the traditional methods of the past. It has fully complied with the statutory invention patent requirements of novelty and progressiveness, and applied for it according to law. Approved this invention patent application, in order to invent invention, to the sense of virtue.

Claims (16)

A video interactive system includes: a display device, which is an output end of a video content playback; a biometric capture device, which is a sensor built into the handheld device; and a video interactive server, which provides a pre- Recording of film and television content, providing live and live video content and providing viewers with feedback to obtain viewers' behavior during film and television playback and analyzing video playback effects; and a video interactive subsystem connected to the display device And the biometric feature capture device and the video interaction server is further connected to the video interaction system, the video interaction subsystem includes: a time trigger module, which detects the time axis of the video play, and when the set time point is to be analyzed At the same time, the capture module is triggered to obtain at least one biometric feature of the viewer; and the capture module automatically collects at least one biometric and reaction of the viewer through the biometric feature capture device. The video interactive system according to claim 1, wherein the display device is a display interface of a handheld device or a personal computer, and is also an external display device. The video interactive system according to claim 1, wherein the biometric capturing device is a camera or a microphone for a sensor built in the handheld device, and the heart rate is calculated by an external connection. Sensor, breath count sensor or sensor for wearable devices. The video interactive system according to claim 1, wherein the video interactive subsystem further comprises: a playing module, which plays the video content of the video interactive server and loses Going out to the display device; the recognition module is to perform echo cancellation and recognition comparison between the feedback message and the facial expression data and the sound data to obtain a corresponding feedback result, and convert the sound decibel and heart rate values into normalized values; The integrated integration module combines the recognized feedback result with the film and television time point to integrate the data, and compares the set timeline event to obtain the statistics of the feedback result of the viewer watching the movie; and the transmission module The whole result is sent back to the video interactive server, so that the film and television dealer or the live master can obtain the viewer behavior during the video playback and analyze the video playback effect. For example, the video interactive system described in claim 4, wherein the biometric is a facial expression, an environmental sound, a voice recognition, or a feedback message of a heart rate count, and a response of a person watching the video content. A method for interacting with a film and television, comprising: Step 1: storing the content of the film and television in the interactive video server; Step 2: triggering the event of the film and television time axis triggering, wherein the triggering event is set to associate the film and television content with at least one film and television timeline, film and television The time axis can correspond to at least one acquisition event, as a subsequent time when the video playing time reaches a set time axis, triggering the biometric capturing device to perform the basis for collecting the biometric characteristics and behavior of the viewer; and step 3, playing the video content and the user; Step 4: Collect feedback information in a manner that does not disturb the viewer to watch the video content; Step 5: Analyze the feedback message and integrate the video time into a unified information result and transmit the unified information. For example, the film and television interactive method described in claim 6 of the patent scope, wherein the step 1 is placed on the video interactive server, the film and television merchant uploads the pre-recorded video content to the video interactive server for storage, or broadcasts the live event to Live video provides video content. For example, in the film and television interactive method described in claim 6, in the third step, the viewer uses the video interactive subsystem to connect to the video interactive server, and play the video content and output it to the display device. For example, in the film and television interactive method described in claim 6, wherein the step 4 of the collection feedback message is when the film and television playing time reaches a set time axis, the film and television interactive subsystem can trigger the control of the biometric capturing device, Disturbing the viewer to watch the video content, automatically collecting the viewer to watch at least one biometric and response feedback message of the film and television. For example, the film and television interactive method described in claim 6 of the patent scope, wherein the step 5 is performed after the film and television interactive subsystem collects the feedback message, and then integrates with the film and television time to process the information result, and finally returns the film and television. The interactive server, feedback and integration information can enable the film and television dealer or live broadcast master to obtain the viewer behavior during the video playback, and analyze the video playback effect. The method for interacting with the film and television according to item 6 of the patent application scope, wherein the feedback message is to acquire relevant biological features, including facial expressions of the viewer, environmental sound extraction, speech recognition or heart rate numerical biometrics, and viewing The reaction of the film and television content. For example, the film and television interactive method described in claim 6 wherein the film timeline is a specific movie clip, a specific playing time point or an interval time. For example, the film and television interaction method described in claim 6 wherein the interval time is a repetition interval or a plurality of intervals. The film and television interactive method according to claim 8 , wherein the collecting event is to trigger at least one biometric capturing device. For example, the video interactive method described in claim 8 of the patent scope, wherein the video interactive subsystem is an APP (Application) of a handheld device, and an APP mode of an iOS, Android or Windows Phone version is also available in Windows, The way a PC application is executed on a Mac or Linux operating system. The method for interacting with the film and television according to Item 8 of the patent application, wherein the biometric feature capturing device is a camera or a microphone for a sensor built in the handheld device, and the heart rate is calculated by an external connection mode. Sensor, breath count sensor or sensor for wearable devices.