WO2015078240A1

WO2015078240A1 - Video control method and user terminal

Info

Publication number: WO2015078240A1
Application number: PCT/CN2014/088431
Authority: WO
Inventors: 肖振威
Original assignee: 乐视网信息技术（北京）股份有限公司
Priority date: 2013-11-27
Filing date: 2014-10-11
Publication date: 2015-06-04
Also published as: CN103618944A

Abstract

Provided are a video control method and a user terminal, which relate to the technical field of mobile terminals, and judge an eyeball state to control the playing of a video by way of continuously analyzing whether a first image contains an eyeball image, thereby realizing a function of automatically controlling the playing of the video according to the watching state of a user. The method comprises: step a, conducting shooting to obtain a first image; step b: analyzing the shot first image, and judging whether the first image contains an eyeball image at an eye opening state; and repeatedly executing steps a and b, and if N continuous first images do not contain eyeball images at the eye opening state, stopping playing a video, where N is an integer which is greater than 1.

Description

Video control method and user terminal

The present application claims priority to Chinese Patent Application No. 2013106141969, entitled "A Video Control Method and User Terminal" on November 27, 2013, the entire contents of which are incorporated herein by reference. in.

Technical field

The present invention relates to the field of mobile terminal technologies, and in particular, to a video control method and a user terminal.

Background technique

With the advent of the era of intelligent mobile terminals, the configuration of mobile terminals is becoming more and more powerful, and the functions are more and more complete. More and more users play video through mobile terminals, and browse downloaded videos or personal recordings in different locations. Video to enjoy the intelligent experience brought by mobile terminals.

However, due to the prior art playback control of the video, it is necessary to enter the control interface of the video player to perform operations such as video pause, video playback, video switching, video fast forward or video fast rewind. Taking video pause as an example, when playing control on a mobile terminal with a touch screen, the user needs to press the play interface of the video on the touch screen to enter the control interface of the video, and then find the corresponding pause button to complete the pause operation of the video. For example, if the user suddenly leaves or falls asleep during the watching of the video, the terminal device cannot obtain the information that the user does not need the video to play, and continues to play, resulting in waste of battery power.

Therefore, it is necessary to provide a method and an end device capable of automatically controlling video playback according to a user's viewing state to overcome the above drawbacks.

Summary of the invention

An embodiment of the present invention provides a video control method and a user terminal, which continuously determines whether an eyeball image is included in a first image to determine an eyeball state, and then controls video playback according to the determination result, thereby automatically controlling the video according to the viewing state of the user. The function of playing, thus achieving the purpose of turning off the playback device in time and saving power.

In order to achieve the above object, embodiments of the present invention adopt the following technical solutions:

An embodiment of the present invention provides a video control method, which is applied to a user terminal provided with a camera in a process of playing a video, and the method includes:

Step a: shooting to obtain a first image;

Step b: analyzing the captured first image, and determining whether the first image includes an eyeball image in a blink state;

Steps a and b are repeatedly executed. If the eye images of the blink state are not included in the N consecutive first images, the video is stopped, where N is an integer greater than 1.

The step b includes:

Determining, according to the first image, whether the face image is included in the first image;

If the face image is included in the first image, determining an eyeball region according to the face image;

At the determined eyeball region, it is determined whether or not the eyeball image of the blink state is included.

The determining, at the determined eyeball region, whether the eyeball image including the blink state is included:

The eyeball region is analyzed using an Adaboost algorithm to obtain an eyeball state, the eyeball state including a blink state and a closed eye state;

Based on the state of the eyeball, it is determined whether or not the eyeball image of the blink state is included.

The analyzing the eyeball region using the Adaboost algorithm to obtain an eyeball state includes:

Aligning the eyeball region from a preset sample library using the Adaboost algorithm, the preset sample library comprising a positive sample set having a blink eye image, and a negative sample set of the closed eye image;

If the eyeball area after the recognition is a blinking eye image, obtaining an eyeball state at the time is the blinking state;

If the eyeball region after the recognition is a closed eye image, the eyeball state at this time is obtained as the closed eye state.

If the face image is not included in the first image, determining that the eye image of the blink state is not included in the first image stops playing the video.

An embodiment of the present invention provides a user terminal, including:

a collecting unit, configured to capture a first image;

An analyzing unit, configured to analyze the captured first image, and determine whether the first image includes an eyeball image in a blink state;

And an execution unit, configured to repeatedly perform steps a and b, and if the eye images of the blink state are not included in the N consecutive first images, stop playing the video, where N is an integer greater than 1.

The analyzing unit is configured to determine, according to the first image, whether the first image includes a face image; and if the first image includes the face image, determine the face image according to the face image An eyeball region; and at the determined eyeball region, determining whether or not the eyeball image of the blink state is included.

The analyzing unit is configured to analyze the eyeball region by using an Adaboost algorithm to obtain an eyeball state, the eyeball state includes a blinking state and a closed eye state; and determining, according to the eyeball state, whether the eyeball of the blinking state is included image.

The analyzing unit is configured to perform comparison and recognition on the eyeball region from a preset sample library by using the Adaboost algorithm, where the preset sample library includes a positive sample set with a blink eye image, and a closed eye image a negative sample set; and if the identified eyeball region is a blink eye image, obtaining an eyeball state at the time is the blink state; and if the aligned eyeball region is a closed eye The image is obtained at the time when the state of the eyeball is the closed eye state.

If the face image is not included in the first image,

The executing unit is configured to determine an eyeball image that does not include the blink state in the first image, and stop playing the video.

An embodiment of the present invention provides a video control method and a user terminal, which are applied to a user terminal provided with a camera to analyze whether an eyeball region is included in a first image, and then in a preset sample library, in a process of playing a video. The area is compared, and the image containing no eyeball is obtained. When the image of the N consecutive images is not included, the video is stopped, and the video is automatically controlled according to the viewing state of the user, thereby avoiding the video playing process. Unnecessary power loss.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only Are some embodiments of the invention, to those of ordinary skill in the art In other words, other drawings can be obtained from these drawings without any creative work.

1 is a schematic flowchart of a video control method according to Embodiment 1 of the present invention;

2 is a schematic flowchart of a video control method according to Embodiment 2 of the present invention;

FIG. 3 is a schematic structural diagram of a user terminal according to Embodiment 3 of the present invention.

detailed description

In the following description, for purposes of illustration and description, reference However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known devices, circuits, and methods are omitted so as not to obscure the description of the invention.

The user terminal may be a wireless terminal or a wired terminal, and the wireless terminal may be a device that provides voice and/or data connectivity to the user, a handheld device with wireless connectivity, or other processing device connected to the wireless modem. The wireless terminal can communicate with one or more core networks via a radio access network (eg, RAN, Radio Access Network), which can be a mobile terminal, such as a mobile phone (or "cellular" phone) and with a mobile terminal The computers, for example, can be portable, pocket-sized, handheld, computer-integrated or in-vehicle mobile devices that exchange language and/or data with the wireless access network.

Embodiment 1

An embodiment of the present invention provides a video control method, as shown in FIG. 1, including:

101. The user terminal captures the first image.

An embodiment of the present invention provides a video control method, which is applied to a user terminal provided with a camera to compare an eyeball region in a preset sample library by analyzing whether an eyeball region is included in a first image during a video playback process. The image that does not include the eyeball is obtained, and the video is stopped when the eyeball image is not included in the first N images, and the function of automatically controlling the video playback according to the viewing state of the user is realized.

Specifically, before the user views the video, the user terminal detects whether the user triggers the specified function button to enable the eyeball detection function, wherein the function button may be a physical button or a virtual The button is intended. If the user terminal detects that the user triggers the specified function button to turn on the eyeball detection function, the camera is used to capture the first image.

102. The user terminal analyzes the captured first image, and determines whether the first image includes an eyeball image in a blink state.

Specifically, the user terminal determines, according to the first image, whether the face image is included in the first image. If the first image does not include the face image, the video is stopped; if the first image includes the face image, the eyeball region is determined according to the face image, and then the eyeball region is analyzed by using the Adaboost algorithm to obtain the eyeball state, wherein the eyeball The state includes a blink state and a closed eye state, and then the user terminal determines whether or not the eyeball image is included according to the eyeball state.

Among them, Adaboost is an iterative algorithm when the user terminal analyzes the eyeball area. The core idea is to train different classifiers (weak classifiers) for the same training set, and then combine these weak classifiers. To form a stronger final classifier.

Further, when the user terminal analyzes the eyeball region by using the Adaboost algorithm and obtains the eyeball state at this time, the eyeball region is first compared from the preset sample library by using the Adaboost algorithm, and the preset sample library includes a positive sample of the blink eye image. Set, as well as a negative sample set of closed eye images. If the eyeball area after the comparison is a blinking eye image, it is determined that the eyeball state is a blinking state at this time; if the aligned eyeball region is a closed eye image, it is determined that the eyeball state is a closed eye state at this time.

103. The user terminal repeatedly performs steps 101 to 102. If the eye images of the blinking state are not included in the N consecutive first images, the video is stopped, where N is an integer greater than 1.

Specifically, after the user terminal captures the first image and analyzes the captured first image, and determines whether the eyeball image is included in the first image, if it is determined that the first image does not include the eyeball image, the counter in the user terminal starts counting. If no image of the eyeball is included in the first N consecutive images, the value of the counter at this time is N, reaching the critical point at which the triggering of the stop of the video is played, and then the stop playback command is generated and the video is stopped.

An embodiment of the present invention provides a video control method, which is applied to a user terminal provided with a camera to obtain a first image by shooting during video playback; and then analyzing the captured first image to determine whether the first image includes 睁The eyeball image of the eye state; finally, if the eyeball image of the blinking state is not included in the first N consecutive images, the video is stopped, wherein, N Is an integer greater than 1. The solution analyzes whether the eyeball region is included in the first image, and then compares the eyeball region in the preset sample library to obtain an eyeball image, and stops when the first N images of the first image are not included. The video realizes the function of automatically controlling the video playback according to the viewing state of the user, and avoids unnecessary power loss during video playback.

Embodiment 2

An embodiment of the present invention provides a video control method, as shown in FIG. 2, including:

201. The user terminal establishes a preset sample library.

Wherein, the preset sample library includes a positive sample set having a blink eye image and a negative sample set of the closed eye image.

Specifically, the user terminal collects the test picture set and establishes a preset sample library before the normal operation, and the preset sample library includes a positive sample set with a blink eye image and a negative sample set of the closed eye image, wherein the negative sample The set consists of a picture saved by the human eye image in the closed-eye state of the human face and an image not including the human eye area; the positive sample set is composed of a blink image, and the blink image is from the entire image containing the blink The face image is taken out. The interception method can be used to capture the blink image using the initial size of 30*20 (the unit is the pixel). When the nesting is performed, the center of the rectangle and the geometric center of the eye should be coincident. If the rectangle is not large enough. The length and width of the rectangular frame are enlarged by a ratio of 1:1.2.

In addition, the user terminal periodically updates the positive sample set and the negative sample set in the preset sample bank to maintain the accuracy when comparing the positive sample set and the negative sample set in the preset sample bank.

202. The user terminal provided with the camera performs shooting during the process of playing the video to obtain the first image.

203. The user terminal determines, according to the first image, whether the first image includes a face image.

Specifically, specifically, the user terminal determines, according to the first image, whether the first image includes a face image. If the first image does not include the face image, the video is stopped; if the first image includes the face image, the eyeball region is determined according to the face image, and then the eyeball image is determined at the determined eyeball region.

204. If the first image includes a face image, the user terminal determines the eyeball region according to the face image.

Specifically, if the first image includes a face image, the user terminal determines the eyeball region according to the face image. Then, using the Adaboost algorithm to analyze the eyeball area, the eyeball state is obtained, and the eyeball state includes the blinking state and the closed eye state. Finally, according to the state of the eyeball, it is judged whether or not the eyeball image is included.

205. The user terminal analyzes an eyeball region by using an Adaboost algorithm to obtain an eyeball state.

Wherein, the eyeball state includes a blinking state and a closed eye state.

Specifically, the user terminal compares the eyeball regions from the preset sample library by using an Adaboost algorithm, and the preset sample library includes a positive sample set having a blink eye image and a negative sample set of the closed eye image; The eyeball area is the blinking eye image, and it is judged that the eyeball state is the blinking state at this time; correspondingly, if the aligned eyeball region is the closed eyed eye image, it is determined that the eyeball state is the closed eye state at this time.

206. The user terminal determines, according to the state of the eyeball, whether or not the eyeball image of the blink state is included.

Specifically, after the user terminal uses the Adaboost algorithm to analyze the eyeball region and obtain the eyeball state, it is obtained that the eyeball region in the face image in the first image indicates a blinking state or a closed eye state.

If it is obtained that the eyeball region in the face image in the first picture indicates the blinking state, the user terminal determines that the eyeball image is included in the first image.

If it is determined that the eyeball region in the face image in the first picture indicates a closed eye state, the user terminal determines that the eyeball image is not included in the first image.

207. Steps 202 to 206 are repeatedly performed. If the eye images of the blink state are not included in the N consecutive first images, the video is stopped, where N is an integer greater than 1.

Exemplarily, the user falls asleep while watching the video using the mobile phone terminal equipped with the camera, and the camera in the mobile phone terminal performs image acquisition to obtain the first image, and after analyzing by the Adaboost algorithm, it is determined that the user is in the closed eye state, that is, the first image. The eyeball image is not included in the image. Further, the mobile terminal performs image collection and analysis every minute. When the first five images are not included in the image, the video playback stop command is triggered, and then the mobile terminal according to the stop command. Stop playing the video.

An embodiment of the present invention provides a video control method, which is applied to a user terminal provided with a camera to obtain a first image by shooting during video playback; and then analyzing the captured first image to determine whether the first image includes 睁The eyeball image of the eye state; finally, if the eyeball image of the blinking state is not included in the N consecutive first images, the video is stopped, where N is an integer greater than 1. The solution analyzes whether the eyeball region is included in the first image, and then compares the eyeball region in the preset sample library to obtain an eyeball image, and stops when the first N images of the first image are not included. The video realizes the function of automatically controlling the video playback according to the viewing state of the user, and avoids unnecessary power loss during video playback.

Embodiment 3

An embodiment of the present invention provides a user terminal, as shown in FIG. 3, including:

The collecting unit 01 is configured to capture and obtain a first image;

The analyzing unit 02 is configured to analyze the captured first image, and determine whether the first image includes an eyeball image in a blink state;

The executing unit 03 is configured to repeatedly perform the steps a and b, and if the first image of the eye image that does not include the blink state in the first N consecutive images does not include the eyeball image in the blink state, the video is stopped, wherein the video is stopped. , N is an integer greater than one.

Further, the analyzing unit 02 is configured to determine, according to the first image, whether a face image is included in the first image; and if the first image includes a face image, determine an eyeball region according to the face image; and determine the eyeball At the area, it is judged whether or not the eyeball image of the blinking state is included.

Further, the analyzing unit 02 is configured to analyze the eyeball region by using the Adaboost algorithm, obtain an eyeball state, the eyeball state includes a blinking state and a closed eye state, and determine whether the eyeball image of the blinking state is included according to the eyeball state.

Further, the analyzing unit 02 is configured to perform an alignment recognition on the eyeball region from the preset sample library by using an Adaboost algorithm, where the preset sample library includes a positive sample set having a blink eye image, and a negative sample set of the closed eye image And if the eyeball area after the recognition is a blinking eye image, the eyeball state is obtained as a blinking state at this time; and if the eyeball region after the recognition is a closed eye image, the eyeball state is closed at this time. Eye state.

Further, if the first image does not include a face image, then

The executing unit 03 is configured to determine an eyeball image that does not include a blink state in the first image, and stop playing the video.

An embodiment of the present invention provides a user terminal, which is applied to a user terminal provided with a camera to obtain a first image by capturing during playing a video; and then analyzing the captured first image to determine whether the first image includes a blink The eyeball image of the state; finally, if the eyeball images of the blinking state are not included in the N consecutive first images, the video is stopped, where N is an integer greater than 1. The solution analyzes whether the eyeball region is included in the first image, and then compares the eyeball region in the preset sample library to obtain an eyeball image, and stops when the first N images of the first image are not included. The video realizes the function of automatically controlling the video playback according to the viewing state of the user, and avoids unnecessary power loss during video playback.

It will be clearly understood by those skilled in the art that for the convenience and brevity of the description, only the division of each functional module described above is exemplified. In practical applications, the above function assignment can be completed by different functional modules as needed. The internal structure of the device is divided into different functional modules to perform all or part of the functions described above. For the specific working process of the system, the device and the unit described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed system, device And methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be used. Combinations can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the appended claims.

Claims

A video control method is characterized in that, in a process of playing a video by a user terminal provided with a camera, the method includes:

Step a: shooting to obtain a first image;

Step b: analyzing the captured first image, and determining whether the first image includes an eyeball image in a blink state;

Steps a and b are repeatedly executed. If the eye images of the blink state are not included in the N consecutive first images, the video is stopped, where N is an integer greater than 1.
The method of claim 1 wherein said step b comprises:

Determining, according to the first image, whether the face image is included in the first image;

If the face image is included in the first image, determining an eyeball region according to the face image;

At the determined eyeball region, it is determined whether or not the eyeball image of the blink state is included.
The method according to claim 2, wherein said determining, at said determined area of said eyeball, whether or not said eyeball image comprising said blinked state comprises:

The eyeball region is analyzed using an Adaboost algorithm to obtain an eyeball state, the eyeball state including a blink state and a closed eye state;

Based on the state of the eyeball, it is determined whether or not the eyeball image of the blink state is included.
The method according to claim 3, wherein the analyzing the eyeball region using the Adaboost algorithm to obtain an eyeball state comprises:

Aligning the eyeball region from a preset sample library using the Adaboost algorithm, the preset sample library comprising a positive sample set having a blink eye image, and a negative sample set of the closed eye image;

If the eyeball area after the recognition is a blinking eye image, obtaining an eyeball state at the time is the blinking state;

If the eyeball region after the recognition is a closed eye image, the eyeball state at this time is obtained as the closed eye state.
The method according to claim 2, wherein if the face image is not included in the first image, determining that the first image does not include the eyeball image of the blink state, and stopping playing the video .
A user terminal, comprising:

a collecting unit, configured to capture a first image;

An analyzing unit, configured to analyze the captured first image, and determine whether the first image includes an eyeball image in a blink state;

And an execution unit, configured to repeatedly perform steps a and b, and if the eye images of the blink state are not included in the N consecutive first images, stop playing the video, where N is an integer greater than 1.
The user terminal according to claim 6, wherein

The analyzing unit is configured to determine, according to the first image, whether the first image includes a face image; and if the first image includes the face image, determine the face image according to the face image An eyeball region; and at the determined eyeball region, determining whether or not the eyeball image of the blink state is included.
A user terminal according to claim 7, wherein

The analyzing unit is configured to analyze the eyeball region by using an Adaboost algorithm to obtain an eyeball state, the eyeball state includes a blinking state and a closed eye state; and determining, according to the eyeball state, whether the eyeball of the blinking state is included image.
A user terminal according to claim 8, wherein

The analyzing unit is configured to perform comparison and recognition on the eyeball region from a preset sample library by using the Adaboost algorithm, where the preset sample library includes a positive sample set with a blink eye image, and a closed eye image a negative sample set; and if the identified eyeball region is a blink eye image, obtaining an eyeball state at the time is the blink state; and if the aligned eyeball region is a closed eye The image is obtained at the time when the state of the eyeball is the closed eye state.
The user terminal according to claim 7, wherein if the face image is not included in the first image,

The executing unit is configured to determine an eyeball image that does not include the blink state in the first image, and stop playing the video.