CN112423059A - Gesture-based video control method, television and computer-readable storage medium - Google Patents
Gesture-based video control method, television and computer-readable storage medium Download PDFInfo
- Publication number
- CN112423059A CN112423059A CN202011073939.2A CN202011073939A CN112423059A CN 112423059 A CN112423059 A CN 112423059A CN 202011073939 A CN202011073939 A CN 202011073939A CN 112423059 A CN112423059 A CN 112423059A
- Authority
- CN
- China
- Prior art keywords
- video
- video frame
- gesture
- determining
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
Abstract
The invention discloses a video control method based on gestures, which comprises the following steps: when a gesture is detected and an instruction corresponding to the gesture is a progress operation instruction, acquiring a video currently played by a television and receiving times corresponding to the progress operation instruction stored on the television on the same day, wherein the receiving times are updated when the television detects the progress operation instruction; determining a target video frame in each video frame in the video, and acquiring a direction and an angle corresponding to the gesture, wherein the target video frame is a highlight video frame in the video; determining the playing rate of the video frames in the video according to the target video frames, the corresponding angles of the gestures and the receiving times; and determining the playing sequence of each video frame in the video according to the direction of the gesture, and playing the video according to the playing sequence and the playing speed. The invention also discloses a television and a computer readable storage medium. The invention shortens the time consumed by the user for inquiring the favorite video.
Description
Technical Field
The invention relates to the technical field of televisions, in particular to a video control method based on gestures, a television and a computer readable storage medium.
Background
With the popularization of networks, people can watch video resources in the networks through televisions.
Due to the numerous resources of videos, users inevitably need to control the progress of the videos to quickly browse the contents of the videos to find out the favorite videos of the users.
At present, a user can control the video to fast forward or fast backward through a gesture, that is, the user controls the playing rate of the video through the gesture. However, the playing speed corresponding to the gesture is preset, which may cause the highlight of the video to be played quickly, so that the user cannot capture the highlight of the video, and the duration for the user to query the favorite video is too long.
Disclosure of Invention
The invention mainly aims to provide a video control method based on gestures, a television and a computer readable storage medium, and aims to solve the problem that the duration of a user for inquiring favorite videos is too long.
In order to achieve the above object, the present invention provides a video control method based on gestures, which includes the following steps:
when a gesture is detected and an instruction corresponding to the gesture is a progress operation instruction, acquiring a video currently played by a television and receiving times corresponding to the progress operation instruction stored on the television on the same day, wherein the receiving times are updated when the progress operation instruction is detected by the television;
determining a target video frame in each video frame in the video, and acquiring a direction and an angle corresponding to the gesture, wherein the target video frame is a highlight video frame in the video;
determining the playing rate of the video frames in the video according to the target video frames, the angle corresponding to the gesture and the receiving times;
and determining the playing sequence of each video frame in the video according to the direction of the gesture, and playing the video according to the playing sequence and the playing speed.
In an embodiment, the step of determining the playing rate of the video frames in the video according to the playing time point corresponding to the target video frame, the angle corresponding to the gesture, and the receiving times includes:
segmenting the video according to the receiving times to obtain each segmentation time point, wherein the segmentation time points comprise the time point when the sub video obtained by segmenting the video starts playing and the time point when the video finishes playing;
determining a time difference value between the playing time point of each target video frame and each subsection time point, and determining a target difference value in each time difference value, wherein the target difference value is smaller than a preset difference value;
and determining the playing speed of the video according to the ratio of the sum of the target time points corresponding to each target difference value to the angle corresponding to the gesture, wherein the target time point is the earliest time point in the time points corresponding to the target difference values.
In an embodiment, the step of determining the playing rate of the video according to the ratio between the sum of the target time points corresponding to each target difference value and the angle corresponding to the gesture includes:
according to the ratio of the sum of the target time points corresponding to each target difference value to the angle corresponding to the gesture;
and acquiring a preset playing speed of the video, and determining the playing speed of the video according to the product of the ratio and the preset playing speed.
In one embodiment, the step of determining a target video frame in each video frame in the video comprises:
acquiring the total times corresponding to the progress operation instruction stored in the television, wherein the total times are updated when the television detects the progress operation instruction;
when the ratio of the receiving times to the total times is larger than or equal to a first preset ratio, determining an audio image frame in each video frame to determine the video frame as a target video frame, wherein the ratio of a first audio amplitude of the audio image frame to a second audio amplitude of an image frame adjacent to the audio image frame is larger than or equal to a second preset ratio, or the ratio of the second audio amplitude to the first audio amplitude is larger than or equal to a second preset ratio; or the like, or, alternatively,
when the ratio of the receiving times to the total times is smaller than a first preset ratio, determining a moving image frame and the audio image frame in each video frame to determine the moving image frame and the audio image frame as a target video frame, wherein the pixel values of pixel points in the moving image frame in a preset proportion change relative to the adjacent image frame of the moving image frame.
In one embodiment, the step of determining an audio image frame in each of the video frames comprises:
acquiring a first video frame and a second video frame which are adjacent in the video;
determining a magnitude ratio between a first audio magnitude of the first video frame and a second audio magnitude of the second video frame;
when the amplitude ratio exceeds a preset range, determining the first video frame and the second video frame as audio video frames, wherein the critical value of the preset range is the reciprocal of the second preset ratio and the second preset ratio;
and re-acquiring the adjacent first video frame and second video frame in the rest video frames, and returning to the step of determining the amplitude ratio between the first audio amplitude of the first video frame and the second audio amplitude of the second video frame.
In one embodiment, the step of determining a moving image frame in each of the video frames comprises:
acquiring a third video frame and a fourth video frame which are adjacent in the video, and determining a second pixel point corresponding to a first pixel point at each preset position in the fourth video frame in the third video frame, wherein the position of the first pixel point in the fourth video frame is the same as the position of the second pixel point in the third video frame;
determining a pixel difference value between the pixel value of the first pixel point and the pixel value of the second pixel point, and determining the number of the pixel difference values which are larger than a preset difference value;
when the ratio of the number to the number of the first pixel points is larger than a preset ratio, determining the third video frame and the fourth video frame as motion video frames;
and re-acquiring adjacent third video frames and fourth video frames from the rest video frames, and returning to the step of determining second pixel points corresponding to the first pixel points at each preset position in the fourth video frames in the third video frames.
In an embodiment, after the step of determining the play rate of the video frames in the video according to the target video frame, the angle corresponding to the gesture, and the receiving times, the method further includes:
acquiring the orientation of a thumb in the gesture;
determining a direction of the gesture based on the orientation of the thumb.
In an embodiment, the television is provided with an image acquisition module, and is characterized in that the video control method based on gestures further includes:
acquiring an image acquired by the image acquisition module in real time;
identifying a hand of a user in the image to obtain a gesture;
and when the recognized gesture matches a preset gesture, determining that the command corresponding to the recognized gesture is a progress operation command, and executing the steps of acquiring the video currently played by the television and the number of times corresponding to the progress operation command stored on the television on the same day.
To achieve the above object, the present invention further provides a television comprising a memory, a processor, and a control program stored in the memory and executable on the processor, wherein the control program, when executed by the processor, implements the steps of the gesture-based video control method as described above.
To achieve the above object, the present invention further provides a readable storage medium storing a control program, which when executed by a processor, implements the steps of the gesture-based video control method as described above.
According to the video control method based on the gesture, the television and the readable storage medium provided by the invention, when the television detects that the command corresponding to the gesture is the progress control command, the receiving times of the currently played video of the television and the progress operation command stored on the same day of the television are obtained, the wonderful video frame is determined as the target video frame in each video in the video, the playing speed of the video frame in the video is determined according to the target video frame, the angle corresponding to the gesture and the receiving times, the television determines the playing sequence of each video frame in the video according to the direction of the gesture, and finally the video is played according to the playing sequence and the playing speed. Because the receiving times of the progress operation instruction represent the current progress operation frequency of the user, and the angle corresponding to the gesture represents the playing speed expected by the user, namely the progress operation frequency of the television set user and the wonderful video frame correct the expected playing speed, the television can reasonably correct the preset playing speed by combining the wonderful part of the video, namely the television plays the wonderful part of the video at the reasonable playing speed, the user is prevented from skipping the wonderful part of the video, and the time consumed by the user for inquiring the favorite video is shortened.
Drawings
Fig. 1 is a schematic diagram of a hardware architecture of a television according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a first exemplary embodiment of a gesture-based video control method according to the present invention;
FIG. 3 is a flowchart illustrating a detailed process of step S30 in the second embodiment of the gesture-based video control method according to the present invention;
fig. 4 is a flowchart illustrating step S20 of the gesture-based video control method according to the third embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The main solution of the embodiment of the invention is as follows: when a gesture is detected and an instruction corresponding to the gesture is a progress operation instruction, acquiring a video currently played by a television and receiving times corresponding to the progress operation instruction stored on the television on the same day, wherein the receiving times are updated when the progress operation instruction is detected by the television; determining a target video frame in each video frame in the video, and acquiring a direction and an angle corresponding to the gesture, wherein the target video frame is a highlight video frame in the video; determining the playing rate of the video frames in the video according to the target video frames, the angle corresponding to the gesture and the receiving times; and determining the playing sequence of each video frame in the video according to the direction of the gesture, and playing the video according to the playing sequence and the playing speed.
Because the receiving times of the progress operation instruction represent the current progress operation frequency of the user, and the angle corresponding to the gesture represents the playing speed expected by the user, namely the progress operation frequency of the television set user and the wonderful video frame correct the expected playing speed, the television can reasonably correct the preset playing speed by combining the wonderful part of the video, namely the television plays the wonderful part of the video at the reasonable playing speed, the user is prevented from skipping the wonderful part of the video, and the time consumed by the user for inquiring the favorite video is shortened.
As an implementation, a television may be as shown in fig. 1.
The embodiment of the invention relates to a television, which comprises: a processor 101, e.g. a CPU, a memory 102, a communication bus 103. Wherein a communication bus 103 is used for enabling the connection communication between these components.
The memory 102 may be a high-speed RAM memory or a non-volatile memory (e.g., a disk memory). As shown in fig. 1, a control program may be included in the memory 103 as a kind of computer storage medium; and the processor 101 may be configured to call the control program stored in the memory 102 and perform the following operations:
when a gesture is detected and an instruction corresponding to the gesture is a progress operation instruction, acquiring a video currently played by a television and receiving times corresponding to the progress operation instruction stored on the television on the same day, wherein the receiving times are updated when the progress operation instruction is detected by the television;
determining a target video frame in each video frame in the video, and acquiring a direction and an angle corresponding to the gesture, wherein the target video frame is a highlight video frame in the video;
determining the playing rate of the video frames in the video according to the target video frames, the angle corresponding to the gesture and the receiving times;
and determining the playing sequence of each video frame in the video according to the direction of the gesture, and playing the video according to the playing sequence and the playing speed.
In one embodiment, the processor 101 may be configured to call a control program stored in the memory 102 and perform the following operations:
segmenting the video according to the receiving times to obtain each segmentation time point, wherein the segmentation time points comprise the time point when the sub video obtained by segmenting the video starts playing and the time point when the video finishes playing;
determining a time difference value between the playing time point of each target video frame and each subsection time point, and determining a target difference value in each time difference value, wherein the target difference value is smaller than a preset difference value;
and determining the playing speed of the video according to the ratio of the sum of the target time points corresponding to each target difference value to the angle corresponding to the gesture, wherein the target time point is the earliest time point in the time points corresponding to the target difference values.
In one embodiment, the processor 101 may be configured to call a control program stored in the memory 102 and perform the following operations:
according to the ratio of the sum of the target time points corresponding to each target difference value to the angle corresponding to the gesture;
and acquiring a preset playing speed of the video, and determining the playing speed of the video according to the product of the ratio and the preset playing speed.
In one embodiment, the processor 101 may be configured to call a control program stored in the memory 102 and perform the following operations:
acquiring the total times corresponding to the progress operation instruction stored in the television, wherein the total times are updated when the television detects the progress operation instruction;
when the ratio of the receiving times to the total times is larger than or equal to a first preset ratio, determining an audio image frame in each video frame to determine the video frame as a target video frame, wherein the ratio of a first audio amplitude of the audio image frame to a second audio amplitude of an image frame adjacent to the audio image frame is larger than or equal to a second preset ratio, or the ratio of the second audio amplitude to the first audio amplitude is larger than or equal to a second preset ratio; or the like, or, alternatively,
when the ratio of the receiving times to the total times is smaller than a first preset ratio, determining a moving image frame and the audio image frame in each video frame to determine the moving image frame and the audio image frame as a target video frame, wherein the pixel values of pixel points in the moving image frame in a preset proportion change relative to the adjacent image frame of the moving image frame.
In one embodiment, the processor 101 may be configured to call a control program stored in the memory 102 and perform the following operations:
acquiring a first video frame and a second video frame which are adjacent in the video;
determining a magnitude ratio between a first audio magnitude of the first video frame and a second audio magnitude of the second video frame;
when the amplitude ratio exceeds a preset range, determining the first video frame and the second video frame as audio video frames, wherein the critical value of the preset range is the reciprocal of the second preset ratio and the second preset ratio;
and re-acquiring the adjacent first video frame and second video frame in the rest video frames, and returning to the step of determining the amplitude ratio between the first audio amplitude of the first video frame and the second audio amplitude of the second video frame.
In one embodiment, the processor 101 may be configured to call a control program stored in the memory 102 and perform the following operations:
acquiring a third video frame and a fourth video frame which are adjacent in the video, and determining a second pixel point corresponding to a first pixel point at each preset position in the fourth video frame in the third video frame, wherein the position of the first pixel point in the fourth video frame is the same as the position of the second pixel point in the third video frame;
determining a pixel difference value between the pixel value of the first pixel point and the pixel value of the second pixel point, and determining the number of the pixel difference values which are larger than a preset difference value;
when the ratio of the number to the number of the first pixel points is larger than a preset ratio, determining the third video frame and the fourth video frame as motion video frames;
and re-acquiring adjacent third video frames and fourth video frames from the rest video frames, and returning to the step of determining second pixel points corresponding to the first pixel points at each preset position in the fourth video frames in the third video frames.
In one embodiment, the processor 101 may be configured to call a control program stored in the memory 102 and perform the following operations:
acquiring the orientation of a thumb in the gesture;
determining a direction of the gesture based on the orientation of the thumb.
In one embodiment, the processor 101 may be configured to call a control program stored in the memory 102 and perform the following operations:
acquiring an image acquired by the image acquisition module in real time;
identifying a hand of a user in the image to obtain a gesture;
and when the recognized gesture matches a preset gesture, determining that the command corresponding to the recognized gesture is a progress operation command, and executing the steps of acquiring the video currently played by the television and the number of times corresponding to the progress operation command stored on the television on the same day.
According to the scheme, when the television detects that the command corresponding to the gesture is the progress control command, the receiving times of the currently played video of the television and the progress operation command stored on the television on the same day are obtained, the highlight video frame is determined as the target video frame in each video in the video, the playing speed of the video frame in the video is determined according to the target video frame, the angle corresponding to the gesture and the receiving times, the playing sequence of each video frame in the video is determined according to the direction of the gesture, and finally the video is played according to the playing sequence and the playing speed. Because the receiving times of the progress operation instruction represent the current progress operation frequency of the user, and the angle corresponding to the gesture represents the playing speed expected by the user, namely the progress operation frequency of the television set user and the wonderful video frame correct the expected playing speed, the television can reasonably correct the preset playing speed by combining the wonderful part of the video, namely the television plays the wonderful part of the video at the reasonable playing speed, the user is prevented from skipping the wonderful part of the video, and the time consumed by the user for inquiring the favorite video is shortened.
Based on the hardware architecture of the television, the embodiment of the gesture-based video control method is provided.
Referring to fig. 2, fig. 2 is a first embodiment of the gesture-based video control method according to the present invention, which includes the following steps:
step S10, when a gesture is detected and an instruction corresponding to the gesture is a progress operation instruction, acquiring a video currently played by the television and the receiving times corresponding to the progress operation instruction stored on the television on the same day, wherein the receiving times are updated when the progress operation instruction is detected by the television;
in this embodiment, the television is provided with an image acquisition module, the image acquisition module may be a camera, and the image acquisition module is arranged on the screen of the television to acquire the gesture of the user. Of course, the television can be connected with an independent image acquisition device, and the image acquisition angle of the image acquisition device is the same as the video display direction of the television, so that the image of the user in front of the television can be acquired frontally.
And after the television plays the video, acquiring the image acquired by the image acquisition module in real time. The television analyzes the acquired image, and if the image comprises the hand of the user, the hand can be identified to obtain the gesture. The television further judges whether the recognized gesture matches a preset gesture. The preset gesture is the gesture stored in the television. And if the gesture matches the preset gesture, determining that the command corresponding to the recognized gesture is a progress operation command. The preset gesture can be set on the television by the user, for example, the user sets the five fingers to be opened to the preset gesture. And if the recognized gesture is also the opening of the five fingers, matching the recognized gesture with a preset gesture.
The progress operation instruction comprises a fast forward instruction and a fast backward instruction. The progress operation instruction may be determined according to an orientation of a thumb in the gesture. If the thumb in the image (in the case of a square image) faces to the right, the thumb of the user points to the beginning of the television playing video, so the progress operation instruction is a fast-backward instruction; and if the thumb in the image faces to the left, the thumb of the user points to the end of the television playing video, and the progress operation instruction is a fast-backward instruction.
And when the television determines that the instruction corresponding to the gesture is the progress operation instruction, the television acquires the currently played video and the receiving times corresponding to the progress operation instruction stored on the television on the same day. Specifically, a video currently played by the television has a playing address, and the television acquires all video frames of the video according to the playing address. When the television detects the progress operation command, the receiving times of the progress operation command, namely the receiving times +1, are recorded. The television is connected with the network, so that the television can be synchronized with the actual time, and when the time on the television is updated to a new day, namely to 00:00, the receiving times are reset to zero, namely the television needs to record the receiving times of the schedule operation instruction of the day again.
Step S20, determining a target video frame in each video frame in the video, and acquiring a direction and an angle corresponding to the gesture, wherein the target video frame is a highlight video frame in the video;
the server can mark each highlight video frame in the video frames in advance, and after the television acquires the television, each highlight video frame can be determined according to the mark to serve as a target video frame. The server can determine the highlight video frame according to the heat value of each video frame in the video, and the heat value can be determined by the operation times of each network user on the video frame. The operations of the network user on the video include, but are not limited to, fast forward, pause, fast rewind. The video frame corresponding to the heat value higher than the threshold value can be determined as the highlight video frame.
And the television acquires an angle corresponding to the gesture. Specifically, the television may determine the angle of the gesture through the open fingers, for example, the thumb must be open in the gesture, and the television acquires the open finger farthest from the thumb in the gesture, so as to determine the angle according to the finger and the thumb, for example, if the open finger farthest from the thumb is a little finger, the angle corresponding to the gesture is 90 degrees, and if the open finger farthest from the thumb is a middle finger, the angle corresponding to the gesture is 45 degrees. The television also determines the direction corresponding to the gesture, specifically, the television acquires the direction of the thumb in the gesture, so that the direction of the gesture is determined according to the direction of the thumb.
Step S30, determining the playing speed of the video frame in the video according to the target video frame, the angle corresponding to the gesture and the receiving times;
the angle corresponding to the gesture can represent the preset playing rate of the video selected by the user, the number of the target video frames can represent the wonderful degree of the video, and the receiving times represent the current progress operating frequency of the user. If the receiving times are more, the fast forward or fast backward speed of the video is over slow, and a user requires a higher playing speed; and the more the number of the playing time points is, the higher the wonderful degree of the video is reflected, the lower the speed of fast backward or fast forward is required, so the television can determine a first coefficient according to the number of the playing time points, determine a second coefficient according to the receiving times, and obtain the current playing speed of the video frames in the video at the preset playing speed corresponding to the angle corresponding to the gesture multiplied by the first coefficient and the second coefficient. The first coefficient is related to the number of target video frames, and the larger the number, the smaller the first coefficient (the first coefficient is larger than zero); the second coefficient is associated with the number of receptions, the greater the second coefficient.
Step S40, determining the playing sequence of each video frame in the video according to the direction of the gesture, and playing the video according to the playing sequence and the playing speed.
After the television determines the playing speed, the playing sequence of each video frame in the video is determined according to the direction of the gesture. The play order may be characterized as fast-backward or fast-forward. When the direction of the thumb is towards the right, the direction of the gesture is towards the left, namely the playing sequence is played from late to early according to the time, namely the video is fast reversed. When the thumb is facing left, the direction of the gesture is facing right, that is, the playing sequence is playing from morning to evening according to time, that is, fast forwarding of the video. After the television determines the playing sequence and the playing speed, the video can be played according to the playing sequence and the playing speed. It should be noted that, if the playing rate is less than or equal to the normal playing rate of the video, the video is played back or played normally according to the normal playing rate; and if the playing speed is greater than the normal playing speed of the video, playing the video according to the playing sequence and the playing speed.
In the technical scheme provided by this embodiment, when the television detects that the instruction corresponding to the gesture is the progress control instruction, the receiving times of the currently played video of the television and the progress operation instruction stored on the television on the same day are obtained, the highlight video frame is determined as the target video frame in each video in the video, the playing rate of the video frame in the video is determined according to the target video frame, the angle corresponding to the gesture and the receiving times, the playing sequence of each video frame in the video is determined according to the direction of the gesture by the television, and finally the video is played according to the playing sequence and the playing rate. Because the receiving times of the progress operation instruction represent the current progress operation frequency of the user, and the angle corresponding to the gesture represents the playing speed expected by the user, namely the progress operation frequency of the television set user and the wonderful video frame correct the expected playing speed, the television can reasonably correct the preset playing speed by combining the wonderful part of the video, namely the television plays the wonderful part of the video at the reasonable playing speed, the user is prevented from skipping the wonderful part of the video, and the time consumed by the user for inquiring the favorite video is shortened.
Referring to fig. 3, fig. 3 is a second embodiment of the gesture-based video control method according to the present invention, and based on the first embodiment, the step S30 includes:
step S31, segmenting the video according to the receiving times to obtain each segmentation time point, wherein the segmentation time points comprise the time point when the sub video obtained by segmenting the video starts playing and the time point when the video finishes playing;
in this embodiment, after the receiving times are obtained, the television acquires the total duration of the video, so as to segment the total duration according to the receiving times. Specifically, the total duration is divided by the receiving times to obtain a segmented duration, so that the total duration process of the video is segmented according to the segmented duration, and a playing time point corresponding to the segmentation point is obtained, wherein the playing time point is the segmented time point. That is, the segment time point includes a time point of starting playing and a time point of ending playing of the sub-video obtained by video segmentation.
Step S32, determining a time difference value between the playing time point of each target video frame and each segmentation time point, and determining a target difference value in each time difference value, wherein the target difference value is smaller than a preset difference value;
the television acquires the playing time point of each target video frame, and determines the time difference between each playing time point and each segmentation time point, namely, a plurality of time differences can be obtained. And comparing each time difference value with a preset difference value by the television, and determining the time difference value smaller than the preset difference value as a target difference value. When the ratio between the number of target differences and the total number of time differences is higher than the threshold, the ratio between the number of target differences and the total number of time differences of the television is the threshold. The threshold may be any suitable value, for example, 10%, that is, the television sorts all the time difference values smaller than the preset difference value from small to large, and the television sequentially extracts the minimum time difference value in the sorting queue as the target difference value until the difference value between the number of the target difference values and the total number of the time difference values is closest to the threshold and is smaller than the threshold.
Step S33, determining a playing rate of the video according to a ratio between a sum of target time points corresponding to each target difference value and an angle corresponding to the gesture, where the target time point is an earliest time point among the time points corresponding to the target difference values.
The target difference value is obtained by segmenting the time point and the playing time point, namely the target difference value corresponds to the two time points, the television takes the earliest time point of the two time points corresponding to the target difference value as the target time point, then the sum of all the target time points is determined, and finally the playing speed of the video is determined by dividing the sum of the target time points by the angle corresponding to the gesture.
Further, the ratio between the sum of the target time points and the angle corresponding to the gesture may be characterized as a fast forward ratio or a fast backward ratio. For example, when the ratio is 2, the fast backward ratio or the fast forward ratio is × 2, that is, 2 times the normal playing rate of the video. Therefore, the television can determine the playing speed of the video according to the product of the ratio of the sum of the target time points and the angle corresponding to the gesture and the preset playing speed, wherein the preset playing speed is the playing speed when the video is normally played. In the technical scheme provided by this embodiment, the television segments the video according to the receiving times to obtain each segmented time point, and then determines the time difference between the playing time point of the target video frame and each segmented time point to determine the target difference in the time difference, so as to accurately determine the playing rate of the video according to the ratio of the sum of the target time points corresponding to each target difference to the angle corresponding to the gesture.
Referring to fig. 4, fig. 4 is a third embodiment of the gesture-based video control method according to the present invention, and based on the first or second embodiment, the step S20 includes:
step S21, acquiring the total times corresponding to the progress operation instruction stored in the television, wherein the total times are updated when the television detects the progress operation instruction;
in this embodiment, when the television resets the number of times of reception, the number of times of reception is counted in the total number of times of the progress operation instruction. In addition, the time corresponding to the total number of times should not exceed a preset time, for example, the total number of times of the television segment recording the progress operation instruction of the last week.
Step S22, when the ratio of the receiving times to the total times is greater than or equal to a first preset ratio, determining an audio image frame in each of the video frames to determine the video frame as a target video frame, where a ratio between a first audio amplitude of the audio image frame and a second audio amplitude of an image frame adjacent to the audio image frame is greater than or equal to a second preset ratio, or a ratio between the second audio amplitude and the first audio amplitude is greater than or equal to a second preset ratio;
step S23, when the ratio of the receiving times to the total times is smaller than a first preset ratio, determining a moving image frame and the audio image frame in each of the video frames to determine as a target video frame, where pixel values of pixels in the moving image frame at a preset ratio change with respect to an adjacent image frame of the moving image frame.
The gesture control of the television segment is divided into a low frequency segment control mode and a high frequency segment control mode. The low-frequency segmentation control mode means that the ratio of the current receiving times to the total times is smaller than a first preset ratio. The high-frequency segment control mode refers to that the ratio of the current receiving times to the total times is greater than or equal to a second preset ratio, and the first preset ratio can be any number of combined values, for example, the first preset ratio is 0.5. And if the control modes corresponding to the gesture control are different, the target video frames are different. If the control mode is the low-frequency segmentation control mode, the target video frame is a motion image frame and an audio image frame; if the control mode is the high-frequency segmentation control mode, the target video frame is an audio image frame.
It should be noted that a ratio between a first audio amplitude of an audio image frame and a second audio amplitude of an image frame adjacent to the audio image frame is greater than or equal to a second preset ratio, or a ratio between the second audio amplitude and the first audio amplitude is greater than or equal to the second preset ratio. The second predetermined ratio may be any suitable value, for example, the second predetermined ratio may be 2, that is, the difference between the audio amplitude of an audio image frame and the audio amplitude of its adjacent image frame is greater than the minimum audio amplitude of the two audio amplitudes. The moving image frame refers to that the pixel values of the pixel points in the moving image frame in a preset proportion are sent and changed relative to the adjacent image frame of the moving image frame.
And after the television obtains the total times, determining the control mode of the gesture control. I.e. the television determines the ratio between the number of receptions and the total number. When the ratio is larger than or equal to a first preset ratio, determining an audio image frame in each video frame to be determined as a target video frame; when the ratio is smaller than a first preset ratio, determining a motion image frame and the audio image frame in each video frame to determine as a target video frame.
In the technical scheme provided by the embodiment, the television accurately determines the highlight image frames in the video according to the progress operation frequency of the user on the television at present.
In one embodiment, step S21 includes:
step C1, acquiring adjacent first video frames and second video frames in the video;
step C2, determining a ratio of the first audio amplitude of the first video frame to the second audio amplitude of the second video frame;
step C3, when the amplitude ratio exceeds a preset range, determining the first video frame and the second video frame as audio video frames, where a critical value of the preset range is the reciprocal of the second preset ratio and the second preset ratio;
step C4, retrieving the adjacent first video frame and second video frame from the remaining video frames, and returning to the step of determining the amplitude ratio between the first audio amplitude of the first video frame and the second audio amplitude of the second video frame.
In this embodiment, the tv segment now acquires a first video frame and a second video frame adjacent to each other in playing time from the video. The television then determines a magnitude ratio between a first audio magnitude of the first video frame and a second audio magnitude of the second video frame. The television is provided with a preset range, and the critical value of the preset range is a second preset ratio and the reciprocal of the second preset ratio, namely when the second preset ratio is 2, the preset range is [0.5, 2 ]. When the amplitude ratio exceeds the preset range, it can be determined that the audio amplitude of the first video frame is more than 2 times of the audio amplitude of the second video frame (the second preset ratio is 2), or the audio amplitude of the second video frame is more than 2 times of the audio amplitude of the first video frame. For example, if the amplitude ratio is less than 0.5, it can be determined that the audio amplitude of the second video frame is more than 2 times the audio amplitude of the first video frame; if the amplitude ratio is greater than 2, it can be determined that the audio amplitude of the first video frame is more than 2 times the audio amplitude of the second video frame.
And determining the first video frame and the second video frame as audio-video frames when the determined amplitude ratio of the television exceeds the preset range. That is, the difference of the audio amplitude of the adjacent video frames is large, and the video frames can be regarded as wonderful video frames.
And the television re-determines the adjacent first video frame and second video frame from the rest video frames, thereby returning to the step of determining the amplitude ratio between the first audio amplitude of the first video frame and the second audio amplitude of the second video frame, namely, the television gradually determines the audio video frames by taking the two adjacent video frames as a unit until all the adjacent video frames are determined. Of course, when the ratio is within the preset range, the television re-determines the adjacent first video frame and second video frame from the remaining video frames, thereby returning to the step of determining the amplitude ratio between the first audio amplitude of the first video frame and the second audio amplitude of the second video frame.
In one embodiment, the step of determining a motion image frame in each of said video frames comprises:
step S1, obtaining a third video frame and a fourth video frame which are adjacent to each other in the video, and determining a second pixel point corresponding to a first pixel point at each preset position in the fourth video frame in the third video frame, wherein the position of the first pixel point in the fourth video frame is the same as the position of the second pixel point in the third video frame;
step S2, determining a pixel difference value between the pixel value of the first pixel point and the pixel value of the second pixel point, and determining the number of the pixel difference values larger than a preset difference value;
step S3, when the ratio between the number and the number of the first pixel points is greater than a preset ratio, determining the third video frame and the fourth video frame as motion video frames;
step S4, re-acquiring adjacent third and fourth video frames from the remaining video frames, and returning to the step of determining, in the third video frame, a second pixel point corresponding to the first pixel point at each preset position in the fourth video frame.
In this embodiment, the television acquires a third video and a fourth video which are adjacent to each other in the video, and determines a second pixel point corresponding to the first pixel point at each preset position in the fourth video frame in the third video frame. Each first pixel point corresponding to each preset position is in the same row or the same column, and can also be a random position. The position of the first pixel point in the fourth video frame is the same as the position of the second pixel point in the third video frame, for example, the position of the first pixel point in the fourth video frame is a third row and a third column, and the position of the second pixel point corresponding to the first pixel point in the third video frame is also the third row and the third column.
The television acquires a pixel difference value between the pixel value of the first pixel point and the pixel value of the second pixel point, and then determines the number of the pixel difference values larger than a preset difference value. And the television determines the ratio of the number of the first pixels to the number of the first pixels, and if the ratio is greater than the preset ratio, the third video frame and the fourth video frame are motion image frames. The predetermined ratio can be any number of combinations, for example, the predetermined ratio is 1: 2. It can be understood that the motion video frame can be regarded as a video frame with a tense plot and a background needing to be replaced in the video, that is, the motion video frame can also be regarded as a highlight video frame.
And the television re-determines the adjacent third video frame and fourth video frame from the rest video frames, thereby returning to the step of determining the second pixel point corresponding to the first pixel point at each preset position in the fourth video frame in the third video frame, namely, the television gradually determines the motion video frame by taking the two adjacent video frames as a unit until all the adjacent video frames are determined. Of course, when the ratio is smaller than the preset ratio, the television redetermines the adjacent third video frame and the fourth video frame from the rest video frames.
The invention also provides a television, which comprises a memory, a processor and a control program stored in the memory and executable on the processor, wherein the control program realizes the steps of the gesture-based video control method according to the above embodiment when being executed by the processor.
The present invention also provides a readable storage medium storing a control program, which when executed by a processor implements the steps of the gesture-based video control method according to the above embodiment.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A video control method based on gestures is characterized by comprising the following steps:
when a gesture is detected and an instruction corresponding to the gesture is a progress operation instruction, acquiring a video currently played by a television and receiving times corresponding to the progress operation instruction stored on the television on the same day, wherein the receiving times are updated when the progress operation instruction is detected by the television;
determining a target video frame in each video frame in the video, and acquiring a direction and an angle corresponding to the gesture, wherein the target video frame is a highlight video frame in the video;
determining the playing rate of the video frames in the video according to the target video frames, the angle corresponding to the gesture and the receiving times;
and determining the playing sequence of each video frame in the video according to the direction of the gesture, and playing the video according to the playing sequence and the playing speed.
2. The method of claim 1, wherein the step of determining the play rate of the video frames in the video according to the target video frame, the corresponding angle of the gesture, and the number of receptions comprises:
segmenting the video according to the receiving times to obtain each segmentation time point, wherein the segmentation time points comprise the time point when the sub video obtained by segmenting the video starts playing and the time point when the video finishes playing;
determining a time difference value between the playing time point of each target video frame and each subsection time point, and determining a target difference value in each time difference value, wherein the target difference value is smaller than a preset difference value;
and determining the playing speed of the video according to the ratio of the sum of the target time points corresponding to each target difference value to the angle corresponding to the gesture, wherein the target time point is the earliest time point in the time points corresponding to the target difference values.
3. The method as claimed in claim 2, wherein the step of determining the playing rate of the video according to the ratio between the sum of the target time points corresponding to each target difference value and the angle corresponding to the gesture comprises:
according to the ratio of the sum of the target time points corresponding to each target difference value to the angle corresponding to the gesture;
and acquiring a preset playing speed of the video, and determining the playing speed of the video according to the product of the ratio and the preset playing speed.
4. The method of claim 1, wherein the step of determining a target video frame for each video frame in the video comprises:
acquiring the total times corresponding to the progress operation instruction stored in the television, wherein the total times are updated when the television detects the progress operation instruction;
when the ratio of the receiving times to the total times is larger than or equal to a first preset ratio, determining an audio image frame in each video frame to determine the video frame as a target video frame, wherein the ratio of a first audio amplitude of the audio image frame to a second audio amplitude of an image frame adjacent to the audio image frame is larger than or equal to a second preset ratio, or the ratio of the second audio amplitude to the first audio amplitude is larger than or equal to a second preset ratio; or the like, or, alternatively,
when the ratio of the receiving times to the total times is smaller than a first preset ratio, determining a moving image frame and the audio image frame in each video frame to determine the moving image frame and the audio image frame as a target video frame, wherein the pixel values of pixel points in the moving image frame in a preset proportion change relative to the adjacent image frame of the moving image frame.
5. The method of claim 4, wherein said step of determining an audio image frame in each of said video frames comprises:
acquiring a first video frame and a second video frame which are adjacent in the video;
determining a magnitude ratio between a first audio magnitude of the first video frame and a second audio magnitude of the second video frame;
when the amplitude ratio exceeds a preset range, determining the first video frame and the second video frame as audio video frames, wherein the critical value of the preset range is the reciprocal of the second preset ratio and the second preset ratio;
and re-acquiring the adjacent first video frame and second video frame in the rest video frames, and returning to the step of determining the amplitude ratio between the first audio amplitude of the first video frame and the second audio amplitude of the second video frame.
6. The method of claim 4, wherein said step of determining a motion image frame in each of said video frames comprises:
acquiring a third video frame and a fourth video frame which are adjacent in the video, and determining a second pixel point corresponding to a first pixel point at each preset position in the fourth video frame in the third video frame, wherein the position of the first pixel point in the fourth video frame is the same as the position of the second pixel point in the third video frame;
determining a pixel difference value between the pixel value of the first pixel point and the pixel value of the second pixel point, and determining the number of the pixel difference values which are larger than a preset difference value;
when the ratio of the number to the number of the first pixel points is larger than a preset ratio, determining the third video frame and the fourth video frame as motion video frames;
and re-acquiring adjacent third video frames and fourth video frames from the rest video frames, and returning to the step of determining second pixel points corresponding to the first pixel points at each preset position in the fourth video frames in the third video frames.
7. The method according to any one of claims 1-6, wherein after the step of determining the play rate of the video frames in the video according to the target video frame, the angle corresponding to the gesture, and the number of receptions, the method further comprises:
acquiring the orientation of a thumb in the gesture;
determining a direction of the gesture based on the orientation of the thumb.
8. The method of any one of claims 1-6, wherein the television is provided with an image acquisition module, the method further comprising:
acquiring an image acquired by the image acquisition module in real time;
identifying a hand of a user in the image to obtain a gesture;
and when the recognized gesture matches a preset gesture, determining that the command corresponding to the recognized gesture is a progress operation command, and executing the steps of acquiring the video currently played by the television and the number of times corresponding to the progress operation command stored on the television on the same day.
9. A television comprising a memory, a processor, and a control program stored in the memory and executable on the processor, the control program when executed by the processor implementing the steps of the gesture based video control method according to any of claims 1-8.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a control program which, when executed by a processor, implements the steps of the gesture based video control method according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011073939.2A CN112423059B (en) | 2020-10-09 | 2020-10-09 | Gesture-based video control method, television, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011073939.2A CN112423059B (en) | 2020-10-09 | 2020-10-09 | Gesture-based video control method, television, and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112423059A true CN112423059A (en) | 2021-02-26 |
| CN112423059B CN112423059B (en) | 2023-08-01 |
Family
ID=74854369
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011073939.2A Active CN112423059B (en) | 2020-10-09 | 2020-10-09 | Gesture-based video control method, television, and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112423059B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116761040A (en) * | 2023-08-22 | 2023-09-15 | 超级芯(江苏)智能科技有限公司 | VR cloud platform interaction method and interaction system |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1148770A (en) * | 1995-08-04 | 1997-04-30 | 索尼公司 | Digital video data coding and decoding device removing caption data noise, and method thereof |
| US20100289743A1 (en) * | 2009-05-15 | 2010-11-18 | AFA Micro Co. | Laser pointer and gesture-based input device |
| US20120179970A1 (en) * | 2011-01-06 | 2012-07-12 | Tivo Inc. | Method and Apparatus For Controls Based on Concurrent Gestures |
| CN103299645A (en) * | 2010-12-22 | 2013-09-11 | 谷歌公司 | Video player with assisted seek |
| US20170171592A1 (en) * | 2015-12-10 | 2017-06-15 | Le Holdings (Beijing) Co., Ltd. | Method and electronic apparatus for adjusting viewing angle of Smart Television playing panorama videos |
| WO2017113600A1 (en) * | 2015-12-30 | 2017-07-06 | 深圳Tcl数字技术有限公司 | Video playing method and device |
| CN107197381A (en) * | 2017-06-12 | 2017-09-22 | 深圳Tcl新技术有限公司 | Temperature curve generation method, device and the readable storage medium storing program for executing of television video |
| CN107295409A (en) * | 2017-08-08 | 2017-10-24 | 广东小天才科技有限公司 | A kind of method, device, terminal device and the storage medium of control video playback |
| US20190149873A1 (en) * | 2017-11-16 | 2019-05-16 | Adobe Systems Incorporated | Handheld controller gestures for virtual reality video playback |
| CN109963196A (en) * | 2017-12-26 | 2019-07-02 | 深圳Tcl数字技术有限公司 | A kind of television program playing method, storage medium and smart television |
-
2020
- 2020-10-09 CN CN202011073939.2A patent/CN112423059B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1148770A (en) * | 1995-08-04 | 1997-04-30 | 索尼公司 | Digital video data coding and decoding device removing caption data noise, and method thereof |
| US20100289743A1 (en) * | 2009-05-15 | 2010-11-18 | AFA Micro Co. | Laser pointer and gesture-based input device |
| CN103299645A (en) * | 2010-12-22 | 2013-09-11 | 谷歌公司 | Video player with assisted seek |
| US20120179970A1 (en) * | 2011-01-06 | 2012-07-12 | Tivo Inc. | Method and Apparatus For Controls Based on Concurrent Gestures |
| US20170171592A1 (en) * | 2015-12-10 | 2017-06-15 | Le Holdings (Beijing) Co., Ltd. | Method and electronic apparatus for adjusting viewing angle of Smart Television playing panorama videos |
| WO2017113600A1 (en) * | 2015-12-30 | 2017-07-06 | 深圳Tcl数字技术有限公司 | Video playing method and device |
| CN107197381A (en) * | 2017-06-12 | 2017-09-22 | 深圳Tcl新技术有限公司 | Temperature curve generation method, device and the readable storage medium storing program for executing of television video |
| CN107295409A (en) * | 2017-08-08 | 2017-10-24 | 广东小天才科技有限公司 | A kind of method, device, terminal device and the storage medium of control video playback |
| US20190149873A1 (en) * | 2017-11-16 | 2019-05-16 | Adobe Systems Incorporated | Handheld controller gestures for virtual reality video playback |
| CN109963196A (en) * | 2017-12-26 | 2019-07-02 | 深圳Tcl数字技术有限公司 | A kind of television program playing method, storage medium and smart television |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116761040A (en) * | 2023-08-22 | 2023-09-15 | 超级芯(江苏)智能科技有限公司 | VR cloud platform interaction method and interaction system |
| CN116761040B (en) * | 2023-08-22 | 2023-10-27 | 超级芯(江苏)智能科技有限公司 | VR cloud platform interaction method and interaction system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112423059B (en) | 2023-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11503345B2 (en) | Apparatus, systems and methods for control of sporting event presentation based on viewer engagement | |
| US20210266615A1 (en) | Smart TV detection of STB user-control actions related to STB-originated content presentation speed | |
| EP1763174B1 (en) | Systems and methods for networking digital video recorders | |
| US9043838B2 (en) | Apparatus, systems and methods for a video thumbnail electronic program guide | |
| WO2017113712A1 (en) | Method and device for video switching | |
| US10200742B2 (en) | System and method of displaying content | |
| US10397634B2 (en) | System and method for synchronized presentation of video timeline metadata | |
| EP2070317B1 (en) | Broadcast receiver and method of providing preferred channel thereof | |
| JP5820986B2 (en) | Video receiving apparatus and image recognition method for received video | |
| CN108174280B (en) | Audio and video online playing method and system | |
| US11190828B1 (en) | Systems and methods for versatile video recording | |
| US20160073168A1 (en) | Channel of top socially-commented audio visual content | |
| CN107370768B (en) | Intelligent television streaming media preview system and method | |
| US10205973B2 (en) | Method and system for linking real-time broadcasting with non-real-time video service using user history information | |
| CN107018440B (en) | Methods, systems, and media for presenting advertisements while buffering video | |
| CN111131883B (en) | Video progress adjusting method, television and storage medium | |
| US11082752B2 (en) | Shot-based view files for trick play mode in a network-based video delivery system | |
| CN112423059A (en) | Gesture-based video control method, television and computer-readable storage medium | |
| US9721615B2 (en) | Non-linear video review buffer navigation | |
| US20170164050A1 (en) | Presenting alternate content | |
| JP5838351B2 (en) | Video receiving apparatus and image recognition method for received video | |
| JP2009296536A (en) | Moving image relay distributing device, moving image relay distributing method and program | |
| CN113038245B (en) | Multimedia content playing device and multimedia content playing method | |
| JP4389667B2 (en) | Video search method and system | |
| EP3276969A1 (en) | Content presentation device and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |