CN110502117A - Screenshot method and electric terminal in electric terminal - Google Patents
Screenshot method and electric terminal in electric terminal Download PDFInfo
- Publication number
- CN110502117A CN110502117A CN201910789435.1A CN201910789435A CN110502117A CN 110502117 A CN110502117 A CN 110502117A CN 201910789435 A CN201910789435 A CN 201910789435A CN 110502117 A CN110502117 A CN 110502117A
- Authority
- CN
- China
- Prior art keywords
- key frame
- frame images
- screenshot
- display interface
- subtitle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
- H04N21/4314—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/278—Subtitling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
Abstract
The present invention provides the screenshot method and electric terminal in a kind of electric terminal, which includes: the predetermined input received for triggering screenshotss, into screenshotss state；Under screenshotss state, the type of current display interface is determined；Eyeball eye position is tracked, and determines eyeball eye position information；Screenshot is generated based on the type of identified current display interface and eyeball eye position information.Using the screenshot method and electric terminal in the electric terminal of exemplary embodiment of the present, the step of capable of effectively simplifying shot operation.
Description
Technical field
All things considered of the present invention is related to electronic technology field, more particularly, is related to the screenshot side in a kind of electric terminal
Method and electric terminal.
Background technique
Currently, triggering screenshotss operation generally by combination button or gesture input in existing screenshot mode
To obtain screenshot.When user wishes to obtain long screenshot, generally requires user and manually select plurality of pictures next life growth screenshot, or
For person user by growth of manual scroll through pages next life screenshot, operating process is relatively complicated.
When user wishes to carry out screenshot to video, shot operation is triggered, in video display process usually to intercept
Corresponding video content is to obtain screenshot.But this screenshot mode may be truncated to fuzzy frame image, and user is to be truncated to
Clearly frame image needs to adjust video playback time, and carries out screenshot again, and it is very inconvenient to operate.
When user wishes to intercept continuous multiple frames image in video, need to continuously repeat the multiple shot operation of execution, process
It is relatively complicated.In addition, when user wishes the multiple image of interception merging into an image, it is also necessary to by means of image procossing
Software is just able to achieve, and is operated very complicated.
Summary of the invention
The screenshot method being designed to provide in a kind of electric terminal and electronics of exemplary embodiment of the present invention are whole
End, to overcome above-mentioned at least one defect.
One aspect according to an exemplary embodiment of the present invention provides the screenshot method in a kind of electric terminal, comprising: connect
The predetermined input for triggering screenshotss is received, into screenshotss state；Under screenshotss state, the type of current display interface is determined；With
Track eyeball eye position, and determine eyeball eye position information；Based on the type and eyeball of identified current display interface
Eye position information generates screenshot.
Optionally, the predetermined input may include at least one in following item: the input of at least one physical button, hand
Gesture input, touch input, voice input.
Optionally it is determined that the step of type of current display interface can include: applied according to belonging to current display interface
Type, and/or, the contents and distribution of current display interface determines the type of current display interface.
Optionally, the type of current display interface may include video class, may include video in video class display interface,
In, based on the type of identified current display interface and eyeball eye position information come the step of generating screenshot can include:
According to eyeball eye position information, the concern position of eyeball sight in video is determined；Based in video include the concern position
At least one key frame images for setting corresponding content generate screenshot.
Optionally, the video can be formed by multiple frame images, and the multiple frame image may include multiple key frame images
With multiple intermediate frame images, any key frame images are a complete picture, and any intermediate frame image includes being relative to right
Answer the image change information of key frame.
Optionally, it is generated and is cut based at least one key frame images in video including the corresponding content in the concern position
The step of figure can include: if determining that the concern position of eyeball sight in video is in video according to eyeball eye position information
Subtitle region then obtains the key frame images at the triggering screenshotss moment；It determines and the key frame figure at the triggering screenshotss moment
The corresponding time value of the last one key frame images as being same scene, and using identified time value as cut-off time；
It extracts since triggering the screenshotss moment from the subtitle file of the video to a plurality of subtitle of the cut-off time；From described more
The complete semantic subtitle of expression one is filtered out in subtitle；By at least one key frame images corresponding with the subtitle filtered out,
Screenshot is generated with connecting method.
Optionally, the step of subtitle of the complete semanteme of expression one is filtered out from a plurality of subtitle can include: according to every
The display time of subtitle in a manner of inverted order, for showing that two temporally adjacent subtitles execute following steps: calculating when showing
Between adjacent two subtitles time interval, if showing that a time posterior subtitle and display time are first in two subtitles
A subtitle between time interval be more than or equal to preset time, then delete a display time posterior subtitle and
Other subtitles shown after showing a time posterior subtitle, if showing a time posterior word in two subtitles
Time interval between the subtitle of curtain and display time formerly is less than preset time, then retains two subtitles；By to guarantor
All subtitles stayed carry out semantic analysis, to filter out the complete semantic subtitle of expression one.
Optionally, at least one described key frame images may include a key frame images, wherein by with the word that filters out
Corresponding at least one key frame images of curtain, the step of screenshot is generated with connecting method can include: in one key frame figure
As on the basis of, screenshot is generated in the form of subtitle superposition.
Optionally, at least one described key frame images may include multiple key frame images, wherein by with the word that filters out
Corresponding at least one key frame images of curtain, the step of screenshot is generated with connecting method can include: by the multiple key frame figure
As splicing in chronological order to generate screenshot, alternatively, choosing one from the multiple key frame images represents key frame images, In
It is described represent key frame images on the basis of, generate screenshot in the form of subtitle superposition.
Optionally, represent key frame images the step of is chosen from the multiple key frame images can include: from described
In multiple key frame images access time value with since triggering the screenshotss moment to the intermediate time of the period of cut-off time most
A close key frame images are as key frame images are represented, alternatively, the access time value from the multiple key frame images
A smallest key frame images as with represent key frame images.
Optionally, the screenshot method can further include: if determining that eyeball sight is regarding according to eyeball eye position information
Concern position in frequency is not subtitle region in video, then identifies object corresponding to the concern position of eyeball sight；It obtains
Take at least one key frame images in the predetermined amount of time since triggering the screenshotss moment；From at least one pass described in acquisition
Search includes the candidate key-frames image of the object in key frame image；It is removed from the candidate key-frames image searched similar
Spend high key frame images；The key frame images that will be remained in candidate key-frames image generate screenshot with connecting method.
Optionally, the type of current display interface may include text class, may include text information in text class display interface,
Wherein, it can be wrapped the step of generating screenshot based on the type of identified current display interface and eyeball eye position information
It includes: according to eyeball eye position information, determining the moving direction of eyeball sight；Switch according to the moving direction of eyeball sight aobvious
Show interface, and obtains the image of corresponding display interface when switching every time；When determining display interface finishing switching, by each display
The image at interface forms screenshot according to display order.
Optionally, the type of current display interface may include control class, may include control in control class display interface,
In, based on the type of identified current display interface and eyeball eye position information come the step of generating screenshot can include:
It according to eyeball eye position information, controls electric terminal and executes at least one operating procedure, at least one described operating procedure
The corresponding display interface of any operation step on include control, wherein according to eyeball eye position information by operation described in
Control executes any operation step；Obtain the figure that electric terminal executes corresponding display interface when each operating procedure
Picture；The image of the corresponding display interface of at least one described operating procedure is sequentially formed into screenshot according to the execution of operating procedure.
Optionally, the step of obtaining the image of corresponding display interface when electric terminal executes any operation step can wrap
It includes: the control position addition mark on the image of the corresponding display interface of any operation step of acquisition；
The image after the mark will be added as the image of the corresponding display interface of any operation step.
One aspect according to an exemplary embodiment of the present invention, provides a kind of electric terminal, including input interface, eyeball with
Track unit and processor, wherein input interface receives the predetermined input for triggering screenshotss, and processor is configured as: in response to
The predetermined input received, control electric terminal enters screenshotss state, and the class of current display interface is determined under screenshotss state
Type, eye tracking element keeps track eyeball eye position, and eyeball eye position information is provided, wherein processor is also configured to
Screenshot is generated based on the type of identified current display interface and eyeball eye position information.
Optionally, the predetermined input may include at least one in following item: the input of at least one physical button, hand
Gesture input, touch input, voice input.
Optionally it is determined that the processing of the type of current display interface can include: applied according to belonging to current display interface
Type, and/or, the contents and distribution of current display interface determines the type of current display interface.
Optionally, eye tracking unit can include: infrared light supply emits infrared light towards the eyes of user；Infrared photography
Head receives by the infrared light of user eyeball pupillary reflex, and provides eyeball eye position information.
Optionally, the type of current display interface may include video class, may include video in video class display interface,
In, the processing of screenshot is generated based on the type of identified current display interface and eyeball eye position information can include:
According to eyeball eye position information, the concern position of eyeball sight in video is determined；Based in video include the concern position
At least one key frame images for setting corresponding content generate screenshot.
Optionally, the video can be formed by multiple frame images, and the multiple frame image may include multiple key frame images
With multiple intermediate frame images, any key frame images are a complete picture, and any intermediate frame image includes being relative to right
Answer the image change information of key frame.
Optionally, it is generated and is cut based at least one key frame images in video including the corresponding content in the concern position
The processing of figure can include: if determining that the concern position of eyeball sight in video is in video according to eyeball eye position information
Subtitle region then obtains the key frame images at the triggering screenshotss moment；It determines and the key frame figure at the triggering screenshotss moment
The corresponding time value of the last one key frame images as being same scene, and using identified time value as cut-off time；
It extracts since triggering the screenshotss moment from the subtitle file of the video to a plurality of subtitle of the cut-off time；From described more
The complete semantic subtitle of expression one is filtered out in subtitle；By at least one key frame images corresponding with the subtitle filtered out,
Screenshot is generated with connecting method.
Optionally, the processing of the complete semantic subtitle of expression one is filtered out from a plurality of subtitle can include: according to every
The display time of subtitle in a manner of inverted order, for showing that two temporally adjacent subtitles execute following processing: calculating when showing
Between adjacent two subtitles time interval, if showing that a time posterior subtitle and display time are first in two subtitles
A subtitle between time interval be more than or equal to preset time, then delete a display time posterior subtitle and
Other subtitles shown after showing a time posterior subtitle, if showing a time posterior word in two subtitles
Time interval between the subtitle of curtain and display time formerly is less than preset time, then retains two subtitles；By to guarantor
All subtitles stayed carry out semantic analysis, to filter out the complete semantic subtitle of expression one.
Optionally, at least one described key frame images may include a key frame images, wherein by with the word that filters out
At least one corresponding key frame images of curtain, the processing of screenshot are generated with connecting method can include: in one key frame figure
As on the basis of, screenshot is generated in the form of subtitle superposition.
Optionally, at least one described key frame images may include multiple key frame images, wherein by with the word that filters out
At least one corresponding key frame images of curtain, the processing of screenshot are generated with connecting method can include: by the multiple key frame figure
As splicing in chronological order to generate screenshot, alternatively, choosing one from the multiple key frame images represents key frame images, In
It is described represent key frame images on the basis of, generate screenshot in the form of subtitle superposition.
Optionally, a processing for representing key frame images is chosen from the multiple key frame images can include: from described
In multiple key frame images access time value with since triggering the screenshotss moment to the intermediate time of the period of cut-off time most
A close key frame images are as key frame images are represented, alternatively, the access time value from the multiple key frame images
A smallest key frame images as with represent key frame images.
Optionally, if the processor can be also configured to determine that eyeball sight exists according to eyeball eye position information
Concern position in video is not subtitle region in video, then identifies object corresponding to the concern position of eyeball sight；
Obtain at least one key frame images in the predetermined amount of time since triggering the screenshotss moment；From described in acquisition at least one
Search includes the candidate key-frames image of the object in key frame images；Phase is removed from the candidate key-frames image searched
Like the high key frame images of degree；The key frame images that will be remained in candidate key-frames image generate screenshot with connecting method.
Optionally, the type of current display interface may include text class, may include text information in text class display interface,
Wherein, the electric terminal can further include display screen, based on the type and eyeball sight of identified current display interface
Location information generates the processing of screenshot can include: according to eyeball eye position information, determines the moving direction of eyeball sight；It presses
According to the moving direction control display screens switch display interface of eyeball sight, and obtain corresponding display interface when switching every time
Image；When determining display interface finishing switching, the image of each display interface is formed into screenshot according to display order.
Optionally, the type of current display interface may include control class, may include control in control class display interface,
In, the processing of screenshot is generated based on the type of identified current display interface and eyeball eye position information can include:
It according to eyeball eye position information, controls electric terminal and executes at least one operating procedure, at least one described operating procedure
The corresponding display interface of any operation step on include control, wherein according to eyeball eye position information by operation described in
Control executes any operation step；Obtain the figure that electric terminal executes corresponding display interface when each operating procedure
Picture；The image of the corresponding display interface of at least one described operating procedure is sequentially formed into screenshot according to the execution of operating procedure.
Optionally, the processing for obtaining the image of corresponding display interface when electric terminal executes any operation step can wrap
It includes: the control position addition mark on the image of the corresponding display interface of any operation step of acquisition；
The image after the mark will be added as the image of the corresponding display interface of any operation step.
Other side according to an exemplary embodiment of the present invention, providing a kind of computer for being stored with computer program can
Storage medium is read, when the computer program realizes the screenshot method in above-mentioned electric terminal when being executed by processor.
Using the screenshot method and electric terminal in the electric terminal of exemplary embodiment of the present, can effectively simplify
The step of shot operation.
Part in following description is illustrated into the other aspect and/or advantage of present general inventive concept, there are also one
Dividing will be apparent by description, or can learn by the implementation of present general inventive concept.
Detailed description of the invention
Pass through the detailed description carried out below with reference to the attached drawing for being exemplarily illustrated embodiment, exemplary embodiment of the present
Above and other objects, features and advantages will become apparent, in which:
Fig. 1 shows the flow chart of the screenshot method in electric terminal according to an exemplary embodiment of the present invention；
Fig. 2 shows the types and eyeball according to an exemplary embodiment of the present invention based on identified current display interface
Eye position information is come the first exemplary process diagram the step of generating screenshot；
Fig. 3 shows the subtitle according to an exemplary embodiment of the present invention that the complete semanteme of expression one is filtered out from a plurality of subtitle
The step of flow chart；
Fig. 4 shows the exemplary diagram according to an exemplary embodiment of the present invention that screenshot is generated in the form of subtitle superposition；
Fig. 5 shows according to an exemplary embodiment of the present invention splice multiple key frame images in chronological order to generate and cut
The exemplary diagram of figure；
The first application that Fig. 6 to Fig. 8 shows the screenshot method in electric terminal according to an exemplary embodiment of the present invention is shown
Example diagram；
The second application that Fig. 9 and Figure 10 shows the screenshot method in electric terminal according to an exemplary embodiment of the present invention is shown
Example diagram；
Figure 11 and Figure 12 shows the third application of the screenshot method in electric terminal according to an exemplary embodiment of the present invention
Exemplary diagram；
Figure 13 shows the type and eye according to an exemplary embodiment of the present invention based on identified current display interface
Ball eye position information is come the second exemplary process diagram the step of generating screenshot；
Figure 14 and Figure 15 shows the 4th application of the screenshot method in electric terminal according to an exemplary embodiment of the present invention
Exemplary diagram；
Figure 16 shows the type and eye according to an exemplary embodiment of the present invention based on identified current display interface
Ball eye position information is come third exemplary process diagram the step of generating screenshot；
Figure 17 shows the 5th of the screenshot method in electric terminal according to an exemplary embodiment of the present invention using exemplary diagram；
Figure 18 shows the type and eye according to an exemplary embodiment of the present invention based on identified current display interface
Ball eye position information is come the 4th exemplary process diagram the step of generating screenshot；
Figure 19 shows the 6th of the screenshot method in electric terminal according to an exemplary embodiment of the present invention using exemplary diagram；
Figure 20 shows the block diagram of electric terminal according to an exemplary embodiment of the present invention.
Specific embodiment
It reference will now be made in detail the embodiment of the present invention, examples of the embodiments are shown in the accompanying drawings, wherein identical mark
Number identical component is referred to always.It will illustrate the embodiment, by referring to accompanying drawing below to explain the present invention.
Fig. 1 shows the flow chart of the screenshot method in electric terminal according to an exemplary embodiment of the present invention.
Referring to Fig.1, in step slo, the predetermined input for triggering screenshotss is received, into screenshotss state.
As an example, predetermined input may include at least one in following item: the input of at least one physical button, gesture
Input, touch input, voice input.It should be understood that the predetermined input for triggering screenshotss is not limited to above-mentioned cited several sides
Formula can also trigger screenshotss by other means.
In step S20, under screenshotss state, the type of current display interface is determined.
In one example, the class of current display interface can be determined according to the type applied belonging to current display interface
Type.Here, the type applied belonging to current display interface can be determined by various modes, and which is not limited by the present invention.
In one example, by taking electric terminal is using android system as an example, active window manager can be passed through
(Activity Manager) determines the type applied belonging to current display interface.
Manager=(ActivityManager) getSystemService (Context.ACTIVITY_SERVI
CE)；String className=info.topActivity.getClassName ()；// complete class name
By the type for parsing above-mentioned class name available applications.It should be understood that the mode of the type of above-mentioned determining application is only
Example can also determine the type of application by other means.
In another example, the type of current display interface can be determined according to the contents and distribution of current display interface.
Here, the contents and distribution of current display interface can be determined by various modes, for example, can be by current display
The description file at interface is parsed the contents and distribution to determine current display interface, and which is not limited by the present invention.
It preferably, can be by above two judgment method being combined to more accurately determine the class of current display interface
Type.
In step s 30, eyeball eye position is tracked, and determines eyeball eye position information.
Here, eyeball eye position is tracked in step S30 under screenshotss state.It should be understood that in the exemplary implementation of the present invention
Step S30 can also be first carried out in example and executes step S20 again, or step S20 and step are performed simultaneously under screenshotss state
S30, the present invention for the execution sequencing between step step S20 and step S30 without limitation.
For example, eyeball eye position can be tracked in several ways so that it is determined that eyeball eye position information.
In one example, eyeball eye position is tracked using the front camera of electric terminal.
For example, electric terminal there can be front camera, under screenshotss state, the front camera of electric terminal can control
It opens, and captures the image of eyes of user, identified by the image to the eyes of user captured to track eyeball sight
Position determines eyeball eye position information.
Here, the image of eyes of user is identified normal known in this field to track the method for eyeball eye position
Know, the present invention repeats no more the content of this part.
In another example, eye is tracked using the eyeball inductor of electric terminal (alternatively referred to as eyeball tracking sensor)
Ball eye position.
For example, electric terminal there can be eyeball inductor, under screenshotss state, the eyeball inductor of electric terminal can control
It opens, which can be traced eyeball eye position, and provide eyeball eye position information.
Here, the method for eyeball inductor tracking eyeball eye position is the common knowledge of this field, and the present invention is to this portion
The content divided repeats no more.
In another example, eyeball eye position is tracked using the infrared light supply and infrared camera of electric terminal.
For example, electric terminal can have infrared light supply and infrared camera.In the case, infrared light supply can be towards user
Eyes emit infrared light, infrared camera is received by the infrared light of user eyeball pupillary reflex, and provides eyeball eye position
Information.
In step s 40, it is generated based on the type of identified current display interface and eyeball eye position information
Screenshot.
Here, the concern position of eyeball sight can be determined according to eyeball eye position information, in addition to this it is possible to
Determine the moving direction of eyeball sight, according to eyeball eye position information to control electricity based on the moving direction of eyeball sight
Sub- terminal executes page turn over operation or slide.
As an example, in exemplary embodiments of the present invention, the type of current display interface may include video class, text class
And control class.It should be understood that the type of current display interface can be not limited to above-mentioned cited three types, it can also include other
Type can also be cut for other kinds of current display interface using the screenshot method in above-mentioned electric terminal
Figure.
The process for generating screenshot is introduced for different types of current display interface separately below.
In the first embodiment, the process for generating screenshot is introduced so that the type of current display interface is video class as an example.
It here, include video in video class display interface.
For the current display interface of video class, screenshot can be generated in the following way: be believed according to eyeball eye position
Breath, determines the concern position (that is, coordinate position of the focus of eyeball sight in video) of eyeball sight in video, is based on
It include at least one key frame images generation screenshot of the corresponding content in concern position in video.
It should be understood that video is formed by multiple frame images, multiple frame images include multiple key frame images and multiple intermediate frames
Image, any key frame images be a complete picture, any intermediate frame image include be relative to correspondence key frame figure
As change information.
For example, identifier can be arranged to the key frame images in video in advance, based on set identifier come from video
Middle extraction key frame images.In addition to this it is possible to key frame images be extracted from video by other means, for example, can
Extract key frame images in the following way: the method based on sampling, the method based on camera lens, the side based on color characteristic
Method, the method based on motion analysis, the method based on cluster.It should be understood that the mode for extracting key frame images is not limited to above-mentioned institute
The several method enumerated can also extract key frame images using other methods from video.
For example, the difference of the concern position according to eyeball sight in video, can be used different modes to generate screenshot.
The first situation determines that the concern position of eyeball sight in video is in video according to eyeball eye position information
Subtitle region.
It in the case, can be according to the complete semantic subtitle of subtitle expression one with the triggering screenshotss moment for including in video
Corresponding at least one key frame images generate screenshot.
It is introduced referring to Fig. 2 when the concern position of eyeball sight in video is subtitle region in video,
The process of screenshot is generated based on the current display interface of video class and eyeball eye position information.
Fig. 2 shows the types and eyeball according to an exemplary embodiment of the present invention based on identified current display interface
Eye position information is come the first exemplary process diagram the step of generating screenshot.
Referring to Fig. 2, in step S401, the key frame images at the triggering screenshotss moment are obtained.
In general, the corresponding subtitle file of a video, a subtitle file include a plurality of subtitle, every subtitle is by display
Time is shown.For example, a certain moment is in the display time of a certain subtitle if video playing is to a certain moment
It is interior, then a certain subtitle, when end time of the video playing to a certain subtitle, this word are shown on the display screen
Curtain is no longer shown.
For example, every subtitle may include time started (beginTime), end time (endTime), title, word
Symbol string, character body (strBody).
As an example, the format of subtitle file can be as follows:
00:00:02,000-> 00:00:7,999
<i>prompt caption information 1</i>
……
00:20:02,000-> 00:20:7,999
<i>prompt caption information 100</i>
……
Wherein, the format of a subtitle can be as follows:
00:20:02,000-> 00:20:7,999
<i>prompt caption information 100</i>
Taking the above example as an example, " 00:20:02, at the beginning of 000 " indicates this subtitle, " 00:20:7,999 " are indicated
The end time of this subtitle, " prompt caption information " is character string, i.e., shown caption content, "<i>" indicate character
The origin identification of string accords with,</i>indicate the end identifier of character string.
After triggering screenshot, a word in the display of triggering screenshotss moment can be searched for from the corresponding subtitle file of video
Curtain, and determine the display time (that is, from period at the beginning of a subtitle to the end time) of a subtitle, from
The key frame images of display time of the search in a subtitle in video, as the key frame figure at the triggering screenshotss moment
Picture.
If searched only for a key frame images, using the key frame images searched as in triggering screenshotss
The key frame images at moment.It, can be by the designated key frame in multiple key frame images if searching multiple key frame images
Image is as the key frame images at the triggering screenshotss moment.
As an example, designated key frame image can be determined in the following way: from the multiple key frame images searched
The immediate key frame images of intermediate time of the display time of middle access time value and a subtitle are closed as specified
Key frame image.Alternatively, the smallest key frame images conduct of access time value refers to from the multiple key frame images searched
Determine key frame images.
If not searching for key frame images, time value is nearest before the display time for being chosen at a subtitle one
A key frame images are as the key frame images at the triggering screenshotss moment.
In step S402, determine crucial for the last one of same scene with the key frame images at the triggering screenshotss moment
The corresponding time value of frame image, and using identified time value as cut-off time.
Here, it is believed that a complete semanteme is generally expressed in a scene of video, accordingly it is contemplated that from identical field
The complete semantic subtitle of search expression one in the key frame images of scape.
For example, the key frame images and time value at the triggering screenshotss moment that can be will acquire are after triggering the screenshotss moment
Other key frame images are compared, and to determine whether scene converts, when determining that scene converts, will be had identical
The corresponding time value of the last one key frame images of scene is as cut-off time.Here, using various image recognition processings
Method determines whether two images convert, and which is not limited by the present invention.
In step S403, extracted from the subtitle file of video a plurality of to cut-off time since triggering the screenshotss moment
Subtitle.
In step s 404, the complete semantic subtitle of expression one is filtered out from a plurality of subtitle.
Fig. 3 shows the subtitle according to an exemplary embodiment of the present invention that the complete semanteme of expression one is filtered out from a plurality of subtitle
The step of flow chart.
The time interval of i-th subtitle and (i-1)-th subtitle is calculated in step S41 referring to Fig. 3.
Here, i-th subtitle and (i-1)-th subtitle can refer to show that two temporally adjacent subtitles, i-th subtitle can refer to show
Show and shows that a time posterior subtitle, (i-1)-th subtitle can refer to show temporally adjacent two in two temporally adjacent subtitles
A time first subtitle is shown in subtitle.
In step S42, determine whether the time interval of i-th subtitle and (i-1)-th subtitle is less than preset time.
If time interval is not less than (being more than or equal to) preset time, S43 is thened follow the steps: deleting i-th subtitle,
Also other subtitles shown after i-th subtitle are also deleted at the same time.
In step S44, so that i=i-1, and return to step S41.
If time interval is less than preset time, S45 is thened follow the steps: language is carried out to i-th subtitle and (i-1)-th subtitle
Justice analysis.
In step S46, determine whether semantic content expressed by two subtitles is identical.
Here, can judge whether semantic content expressed by two subtitles is identical using various semantic analysis.Example
Such as, semantic analysis can be carried out using Recognition with Recurrent Neural Network, however, the present invention is not limited thereto can also carry out by other means
Semantic analysis.
If i-th subtitle and (i-1)-th subtitle are not the same semantic contents of expression, S47 is thened follow the steps: deleting i-th
Subtitle, at the same time still deletes other subtitles shown after i-th subtitle.
If i-th subtitle and (i-1)-th subtitle express same semantic content, thens follow the steps S48: retaining i-th word
Curtain.
In step S49, determine whether i is equal to 2.Here, the initial value of i is n, and n in above-mentioned steps S403 by determining
The quantity of a plurality of subtitle out, for example, nth subtitle refers to showing a subtitle of the time after in a plurality of subtitle.
That is being to be sentenced according to the display time of every subtitle in a manner of inverted order to carry out semanteme in exemplary embodiments of the present invention
Disconnected.
If i is not equal to 2, S51 is thened follow the steps: so that i=i-1, and return to step S41.
If i is equal to 2, the semantic analysis to a plurality of subtitle is completed, the subtitle remained at this time is to filter out
The complete semantic subtitle of expression one.
It should be understood that subtitle semanteme screening mode shown in Fig. 3 is merely illustrative, the invention is not limited thereto, can also pass through other
Mode is analyzed a plurality of subtitle to be screened out from it the complete semantic subtitle of expression one.For example, being directed to stream shown in Fig. 3
Journey can also retain i-th subtitle, and directly execute step S49, when time interval is less than preset time when i is equal to
When 2, the judgement of the time interval of a plurality of subtitle is completed in expression, semantic by carrying out to all subtitles remained at this time
Analysis, to filter out the complete semantic subtitle of expression one.
Fig. 2 is returned to, in step S405, by least one key frame images corresponding with the subtitle filtered out, with splicing
Mode generates screenshot.
For example, it may be determined that the subtitle filtered out corresponding display period, the display period can refer to from the word filtered out
Show that the time shows a word of the time after into the subtitle filtered out at the beginning of the preceding subtitle in curtain
Period between the end time of curtain.Search is at least one key frame images conduct of the display period from video
At least one key frame images corresponding with the subtitle filtered out, at least one key frame images searched spliced with
Generate screenshot.
A kind of situation, at least one key frame images include a key frame images.
In the case, screenshot can be generated in the form of subtitle superposition on the basis of a key frame images.
Fig. 4 shows the exemplary diagram according to an exemplary embodiment of the present invention that screenshot is generated in the form of subtitle superposition.
For shown in Fig. 4, the side of subtitle, the expression one that will be filtered out can be shown in a key frame images
Complete semantic a plurality of subtitle is sequentially overlapped by display time sequencing, and the display area of a plurality of subtitle is not overlapped.This is closed
Key frame image and superimposed a plurality of subtitle are formed by image as screenshot.At this point, in screenshot i.e. include can be clear
The key frame images of clear reflecting video content further comprise the complete semantic a plurality of subtitle of the expression one under the scene.
Another situation, at least one key frame images include multiple key frame images.
In the case, multiple key frame images can be spliced in chronological order to generate screenshot (as shown in Figure 5).As
Example can select splicing style according to the semantic style of a plurality of subtitle.
In addition to this, it for multiple key frame images are included the case where, can also use above-mentioned in the form of subtitle superposition
To generate screenshot.
In the case, one can be chosen from multiple key frame images and represent key frame images, representing key frame images
On the basis of, screenshot is generated in the form of subtitle superposition.
As an example, one key frame images can be represented in the following manner to choose from multiple key frame images: from more
Since access time value and the intermediate time of the period triggering the screenshotss moment to cut-off time most connect in a key frame images
A close key frame images are used as and represent key frame images.Alternatively, access time value is the smallest from multiple key frame images
One key frame images as with represent key frame images.
The above-mentioned screenshot process for video class display interface is introduced below by several embodiments.
Fig. 6-Fig. 8 shows first of the screenshot method in electric terminal according to an exemplary embodiment of the present invention using example
Figure.
In the present example it is assumed that television set " pretty good " is currently viewed in user, when receiving for triggering screenshotss
When predetermined input, video playing picture is picture shown in fig. 6, by tracking eyeball eye position, determines eyeball sight at this time
Concern position in video is subtitle region in video (for example, frame favored area shown in Fig. 6).
Assuming that the complete semantic subtitle of the expression one filtered out shares five subtitles, it is as follows:
00:00:02,000-> 00:00:6,999
<i>su Ming is beautiful, and you are a girl</i>
00:00:08,000-> 00:00:13,999
<i>how you can be with your two elder brothers' ratios</i>
00:00:14,045-> 00:00:18,678
<i>we are only responsible for you and support you by 18 years old</i>
00:00:20,045-> 00:00:24,678
<i>it also to get married after you</i>
00:00:26,066-> 00:00:30,678
<i>to it is old we do not need you yet and support</i>
From video search in above-mentioned five subtitles corresponding display period (00:00:02,000 to 00:00:30,
678) key frame images.
Assuming that searching only for a key frame images (as shown in Figure 7) to screen then on the basis of the key frame images
Five subtitles out are overlapped to obtain screenshot (as shown in Figure 8).
The second application that Fig. 9 and Figure 10 shows the screenshot method in electric terminal according to an exemplary embodiment of the present invention is shown
Example diagram.
In the present example it is assumed that when receiving the predetermined input for triggering screenshotss, view that user is currently viewed
Frequency picture is picture shown in Fig. 9, by tracking eyeball eye position, determines the concern position of eyeball sight in video at this time
For subtitle region in video.
Assuming that the complete semantic subtitle of the expression one filtered out shares two subtitles, it is as follows:
00:10:02,000-> 00:10:12,999
<i>although we say with smile mutually " seeing you later " 1</i>
00:10:14,000-> 00:10:22,999
<i>but we know separation fairly well and say goodbye to 2 forever</i>
Here it is possible in video search in above-mentioned two subtitles corresponding display period (00:10:02,000 to 00:
10:22,999) key frame images.In addition to this it is possible to be searched for and every respectively respectively for the every subtitle filtered out
The corresponding key frame images of subtitle are made for example, the key frame images of the display time in this subtitle can be searched for from video
For key frame images corresponding with this subtitle.Here, method shown in the step S401 of above-mentioned Fig. 2 can be used determine with
The corresponding key frame images of any bar subtitle, the present invention repeat no more the content of this part.
Assuming that two subtitles are respectively provided with corresponding key frame images, in the case, can by two key frame images into
Row picture mosaic is to generate screenshot (as shown in Figure 10).
Figure 11 and Figure 12 shows the third application of the screenshot method in electric terminal according to an exemplary embodiment of the present invention
Exemplary diagram.
In the present example it is assumed that when receiving the predetermined input for triggering screenshotss, view that user is currently viewed
Frequency picture is picture shown in Figure 11, by tracking eyeball eye position, determines the concern position of eyeball sight in video at this time
It is set to subtitle region in video.
Assuming that the complete semantic subtitle of the expression one filtered out shares four subtitles, and every subtitle is respectively provided with correspondence
Key frame images, four key frame images can be carried out to picture mosaics to generate screenshot (as shown in figure 12).
Same key frame images are corresponded to if there is two or two or more subtitles, it can also be using institute in this example
The mode shown generates screenshot, and only there are identical key frame images in screenshot, that is, subtitle is different but picture is identical.
It should be understood that subtitle superposition mode shown in above-mentioned example and image mosaic mode are merely illustrative, the present invention is not
It is limited to this, screenshot can also be generated using other modes.
Second situation determines that the concern position of eyeball sight in video is not video according to eyeball eye position information
Middle subtitle region.
In the case, it is believed that the concern of user eyeball sight is certain an object in video, at this time can be by eye
The concern position of ball sight in video is identified to determine eyeball sight object of interest, and according to including institute in video
At least one key frame images of the object of concern generate screenshot.
It is introduced referring to Figure 13 when the concern position of eyeball sight in video is not subtitle region in video
When, the process of screenshot is generated based on the current display interface of video class and eyeball eye position information.
Figure 13 shows the type and eye according to an exemplary embodiment of the present invention based on identified current display interface
Ball eye position information is come the second exemplary process diagram the step of generating screenshot.
Referring to Fig.1 3, in step S411, identify object corresponding to the concern position of eyeball sight.
Here, object can refer to personage or article in video.For example, can using various recognition processing methods come pair
Video is identified to determine object corresponding to the concern position of eyeball sight.
In step S412, at least one key frame figure in the predetermined amount of time since triggering the screenshotss moment is obtained
Picture.
For example, at least one in the predetermined amount of time since triggering the screenshotss moment can be in from search time value in video
A key frame images.Here, the time span of predetermined amount of time can be set according to demand, and the present invention does not limit this
It is fixed.
In step S413, search includes the candidate of identified object from least one key frame images of acquisition
Key frame images.
For example, can be identified using various recognition processing methods at least one key frame images, with determination
It whether include identified object at least one key frame images, the key frame images of the object comprising being identified are true
It is set to candidate key-frames image.Here, the quantity of the candidate key-frames image searched can be more than or equal to 1.
In step S414, the high key frame images of similarity are removed from the candidate key-frames image searched.
For example, duplicate removal processing can be carried out to the candidate key-frames image searched, the high key frame images of similarity are gone
It removes, retains identified object as far as possible and be in different angle or the key frame images under different scenes.
In step S415, the key frame images that will be remained in candidate key-frames image are generated with connecting method and are cut
Figure.
For example, the key frame images remained can be carried out with picture mosaic to generate screenshot, various connecting methods can be used
Screenshot is generated, as an example, can be spliced according to frame style, which is not limited by the present invention.
Here, it since the key frame images for generating screenshot are clearly image, can be effectively ensured obtained
The clarity of screenshot.
Figure 14 and Figure 15 shows the 4th application of the screenshot method in electric terminal according to an exemplary embodiment of the present invention
Exemplary diagram.
In the present example it is assumed that when receiving the predetermined input for triggering screenshotss, view that user is currently viewed
Frequency picture is picture shown in Figure 14, by tracking eyeball eye position, determines the concern position of eyeball sight in video at this time
Setting is not subtitle region in video, carries out identifying that its determining object of interest is by the concern position to eyeball sight
Personage in video (for example, frame favored area shown in Figure 14).
At least one key frame images in the predetermined amount of time since triggering the screenshotss moment are obtained, at least one pass
Key frame image carries out screening and duplicate removal processing, and by all angles comprising the personage remained, the key of each expression
Frame image is spliced to generate screenshot.
It should be understood that in addition to above by mode of the eyeball eye position to generate screenshot is tracked, it can also be in screenshotss shape
The selection operation for being used to determine object that user's input is received under state, object is determined in response to the selection operation, then pass through
Above-mentioned steps S412~S415 is executed to obtain screenshot.As an example, the selection operation may include enterprising in current display interface
Row circle selection operation, with selected object.
In a second embodiment, the process for generating screenshot is introduced so that the type of current display interface is text class as an example.
It here, include text information in text class display interface.It is answered as an example, text class display interface may include but be not limited to communication
The browser interface of chat interface or picture and text.
It is introduced referring to Figure 16 based on the current display interface of text class and eyeball eye position information and is generated
The process of screenshot.
Figure 16 shows the type and eye according to an exemplary embodiment of the present invention based on identified current display interface
Ball eye position information is come third exemplary process diagram the step of generating screenshot.
Referring to Fig.1 6, in step s 51, according to eyeball eye position information, determine the moving direction of eyeball sight.
Here, the method that the moving direction of eyeball sight is determined according to eyeball eye position information is normal known in this field
Know, the present invention repeats no more the content of this part.
In step S52, switch display interface according to the moving direction of eyeball sight, and obtains corresponding when switching every time
Display interface image.
It, can be according to the movement of eyeball sight if the moving direction of eyeball sight is consistent with the switching mode of display interface
Direction switches display interface.
For example, the case where switching mode for display interface is left and right page turning, if the moving direction of eyeball sight is
It is moved to the left and perhaps moves right, control electric terminal and switch to the left from current display interface or switch to the right.For aobvious
The case where switching mode for showing interface is upper and lower page turning, if the moving direction of eyeball sight is to move up or to moving down
It is dynamic, then it controls current display interface and scrolls up or scroll down through.
For example, the image of corresponding display interface when switching every time can be directly intercepted in each switching, in addition to this,
Record screen function can also be opened while according to the moving direction of eyeball sight to switch display interface, from the video of recording
Key frame images are extracted, the image of corresponding display interface when obtaining switching every time.
In step S53, when determining display interface finishing switching, by the image of each display interface according to display order shape
At screenshot.
For example, when meeting following condition for the moment, it may be determined that display interface finishing switching.
The concern position of eyeball sight does not change in the given time；Corresponding display circle when by each switching
Text information in face carries out semantic analysis, determines that completion has been displayed in the complete semantic text information of expression one；Detect text
The paragraph of information terminates；Detect that the spacing distance in display interface between two neighboring paragraph is greater than preset distance value；Detection
To having been switched to end interface.
It here,, can be by above-mentioned if current display interface can switch for the current display interface of text class
Method shown in Figure 16 carries out screenshot according to the moving direction of eyeball sight, if current display interface cannot switch,
The image of current display interface is then directly intercepted as final screenshot.
Figure 17 shows the 5th of the screenshot method in electric terminal according to an exemplary embodiment of the present invention using exemplary diagram.
In the present example it is assumed that current display interface is the product about Samsung Intelligent laser projection instrument under screenshotss state
Interface (image of the leftmost side in such as Figure 17) is introduced, determines that the type of current display interface is text class by various modes.It is false
If the case where switching mode of current display interface is left and right page turning, when the moving direction for determining user eyeball sight is to move right
When dynamic, control electric terminal switches to the right from current display interface, and obtains the image of corresponding display interface when switching every time
(image of centre and the rightmost side in such as Figure 17).
Text information when by each switching in corresponding display interface carries out semantic analysis, determines that expression one is complete
Completion has been displayed in semantic text information, for example, being determined by semantic analysis through three display interfaces pair shown in Figure 17
Completion has been displayed in the product introduction of Samsung Intelligent laser projection instrument, at this point, will acquire each switching when corresponding display interface
Image according to display order formed screenshot (as shown in figure 17).
In the third embodiment, the process for generating screenshot is introduced so that the type of current display interface is control class as an example.
It here, include control in control class display interface.As an example, control may include icon or menu item on display interface.
It is introduced referring to Figure 18 based on the current display interface of control class and eyeball eye position information and is generated
The process of screenshot.
Figure 18 shows the type and eye according to an exemplary embodiment of the present invention based on identified current display interface
Ball eye position information is come the 4th exemplary process diagram the step of generating screenshot.
Referring to Fig.1 8, in step S61, according to eyeball eye position information, controls electric terminal and execute at least one behaviour
Make step.It here, include control on the corresponding display interface of any operation step at least one operating procedure, according to eyeball
Eye position information executes any operation step by operational controls.
For example, control electric terminal executes the process of any operation step are as follows: determined when according to eyeball eye position information
Eyeball sight the concern position in the corresponding display interface of any operation step be display interface in control region when,
It is automatically brought into operation the control on the display interface included, to execute any operation step automatically.
In step S62, the image that electric terminal executes corresponding display interface when each operating procedure is obtained.
For example, can directly intercept the image of corresponding display interface when electric terminal executes each operating procedure.In
In one preferred embodiment, electric terminal can also be obtained in the following manner and executes corresponding display circle when any operation step
The image in face, for example, the control position addition mark on the image of the corresponding display interface of any operation step of acquisition
Know, using the image after addition mark as the image of the corresponding display interface of any operation step.
In step S63, by the image of the corresponding display interface of at least one operating procedure according to the execution of operating procedure
Sequentially form screenshot.
Figure 19 shows the 6th of the screenshot method in electric terminal according to an exemplary embodiment of the present invention using exemplary diagram.
In the present example it is assumed that current display interface is that the desktop of electric terminal is (most left in such as Figure 19 under screenshotss state
The image of side), determine that the type of current display interface is control class by various modes.When true according to eyeball eye position information
Determine concern position of the eyeball sight in display interface be display interface in control (icon " is set ") region when, automatically
The control is operated, control electric terminal enters set interface (second image in left side in such as Figure 19).When according to eyeball sight position
Confidence, which ceases, determines that concern position of the eyeball sight in set interface is control (" showing " menu item) place in set interface
When region, it is automatically brought into operation the control, control electric terminal enters display set interface (second image in right side in such as Figure 19).When
Determine that concern position of the eyeball sight in display set interface is controlled to show in set interface according to eyeball eye position information
When part (i.e. " brightness " menu item) region, it is automatically brought into operation the control, control electric terminal enters brightness regulation interface (as schemed
The image of the rightmost side in 19).Concern position of the eyeball sight in brightness regulation interface is determined according to eyeball eye position information
For in brightness regulation interface for manually adjusting the control region of brightness size, since the brightness regulation interface is final boundary
Face can identify on the image at above-mentioned each interface at this time, and execute sequence according to operating procedure in the addition of control position
It is formed screenshot (as shown in figure 19).
In a preferred embodiment, can also the picture mosaic style to the screenshot of generation be adjusted.For example, can receive user
To the selection operation (for example, long press operation) of the screenshot of generation, at least one option, Mei Gexuan are shown in response to the selection operation
A kind of corresponding picture mosaic style of item, changes the picture mosaic style of the screenshot of generation according to user to the selection of at least one option, that is,
The picture mosaic style of the screenshot of generation is adjusted to picture mosaic style corresponding to selected option.
In addition, the selected picture mosaic style of user can be recorded also, when generating screenshot next time, the user institute of record is selected
The picture mosaic style of selection generates screenshot, to meet the individual demand of user.
Figure 20 shows the block diagram of electric terminal according to an exemplary embodiment of the present invention.
As shown in figure 20, electric terminal according to an exemplary embodiment of the present invention includes: input interface 10,20 and of processor
Eye tracking unit 30.
Particularly, input interface 10 receives the predetermined input for triggering screenshotss.
As an example, predetermined input may include but be not limited at least one in following item: at least one physical button
Input, gesture input, touch input, voice input.
Processor 20 controls electric terminal and enters screenshotss state, and in screenshotss state in response to the predetermined input received
The type of lower determining current display interface.
For example, processor 20 can determine the class of current display interface according to the type applied belonging to current display interface
Type.Alternatively, processor 20 can also determine the type of current display interface according to the contents and distribution of current display interface.It is preferred that
Ground, processor 20 can be by combining above two judgment method to more accurately determine the type of current display interface.
Eye tracking unit 30 tracks eyeball eye position, and provides eyeball eye position information.
Here, eye tracking unit 30 tracks eyeball eye position under screenshotss state.
In one example, eye tracking unit 30 can be the front camera of electric terminal.
For example, processor 20 can control the front camera of electric terminal to open, and control preposition take the photograph under screenshotss state
As the image of head capture eyes of user, processor 20 is identified to determine eyeball by the image to the eyes of user captured
Eye position information.
In another example, eye tracking unit 30 can be the eyeball inductor of electric terminal.
For example, processor 20 can control the eyeball inductor of electric terminal to open, the eyeball inductor under screenshotss state
Traceable eyeball eye position, and eyeball eye position information is provided.
In another example, eye tracking unit 30 may include infrared light supply and infrared camera.
In the case, infrared light supply can emit infrared light towards the eyes of user, and infrared camera is received by user's eye
The infrared light of ball pupillary reflex, and eyeball eye position information is provided.
The eyeball view that processor 20 is provided based on the type and eye tracking unit 30 of identified current display interface
Line position information generates screenshot.
As an example, in exemplary embodiments of the present invention, the type of current display interface may include video class, text class
And control class.The process for generating screenshot is introduced for different types of current display interface separately below.
In the first embodiment, the process for generating screenshot is introduced so that the type of current display interface is video class as an example.
It here, include video in video class display interface.
In the case, it is generated and is cut based on the type of identified current display interface and eyeball eye position information
The processing of figure can include: according to eyeball eye position information, the concern position of eyeball sight in video is determined, based in video
At least one key frame images including the corresponding content in concern position generate screenshot.
It should be understood that video is formed by multiple frame images, multiple frame images include multiple key frame images and multiple intermediate frames
Image, any key frame images be a complete picture, any intermediate frame image include be relative to correspondence key frame figure
As change information.
For example, processor 20 can pass through different modes next life according to the difference of the concern position of eyeball sight in video
At screenshot.
The first situation determines that the concern position of eyeball sight in video is in video according to eyeball eye position information
Subtitle region.
In the case, processor 20 can generate screenshot by following processing.
Obtain the key frame images at the triggering screenshotss moment；Determine that with the key frame images at the triggering screenshotss moment be identical
The corresponding time value of the last one key frame images of scene, and using identified time value as cut-off time；From video
It extracts since triggering the screenshotss moment in subtitle file to a plurality of subtitle of the cut-off time；Table is filtered out from a plurality of subtitle
Up to a complete semantic subtitle；By at least one key frame images corresponding with the subtitle filtered out, is generated and cut with connecting method
Figure.
Preferably, the processing of the complete semantic subtitle of expression one is filtered out from a plurality of subtitle can include: according to every word
The display time of curtain in a manner of inverted order, handles for the two subtitles execution for showing temporally adjacent is following: calculating display time phase
The time interval of two adjacent subtitles, if showing a time posterior subtitle in two subtitles and showing one of time formerly
Time interval between subtitle is more than or equal to preset time, then deletes a display time posterior subtitle and aobvious
Show other subtitles shown after a time posterior subtitle, if show in two subtitles a time posterior subtitle and
It shows that the time interval between the subtitle of time formerly is less than preset time, then retains two subtitles；By to retaining
All subtitles come carry out semantic analysis, to filter out the complete semantic subtitle of expression one.
A kind of situation, at least one key frame images include a key frame images.
In the case, it by least one key frame images corresponding with the subtitle filtered out, is generated and is cut with connecting method
The processing of figure a can include: on the basis of key frame images, generate screenshot in the form of subtitle superposition.
Another situation, at least one key frame images include multiple key frame images.
In the case, it by least one key frame images corresponding with the subtitle filtered out, is generated and is cut with connecting method
The processing of figure can include: splice multiple key frame images to generate screenshot in chronological order, alternatively, from multiple key frame images
Middle selection one represents key frame images, on the basis of representing key frame images, generates screenshot in the form of subtitle superposition.
For example, choosing a processing for representing key frame images from multiple key frame images can include: from multiple key frames
In image access time value with since triggering the screenshotss moment to intermediate time immediate one of the period of cut-off time
Key frame images are as key frame images are represented, alternatively, access time is worth a smallest key from multiple key frame images
Frame image as with represent key frame images.
Second situation determines that the concern position of eyeball sight in video is not video according to eyeball eye position information
Middle subtitle region.
In the case, processor 20 can generate screenshot by handling as follows.
Identify object corresponding to the concern position of eyeball sight；Obtain the predetermined amount of time since triggering the screenshotss moment
At least one interior key frame images；Search includes the candidate key of the object from least one key frame images of acquisition
Frame image；The high key frame images of similarity are removed from the candidate key-frames image searched；It will be in candidate key-frames image
The key frame images remained generate screenshot with connecting method.
In a second embodiment, the process for generating screenshot is introduced so that the type of current display interface is text class as an example.
It here, include text information in text class display interface.It is answered as an example, text class display interface may include but be not limited to communication
The browser interface of chat interface or picture and text.
Preferably, electric terminal according to an exemplary embodiment of the present invention can further include display screen, on the display screen
It can show current display interface.
In the case, processor 20 can generate screenshot by handling as follows: according to eyeball eye position information, determine
The moving direction of eyeball sight；According to the moving direction control display screens switch display interface of eyeball sight, and obtain each
The image of corresponding display interface when switching；When determining display interface finishing switching, by the image of each display interface according to aobvious
Show and sequentially forms screenshot.
In the third embodiment, the process for generating screenshot is introduced so that the type of current display interface is control class as an example.
It here, include control in control class display interface.As an example, control may include icon or menu item on display interface.
In the case, processor 20 can generate screenshot by handling as follows: according to eyeball eye position information, control
Electric terminal executes at least one operating procedure, on the corresponding display interface of any operation step at least one operating procedure
Including control, wherein execute any operation step by operating the control according to eyeball eye position information；Obtain electronics
Terminal executes the image of corresponding display interface when each operating procedure；By the corresponding display interface of at least one operating procedure
Image sequentially forms screenshot according to the execution of operating procedure.
In a preferred embodiment, it obtains electric terminal and executes the image of corresponding display interface when any operation step
Processing can include: the control position addition mark on the image of the corresponding display interface of any operation step of acquisition；
Using the image after addition mark as the image of the corresponding display interface of any operation step.
An exemplary embodiment of the present invention also provides a kind of computer-readable storage medium for being stored with computer program
Matter.The computer-readable recording medium storage has makes processor execute cutting in above-mentioned electric terminal when being executed by a processor
The computer program of drawing method.The computer readable recording medium is the arbitrary number that can store the data read by computer system
According to storage device.The example of computer readable recording medium includes: read-only memory, random access memory, CD-ROM, magnetic
Band, floppy disk, optical data storage devices and carrier wave (such as being transmitted through wired or wireless transmission path by the data of internet).
Using the screenshot method and electric terminal in the electric terminal of exemplary embodiment of the present, pass through semantic analysis
Or image recognition, to obtain accurately long screenshot, to realize intelligent screenshot.
The foregoing describe each exemplary embodiments of the invention, it should be appreciated that foregoing description is merely exemplary, and exhaustive
Property, the present invention is not limited to disclosed each exemplary embodiments.Without departing from the scope and spirit of the invention, right
Many modifications and changes are obvious for those skilled in the art.Therefore, protection of the invention
Range should be subject to the scope of the claims.
Claims (16)
1. the screenshot method in a kind of electric terminal characterized by comprising
The predetermined input for triggering screenshotss is received, into screenshotss state；
Under screenshotss state, the type of current display interface is determined；
Eyeball eye position is tracked, and determines eyeball eye position information；
Screenshot is generated based on the type of identified current display interface and eyeball eye position information.
2. screenshot method as described in claim 1, which is characterized in that the predetermined input includes at least one in following item
: input, gesture input, touch input, the voice input of at least one physical button.
3. screenshot method as described in claim 1, which is characterized in that the step of determining the type of current display interface include:
The type applied according to belonging to current display interface, and/or, the contents and distribution of current display interface determines current display
The type at interface.
4. screenshot method as described in claim 1, which is characterized in that the type of current display interface includes video class, video
It include video in class display interface,
Wherein, based on the type of identified current display interface and eyeball eye position information come packet the step of generating screenshot
It includes:
According to eyeball eye position information, the concern position of eyeball sight in video is determined；
Screenshot is generated based at least one key frame images in video including the corresponding content in the concern position.
5. screenshot method as claimed in claim 4, which is characterized in that the video is formed by multiple frame images, the multiple
Frame image includes multiple key frame images and multiple intermediate frame images, and any key frame images are a complete picture, Ren Yizhong
Between frame image include be image change information relative to corresponding key frame.
6. screenshot method as claimed in claim 5, which is characterized in that based on corresponding interior including the concern position in video
Hold at least one key frame images generate screenshot the step of include:
If determining that the concern position of eyeball sight in video is subtitle location in video according to eyeball eye position information
Domain then obtains the key frame images at the triggering screenshotss moment；
Determine that with the key frame images at the triggering screenshotss moment be the last one key frame images of same scene corresponding time
Value, and using identified time value as cut-off time；
It extracts since triggering the screenshotss moment from the subtitle file of the video to a plurality of subtitle of the cut-off time；
The complete semantic subtitle of expression one is filtered out from a plurality of subtitle；
By at least one key frame images corresponding with the subtitle filtered out, screenshot is generated with connecting method.
7. screenshot method as claimed in claim 6, which is characterized in that filter out one complete language of expression from a plurality of subtitle
Justice subtitle the step of include:
According to the display time of every subtitle in a manner of inverted order, following steps are executed for two temporally adjacent subtitles of display:
The time interval for showing two temporally adjacent subtitles is calculated, if showing a time posterior subtitle in two subtitles and showing
Show that the time interval between the subtitle of time formerly is more than or equal to preset time, then deletes the display time posterior one
Subtitle and other subtitles shown after showing a time posterior subtitle, if showing that the time exists in two subtitles
Time interval between a first subtitle of a subtitle and display time afterwards is less than preset time, then retains two words
Curtain；
By carrying out semantic analysis to all subtitles remained, to filter out the complete semantic subtitle of expression one.
8. screenshot method as claimed in claim 6, which is characterized in that at least one described key frame images include a key
Frame image,
Wherein, it by least one key frame images corresponding with the subtitle filtered out, is wrapped with the step of connecting method generation screenshot
It includes: on the basis of one key frame images, generating screenshot in the form of subtitle superposition.
9. screenshot method as claimed in claim 6, which is characterized in that at least one described key frame images include multiple keys
Frame image,
Wherein, it by least one key frame images corresponding with the subtitle filtered out, is wrapped with the step of connecting method generation screenshot
It includes: the multiple key frame images being spliced in chronological order to generate screenshot,
Alternatively, choosing one from the multiple key frame images represents key frame images, in the base for representing key frame images
On plinth, screenshot is generated in the form of subtitle superposition.
10. screenshot method as claimed in claim 9, which is characterized in that choose one from the multiple key frame images and represent
The step of key frame images includes:
From the multiple key frame images access time value with since triggering the screenshotss moment to the period of cut-off time
The immediate key frame images of intermediate time, which are used as, represents key frame images,
Alternatively, from the multiple key frame images the smallest key frame images of access time value as with represent key frame
Image.
11. screenshot method as claimed in claim 6, which is characterized in that the screenshot method further include:
If determining that the concern position of eyeball sight in video is not subtitle place in video according to eyeball eye position information
Region then identifies object corresponding to the concern position of eyeball sight；
Obtain at least one key frame images in the predetermined amount of time since triggering the screenshotss moment；
Search includes the candidate key-frames image of the object from least one key frame images described in acquisition；
The high key frame images of similarity are removed from the candidate key-frames image searched；
The key frame images that will be remained in candidate key-frames image generate screenshot with connecting method.
12. screenshot method as described in claim 1, which is characterized in that the type of current display interface includes text class, text
It include text information in class display interface,
Wherein, based on the type of identified current display interface and eyeball eye position information come packet the step of generating screenshot
It includes:
According to eyeball eye position information, the moving direction of eyeball sight is determined；
Switch display interface according to the moving direction of eyeball sight, and obtains the figure of corresponding display interface when switching every time
Picture；
When determining display interface finishing switching, the image of each display interface is formed into screenshot according to display order.
13. screenshot method as described in claim 1, which is characterized in that the type of current display interface includes control class, control
It include control in class display interface,
Wherein, based on the type of identified current display interface and eyeball eye position information come packet the step of generating screenshot
It includes:
According to eyeball eye position information, controls electric terminal and execute at least one operating procedure, at least one described operation step
It include control on the corresponding display interface of any operation step in rapid, wherein operation is passed through according to eyeball eye position information
The control executes any operation step；
Obtain the image that electric terminal executes corresponding display interface when each operating procedure；
The image of the corresponding display interface of at least one described operating procedure is sequentially formed into screenshot according to the execution of operating procedure.
14. screenshot method as claimed in claim 13, which is characterized in that obtain electric terminal execute any operation step when pair
The step of image for the display interface answered includes:
The control position addition mark on the image of the corresponding display interface of any operation step of acquisition；
The image after the mark will be added as the image of the corresponding display interface of any operation step.
15. a kind of electric terminal, which is characterized in that including input interface, eye tracking unit and processor,
Wherein, input interface receives the predetermined input for triggering screenshotss,
Processor is configured as: in response to the predetermined input received, being controlled electric terminal and is entered screenshotss state, and in screenshotss shape
The type of current display interface is determined under state,
Eye tracking element keeps track eyeball eye position, and eyeball eye position information is provided,
Wherein, processor is also configured to type and eyeball eye position information based on identified current display interface
To generate screenshot.
16. a kind of computer readable storage medium for being stored with computer program, which is characterized in that when the computer program exists
The screenshot method in the electric terminal as described in any one of claim 1 to 14 is realized when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910789435.1A CN110502117B (en) | 2019-08-26 | 2019-08-26 | Screenshot method in electronic terminal and electronic terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910789435.1A CN110502117B (en) | 2019-08-26 | 2019-08-26 | Screenshot method in electronic terminal and electronic terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110502117A true CN110502117A (en) | 2019-11-26 |
| CN110502117B CN110502117B (en) | 2023-04-25 |
Family
ID=68589432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910789435.1A Active CN110502117B (en) | 2019-08-26 | 2019-08-26 | Screenshot method in electronic terminal and electronic terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110502117B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111010610A (en) * | 2019-12-18 | 2020-04-14 | 维沃移动通信有限公司 | Video screenshot method and electronic equipment |
| CN112261453A (en) * | 2020-10-22 | 2021-01-22 | 北京小米移动软件有限公司 | Method, device and storage medium for transmitting subtitle splicing map |
| CN113723397A (en) * | 2020-05-26 | 2021-11-30 | 华为技术有限公司 | Screen capturing method and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105446673A (en) * | 2014-07-28 | 2016-03-30 | 华为技术有限公司 | Screen display method and terminal device |
| CN107229402A (en) * | 2017-05-22 | 2017-10-03 | 努比亚技术有限公司 | Dynamic screenshotss method, device and the readable storage medium storing program for executing of terminal |
| CN108664288A (en) * | 2018-05-14 | 2018-10-16 | 维沃移动通信有限公司 | A kind of image interception method and mobile terminal |
-
2019
- 2019-08-26 CN CN201910789435.1A patent/CN110502117B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105446673A (en) * | 2014-07-28 | 2016-03-30 | 华为技术有限公司 | Screen display method and terminal device |
| CN107229402A (en) * | 2017-05-22 | 2017-10-03 | 努比亚技术有限公司 | Dynamic screenshotss method, device and the readable storage medium storing program for executing of terminal |
| CN108664288A (en) * | 2018-05-14 | 2018-10-16 | 维沃移动通信有限公司 | A kind of image interception method and mobile terminal |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111010610A (en) * | 2019-12-18 | 2020-04-14 | 维沃移动通信有限公司 | Video screenshot method and electronic equipment |
| CN111010610B (en) * | 2019-12-18 | 2022-01-28 | 维沃移动通信有限公司 | Video screenshot method and electronic equipment |
| CN113723397A (en) * | 2020-05-26 | 2021-11-30 | 华为技术有限公司 | Screen capturing method and electronic equipment |
| WO2021238740A1 (en) * | 2020-05-26 | 2021-12-02 | 华为技术有限公司 | Screen capture method and electronic device |
| CN113723397B (en) * | 2020-05-26 | 2023-07-25 | 华为技术有限公司 | Screen capturing method and electronic equipment |
| CN112261453A (en) * | 2020-10-22 | 2021-01-22 | 北京小米移动软件有限公司 | Method, device and storage medium for transmitting subtitle splicing map |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110502117B (en) | 2023-04-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10979761B2 (en) | Intelligent video interaction method | |
| Del Molino et al. | Summarization of egocentric videos: A comprehensive survey | |
| Tonomura et al. | Structured video computing | |
| KR102290419B1 (en) | Method and Appratus For Creating Photo Story based on Visual Context Analysis of Digital Contents | |
| CN110502117A (en) | Screenshot method and electric terminal in electric terminal | |
| WO2001054399A2 (en) | Multi-modal video target acquisition and re-direction system and method | |
| US20200374491A1 (en) | Forensic video exploitation and analysis tools | |
| JP5506324B2 (en) | Similar image search system and similar image search method | |
| CN108337471B (en) | Video picture processing method and device | |
| CN111970456B (en) | Shooting control method, device, equipment and storage medium | |
| KR20090093904A (en) | Apparatus and method for scene variation robust multimedia image analysis, and system for multimedia editing based on objects | |
| EP3905188A1 (en) | Information processing device, information processing method, and program | |
| US20050251741A1 (en) | Methods and apparatus for capturing images | |
| JP3372096B2 (en) | Image information access device | |
| JP5826513B2 (en) | Similar image search system | |
| KR20180075506A (en) | Information processing apparatus, information processing method, and program | |
| CN111062284A (en) | Visual understanding and diagnosing method of interactive video abstract model | |
| JP2015099603A (en) | Image display system | |
| JP2023538981A (en) | emoticon generator | |
| Liu et al. | Video key-frame extraction for smart phones | |
| KR102507873B1 (en) | Method for generating customized video based on objects and service server using the same | |
| JP2004185424A (en) | Presentation recording device | |
| WO2024062971A1 (en) | Information processing device, information processing method, and information processing program | |
| Saini et al. | Automated Video Mashups: Research and Challenges | |
| CN117055732A (en) | Picture switching method, switching device and intelligent conference equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |