CN102572217A - Visual-attention-based multimedia processing method and device - Google Patents
Visual-attention-based multimedia processing method and device Download PDFInfo
- Publication number
- CN102572217A CN102572217A CN2011104538310A CN201110453831A CN102572217A CN 102572217 A CN102572217 A CN 102572217A CN 2011104538310 A CN2011104538310 A CN 2011104538310A CN 201110453831 A CN201110453831 A CN 201110453831A CN 102572217 A CN102572217 A CN 102572217A
- Authority
- CN
- China
- Prior art keywords
- sight line
- focal position
- associated region
- line associated
- sight
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 50
- 230000000007 visual effect Effects 0.000 claims description 37
- 239000000463 material Substances 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 239000000203 mixture Substances 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 210000005252 bulbus oculi Anatomy 0.000 description 3
- 210000001508 eye Anatomy 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000012015 optical character recognition Methods 0.000 description 2
- 210000001747 pupil Anatomy 0.000 description 2
- 230000002146 bilateral effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a visual-attention-based multimedia processing method and a visual-attention-based multimedia processing device, and relates to the technical field of multimedia processing. Control over multimedia display is finished by confirming an eye catcher of a user under the condition of no influence on the use feeling of the user. The method comprises the following steps of: detecting an eye catcher position corresponding to a watcher in a display screen; acquiring a sight correlation area corresponding to the eye catcher position; and performing video enhancement processing on a video image corresponding to the sight correlation area. The embodiment of the invention is mainly applied to the multimedia processing.
Description
Technical field
The present invention relates to the multimedia processing technology field, relate in particular to a kind of multi-media processing method and device based on visual attention location.
Background technology
Along with the user is increasingly high to the requirement of audio frequency and video experience sense, the mode that audio frequency and video are handled more and more relies on user's intention.At present, the processing mode of audio frequency and video is specially the artificial processing scheme of setting, and through background program with audio-video document according to handled scheme handled, the audio-video document after will handling then shows.Audio-video document is handled the intention that needs the perfect processing scheme of setting just can meet the user through this processing mode.
Summary of the invention
Embodiments of the invention provide a kind of multi-media processing method and device based on visual attention location, have realized under the situation that does not influence user's use experience, accomplish the control to multimedia display through the sight line focus of confirming the user.
For achieving the above object, embodiments of the invention adopt following technical scheme:
A kind of multi-media processing method based on visual attention location comprises:
Detect the corresponding sight line focal position of beholder in the display screen;
According to said sight line focal position, obtain the sight line associated region corresponding with said sight line focal position;
The video image corresponding to said sight line associated region carries out the video enhancement process.
A kind of multimedia processing apparatus based on visual attention location comprises:
Detecting unit is used to detect the corresponding sight line focal position of beholder in the display screen;
Acquiring unit is used for according to said sight line focal position, obtains the sight line associated region corresponding with said sight line focal position;
Adjustment unit is used for the corresponding video image of said sight line associated region is carried out the video enhancement process.Multi-media processing method and device that the embodiment of the invention provides based on visual attention location; Through obtaining beholder's visual focus position; And confirm the zone that the beholder is watching according to the sight line associated region that the visual focus position obtains the beholder; Directly said sight line associated region is adjusted to satisfy sense of experience of users then, realized under the situation that does not influence user's use experience, accomplish control multimedia display through the sight line focus of confirming the user.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art; To do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below; Obviously, the accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills; Under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the flow chart of a kind of multi-media processing method based on visual attention location in the embodiment of the invention 1;
Fig. 2 is the flow chart of a kind of multi-media processing method based on visual attention location in the embodiment of the invention 2;
Fig. 3 is the composition frame chart of a kind of multimedia processing apparatus based on visual attention location in the embodiment of the invention 3;
Fig. 4 is that another kind in the embodiment of the invention 3 is based on the composition frame chart of the multimedia processing apparatus of visual attention location;
Fig. 5 is that another kind in the embodiment of the invention 3 is based on the composition frame chart of the multimedia processing apparatus of visual attention location;
Fig. 6 is that another kind in the embodiment of the invention 3 is based on the composition frame chart of the multimedia processing apparatus of visual attention location;
Be that another kind in the embodiment of the invention 3 is based on the composition frame chart of the multimedia processing apparatus of visual attention location during Fig. 7.
Embodiment
To combine the accompanying drawing in the embodiment of the invention below, the technical scheme in the embodiment of the invention is carried out clear, intactly description, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills are not making the every other embodiment that is obtained under the creative work prerequisite, all belong to the scope of the present invention's protection.
Embodiment 1
The embodiment of the invention provides a kind of multi-media processing method based on visual attention location, and is as shown in Figure 1, and this method comprises:
101, detect the corresponding sight line focal position of beholder in the display screen.
Wherein, detect the corresponding sight line focal position of beholder and can pass through pupil-corneal reflection vector method, implementation is following:
Shine people face with an infrared secondary light source, form reflection image at the eyes anterior corneal surface, this reflection image is called as pul (Purkinje) spot by the emperor himself.Human eye is looked the screen diverse location staring at, and corresponding rotation can take place eyeball, supposes under beholder's motionless situation; Since the fixed-site of infrared light emitting diode, and eyeball is an approximate spheroid, so when eyeball moves; Can think that the admire absolute position of spot of pul is constant; And corresponding variation will take place in the position of iris and pupil, and the admire relative position relation of spot and pupil and iris of pul also changes like this, the confirming and can realize through image processing of this relative position relation; Relative position relation by them can draw the direction of sight line then, and obtains the sight line focal position thus.
Implementation method based on the corresponding sight line focal position of above-mentioned detection beholder; The implementation method of the corresponding sight line focal position of beholder specifically comprises and uses above-mentioned pupil-corneal reflection vector method that the corresponding sight line focal position of said a plurality of beholders is detected in the said detection display screen, and obtains all and be in the sight line focal position in the display screen.
The beholder also can realize through alternate manner corresponding sight line focal position in the said detection display screen; The embodiment of the invention does not limit this; The concrete implementation method of said alternate manner is for well known to a person skilled in the art technology, and the present invention repeats no more to this.
102,, obtain the sight line associated region corresponding with said sight line focal position according to said sight line focal position.
Wherein, said according to said sight line focal position, obtain the sight line associated region corresponding with said sight line focal position can but be not limited in the following manner and realize.Be specially:
Obtain the sight line central area according to said sight line focal position, said sight line central area is for being the zone at center with the sight line focal position; Zone association relation according to said sight line central area and setting in advance generates the sight line associated region.
Wherein, the size of said sight line central area is to be provided with in advance, and specifically can be set to the sight line focal position is the center; 1/9 of whole screen width; 1/5 of height, the user also can be provided with according to actual needs voluntarily, and the embodiment of the invention does not limit this.
Wherein, The said zone association relation that is provided with in advance is any relation in the following relation, and this relation belongs to same paragraph for the interior literal of said sight line associated region and identical or close, the said sight line associated region of said sight line central area image pixel and close, the said sight line associated region of said sight line central area picture material and identical or close, the said sight line associated region of said sight line central area picture shape and the literal in the said sight line central area.The user can choose one or more zone association relations according to actual needs, and the embodiment of the invention does not limit this.
103, the corresponding video image of sight line associated region is carried out the video enhancement process.
As for example, the corresponding video image of sight line associated region is carried out the video enhancement process can realize through following two kinds of methods, specifically comprise:
First method: the image information in the said sight line associated region is carried out image enhancement processing.
Wherein, Saidly the corresponding video image of said sight line associated region is carried out image enhancement processing specifically comprise needs are presented at the processing such as sharpening that video content on the display screen is directed to the image in the said sight line associated region, make that this video content can be more clear after showing through display screen.
Second method: the video information in the said sight line associated region is carried out the coding and decoding video enhancement process.
Wherein, saidly video information in the said sight line associated region carried out the coding and decoding video enhancement process specifically comprise:
In the video coding end, when video file is encoded, when being encoded, distributes the image in the sight line associated region more yardage and computational resource, when being encoded, distributes the image in the non-sight line associated region less yardage and computational resource.
In the video decode end, when video file is decoded, the video file after encoding is decoded in conjunction with the bilateral filtering technology.
Be appreciated that; The corresponding video image of sight line associated region is carried out the video enhancement process also can have condition of different to different application scenes; For example: in realizing associated region, not only exist video image also to have the alphabetic character zone, can pass through OCR (Optical Character Recognition, optical character identification) technology; Literal is extracted; And the image that extracts behind the literal carried out the video enhancement process, and then that the image after the enhancement process is superimposed with the literal that identifies, the corresponding video image of this sight line associated region of reconstruct.In addition, also have other implementation, those of ordinary skills other the implementation that can expect also within the protection range of the embodiment of the invention.
In addition; Need to prove, after the corresponding video image of sight line associated region is carried out the video enhancement process, in order to improve user's use experience; Can repeated execution of steps 101 to step 103, so that the audio frequency and video that the user is paid close attention to show that adjustment reaches optimum.
The multi-media processing method that the embodiment of the invention provides based on visual attention location; Through obtaining beholder's visual focus position; And confirm the zone that the beholder is watching according to the sight line associated region that the visual focus position obtains the beholder; Directly said sight line associated region is adjusted to satisfy sense of experience of users then, realized under the situation that does not influence user's use experience, accomplish control multimedia display through the sight line focus of confirming the user.
Embodiment 2
A kind of multi-media processing method based on visual attention location is provided in the embodiment of the invention, and as shown in Figure 2, this method comprises:
201, detect the corresponding sight line focal position of beholder in the display screen.When said sight line focal position was a beholder's sight line focal position, then execution in step 202; When said sight line focal position is the corresponding a plurality of sight lines focal position of a plurality of beholders, then execution in step 203 or execution in step 204.
Wherein, the implementation of the beholder's that said real-time reception picture pick-up device is caught sight line focal position is identical with the associated description of said step 101, and the embodiment of the invention repeats no more to this.
202, according to a said beholder's sight line focal position, obtain the sight line associated region corresponding with a said beholder's sight line focal position, and execution in step 207.
Wherein, said sight line focal position according to a said beholder obtains identically with the associated description of the implementation method of the corresponding sight line associated region in a said beholder's sight line focal position and said step 102, and the embodiment of the invention repeats no more to this.
203, obtain a plurality of sight line associated regions corresponding respectively according to said a plurality of sight lines focal position, and said a plurality of sight line associated regions are merged the sight line associated region after obtaining merging, and execution in step 207 with said a plurality of sight lines focal position.
Wherein, Saidly obtain in a plurality of sight line associated regions corresponding sight line focal position respectively according to a said beholder with said a plurality of sight lines focal position according to said a plurality of sight lines focal position; Obtain identically with the associated description of the implementation method of the corresponding sight line associated region in a said beholder's sight line focal position and said step 102, the embodiment of the invention repeats no more to this.
What be worth explanation is, said said a plurality of sight line associated regions is merged, and the sight line associated region after obtaining merging can be realized in the following manner, specifically comprises:
With said a plurality of sight line associated regions together according to separately sight line associated region position grouping; Generate a new sight line associated region as the sight line associated region after merging, the sight line associated region after the said merging has covered said a plurality of sight line associated region.
204, obtain said a plurality of beholders' rights of using through recognition of face, and confirm whether said a plurality of beholders' rights of using are identical.If said a plurality of beholders' rights of using are different, then execution in step 205; If said a plurality of beholders' rights of using are identical, then execution in step 206.
Wherein, Obtaining of said a plurality of beholders' rights of using can combine the authority in the database of said multimedia processing system that realization is set through face identification method; Can also adopt the mode of human eye iris recognition; Concrete implementation is for well known to a person skilled in the art technology, and the embodiment of the invention no longer is described in detail at this.
205, the beholder's of high rights of using sight line focal position obtains corresponding sight line associated region according to having, and with the sight line associated region that obtains as the corresponding sight line associated region in said a plurality of beholders' sight line focal position, and execution in step 207.
206, obtain a plurality of sight line associated regions corresponding respectively according to said a plurality of sight lines focal position with said a plurality of sight lines focal position; Overlap the zone if said a plurality of sight line associated region exists, confirm that then said coincidence zone is the corresponding sight line associated region in said a plurality of sight lines focal position; Do not overlap the zone if said a plurality of sight line associated regions do not exist, then definite sight line focal position corresponding sight line associated region nearest from picture pick-up device picture center is the corresponding sight line associated region in said a plurality of sight lines focal position, and execution in step 208.
207, the corresponding video image of said sight line associated region is carried out the video enhancement process.
Wherein, said that the corresponding video image of said sight line associated region is carried out in video enhancement process and the step 103 relevant description is identical, can specifically be applied in the following scene, comprising:
Optional, said video image to said sight line associated region correspondence carries out the video enhancement process and can be the switching of main screen and auxiliary screen.For example, in video conference, MCU (Multipoint Control Unit, multipoint control unit) is transferred to the terminal to many pictures, shows in the local terminal.The camera of end detects beholder's sight line, gets access to beholder's sight line focal position, is that the center obtains the sight line associated region according to said sight line focal position.If in the time decision that is provided with in advance, beholder's sight line focal position does not move to outside the said sight line associated region, then is sent to MCU to said sight line associated region positional information; MCU is through this positional information of contrast, if this position is not at key frame, and at auxiliary image; Then amplify auxiliary image, become key frame and show, and its sound is amplified; Key frame is reduced into auxiliary image and shows, and its sound is reduced.
Optional, it can be the enhancing to the image frame per second that said video image to said sight line associated region correspondence carries out the video enhancement process.For example, detect beholder's sight line focal position at video camera, be reported to MCU, MCU calculates the sight line focal position that reports for 2 times, subtracts each other according to its corresponding horizontal ordinate, draws the situation of movement of sight line focal position on abscissa and ordinate.Carry out 3 such operations, if the situation of movement that calculates for 3 times is identical, the direction that moves of judgement place beholder sight line then, otherwise proceed to detect.According to the direction that beholder's sight line moves, carry out the adjustment of captions broadcasting speed: identical with the captions moving direction, represent that then the captions broadcasting speed is too fast, need to reduce the captions translational speed, otherwise need to accelerate the captions translational speed.After the adjustment of captions translational speed, detect again, carry out the adjustment of captions translational speed according to testing result again.Through detection, the adjustment that does not stop, adjust to and beholder's sight line position is in the middle of the screen and no longer mobile, then the adjustment of captions translational speed finishes.
Optional, it can be the enhancing to the audio/video encoding/decoding resource that said video image to said sight line associated region correspondence carries out the video enhancement process.For example, detect the current sight line focal position of beholder at video camera, and obtain the sight line associated region: said sight line associated region is marked; Be sent to MCU to said sight line associated region information coordinate; MCU strengthens the image coding and decoding in the said sight line associated region according to said sight line associated region information, strengthens the encoding and decoding effect with higher pixel, wideer colour gamut, higher transmission bandwidth; Reach better real effect, promote user's visual experience.User's sight line moves, and then move with user's sight line and mobile in this zone, and the image effect in user's sight line is for more excellent.
Need to prove; Except that above-mentioned video image to said sight line associated region correspondence is carried out the video enhancement process; Said video image to said sight line associated region correspondence carries out the video enhancement process and also can carry out according to other method, and the embodiment of the invention does not limit this.
In addition; Need to prove; Before said video image to said sight line associated region correspondence carries out the video enhancement process; Can the relevant information of said sight line associated region be sent to remote server, so that said remote server carries out the adjustment that audio frequency and video show according to the relevant information of said sight line associated region to said sight line associated region.
Wherein, The relevant information of said sight line associated region can be the relevant information of the said sight line focus area corresponding with a beholder's sight line focal position; Can be the relevant information of the sight line focus area after the said merging; Can also can be the said relevant information that overlaps the zone for having the beholder's of high rights of using the corresponding sight line associated region in sight line focal position.Specifically can comprise the information such as centre coordinate, boundary sizes of said sight line associated region, the user can be provided with and add according to actual needs voluntarily, and the embodiment of the invention is not enumerated at this one by one.
Wherein, said remote server can be MCU, and said relevant information with said sight line associated region sends to remote server can realize that the embodiment of the invention does not limit this through communication channels such as IP networks.
The multi-media processing method that the embodiment of the invention provides based on visual attention location; Through obtaining beholder's visual focus position; And confirm the zone that the beholder is watching according to the sight line associated region that the visual focus position obtains the beholder; Directly said sight line associated region is adjusted to satisfy sense of experience of users then, realized under the situation that does not influence user's use experience, accomplish control multimedia display through the sight line focus of confirming the user.
And said multi-media processing method based on visual attention location can also carry out different processing according to beholder's quantity, has improved the service efficiency of equipment, has promoted user's use experience.
And; Relevant information through with said beholder's sight line associated region sends to remote server; So that remote server can be handled the encoding and decoding of the source end of multimedia file,, make the user can obtain better use experience for the user provides better audio frequency and video display effect.
Embodiment 3
The embodiment of the invention provides a kind of multimedia processing apparatus based on visual attention location, and is of Fig. 3, and this device comprises: receiving element 31, acquiring unit 32, adjustment unit 33.
Receiving element 31 is used to detect the corresponding sight line focal position of beholder in the display screen.
Acquiring unit 32 is used for according to said sight line focal position, obtains the sight line associated region corresponding with said sight line focal position.
Further, as shown in Figure 4, said acquiring unit 32 comprises: first acquisition module 321, second acquisition module 322.
Further, as shown in Figure 5, said second acquisition module comprises: authority is confirmed submodule 3221, the definite submodule 3222 in zone.
Authority is confirmed submodule 3221, is used for confirming whether said a plurality of beholders' rights of using are identical.
Further, as shown in Figure 6, said adjustment unit 33 also comprises: first enforcement module 331, second enforcement module 332.
Further, as shown in Figure 7, this device also comprises: transmitting element 34.
Transmitting element 34 is used for the relevant information of said sight line associated region is sent to remote server, so that said remote server carries out the adjustment that audio frequency and video show according to the relevant information of said sight line associated region to said sight line associated region.
Further, said acquiring unit 32 also is used for obtaining the sight line central area according to said sight line focal position, and said sight line central area is for being the zone at center with the sight line focal position; Zone association relation according to said sight line central area and setting in advance generates the sight line associated region.
The multimedia processing apparatus that the embodiment of the invention provides based on visual attention location; Through obtaining beholder's visual focus position; And confirm the zone that the beholder is watching according to the sight line associated region that the visual focus position obtains the beholder; Directly said sight line associated region is adjusted to satisfy sense of experience of users then, realized under the situation that does not influence user's use experience, accomplish the control that audio frequency and video are shown through the sight line focus of confirming the user.
And said multi-media processing method based on visual attention location can also carry out different processing according to beholder's quantity, has improved the service efficiency of equipment, has promoted user's use experience.
And; Relevant information through with said beholder's sight line associated region sends to remote server; So that remote server can be handled the encoding and decoding of the source end of audio frequency and video,, make the user can obtain better use experience for the user provides better audio frequency and video display effect.
Through the description of above execution mode, the those skilled in the art can be well understood to the present invention and can realize by the mode that software adds essential common hardware, can certainly pass through hardware, but the former is better execution mode under a lot of situation.Based on such understanding; The part that technical scheme of the present invention contributes to prior art in essence in other words can be come out with the embodied of software product, and this computer software product is stored in the storage medium that can read, like the floppy disk of computer; Hard disk or CD etc.; Comprise some instructions with so that computer equipment (can be personal computer, server, the perhaps network equipment etc.) carry out the described method of each embodiment of the present invention.
The above; Be merely embodiment of the present invention, but protection scope of the present invention is not limited thereto, any technical staff who is familiar with the present technique field is in the technical scope that the present invention discloses; Can expect easily changing or replacement, all should be encompassed within protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection range of said claim.
Claims (13)
1. the multi-media processing method based on visual attention location is characterized in that, comprising:
Detect the corresponding sight line focal position of beholder in the display screen;
According to said sight line focal position, obtain the sight line associated region corresponding with said sight line focal position;
The video image corresponding to said sight line associated region carries out the video enhancement process.
2. method according to claim 1 is characterized in that, and is said according to said sight line focal position when collecting a plurality of sight lines focal position of a plurality of beholders' correspondences, obtains the sight line associated region corresponding with said sight line focal position and comprises:
Obtain a plurality of sight line associated regions corresponding respectively according to said a plurality of sight lines focal position, and said a plurality of sight line associated regions are merged the sight line associated region after obtaining merging with said a plurality of sight lines focal position; Perhaps
Obtain said a plurality of beholders' rights of using through recognition of face, and, obtain and the corresponding sight line associated region in said a plurality of sight lines focal position according to said a plurality of beholders' rights of using and said a plurality of sight lines focal position.
3. method according to claim 2 is characterized in that, said rights of using and said a plurality of sight lines focal position according to said a plurality of beholders obtain and the corresponding sight line associated region in said a plurality of sight lines focal position, comprising:
Whether the rights of using of confirming said a plurality of beholders are identical;
If said a plurality of beholders' rights of using are different; Then the beholder's of high rights of using sight line focal position obtains corresponding sight line associated region according to having, and with the sight line associated region that obtains as the corresponding sight line associated region in said a plurality of beholders' sight line focal position;
If said a plurality of beholders' rights of using are identical, then obtain a plurality of sight line associated regions corresponding respectively with said a plurality of sight lines focal position according to said a plurality of sight lines focal position; Overlap the zone if said a plurality of sight line associated region exists, confirm that then said coincidence zone is the corresponding sight line associated region in said a plurality of sight lines focal position; Do not overlap the zone if said a plurality of sight line associated regions do not exist, then definite sight line focal position corresponding sight line associated region nearest from picture pick-up device picture center is the corresponding sight line associated region in said a plurality of sight lines focal position.
4. method according to claim 1 is characterized in that, said sight line associated region is carried out the video enhancement process comprise:
Image information in the said sight line associated region is carried out image enhancement processing; Perhaps
Video information in the said sight line associated region is carried out the coding and decoding video enhancement process.
5. method according to claim 1 is characterized in that, according to said sight line focal position, obtains after the sight line associated region corresponding with said sight line focal position, also comprises:
The relevant information of said sight line associated region is sent to remote server, so that said remote server carries out the video enhancement process according to the relevant information of said sight line associated region to said sight line associated region.
6. according to each described method of claim 1-5, it is characterized in that, said according to said sight line focal position, obtain the sight line associated region corresponding and also comprise with said sight line focal position:
Obtain the sight line central area according to said sight line focal position, said sight line central area is for being the zone at center with the sight line focal position;
Zone association relation according to said sight line central area and setting in advance generates the sight line associated region.
7. method according to claim 6 is characterized in that, the said zone association relation that is provided with in advance is any relation in the following relation, and this relation is:
Literal in said sight line associated region and identical or close, the said sight line associated region of said sight line central area image pixel and close, the said sight line associated region of said sight line central area picture material and identical or close, the said sight line associated region of said sight line central area picture shape and the literal in the said sight line central area belong to same paragraph.
8. the multimedia processing apparatus based on visual attention location is characterized in that, comprising:
Detecting unit is used to detect the corresponding sight line focal position of beholder in the display screen;
Acquiring unit is used for according to said sight line focal position, obtains the sight line associated region corresponding with said sight line focal position;
Adjustment unit is used for the corresponding video image of said sight line associated region is carried out the video enhancement process.
9. the multimedia processing apparatus based on visual attention location according to claim 8 is characterized in that, said acquiring unit comprises:
First acquisition module; Be used for when said sight line focal position is a plurality of sight lines focal position of a plurality of beholders' correspondences; Obtain a plurality of sight line associated regions corresponding respectively according to said a plurality of sight lines focal position with said a plurality of sight lines focal position; And said a plurality of sight line associated regions are merged the sight line associated region after obtaining merging;
Second acquisition module; Be used for when said sight line focal position is a plurality of sight lines focal position of a plurality of beholders' correspondences; Obtain said a plurality of beholders' rights of using through recognition of face; And, obtain the sight line associated region corresponding with said sight line focal position according to said a plurality of beholders' rights of using and said a plurality of sight lines focal position.
10. the multimedia processing apparatus based on visual attention location according to claim 9 is characterized in that, said second acquisition module comprises:
Authority is confirmed submodule, is used for confirming whether said a plurality of beholders' rights of using are identical;
Submodule is confirmed in the zone; Be used for not simultaneously in said a plurality of beholders' rights of using; The beholder's of high rights of using sight line focal position obtains corresponding sight line associated region according to having, and with the sight line associated region that obtains as the corresponding sight line associated region in said a plurality of beholders' sight line focal position;
Submodule is confirmed in said zone, also is used for rights of using said a plurality of beholders when identical, obtains a plurality of sight line associated regions corresponding with said a plurality of sight lines focal position respectively according to said a plurality of sight lines focal position; Overlap the zone if said a plurality of sight line associated region exists, confirm that then said coincidence zone is the corresponding sight line associated region in said a plurality of sight lines focal position; Do not overlap the zone if said a plurality of sight line associated regions do not exist, then definite sight line focal position corresponding sight line associated region nearest from picture pick-up device picture center is the corresponding sight line associated region in said a plurality of sight lines focal position.
11. the multimedia processing apparatus based on visual attention location according to claim 8 is characterized in that, said adjustment unit comprises:
First enforcement module is used for the image information in the said sight line associated region is carried out image enhancement processing;
Second enforcement module is used for the video information in the said sight line associated region is carried out the coding and decoding video enhancement process.
12. the multimedia processing apparatus based on visual attention location according to claim 8 is characterized in that, this device also comprises:
Transmitting element is used for the relevant information of said sight line associated region is sent to remote server, so that said remote server carries out the video enhancement process according to the relevant information of said sight line associated region to said sight line associated region.
13. each described multimedia processing apparatus according to Claim 8-12 based on visual attention location; It is characterized in that; Said acquiring unit also is used for obtaining the sight line central area according to said sight line focal position, and said sight line central area is for being the zone at center with the sight line focal position; Zone association relation according to said sight line central area and setting in advance generates the sight line associated region.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110453831.0A CN102572217B (en) | 2011-12-29 | 2011-12-29 | Visual-attention-based multimedia processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110453831.0A CN102572217B (en) | 2011-12-29 | 2011-12-29 | Visual-attention-based multimedia processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102572217A true CN102572217A (en) | 2012-07-11 |
| CN102572217B CN102572217B (en) | 2014-08-20 |
Family
ID=46416608
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110453831.0A Expired - Fee Related CN102572217B (en) | 2011-12-29 | 2011-12-29 | Visual-attention-based multimedia processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102572217B (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103685907A (en) * | 2012-09-26 | 2014-03-26 | 联想(北京)有限公司 | Image acquisition method and electronic equipment |
| WO2015035823A1 (en) * | 2013-09-16 | 2015-03-19 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Image collection with increased accuracy |
| CN105487787A (en) * | 2015-12-09 | 2016-04-13 | 东莞酷派软件技术有限公司 | Terminal operation method and device based on iris recognition and terminal |
| CN105828165A (en) * | 2016-04-29 | 2016-08-03 | 维沃移动通信有限公司 | Method and terminal for acquiring caption |
| CN106060658A (en) * | 2016-05-27 | 2016-10-26 | 青岛海信电器股份有限公司 | Image processing method and device |
| CN106165402A (en) * | 2014-04-22 | 2016-11-23 | 索尼公司 | Information reproduction apparatus, information regeneration method, information record carrier and information recording method |
| CN106485790A (en) * | 2016-09-30 | 2017-03-08 | 珠海市魅族科技有限公司 | Method and device that a kind of picture shows |
| CN108476305A (en) * | 2017-03-21 | 2018-08-31 | 深圳市大疆创新科技有限公司 | A kind of image transfer method, device and equipment |
| CN108650500A (en) * | 2018-04-02 | 2018-10-12 | 北京奇艺世纪科技有限公司 | A kind of panoramic video processing method and processing device |
| CN109218803A (en) * | 2018-09-28 | 2019-01-15 | Oppo广东移动通信有限公司 | Video source modeling control method, device and electronic equipment |
| CN109471579A (en) * | 2018-11-13 | 2019-03-15 | 努比亚技术有限公司 | Terminal screen arrangement information method of adjustment, device, mobile terminal and storage medium |
| CN109660863A (en) * | 2017-10-10 | 2019-04-19 | 中国移动通信集团湖北有限公司 | Visual attention location method for detecting area, device, equipment and computer storage medium |
| CN110135370A (en) * | 2019-05-20 | 2019-08-16 | 北京百度网讯科技有限公司 | The method and device of face In vivo detection, electronic equipment, computer-readable medium |
| CN110554816A (en) * | 2019-07-25 | 2019-12-10 | 华为技术有限公司 | Interface generation method and equipment |
| CN111193938A (en) * | 2020-01-14 | 2020-05-22 | 腾讯科技（深圳）有限公司 | Video data processing method, device and computer readable storage medium |
| CN111311713A (en) * | 2020-02-24 | 2020-06-19 | 咪咕视讯科技有限公司 | Cartoon processing method, cartoon display device, cartoon terminal and cartoon storage medium |
| CN115022616A (en) * | 2022-08-08 | 2022-09-06 | 太原理工大学 | Image focusing enhancement display device and display method based on human eye tracking |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101067716A (en) * | 2007-05-29 | 2007-11-07 | 南京航空航天大学 | Enhanced real natural interactive helmet with sight line follow-up function |
| CN101311882A (en) * | 2007-05-23 | 2008-11-26 | 华为技术有限公司 | Eye tracking human-machine interaction method and apparatus |
| CN101635861A (en) * | 2008-07-02 | 2010-01-27 | 索尼株式会社 | Display device and display method |
| US20100182340A1 (en) * | 2009-01-19 | 2010-07-22 | Bachelder Edward N | Systems and methods for combining virtual and real-time physical environments |
| US20100245387A1 (en) * | 2005-04-11 | 2010-09-30 | Systems Technology, Inc. | Systems and methods for combining virtual and real-time physical environments |
-
2011
- 2011-12-29 CN CN201110453831.0A patent/CN102572217B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100245387A1 (en) * | 2005-04-11 | 2010-09-30 | Systems Technology, Inc. | Systems and methods for combining virtual and real-time physical environments |
| CN101311882A (en) * | 2007-05-23 | 2008-11-26 | 华为技术有限公司 | Eye tracking human-machine interaction method and apparatus |
| CN101067716A (en) * | 2007-05-29 | 2007-11-07 | 南京航空航天大学 | Enhanced real natural interactive helmet with sight line follow-up function |
| CN101635861A (en) * | 2008-07-02 | 2010-01-27 | 索尼株式会社 | Display device and display method |
| US20100182340A1 (en) * | 2009-01-19 | 2010-07-22 | Bachelder Edward N | Systems and methods for combining virtual and real-time physical environments |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103685907A (en) * | 2012-09-26 | 2014-03-26 | 联想(北京)有限公司 | Image acquisition method and electronic equipment |
| WO2015035823A1 (en) * | 2013-09-16 | 2015-03-19 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Image collection with increased accuracy |
| US10002293B2 (en) | 2013-09-16 | 2018-06-19 | Beijing Zhigu Rui Tuo Tech Co., Ltd. | Image collection with increased accuracy |
| CN106165402A (en) * | 2014-04-22 | 2016-11-23 | 索尼公司 | Information reproduction apparatus, information regeneration method, information record carrier and information recording method |
| CN105487787A (en) * | 2015-12-09 | 2016-04-13 | 东莞酷派软件技术有限公司 | Terminal operation method and device based on iris recognition and terminal |
| CN105828165A (en) * | 2016-04-29 | 2016-08-03 | 维沃移动通信有限公司 | Method and terminal for acquiring caption |
| CN105828165B (en) * | 2016-04-29 | 2019-05-17 | 维沃移动通信有限公司 | A kind of method and terminal obtaining subtitle |
| CN106060658A (en) * | 2016-05-27 | 2016-10-26 | 青岛海信电器股份有限公司 | Image processing method and device |
| CN106060658B (en) * | 2016-05-27 | 2019-06-14 | 青岛海信电器股份有限公司 | A kind of image processing method and device |
| CN106485790A (en) * | 2016-09-30 | 2017-03-08 | 珠海市魅族科技有限公司 | Method and device that a kind of picture shows |
| CN108476305A (en) * | 2017-03-21 | 2018-08-31 | 深圳市大疆创新科技有限公司 | A kind of image transfer method, device and equipment |
| CN109660863A (en) * | 2017-10-10 | 2019-04-19 | 中国移动通信集团湖北有限公司 | Visual attention location method for detecting area, device, equipment and computer storage medium |
| CN109660863B (en) * | 2017-10-10 | 2021-07-20 | 中国移动通信集团湖北有限公司 | Visual attention area detection method, device, equipment and computer storage medium |
| CN108650500A (en) * | 2018-04-02 | 2018-10-12 | 北京奇艺世纪科技有限公司 | A kind of panoramic video processing method and processing device |
| CN108650500B (en) * | 2018-04-02 | 2019-11-22 | 北京奇艺世纪科技有限公司 | A kind of panoramic video processing method and processing device |
| CN109218803A (en) * | 2018-09-28 | 2019-01-15 | Oppo广东移动通信有限公司 | Video source modeling control method, device and electronic equipment |
| CN109471579A (en) * | 2018-11-13 | 2019-03-15 | 努比亚技术有限公司 | Terminal screen arrangement information method of adjustment, device, mobile terminal and storage medium |
| CN110135370A (en) * | 2019-05-20 | 2019-08-16 | 北京百度网讯科技有限公司 | The method and device of face In vivo detection, electronic equipment, computer-readable medium |
| US11188771B2 (en) | 2019-05-20 | 2021-11-30 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Living-body detection method and apparatus for face, and computer readable medium |
| CN110554816A (en) * | 2019-07-25 | 2019-12-10 | 华为技术有限公司 | Interface generation method and equipment |
| US11947781B2 (en) | 2019-07-25 | 2024-04-02 | Huawei Technologies Co., Ltd. | Automatically adjusting a layout of a visual element on a to-be-generated interface and quickly generating an interface |
| CN110554816B (en) * | 2019-07-25 | 2024-05-07 | 华为技术有限公司 | Interface generation method and device |
| CN111193938A (en) * | 2020-01-14 | 2020-05-22 | 腾讯科技（深圳）有限公司 | Video data processing method, device and computer readable storage medium |
| CN111193938B (en) * | 2020-01-14 | 2021-07-13 | 腾讯科技（深圳）有限公司 | Video data processing method, device and computer readable storage medium |
| CN111311713A (en) * | 2020-02-24 | 2020-06-19 | 咪咕视讯科技有限公司 | Cartoon processing method, cartoon display device, cartoon terminal and cartoon storage medium |
| CN115022616A (en) * | 2022-08-08 | 2022-09-06 | 太原理工大学 | Image focusing enhancement display device and display method based on human eye tracking |
| CN115022616B (en) * | 2022-08-08 | 2022-12-02 | 太原理工大学 | Image focusing enhancement display device and display method based on human eye tracking |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102572217B (en) | 2014-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102572217A (en) | Visual-attention-based multimedia processing method and device | |
| US11009945B2 (en) | Method for operating an eye tracking device for multi-user eye tracking and eye tracking device | |
| US10089769B2 (en) | Augmented display of information in a device view of a display screen | |
| CN110049324B (en) | Video encoding method, system, device, and computer-readable storage medium | |
| US11711588B2 (en) | Video delivery | |
| US10313633B2 (en) | Methods and system for simulated 3D videoconferencing | |
| US20140063176A1 (en) | Adjusting video layout | |
| US20150181265A1 (en) | Network synchronized camera settings | |
| CN104394363B (en) | A kind of Online class director method and system | |
| US8130257B2 (en) | Speaker and person backlighting for improved AEC and AGC | |
| EP2963921A1 (en) | Multi-terminal conference communication processing method and apparatus | |
| CN104469398B (en) | A kind of Internet video picture processing method and device | |
| CN104335243A (en) | Processing panoramic pictures | |
| CN105933637A (en) | Video communication method and system | |
| KR101540113B1 (en) | Method, apparatus for gernerating image data fot realistic-image and computer-readable recording medium for executing the method | |
| US9088693B2 (en) | Providing direct eye contact videoconferencing | |
| CN111355924A (en) | Method for detecting face scrambling code of special person based on video intelligent analysis | |
| CN111246224A (en) | Video live broadcast method and video live broadcast system | |
| CN108632563A (en) | Dynamic visual telephone system and its application method | |
| CN104811802A (en) | Image playing method and apparatus | |
| US10740624B2 (en) | Method for monitoring consumption of content | |
| US11310465B1 (en) | Video conference teminal and system there of | |
| US20210303853A1 (en) | Systems and methods for automated tracking on a handheld device using a remote camera | |
| WO2020131033A1 (en) | Systems and methods for automated tracking using a client device | |
| Sainio et al. | Eye-controlled region of interest HEVC encoding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20170531 Address after: 510640 Guangdong City, Tianhe District Province, No. five, road, public education building, unit 371-1, unit 2401 Patentee after: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20170719 Address after: 4 Building 99, tower 210000, Gulou District, Jiangsu, Nanjing, Zhongshan North Road Patentee after: NANJING RUICHI DINGXIN TECHNOLOGY CO.,LTD. Address before: 510640 Guangdong City, Tianhe District Province, No. five, road, public education building, unit 371-1, unit 2401 Patentee before: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20211124 Address after: 210000 Room 501, unit 3, No. 5, Dinghuaimen, Gulou District, Nanjing, Jiangsu Province Patentee after: Huang Zhenqiang Address before: 210000 4th floor, 99 Zhongshan North Road, Gulou District, Nanjing City, Jiangsu Province Patentee before: NANJING RUICHI DINGXIN TECHNOLOGY CO.,LTD. |
|
| TR01 | Transfer of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20140820 |