CN102841676A - Webpage browsing control system and method - Google Patents
Webpage browsing control system and method Download PDFInfo
- Publication number
- CN102841676A CN102841676A CN201110171031XA CN201110171031A CN102841676A CN 102841676 A CN102841676 A CN 102841676A CN 201110171031X A CN201110171031X A CN 201110171031XA CN 201110171031 A CN201110171031 A CN 201110171031A CN 102841676 A CN102841676 A CN 102841676A
- Authority
- CN
- China
- Prior art keywords
- lip
- video image
- web page
- image
- profile
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention provides a webpage browsing control system which is applied to an electronic device. The association between standard lip language video images and webpage operation commands, which is stored by the electronic device, is set for the system. When the electronic device displays a webpage for a user to browse, a pickup device picks up a video image of the user; the system identifies a lip profile in each frame of image and a lip movement video image of the user is formed according to the lip profile in each frame of image; and then, the system extracts each section of standard lip language video image and a characteristic matrix of the variation of the lip profiles in the lip movement video image, searches the standard lip language video image matched with the lip movement video image according to the characteristic matrix and executes the webpage operation command associated with the standard lip language video image to correspondingly control a webpage. The invention also provides a webpage browsing control method.
Description
Technical field
The present invention relates to a kind of web page browsing control system and method.
Background technology
At present, the user is utilizing electronic installation, mobile phone for example, and computing machines etc. in the process of browsing page, generally are through keyboard or Touch Screen operation web page, for example spur webpage up and down, operations such as webpage are closed in page turning.Because the restriction of keyboard or touch control screen sensitivity the time-consuming situation of effort again of simple web page operation usually can occur carrying out, or frequent operation causes keyboard wearing and tearing or Touch Screen cut to occur.
Summary of the invention
In view of above content, be necessary to propose a kind of web page browsing control system and method, in the time of can breaking away from the process of utilizing electronic installation browsing page operation web page to the dependence of keyboard or Touch Screen.
A kind of web page browsing control method, this method comprises: related between standard lip reading video image and the web page operation instruction of memory stores of electronic installation (A) is set; (B) display web page supplies the user to browse on the display of electronic installation, utilizes camera head acquisition user's video image, and this video image is stored in said storer; (C) discern human face region in each frame image of this video image; (D) discern lip-region in the human face region and lip profile in each frame image; (E) combine every frame image time information corresponding to form user's the moving video image of lip the lip profile in every frame image; (F) eigenmatrix of each segment standard lip reading video image lip profile variations of extraction memory stores; And extract the eigenmatrix that lip moves lip profile variations in the video image, and search the standard lip reading video image that is complementary with the moving video image of this lip according to said eigenmatrix; And (G) from storer, obtain the web page operation that is associated with this standard lip reading video image and instruct, carry out this web page operation instruction webpage is carried out corresponding control.
A kind of web page browsing control system is applied to electronic installation.The memory stores of this electronic installation has standard lip video image data and web page operation instruction set.This system comprises: association is provided with module, face recognition module, lip profile identification module, lip variation tracing module, lip reading matching module and instruction execution module.Association is provided with related between standard lip reading video image and the web page operation instruction of memory stores that module is provided with electronic installation.When display web page supplied the user to browse on the display of electronic installation, camera head captured user's video image, and this video image is stored in said storer.Human face region in each frame image of this video image of face recognition module identification.Lip-region and lip profile in each frame image of lip profile identification module identification in the human face region.Lip changes tracing module and combines every frame image time information corresponding to form user's the moving video image of lip the lip profile in every frame image.The lip reading matching module extracts the eigenmatrix of each segment standard lip reading video image lip profile variations of memory stores; And extract the eigenmatrix that lip moves lip profile variations in the video image, and search the standard lip reading video image that is complementary with the moving video image of this lip according to said eigenmatrix.The instruction execution module obtains the web page operation instruction that is associated with this standard lip reading video image from storer, carry out this web page operation instruction webpage is carried out corresponding control.
Compared to prior art; Web page browsing control system provided by the present invention and method are utilized lip reading control web page operation; The user does not need keyboard or Touch Screen operation web page; Both made things convenient for user's browsing page, and can avoid again the problem of cut occurring because of frequent operation causes keyboard wearing and tearing or Touch Screen.
Description of drawings
Fig. 1 is the functional block diagram of web page browsing control system preferred embodiments of the present invention.
Fig. 2 is the process flow diagram of web page browsing control method preferred embodiments of the present invention.
Fig. 3 is a synoptic diagram of confirming lip position in the image.
The main element symbol description
| |
100 |
| The web page |
10 |
| |
20 |
| |
30 |
| Standard lip |
31 |
| The web page operation instruction set | 32 |
| |
40 |
| |
50 |
| Association is provided with |
11 |
| |
12 |
| Lip |
13 |
| Lip |
14 |
| The lip |
15 |
| The |
16 |
Following embodiment will combine above-mentioned accompanying drawing to further specify the present invention.
Embodiment
Consulting shown in Figure 1ly, is the functional block diagram of web page browsing control system 10 preferred embodiments of the present invention.This web page browsing control system 10 is applied to electronic installation 100.This electronic installation 100 can be mobile phone, and computing machine or other can show and the device of browsing page.In this embodiment, this electronic installation 100 also comprises camera head 20, storer 30, processor 40 and display 50.In other embodiments, said camera head 20 also can be independent of electronic installation 100.
Each segment standard lip reading video image that web page browsing control system 10 is provided with storer 30 storage related with between web page operation instructs.The video image of lip reading " toward lastrow " of for example setting up standard is associated with the instruction of " delegation moves up ".Storer 30 is also stored the related information that is provided with.
The user's of camera head 20 acquisition browsing pages video image.Display 50 shows this video image.Web page browsing control system 10 is analyzed the lip change procedure that this video image obtains the user; The moving video image of lip of forming this user in conjunction with the time message of taking each frame image in this video image; In standard lip reading video data 31, search the standard lip reading video image that is complementary with the moving video image of this lip; And carry out the web page operation instruction that is associated with this standard lip reading video image, to realize through lip reading control web page operation.
Consult shown in Figure 1ly, web page browsing control system 10 comprises that association is provided with module 11, face recognition module 12, lip profile identification module 13, lip and changes tracing module 14, lip reading matching module 15 and instruction execution module 16.The alleged module of the present invention can be the hardware chip that is made up of a plurality of electronic devices and components, also can be the computer program code segments of being made up of the series of computation instruction.The described module of present embodiment is that a kind of device 40 of can being processed is performed and can accomplish the computer program code segments of fixed function, and it is stored in the storer 30.
Association is provided with module 11 be used to set up standard related between lip reading video image and the web page operation instruction.For example, association is provided with the set up standard video image of lip reading " toward lastrow " of module 11 and is associated with the instruction of " delegation moves up ", and the video image that the lip reading that sets up standard " is closed webpage " is associated with the instruction of " closing current web page ", or the like.
Lip profile identification module 13 is used for discerning lip-region and the lip profile in each frame image human face region.Consult shown in Figure 3; Can confirm that following 1/3 zone of human face region is lip-region in every frame image; 13 pairs of lip-region of lip profile identification module are carried out image and are strengthened (strengthening like contrast) to extract the unique point of lip-region; For example the coordinate on left and right sides labial angle, upper lip summit and lower lip summit is formed the lip profile in this frame image.
Lip changes tracing module 14 and is used for combining every frame image time information corresponding to form user's the moving video image of lip the lip profile of every frame image.For example, combine the time order and function of this 10 frame image to form user's the moving video image of lip in proper order the lip profile in the above-mentioned 10 frame images.
Lip reading matching module 15 is used to extract the eigenmatrix of each segment standard lip reading video image lip profile variations of storer 30 storages, and extracts the eigenmatrix of lip profile variations in the moving video image of lip.This eigenmatrix is based on the unique point of lip profile in each frame image, and for example the changes in coordinates on left and right sides labial angle, upper lip summit and lower lip summit makes up and forms.
Lip reading matching module 15 also is used for searching the standard lip reading video image that is complementary with the moving video image of this lip according to eigenmatrix.For example; A certain segment standard lip reading video image; For example the eigenmatrix similarity of lip profile variations reaches preset ratio (for example 80%) in the moving video image of the eigenmatrix of the lip profile variations of the video image of standard lip reading " toward lastrow " and this lip, and then lip reading matching module 15 judges that these standard lip reading video images and this lip move video image and be complementary.
Consulting shown in Figure 2ly, is the process flow diagram of web page browsing control method preferred embodiments of the present invention.
Step S201, association is provided with set up standard related between the instruction of lip reading video image and web page operation of module 11.For example, association is provided with the set up standard video image of lip reading " toward lastrow " of module 11 and is associated with the instruction of " delegation moves up ", and the video image that the lip reading that sets up standard " is closed webpage " is associated with the instruction of " closing current web page ", or the like.
Step S202, electronic installation 100 display web page on display 50 supplies the user to browse, and camera head 20 captures users' video image, and this video image is stored in storer 30.
Step S203, the human face region in each frame image of face recognition module 12 these video images of identification.For example, form by 10 frame images, then the human face region in face recognition module 12 these 10 frame images of identification as if this video image.
Step S204, lip-region and lip profile in lip profile identification module 13 each frame image of identification in the human face region.Consult shown in Figure 3; Can confirm that following 1/3 zone of human face region is lip-region in every frame image; Lip-region is carried out image strengthen (strengthening) to extract the unique point of lip-region like contrast; For example the coordinate on left and right sides labial angle, upper lip summit and lower lip summit is formed the lip profile in this frame image.Lip changes tracing module 14 and combines every frame image time information corresponding to form user's the moving video image of lip the lip profile in every frame image.For example, combine the time order and function of this 10 frame image to form user's the moving video image of lip in proper order the lip profile in the above-mentioned 10 frame images.
Step S205, the eigenmatrix of lip profile variations in each segment standard lip reading video image of lip reading matching module 15 extraction storeies 30 storages, and extract the eigenmatrix that this lip moves lip profile variations in the video image.This eigenmatrix is based on the unique point of lip profile in each frame image, and for example the changes in coordinates on left and right sides labial angle, upper lip summit and lower lip summit makes up and forms.
Step S206, lip reading matching module 15 search the standard lip reading video image that is complementary with the moving video image of this lip according to said eigenmatrix.For example, lip reading matching module 15 compares the eigenmatrix of lip profile variations in the eigenmatrix of lip profile variations in the moving video image of this lip and each the segment standard lip reading video image one by one.
Step S207, lip reading matching module 15 judge whether to find the standard lip reading video image that is complementary with the moving video image of this lip.For example; Lip reading matching module 15 judges whether to find certain segment standard lip reading video image, in this standard lip reading video image in the moving video image of the eigenmatrix of lip profile variations and this lip the similarity of the eigenmatrix of lip profile variations reach preset ratio (for example 80%).If do not find the standard lip reading video image that is complementary with the moving video image of this lip, then flow process finishes; If find the standard lip reading video image that is complementary with the moving video image of this lip; For example in the video image of standard lip reading " toward lastrow " in the moving video image of the eigenmatrix of lip profile variations and this lip the eigenmatrix similarity of lip profile variations reach 80%, execution in step S208 then.
Step S208, instruction execution module 16 obtain the web page operation instruction that is associated with this standard lip reading video image from storer 30.For example, the web page operation instruction " delegation moves up " that instruction execution module 16 obtains and execution is associated with the video image of standard lip reading " toward lastrow ".
Step S209, instruction execution module 16 carry out this web page operation instruction webpage are carried out corresponding control.For example, instruction execution module 16 is carried out the operational order of " delegation moves up ", the current web page content that electronic installation 100 the is shown delegation that moves up.
Claims (8)
1. a web page browsing control method is characterized in that, this method comprises:
Association is provided with step: related between standard lip reading video image and the web page operation instruction of memory stores of electronic installation is set;
Video image acquisition step: display web page supplies the user to browse on the display of electronic installation, utilizes camera head acquisition user's video image, and this video image is stored in said storer;
Recognition of face step: discern the human face region in each frame image of this video image;
Lip profile identification step: discern lip-region in the human face region and lip profile in each frame image;
Lip changes tracing step: combine every frame image time information corresponding to form user's the moving video image of lip the lip profile in every frame image;
Lip reading coupling step: the eigenmatrix that extracts each segment standard lip reading video image lip profile variations; And extract the eigenmatrix that lip moves lip profile variations in the video image, and search the standard lip reading video image that is complementary with the moving video image of this lip according to said eigenmatrix; And
Instruction execution in step: from storer, obtain the web page operation instruction that is associated with this standard lip reading video image, carry out this web page operation instruction webpage is carried out corresponding control.
2. web page browsing control method as claimed in claim 1 is characterized in that, following 1/3 zone of human face region is a lip-region in every frame image.
3. web page browsing control method as claimed in claim 1; It is characterized in that; Said lip profile identification step comprises: after the lip-region in every frame image is carried out the image enhancing, extract the characteristic point coordinates of lip-region and form the lip profile in this frame image.
4. web page browsing control method as claimed in claim 3 is characterized in that, the eigenmatrix of lip profile variations is based in each frame image of the moving video image of this lip the characteristic point coordinates of lip profile and changes to make up and form in the moving video image of said lip.
5. a web page browsing control system is characterized in that, this system comprises:
Association is provided with module, is used to be provided with related between standard lip reading video image and the web page operation instruction of memory stores of electronic installation;
The video image acquisition module is used for when display web page on the display of electronic installation supplies the user to browse, utilizing camera head acquisition user's video image, and this video image is stored in said storer;
Face recognition module is used for discerning the human face region of each frame image of this video image;
Lip profile identification module is used for discerning lip-region and the lip profile in each frame image human face region;
Lip changes tracing module, is used for combining every frame image time information corresponding to form user's the moving video image of lip the lip profile of every frame image;
The lip reading matching module; Be used to extract the eigenmatrix of each segment standard lip reading video image lip profile variations; And extract the eigenmatrix that lip moves lip profile variations in the video image, and search the standard lip reading video image that is complementary with the moving video image of this lip according to said eigenmatrix; And
The instruction execution module is used for obtaining the web page operation instruction that is associated with this standard lip reading video image from storer, carries out this web page operation instruction webpage is carried out corresponding control.
6. web page browsing control system as claimed in claim 5 is characterized in that, following 1/3 zone of human face region is a lip-region in every frame image.
7. web page browsing control system as claimed in claim 5 is characterized in that, said lip profile identification module is after the lip-region in every frame image is carried out the image enhancing, to extract the characteristic point coordinates of lip-region and form the lip profile in this frame image.
8. web page browsing control system as claimed in claim 5 is characterized in that, the eigenmatrix of lip profile variations is based in each frame image of the moving video image of this lip the characteristic point coordinates of lip profile and changes to make up and form in the moving video image of said lip.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110171031XA CN102841676A (en) | 2011-06-23 | 2011-06-23 | Webpage browsing control system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110171031XA CN102841676A (en) | 2011-06-23 | 2011-06-23 | Webpage browsing control system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102841676A true CN102841676A (en) | 2012-12-26 |
Family
ID=47369117
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110171031XA Pending CN102841676A (en) | 2011-06-23 | 2011-06-23 | Webpage browsing control system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102841676A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103136357A (en) * | 2013-03-06 | 2013-06-05 | 广东欧珀移动通信有限公司 | Method, device and mobile equipment for positioning webpages by face recognition technology |
| CN103716309A (en) * | 2013-12-17 | 2014-04-09 | 华为技术有限公司 | Security authentication method and terminal |
| CN103940042A (en) * | 2014-04-14 | 2014-07-23 | 美的集团股份有限公司 | Control equipment and control method |
| CN104199545A (en) * | 2014-08-28 | 2014-12-10 | 青岛海信移动通信技术股份有限公司 | Method and device for executing preset operations based on mouth shapes |
| CN104219371A (en) * | 2013-05-31 | 2014-12-17 | 中国电信股份有限公司 | Method and device for telephone dialing using lip reading technique |
| CN104238732A (en) * | 2013-06-24 | 2014-12-24 | 由田新技股份有限公司 | Device, method and computer readable recording medium for detecting facial movements to generate signals |
| CN105989329A (en) * | 2014-12-11 | 2016-10-05 | 由田新技股份有限公司 | Method and device for detecting use of handheld device by person |
| CN105989328A (en) * | 2014-12-11 | 2016-10-05 | 由田新技股份有限公司 | Method and device for detecting use of handheld device by person |
| CN106504751A (en) * | 2016-08-01 | 2017-03-15 | 深圳奥比中光科技有限公司 | Self adaptation lip reading exchange method and interactive device |
| WO2018113649A1 (en) * | 2016-12-21 | 2018-06-28 | 深圳市掌网科技股份有限公司 | Virtual reality language interaction system and method |
| CN109241907A (en) * | 2018-09-03 | 2019-01-18 | 北京旷视科技有限公司 | Mask method, device and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1710595A (en) * | 2005-06-16 | 2005-12-21 | 上海交通大学 | Mouth-corner positioning method |
| CN1896918A (en) * | 2005-07-15 | 2007-01-17 | 英华达(上海)电子有限公司 | Method for controlling input on manual equipment by face expression |
| CN101101752A (en) * | 2007-07-19 | 2008-01-09 | 华中科技大学 | Monosyllabic language lip-reading recognition system based on vision character |
| CN102033696A (en) * | 2009-09-24 | 2011-04-27 | 株式会社泛泰 | Apparatus and method for controlling picture using image recognition |
-
2011
- 2011-06-23 CN CN201110171031XA patent/CN102841676A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1710595A (en) * | 2005-06-16 | 2005-12-21 | 上海交通大学 | Mouth-corner positioning method |
| CN1896918A (en) * | 2005-07-15 | 2007-01-17 | 英华达(上海)电子有限公司 | Method for controlling input on manual equipment by face expression |
| CN101101752A (en) * | 2007-07-19 | 2008-01-09 | 华中科技大学 | Monosyllabic language lip-reading recognition system based on vision character |
| CN102033696A (en) * | 2009-09-24 | 2011-04-27 | 株式会社泛泰 | Apparatus and method for controlling picture using image recognition |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103136357A (en) * | 2013-03-06 | 2013-06-05 | 广东欧珀移动通信有限公司 | Method, device and mobile equipment for positioning webpages by face recognition technology |
| CN104219371A (en) * | 2013-05-31 | 2014-12-17 | 中国电信股份有限公司 | Method and device for telephone dialing using lip reading technique |
| CN104238732A (en) * | 2013-06-24 | 2014-12-24 | 由田新技股份有限公司 | Device, method and computer readable recording medium for detecting facial movements to generate signals |
| CN104238732B (en) * | 2013-06-24 | 2019-10-25 | 由田新技股份有限公司 | Device, method and computer readable recording medium for detecting facial movements to generate signals |
| CN103716309B (en) * | 2013-12-17 | 2017-09-29 | 华为技术有限公司 | A kind of safety certifying method and terminal |
| CN103716309A (en) * | 2013-12-17 | 2014-04-09 | 华为技术有限公司 | Security authentication method and terminal |
| CN103940042A (en) * | 2014-04-14 | 2014-07-23 | 美的集团股份有限公司 | Control equipment and control method |
| CN103940042B (en) * | 2014-04-14 | 2016-07-06 | 美的集团股份有限公司 | Control equipment and control method |
| CN104199545A (en) * | 2014-08-28 | 2014-12-10 | 青岛海信移动通信技术股份有限公司 | Method and device for executing preset operations based on mouth shapes |
| CN105989328A (en) * | 2014-12-11 | 2016-10-05 | 由田新技股份有限公司 | Method and device for detecting use of handheld device by person |
| CN105989329A (en) * | 2014-12-11 | 2016-10-05 | 由田新技股份有限公司 | Method and device for detecting use of handheld device by person |
| CN106504751A (en) * | 2016-08-01 | 2017-03-15 | 深圳奥比中光科技有限公司 | Self adaptation lip reading exchange method and interactive device |
| WO2018113649A1 (en) * | 2016-12-21 | 2018-06-28 | 深圳市掌网科技股份有限公司 | Virtual reality language interaction system and method |
| CN109241907A (en) * | 2018-09-03 | 2019-01-18 | 北京旷视科技有限公司 | Mask method, device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102841676A (en) | Webpage browsing control system and method | |
| US9104242B2 (en) | Palm gesture recognition method and device as well as human-machine interaction method and apparatus | |
| KR101603017B1 (en) | Gesture recognition device and gesture recognition device control method | |
| US8379987B2 (en) | Method, apparatus and computer program product for providing hand segmentation for gesture analysis | |
| WO2014045953A1 (en) | Information processing device and method, and program | |
| WO2010049790A1 (en) | Method, apparatus and computer program product for providing adaptive gesture analysis | |
| US20150242114A1 (en) | Electronic device, method and computer program product | |
| US20210125582A1 (en) | Media rendering with orientation metadata | |
| US9134833B2 (en) | Electronic apparatus, method, and non-transitory computer-readable storage medium | |
| CN103824481A (en) | Method and device for detecting user recitation | |
| KR20200120739A (en) | Methods, devices, devices and computer-readable storage media for public handwriting recognition | |
| CN111857508B (en) | Task management method and device and electronic equipment | |
| KR101631011B1 (en) | Gesture recognition apparatus and control method of gesture recognition apparatus | |
| CN104090761A (en) | Screenshot application device and method | |
| CN104808794A (en) | Method and system for inputting lip language | |
| US20160098594A1 (en) | Electronic apparatus, processing method and storage medium | |
| CN107004073A (en) | The method and electronic equipment of a kind of face verification | |
| CN103106388B (en) | Method and system of image recognition | |
| US11429985B2 (en) | Information processing device calculating statistical information | |
| US20130301921A1 (en) | Electronic device and handwritten document search method | |
| EP2630616A1 (en) | Method and apparatus for providing hand detection | |
| CN112364799A (en) | Gesture recognition method and device | |
| CN105095860A (en) | Method and device for character segmentation | |
| CN106648042B (en) | Identification control method and device | |
| CN111656313A (en) | Screen display switching method, display device and movable platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20121226 |