CN102830801A - Method and system for controlling digital signage by utilizing gesture recognition - Google Patents
Method and system for controlling digital signage by utilizing gesture recognition Download PDFInfo
- Publication number
- CN102830801A CN102830801A CN2012102756616A CN201210275661A CN102830801A CN 102830801 A CN102830801 A CN 102830801A CN 2012102756616 A CN2012102756616 A CN 2012102756616A CN 201210275661 A CN201210275661 A CN 201210275661A CN 102830801 A CN102830801 A CN 102830801A
- Authority
- CN
- China
- Prior art keywords
- image
- mouse
- user images
- threshold range
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to a method and a system for controlling a digital signage by utilizing gesture recognition. According to the method and the system, the mouse click operation is realized by the recognition of the user gesture. The method comprises the steps of displaying the information needing to be clicked on the above position of a screen; continuously acquiring a user image, processing the acquired each frame of user image to obtain a hand binary image and a corresponding external rectangular image; selecting a pixel point of any one fixed point of the external rectangular image as a palm point, and mapping the position of the palm point on the user image to be as the position at which a mouse of a system is positioned; and when the mouse is moved to the position on which the information can be clicked, and conducting click operation after the scheduled time is stopped.
Description
Technical field
The present invention relates to the interactive field of man-machine vision, refer in particular to a kind of gesture identification control figure label that utilizes and carry out click method of operating and system.
Background technology
Current main-stream formal advertising display form mainly contains two kinds: non-interactive digital label and the interactive digital label that utilizes touch-screen.
Non-interactive digital label mainly adopts the mode of regularly switching formal advertising; Before the user at a time rests on said digital signage, can only passively accept the information that show on the label this moment, and can't see other information; If the user loses interest in, just possibly leave at once; Even the user just in time sees own information of interest, also can't repeat careful survey.This mode user's convenience is relatively poor, and bandwagon effect is bad.
Utilize the interactive digital label of touch-screen, can answer user's request and select information specific, just can let the user find information of interest at short notice, also can let the user repeat careful survey, user friendliness is high, and bandwagon effect is good.But adopt touch-screen can cause cost higher, particularly formal advertising often needs bigger screen, and the price of touch-screen is along with the size of showing screen is exponential growth.
In order to address this problem, to the invention provides a kind of gesture identification control figure label that utilizes and carry out click method of operating and system.
Summary of the invention
The present invention utilizes the method for gesture identification control figure label, may further comprise the steps:
Step 1: the information that will click is presented at the top position, viewing area;
Step 2: the continuous collecting user images, from each framed user's image, extract colour of skin bianry image, wherein each area of skin color screened according to the size of the ratio of shared user images, obtain the staff bianry image;
Wherein, From said user images, extract colour of skin bianry image; Be to convert said user images into the HSV image, said HSV image is regulated according to the threshold range of pre-set H, S, V component respectively, obtain three component bianry images; Three component images are intersected computing, finally obtain colour of skin bianry image; Said H component threshold range is [0,0.156] ∪ [0.609,1.000], and said S component threshold range is [0.156 ~ 0.664], and said V component threshold range is [0.117,0.742].
Because the information that needs to click is positioned at the display screen top position; User zone such as staff zone and people's face in the click process does not have overlapping; So the staff area of skin color that comprises independent completion in the said colour of skin bianry image and other area of skin color screen according to the size of the ratio of shared user images each area of skin color, promptly calculate the area ratio of each area of skin color and said user images; Keep area than the area of skin color in threshold range C; With remaining area of skin color assignment is " 0 ", to remove other area of skin color, obtains the staff bianry image.Wherein, threshold value C preestablishes and can regulate, and in the present embodiment, threshold range C is [0.030,0.150].
Preferably; After collecting said user images, before carry out step 2, earlier said user images is carried out simple smooth and handle; Utilize the mean shift segmentation method that it is divided into more tangible several zones, border; Can reach like this and reduce image, reduce computational complexity, improve ageing purpose.
Step 3: said staff bianry image is expanded corrosion obtain relative smoothed image, from said relative smoothed image, detect staff profile chained list, from staff profile chained list, extract circumscribed area, obtain the boundary rectangle image;
Step 4: the pixel of fixing a bit arbitrarily of choosing said boundary rectangle image is the mouse position and demonstrates mouse the location map of palm point on said user images as the palm point.Be specially, pixel on the said user images and the pixel on the display screen are shone upon in proportion, the position of the pixel that the palm point is mapped to is as the position of mouse, and mouse is shown.
Preferably, the pixel at each 1/5 place of upper left corner length and width that can choose boundary rectangle is as the palm point.
Step 5: when said mouse moves to the information place that can click, and rest on this information after the schedule time, trigger clicking operation.
Corresponding, the present invention utilizes the system of gesture identification control figure label, comprises with the lower part:
Camera and display screen, camera are used to gather user images;
Graphics processing unit is used to receive said user images, and according to above method it is handled, and obtains the boundary rectangle image;
The mouse mappings unit; Be used to receive said boundary rectangle image; The pixel of fixing a bit arbitrarily of choosing said boundary rectangle is the mouse position with the location map of palm point on said user images, and on display screen, mouse is shown as the palm point;
Click control module, be used to receive the positional information of said mouse, when said mouse moves to the information place that can click, and rest on this information after the schedule time, send and click order to the display message control module;
The display message control module, the information that needs are clicked is presented at the top position of display screen, receives said click order and accomplishes corresponding clicking operation.
Description of drawings
Fig. 1 utilizes the method flow diagram of visual gesture identification control digital signage for the present invention
Fig. 2 utilizes the system schematic of visual gesture identification control digital signage for the present invention
Fig. 3 utilizes the working method synoptic diagram of visual gesture identification control digital signage for the present invention
Embodiment
Describe embodiments of the invention below in detail, the example of said embodiment is shown in the drawings, and wherein identical from start to finish or similar label is represented identical or similar elements or the element with identical or similar functions.Be exemplary through the embodiment that is described with reference to the drawings below, only be used to explain the present invention, and can not be interpreted as limitation of the present invention.
As shown in Figure 1, the embodiment of the invention is utilized the method for gesture identification control figure label, and it mainly may further comprise the steps:
Step 1: the information that will click is presented at the top position of viewing area;
Step 2: the continuous collecting user images, from each framed user's image, extract colour of skin bianry image, wherein each area of skin color screened according to the size of the ratio of shared user images, obtain the staff bianry image, be specially:
From said user images, extract colour of skin bianry image; Be to convert said staff bianry image into the HSV image (the HSV image has H (tone), S (saturation degree), three components of V (brightness); The numerical value of each component is all between 0 ~ 1); Each HSV image is regulated according to the threshold range of pre-set H, S, V component respectively; With the pixel assignment in threshold range is " 1 ", and the pixel assignment outside threshold range is " 0 ", obtains three components bianry image separately; Bianry image to three components intersects computing at last, finally obtains colour of skin bianry image.
Wherein, the threshold range of three components preestablishes and can regulate, said H component, and best threshold range is [0,0.156] ∪ [0.609,1.000]; Said S component, best threshold range is [0.156 ~ 0.664]; Said V component, best threshold range is [0.117,0.742].
Because the information that needs to click is positioned at the display screen top position; User zone such as staff zone and people's face in the click process does not have overlapping, so comprise the staff area of skin color and other area of skin color of independent completion in the said colour of skin bianry image, each area of skin color is screened according to the size of the ratio of shared user images; Promptly calculate the area ratio of each area of skin color and said user images; Keeping area than the area of skin color in threshold range C, is " 0 " with remaining area of skin color assignment, to remove other area of skin color; Keep the staff area of skin color, obtain the staff bianry image.Wherein, threshold value C preestablishes and can regulate, and in the present embodiment, threshold range C is [0.030,0.150].
Preferably; After collecting said user images, before carry out step 2, earlier said user images is carried out simple smooth and handle; Utilize the mean shift segmentation method that it is divided into significantly zone, several borders; Can reach like this and reduce image, reduce computational complexity, improve ageing purpose.
Step 3: said staff bianry image is expanded corrosion obtain relative smoothed image, from said relative smoothed image, detect staff profile chained list, from staff profile chained list, extract circumscribed area, obtain the boundary rectangle image;
Step 4: the pixel of fixing a bit arbitrarily of choosing said boundary rectangle image is the mouse position and demonstrates mouse the location map of palm point on said user images as the palm point.Be specially, pixel on the said user images and the pixel on the display screen are shone upon in proportion, the position of the pixel that the palm point is mapped to is as the position of mouse, and mouse is shown.
Wherein, The said pixel of fixing a bit arbitrarily; Refer to the fixing a certain pixel of relative position on said boundary rectangle image; For example all choose boundary rectangle topmost the pixel of middle is as the palm point at every turn, and the pixel of perhaps at every turn all choosing the upper left corner of said boundary rectangle is as the palm point.Preferably, because most of users are accustomed to using the right hand,, the user carries out when mutual when using the right hand and system; The always relative palm in the position of elbow is kept right; Simultaneously, the palm position always is in elbow position top, so the pixel at each 1/5 place of upper left corner length and width that can choose said boundary rectangle is as the palm point; Can make mouse better to the following property of user's gesture like this, user's experience property can be better.
Step 5: when said mouse moves to the information place that can click, and rest on this information after the schedule time kind, trigger clicking operation.The said schedule time is generally several seconds, can set according to practical application.
Corresponding, a kind of system that utilizes gesture identification control figure label of the present invention, as shown in Figure 2, comprise with the lower part:
Camera and display screen, camera are used to gather user images;
Graphics processing unit is used to receive said user images, and according to above method it is handled, and obtains the boundary rectangle image;
The mouse mappings unit; Be used to receive said boundary rectangle image; The pixel of fixing a bit arbitrarily of choosing said boundary rectangle is the mouse position with the location map of palm point on said user images, and on display screen, mouse is shown as the palm point;
Click control module, be used to receive the positional information of said mouse, when said mouse moves to the information place that can click, and rest on this information after the schedule time, send and click order to the display message control module; The said schedule time is generally several seconds, can set according to practical application.
The display message control module, the information that needs are clicked is presented at the display screen top position, receives said click order and accomplishes corresponding clicking operation.
In the present embodiment; Utilize the working method of the digital signage that gesture identification controls as shown in Figure 3: the display screen top shows 4 placard thumbnails; The left side, top shows a left the arrow icon; The right, top shows a right the arrow icon, and lower position shows the complete enlarged drawing of the current placard of watching in the display screen.System is through camera collection user's image; And user's gesture discerned according to the method described above; The user comes mouse beacon to move through the position of moving hand, when mouse moves to any in the placard thumbnail of top, and after stopping for 2 seconds; System carries out clicking operation, and the complete enlarged drawing of this placard is shown; Same, after mouse moved to top left arrow or right the arrow icon and stopped for 2 seconds, system carried out clicking operation, turns over one page forward or backward.
Further, can the click information place on mouse moves to, system just demonstrates a progress ring and representes progress on display screen; When mouse was docked to 2 seconds, it is full that the progress ring changes, and triggers clicking operation; Such design can offer the custom system feedback constantly, guarantees dynamic interaction experience.Certainly, can also adopt other mode, point out the user, repeat no more here such as the mode of progress bar or voice.
Each functional unit in embodiments of the present invention can be integrated in the processing module, also can be that the independent physics in each unit exists, and also can be that two or more unit are integrated in the module.Above-mentioned integrated module both can adopt the form of hardware to realize, also can adopt the form of software function module to realize that the present invention does not limit it.
Embodiment shown in above foundation is graphic has specified structure of the present invention, characteristic and action effect; The above is merely preferred embodiment of the present invention; But the present invention does not limit practical range with shown in the drawing, every change of doing according to conception of the present invention, or be revised as the equivalent embodiment of equivalent variations; When not exceeding instructions yet and illustrating contained spiritual, all should be in protection scope of the present invention.
Claims (8)
1. method of utilizing gesture identification control figure label is characterized in that may further comprise the steps:
Step 1: the information that will click is presented at the top position of viewing area;
Step 2: the continuous collecting user images, from each framed user's image, extract colour of skin bianry image, wherein each area of skin color screened according to the size of the ratio of shared user images, obtain the staff bianry image;
Step 3: said staff bianry image is expanded corrosion obtain relative smoothed image, from said relative smoothed image, detect staff profile chained list, from staff profile chained list, extract circumscribed area, obtain the boundary rectangle image;
Step 4: the pixel of fixing a bit arbitrarily of choosing said boundary rectangle image is the mouse position and demonstrates mouse the location map of palm point on said user images as the palm point;
Step 5: when said mouse moves to the information place that can click, and rest on this information after the schedule time, trigger clicking operation.
2. the method for utilizing gesture identification control figure label according to claim 1 is characterized in that: from said user images, extracting colour of skin bianry image, is to regulate according to the threshold range of H, S, V component respectively after converting said user images into the HSV image; Obtain three component bianry images, intersect computing again, obtain said colour of skin bianry image; Said H component threshold range is [0; 0.156] ∪ [0.609,1.000], said S component threshold range is [0.156 ~ 0.664]; Said V component threshold range is [0.117,0.742].
3. the method for utilizing gesture identification control figure label according to claim 1 is characterized in that: after collecting said user images, before carry out step 2, utilize the mean shift segmentation method that it is divided into significantly zone, several borders earlier.
4. the method for utilizing gesture identification control figure label according to claim 1 is characterized in that: the pixel at each 1/5 place of upper left corner length and width of choosing boundary rectangle is as the palm point.
5. system that utilizes gesture identification control figure label is characterized in that:
Camera and display screen, camera are used to gather user images;
Graphics processing unit is used to receive said user images, from each framed user's image, extracts colour of skin bianry image, and wherein each area of skin color screened according to the size of the ratio of shared user images, obtains the staff bianry image; Said staff bianry image is expanded corrosion obtain relative smoothed image, from said relative smoothed image, detect staff profile chained list, from staff profile chained list, extract circumscribed area, obtain the boundary rectangle image;
The mouse mappings unit; Be used to receive said boundary rectangle image; The pixel of fixing a bit arbitrarily of choosing said boundary rectangle image is the mouse position with the location map of palm point on said user images, and on display screen, mouse is shown as the palm point;
Click control module, be used to receive the positional information of said mouse, when said mouse moves to the information place that can click, and rest on this information after the schedule time, send and click order to the display message control module;
The display message control module, the information that needs are clicked is presented at the top position of display screen, receives said click order and accomplishes corresponding clicking operation.
6. the system that utilizes gesture identification control figure label according to claim 5 is characterized in that: said graphics processing unit extracts colour of skin bianry image from said user images, is to regulate according to the threshold range of H, S, V component respectively after converting said user images into the HSV image; Obtain three component bianry images, intersect computing again, obtain said colour of skin bianry image; Said H component threshold range is [0; 0.156] ∪ [0.609,1.000], said S component threshold range is [0.156 ~ 0.664]; Said V component threshold range is [0.117,0.742].
7. the system that utilizes gesture identification control figure label according to claim 5; It is characterized in that: after said graphics processing unit receives said user images; Before carry out step 2, utilize the mean shift segmentation method that it is divided into significantly zone, several borders earlier.
8. the system that utilizes gesture identification control figure label according to claim 5 is characterized in that: the pixel at each 1/5 place of the upper left corner length and width of said mouse mappings unit selection boundary rectangle is as the palm point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012102756616A CN102830801A (en) | 2012-08-03 | 2012-08-03 | Method and system for controlling digital signage by utilizing gesture recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012102756616A CN102830801A (en) | 2012-08-03 | 2012-08-03 | Method and system for controlling digital signage by utilizing gesture recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102830801A true CN102830801A (en) | 2012-12-19 |
Family
ID=47333968
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012102756616A Pending CN102830801A (en) | 2012-08-03 | 2012-08-03 | Method and system for controlling digital signage by utilizing gesture recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102830801A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902740A (en) * | 2014-04-22 | 2014-07-02 | 锤子科技（北京）有限公司 | Short message authentication code residing method and device |
| CN105353959A (en) * | 2015-10-30 | 2016-02-24 | 小米科技有限责任公司 | Method and apparatus for controlling list to slide |
| CN113191403A (en) * | 2021-04-16 | 2021-07-30 | 上海戏剧学院 | Generation and display system of theater dynamic poster |
| CN113221834A (en) * | 2021-06-01 | 2021-08-06 | 北京字节跳动网络技术有限公司 | Terminal control method and device, terminal and storage medium |
-
2012
- 2012-08-03 CN CN2012102756616A patent/CN102830801A/en active Pending
Non-Patent Citations (4)
| Title |
|---|
| 栾雄: "人机交互中的手指定位研究与应用", 《硕士学位论文》 * |
| 王茂吉: "基于视觉的静态手势识别系统", 《硕士学位论文》 * |
| 邹利华: "检测彩色图像中人头数的人脸分割算法", 《微计算机信息》 * |
| 齐苏敏等: "基于自适应活动轮廓模型的实时手势跟踪", 《计算机科学》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902740A (en) * | 2014-04-22 | 2014-07-02 | 锤子科技（北京）有限公司 | Short message authentication code residing method and device |
| US10795942B2 (en) | 2014-04-22 | 2020-10-06 | Beijing Bytedance Network Technology Co Ltd. | Method and device for residing short message verification code |
| CN105353959A (en) * | 2015-10-30 | 2016-02-24 | 小米科技有限责任公司 | Method and apparatus for controlling list to slide |
| CN113191403A (en) * | 2021-04-16 | 2021-07-30 | 上海戏剧学院 | Generation and display system of theater dynamic poster |
| CN113221834A (en) * | 2021-06-01 | 2021-08-06 | 北京字节跳动网络技术有限公司 | Terminal control method and device, terminal and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10698560B2 (en) | Organizing digital notes on a user interface | |
| CN103297854B (en) | A kind of web page focus control method | |
| CN102662566B (en) | Screen content amplification display method and terminal | |
| TWI659354B (en) | Computer device having a processor and method of capturing and recognizing notes implemented thereon | |
| CN109271983B (en) | Display method and display terminal for identifying object in screenshot | |
| JP6523706B2 (en) | Display device, mobile device, system including the same, and image quality matching method thereof | |
| CN108255387B (en) | Quick contrast interaction method for images of mobile terminal of touch screen | |
| CN102830800B (en) | Method and system for controlling digital signage by utilizing gesture recognition | |
| CN103869946A (en) | Display control method and electronic device | |
| EP3042273A1 (en) | Using swipe gestures to change displayed applications | |
| CN104199550A (en) | Man-machine interactive type virtual touch device, system and method | |
| US20140043255A1 (en) | Electronic device and image zooming method thereof | |
| CN101739210B (en) | Reminding method for image browsing and device thereof | |
| JP2010134897A (en) | Drawing device, drawing method, program and recording medium | |
| CN102063231A (en) | Non-contact electronic whiteboard system and detection method based on image detection | |
| CN102830801A (en) | Method and system for controlling digital signage by utilizing gesture recognition | |
| CN103399695B (en) | Quadrangle frame identification method and device for intelligent wireless communication terminal | |
| WO2013134975A1 (en) | Method of Scene Recognition Based Virtual Touch Event | |
| CN104199547A (en) | Man-machine interactive type virtual touch device, system and method | |
| WO2017032193A1 (en) | User interface layout adjustment method and apparatus | |
| US20150220800A1 (en) | Note capture, recognition, and management with hints on a user interface | |
| US20150121270A1 (en) | Information processing method and electronic device | |
| CN103076952A (en) | Amplifying method and amplifying device for display contents in display interface of intelligent camera | |
| Hartanto et al. | Real time hand gesture movements tracking and recognizing system | |
| US20130169660A1 (en) | Image editing system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20121219 |