CN112347902A - Method for performing touch interaction by using camera, electronic equipment and storage medium - Google Patents
Method for performing touch interaction by using camera, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112347902A CN112347902A CN202011216322.1A CN202011216322A CN112347902A CN 112347902 A CN112347902 A CN 112347902A CN 202011216322 A CN202011216322 A CN 202011216322A CN 112347902 A CN112347902 A CN 112347902A
- Authority
- CN
- China
- Prior art keywords
- rgb
- value
- average value
- camera
- fixed points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000009471 action Effects 0.000 claims description 11
- 239000011159 matrix material Substances 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method for touch interaction by using a camera, which comprises the following steps: extracting brightness values, extracting RGB values of each fixed point on the camera, and calculating an RGB average value; judging the operation type, comparing the RGB average value with the RGB numerical value of each fixed point, judging the clicking operation, comparing the RGB average value with the RGB numerical value of the corresponding longitudinal fixed point, judging the up-down sliding operation, and comparing the RGB average value with the RGB numerical value of the corresponding transverse fixed point, judging the left-right sliding operation. The invention relates to an electronic device and a storage medium for executing a method for touch interaction by using a camera. According to the invention, simple clicking and touch interaction of up-down and left-right sliding is realized by using the camera under the condition without a touch screen, so that the clicking and sliding effects same as those of touch screen equipment are achieved.
Description
Technical Field
The present invention relates to the field of electronic technologies, and in particular, to a method, an electronic device, and a storage medium for performing touch interaction by using a camera.
Background
In our life, the scenes of touch, display and camera are everywhere, and people can not leave touch, display and camera. However, there are some special scenarios, and there are some devices with weaker interactivity, such as an advertisement machine inside an elevator, an attendance punched-card machine, and the like, and these devices basically have no touch function due to cost considerations or product positioning. When the devices need to be operated and set, the external devices need to be operated and controlled, for example, a mouse and a keyboard are connected. In order to realize touch control, all current solutions are to add a touch module above a display screen, the technology of the touch module is already mature, but in some scenes with weak interactivity, a touch module is added separately for less interaction, which is very cost-ineffective and relatively high in cost.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide a method for performing touch interaction by using a camera, which comprises the steps of analyzing a shielded image generated when a finger acquired by the camera slides on the camera, judging an instruction of the finger by using the change of RGB value of the image, further realizing corresponding operation, and realizing the purpose of realizing a common operation command by using the camera.
The invention provides a method for touch interaction by using a camera, which comprises the following steps:
extracting brightness values, extracting RGB values of each fixed point on the camera, and calculating an RGB average value;
judging the operation type, comparing the RGB average value with the RGB numerical value of each fixed point, judging the clicking operation, comparing the RGB average value with the RGB numerical value of the corresponding longitudinal fixed point, judging the up-down sliding operation, and comparing the RGB average value with the RGB numerical value of the corresponding transverse fixed point, judging the left-right sliding operation.
Further, in the operation type determining step, in a first preset time, if a difference between the RGB value of each fixed point intercepted by the camera and the RGB average value is not greater than a brightness threshold, and after a second preset time, if a difference between the RGB value of each fixed point intercepted by the camera and the RGB average value is greater than the brightness threshold, it is determined that the current action is a single click.
Further, the fixed points are arranged in a fixed point matrix on the camera.
Further, dividing all horizontal fixed points in the fixed point matrix into a group to obtain a plurality of rows of fixed points, and dividing all vertical fixed points in the fixed point matrix into a group to obtain a plurality of columns of fixed points.
Further, in the operation type judging step, several pictures are captured at a frequency of every second, and if the difference value between the RGB value of the first row of fixed points of one picture and the RGB average value is greater than the difference value between the RGB value of the last row of fixed points and the RGB average value, and the difference value between the RGB value of the last row of fixed points and the RGB average value is not greater than the brightness threshold, it is determined that the current action is downslide; and if the difference value between the RGB value of the first row of fixed points and the RGB average value in the plurality of pictures is smaller than the difference value between the RGB value of the last row of fixed points and the RGB average value, and the difference value between the RGB value of the first row of fixed points and the RGB average value is not larger than the brightness threshold value, judging that the current action is upward sliding.
Further, in the operation type judging step, if a difference value between the RGB values of the first row of fixed points of one picture and the RGB average value is greater than a difference value between the RGB values of the last row of fixed points and the RGB average value, and the difference value between the RGB values of the last row of fixed points and the RGB average value is not greater than the brightness threshold, it is determined that the current motion is a right slip; and if the difference value between the RGB value of the first row of fixed points in the plurality of pictures and the RGB average value is smaller than the difference value between the RGB value of the last row of fixed points and the RGB average value, and the difference value between the RGB value of the first row of fixed points and the RGB average value is not larger than the brightness threshold value, judging that the current motion is taken as left sliding.
An electronic device, comprising: a processor;
a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising instructions for performing a method for touch interaction with a camera.
A computer-readable storage medium having stored thereon a computer program for execution by a processor of a method for touch interaction with a camera.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a method for touch interaction by using a camera, which comprises the following steps: extracting brightness values, extracting RGB values of each fixed point on the camera, and calculating an RGB average value; judging the operation type, comparing the RGB average value with the RGB numerical value of each fixed point, judging the clicking operation, comparing the RGB average value with the RGB numerical value of the corresponding longitudinal fixed point, judging the up-down sliding operation, and comparing the RGB average value with the RGB numerical value of the corresponding transverse fixed point, judging the left-right sliding operation. The invention relates to an electronic device and a storage medium for executing a method for touch interaction by using a camera. According to the invention, simple clicking and touch interaction of up-down and left-right sliding is realized by using the camera under the condition without a touch screen, so that the clicking and sliding effects same as those of touch screen equipment are achieved.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood and to implement them in accordance with the contents of the description, the following detailed description is given with reference to the preferred embodiments of the present invention and the accompanying drawings. The detailed description of the present invention is given in detail by the following examples and the accompanying drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flowchart of a method for touch interaction with a camera according to the present invention;
fig. 2 is a picture taken by the camera of the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and the detailed description, and it should be noted that any combination of the embodiments or technical features described below can be used to form a new embodiment without conflict.
A method for touch interaction by using a camera, as shown in fig. 1, includes the following steps:
extracting brightness values, extracting RGB values of each fixed point on the camera, and calculating an RGB average value; as shown in fig. 2, the fixed points are arranged on the camera as a fixed point matrix, all horizontal fixed points in the fixed point matrix are divided into one group to obtain a plurality of rows of fixed points, and all vertical fixed points in the fixed point matrix are divided into one group to obtain a plurality of columns of fixed points. In this embodiment, the number of the fixed points is 20, RGB values of each fixed point in fig. 2 are extracted, and 20 fixed points are grouped from R1G1B1 to R20G20B20, where four horizontal points are a group of X1, X2, X3, X4, and X5, and five vertical points are a group of Y1, Y2, Y3, and Y4.
Judging the operation type, comparing the RGB average value with the RGB numerical value of each fixed point, judging the clicking operation, comparing the RGB average value with the RGB numerical value of the corresponding longitudinal fixed point, judging the up-down sliding operation, and comparing the RGB average value with the RGB numerical value of the corresponding transverse fixed point, judging the left-right sliding operation.
In an embodiment, within the first preset time, in this embodiment, the first preset time is 1 second. When the camera is pressed by the mobile phone for 1 second, the picture tends to be stable and uniform, if the difference value between the RGB value of each fixed point intercepted by the camera and the RGB average value is not greater than the brightness threshold value (unit%), a calculation program is activated, the light incoming quantity of the current camera can be known to be 0, and after the second preset time, if the difference value between the RGB value of each fixed point intercepted by the camera and the RGB average value is greater than the brightness threshold value, the light is restored to be normal, the current action is judged to be that the camera is pressed and then moved away, and the identified action is defined as clicking.
Screenshot is conducted at a frequency of a plurality of pictures per second, if the difference value between the RGB value of the first row of fixed points (namely X1) and the RGB average value of one picture is larger than the difference value between the RGB value of the last row of fixed points (namely X5) and the RGB average value, and the difference value between the RGB value of the last row of fixed points and the RGB average value is not larger than a brightness threshold value, the finger is considered to slide from top to bottom, the upper part of the finger is firstly exposed because the finger slides downwards, the color of each point of the X1 group is different, the current action is judged to slide downwards, and the system executes the action of sliding downwards at the moment; if the difference value between the RGB value of the first row of fixed points and the RGB average value is smaller than the difference value between the RGB value of the last row of fixed points and the RGB average value, and the difference value between the RGB value of the first row of fixed points and the RGB average value is not larger than the brightness threshold value, the finger is considered to slide from bottom to top, because the finger slides upwards, the lower part is exposed firstly, the color of each point of the X5 group is different, and the current action is judged to slide downwards.
If the difference value between the RGB value of the first row of fixed points (namely Y1) of one picture and the RGB average value is larger than the difference value between the RGB value of the last row of fixed points (namely Y4) and the RGB average value, and the difference value between the RGB value of the last row of fixed points and the RGB average value is not larger than the brightness threshold value, the finger slides from left to right, the current motion is judged to be right sliding, and the system executes the motion of sliding right at the moment; if the difference value between the RGB value of the first row of fixed points and the RGB average value in the plurality of pictures is smaller than the difference value between the RGB value of the last row of fixed points and the RGB average value, and the difference value between the RGB value of the first row of fixed points and the RGB average value is not larger than the brightness threshold value, the finger slides from right to left, the current motion is judged as left sliding, and at the moment, the system executes the motion of sliding left.
An electronic device, comprising: a processor;
a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising instructions for performing a method for touch interaction with a camera.
A computer-readable storage medium having stored thereon a computer program for execution by a processor of a method for touch interaction with a camera.
The invention provides a method for touch interaction by using a camera, which comprises the following steps: extracting brightness values, extracting RGB values of each fixed point on the camera, and calculating an RGB average value; judging the operation type, comparing the RGB average value with the RGB numerical value of each fixed point, judging the clicking operation, comparing the RGB average value with the RGB numerical value of the corresponding longitudinal fixed point, judging the up-down sliding operation, and comparing the RGB average value with the RGB numerical value of the corresponding transverse fixed point, judging the left-right sliding operation. The invention relates to an electronic device and a storage medium for executing a method for touch interaction by using a camera. According to the invention, simple clicking and touch interaction of up-down and left-right sliding is realized by using the camera under the condition without a touch screen, so that the clicking and sliding effects same as those of touch screen equipment are achieved.
The foregoing is merely a preferred embodiment of the invention and is not intended to limit the invention in any manner; those skilled in the art can readily practice the invention as shown and described in the drawings and detailed description herein; however, those skilled in the art should appreciate that they can readily use the disclosed conception and specific embodiments as a basis for designing or modifying other structures for carrying out the same purposes of the present invention without departing from the scope of the invention as defined by the appended claims; meanwhile, any changes, modifications, and evolutions of the equivalent changes of the above embodiments according to the actual techniques of the present invention are still within the protection scope of the technical solution of the present invention.
Claims (8)
1. A method for touch interaction by using a camera is characterized by comprising the following steps:
extracting brightness values, extracting RGB values of each fixed point on the camera, and calculating an RGB average value;
judging the operation type, comparing the RGB average value with the RGB numerical value of each fixed point, judging the clicking operation, comparing the RGB average value with the RGB numerical value of the corresponding longitudinal fixed point, judging the up-down sliding operation, and comparing the RGB average value with the RGB numerical value of the corresponding transverse fixed point, judging the left-right sliding operation.
2. The method for touch interaction with a camera of claim 1, wherein: in the operation type judging step, in a first preset time, if the difference value between the RGB value of each fixed point intercepted by the camera and the RGB average value is not larger than a brightness threshold value, and after a second preset time, if the difference value between the RGB value of each fixed point intercepted by the camera and the RGB average value is larger than the brightness threshold value, the current action is judged to be single click.
3. The method for touch interaction with a camera of claim 2, wherein: the fixed points are arranged on the camera to form a fixed point matrix.
4. The method for touch interaction with a camera of claim 3, wherein: dividing all horizontal fixed points in the fixed point matrix into a group to obtain a plurality of rows of fixed points, and dividing all vertical fixed points in the fixed point matrix into a group to obtain a plurality of columns of fixed points.
5. The method for touch interaction with a camera of claim 4, wherein: in the operation type judging step, several pictures are captured at a frequency of every second, and if the difference value between the RGB value of the first row of fixed points of one picture and the RGB average value is larger than the difference value between the RGB value of the last row of fixed points and the RGB average value, and the difference value between the RGB value of the last row of fixed points and the RGB average value is not larger than the brightness threshold value, the current action is judged to slide downwards; and if the difference value between the RGB value of the first row of fixed points and the RGB average value in the plurality of pictures is smaller than the difference value between the RGB value of the last row of fixed points and the RGB average value, and the difference value between the RGB value of the first row of fixed points and the RGB average value is not larger than the brightness threshold value, judging that the current action is upward sliding.
6. The method for touch interaction with a camera of claim 4, wherein: in the operation type judging step, if the difference value between the RGB numerical value of the first row of fixed points of one picture and the RGB average value is larger than the difference value between the RGB numerical value of the last row of fixed points and the RGB average value, and the difference value between the RGB numerical value of the last row of fixed points and the RGB average value is not larger than the brightness threshold value, judging that the current motion is taken as right sliding; and if the difference value between the RGB value of the first row of fixed points in the plurality of pictures and the RGB average value is smaller than the difference value between the RGB value of the last row of fixed points and the RGB average value, and the difference value between the RGB value of the first row of fixed points and the RGB average value is not larger than the brightness threshold value, judging that the current motion is taken as left sliding.
7. An electronic device, characterized by comprising: a processor;
a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising instructions for carrying out the method according to any one of claims 1-6.
8. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program is executed by a processor for performing the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011216322.1A CN112347902A (en) | 2020-11-04 | 2020-11-04 | Method for performing touch interaction by using camera, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011216322.1A CN112347902A (en) | 2020-11-04 | 2020-11-04 | Method for performing touch interaction by using camera, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112347902A true CN112347902A (en) | 2021-02-09 |
Family
ID=74428274
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011216322.1A Pending CN112347902A (en) | 2020-11-04 | 2020-11-04 | Method for performing touch interaction by using camera, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112347902A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112929778A (en) * | 2021-02-10 | 2021-06-08 | 歌尔科技有限公司 | Wireless earphone control method and device, electronic equipment and wireless earphone |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100952603B1 (en) * | 2009-11-05 | 2010-04-15 | 팬터로그인터액티브 주식회사 | Screen scroll method and device using camera |
| CN104978131A (en) * | 2015-06-18 | 2015-10-14 | Tcl移动通信科技（宁波）有限公司 | E-book page turning detection method and E-book page turning detection system based on camera image |
| CN105894497A (en) * | 2016-03-25 | 2016-08-24 | 惠州Tcl移动通信有限公司 | Camera-based key detection method and system, and mobile terminal |
| US20190196708A1 (en) * | 2017-12-22 | 2019-06-27 | Astro HQ LLC | Camera-detected touch input |
-
2020
- 2020-11-04 CN CN202011216322.1A patent/CN112347902A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100952603B1 (en) * | 2009-11-05 | 2010-04-15 | 팬터로그인터액티브 주식회사 | Screen scroll method and device using camera |
| CN104978131A (en) * | 2015-06-18 | 2015-10-14 | Tcl移动通信科技（宁波）有限公司 | E-book page turning detection method and E-book page turning detection system based on camera image |
| CN105894497A (en) * | 2016-03-25 | 2016-08-24 | 惠州Tcl移动通信有限公司 | Camera-based key detection method and system, and mobile terminal |
| US20190196708A1 (en) * | 2017-12-22 | 2019-06-27 | Astro HQ LLC | Camera-detected touch input |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112929778A (en) * | 2021-02-10 | 2021-06-08 | 歌尔科技有限公司 | Wireless earphone control method and device, electronic equipment and wireless earphone |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103513946B (en) | Control the method and device of display | |
| US10237622B2 (en) | Page turning method, page turning apparatus and terminal as well as computer readable medium | |
| JP6982194B2 (en) | Video feature extraction method and equipment | |
| CN113032268B (en) | Software testing method, device and storage medium | |
| CN113268303A (en) | Interface element configuration method and device, storage medium and electronic equipment | |
| CN107122104B (en) | Data display method and device | |
| CN112612436A (en) | Screen projection switching method, device, equipment and storage medium | |
| CN114003160A (en) | Data visualization display method and device, computer equipment and storage medium | |
| CN104572223B (en) | The method for sorting and device of a kind of desktop icons | |
| CN112347902A (en) | Method for performing touch interaction by using camera, electronic equipment and storage medium | |
| CN113053313A (en) | OLED screen ghost reduction method, system, device and storage medium | |
| JP4980486B1 (en) | Moving image region determination apparatus or method thereof | |
| CN110471700B (en) | Graphic processing method, apparatus, storage medium and electronic device | |
| CN111258434A (en) | Method, device, equipment and storage medium for inserting pictures into chat interface | |
| CN111274152B (en) | Application program testing method, device, equipment and storage medium | |
| US8922491B2 (en) | Method and apparatus enabling keystroke acceleration | |
| CN116521037B (en) | Word processing method and system | |
| CN112667343A (en) | Interface adjusting method, device, equipment and storage medium | |
| CN109246468B (en) | Video list switching method, device and storage medium based on education system | |
| CN104035652A (en) | Control method and control device for user interface display | |
| CN104407763A (en) | Content input method and system | |
| CN111047032A (en) | Image detection acceleration method and device and storage medium | |
| CN108108212B (en) | Processing method and device for application program list page and terminal | |
| CN114442973A (en) | Screen display method and device, computer equipment and storage medium | |
| CN110837289A (en) | Power saving method and device applied to OLED screen equipment and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 238, room 406, 1 Yichuang street, Huangpu District, Guangzhou, Guangdong 510700 Applicant after: Guangzhou langguo Electronic Technology Co.,Ltd. Address before: 510700 15 / F, unit 1, building 2, Yunsheng Science Park, No. 11, Zhongmian Road, Huangpu District, Guangzhou City, Guangdong Province Applicant before: GUANGZHOU LANGO ELECTRONIC SCIENCE & TECHNOLOGY Co.,Ltd. |