CN114138162A - Intelligent transparent office table interaction method - Google Patents
Intelligent transparent office table interaction method Download PDFInfo
- Publication number
- CN114138162A CN114138162A CN202111189457.8A CN202111189457A CN114138162A CN 114138162 A CN114138162 A CN 114138162A CN 202111189457 A CN202111189457 A CN 202111189457A CN 114138162 A CN114138162 A CN 114138162A
- Authority
- CN
- China
- Prior art keywords
- area
- bright spots
- effective bright
- transparent
- fingers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 230000003993 interaction Effects 0.000 title claims abstract description 58
- 238000012545 processing Methods 0.000 claims abstract description 48
- 238000004891 communication Methods 0.000 claims abstract description 30
- 238000001514 detection method Methods 0.000 claims abstract description 5
- 238000012544 monitoring process Methods 0.000 claims abstract description 4
- 230000008034 disappearance Effects 0.000 claims description 48
- 230000008569 process Effects 0.000 claims description 40
- 238000003860 storage Methods 0.000 claims description 24
- 238000003384 imaging method Methods 0.000 claims description 22
- 230000002452 interceptive effect Effects 0.000 claims description 22
- 238000013507 mapping Methods 0.000 claims description 13
- 230000009471 action Effects 0.000 claims description 8
- 238000006073 displacement reaction Methods 0.000 claims description 7
- 238000003825 pressing Methods 0.000 claims description 6
- 238000013519 translation Methods 0.000 claims description 5
- 230000008901 benefit Effects 0.000 abstract description 5
- 230000002093 peripheral effect Effects 0.000 abstract description 2
- 239000011521 glass Substances 0.000 description 23
- 239000012780 transparent material Substances 0.000 description 13
- 241000699666 Mus <mouse, genus> Species 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 230000002708 enhancing effect Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 238000004886 process control Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000002834 transmittance Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000699660 Mus musculus Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000005282 brightening Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003670 easy-to-clean Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005530 etching Methods 0.000 description 1
- 238000001125 extrusion Methods 0.000 description 1
- 239000007888 film coating Substances 0.000 description 1
- 238000009501 film coating Methods 0.000 description 1
- 239000005357 flat glass Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000010147 laser engraving Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 229920003023 plastic Polymers 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 229920005989 resin Polymers 0.000 description 1
- 239000011347 resin Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004381 surface treatment Methods 0.000 description 1
- 239000005341 toughened glass Substances 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47B—TABLES; DESKS; OFFICE FURNITURE; CABINETS; DRAWERS; GENERAL DETAILS OF FURNITURE
- A47B13/00—Details of tables or desks
- A47B13/08—Table tops; Rims therefor
- A47B13/12—Transparent tops, e.g. with lighting means under the table top
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47B—TABLES; DESKS; OFFICE FURNITURE; CABINETS; DRAWERS; GENERAL DETAILS OF FURNITURE
- A47B21/00—Tables or desks for office equipment, e.g. typewriters, keyboards
- A47B21/007—Tables or desks for office equipment, e.g. typewriters, keyboards with under-desk displays, e.g. displays being viewable through a transparent working surface of the table or desk
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0486—Drag-and-drop
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47B—TABLES; DESKS; OFFICE FURNITURE; CABINETS; DRAWERS; GENERAL DETAILS OF FURNITURE
- A47B21/00—Tables or desks for office equipment, e.g. typewriters, keyboards
- A47B21/007—Tables or desks for office equipment, e.g. typewriters, keyboards with under-desk displays, e.g. displays being viewable through a transparent working surface of the table or desk
- A47B2021/0076—Tables or desks for office equipment, e.g. typewriters, keyboards with under-desk displays, e.g. displays being viewable through a transparent working surface of the table or desk the screen being incorporated in the desk top
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
An intelligent transparent desk interaction method comprises the following steps: presetting a touch operation event on a transparent desktop, defining an operation logic instruction set, defining a judgment criterion of effective bright spots, defining an operable area on the transparent desktop, starting an enhanced light source, continuously acquiring an image of the area of the transparent desktop below the transparent desktop by an image acquisition module, continuously monitoring state information of the effective bright spots appearing in the operable area in the image by a processing control module, determining a corresponding touch operation event according to the bright spot state information, and transmitting a control instruction corresponding to the touch operation event to the intelligent terminal by a communication module. The invention relates to a computer and an intelligent terminal peripheral equipment interaction method, which can realize touch interaction with an intelligent terminal on a fully transparent desktop, and has the advantages of low input signal detection difficulty, reliable identification, support of multi-point touch and low cost.
Description
Technical Field
The invention relates to an intelligent terminal peripheral equipment interaction method, in particular to an intelligent transparent office table interaction method.
Background
Since office work and study are usually performed on the desktop, the desktop is directly used as an interactive device of an intelligent terminal such as a computer, which brings extremely high convenience, and therefore, the desktop is always a popular research direction. The difficulty of directly using a desktop as an interactive device is that a common desktop lacks a medium required for interaction, and a common scheme mainly includes the following two types in order to make the desktop have an interactive capability.
One solution is to provide the desktop itself with an interactive medium, for example, a touch sensitive panel or a display panel is added on the desktop, so that the desktop becomes a large-sized touch display screen and has input and output interactive capabilities. However, the cost of the large-size touch display screen is very high, the use environment of the desktop is relatively complex, and the touch display screen of the desktop can be damaged by heavy objects, extrusion, collision, impact and other abnormal loads. The touch display screen has the characteristics that once the touch display screen is damaged, the rejection rate is high, the maintenance cost is high, and therefore the defect of the scheme is obvious.
Another solution is to provide interactive capability to the desktop through projection and infrared sensing, for example, the invention patent "information input device and method based on projection and infrared sensing" of samsung electronic corporation, patent No. CN201310739468.8, the main embodiments of the invention are: by using a projection method, a virtual keyboard and mouse interaction area is mapped on the surface of a desktop or other objects through visible light and infrared laser, when any position of the virtual interaction area is clicked by a finger or other touch equipment, an infrared camera unit can detect that a light field is blocked, so that the position information of a click event is identified, the touch and interaction event corresponding to the click event is judged through the position information, and then the touch and interaction event is transmitted to an operated terminal through a communication module, so that the interactive operation of the operated terminal is realized. The most obvious shortcoming of the scheme is that when the scheme is adopted to carry out multi-point touch operation, particularly keyboard input operation, straight line shielding can be formed between operating fingers on an infrared laser light path, so that frequent false recognition and false operation are caused, the input accuracy is greatly reduced, and the interaction experience is poor.
Disclosure of Invention
The invention aims to provide an intelligent transparent office table interaction method which is used for realizing touch interaction with an intelligent terminal on a transparent desktop of an intelligent transparent office table. The intelligent transparent office table is provided with a fully transparent table top, due to the light guide characteristic of a transparent material, after the inside of the transparent table top is illuminated by an enhanced light source, when a finger contacts the upper surface of the table top, the skin of the finger of a contact part can be illuminated by the light transmitted from the inside of the table top to form a bright spot, the time and the position information of the bright spot can be recorded by means of an upward shooting image acquisition module arranged below the table top, the continuous sampling can accurately acquire the click, touch, movement and other events of the finger on the table top in real time, the events are analyzed and mapped into a control instruction of an intelligent terminal through a processing control module, and the interaction with the intelligent terminal can be realized.
The technical scheme for realizing the aim of the invention is as follows:
an intelligent transparent office table interaction method is used for realizing touch interaction with an intelligent terminal on a transparent table top of an intelligent transparent office table, the intelligent transparent office table comprises a transparent table top, an enhanced light source, an image acquisition module, a processing control module, a storage module and a communication module, the enhanced light source is used for illuminating the inside of the transparent table top, the image acquisition module is arranged below the transparent table top and can acquire images of the area where the transparent table top is located by upward shooting from the lower part of the transparent table top, the acquired images can be analyzed and processed by the processing control module, and the storage module and the communication module are both connected with the processing control module
S1: presetting touch operation actions performed on a transparent desktop, when the enhanced light source is started, if a finger contacts the upper surface of the transparent desktop, the skin of the finger in a contact area can be illuminated, forming obvious bright spots in an image collected by the image collection module from the lower part of the transparent desktop, wherein different touch operation actions correspond to different bright spot state information, the state information of the bright spots comprises the number of the bright spots and corresponding time sequence data, the position of each bright spot and corresponding time sequence data, correspondingly presetting the state information of the bright spots and the touch operation actions performed on the transparent desktop to obtain a touch operation event set mapped by the state information of the bright spots, and storing the touch operation event set and the mapping relation in the storage module;
s2: defining touch operation events on the transparent desktop to obtain a corresponding relation set of various touch operation events and control instructions, wherein the corresponding relation set is called a preset operation logic instruction set, and the preset operation logic instruction set is stored in the storage module;
s3: defining a judgment criterion of effective bright spots, wherein the judgment criterion is used for judging whether the bright spots in the region where the transparent desktop is located in the image acquired by the image acquisition module are the bright spots generated by the contact of finger tips with the transparent desktop or not;
s4: an operable area is defined on the transparent desktop, and the position and size information of the operable area is stored in the storage module, wherein the operable area is an effective area on the transparent desktop, and interactive operation can be carried out on the operable area;
s5: turning on the enhanced light source to make the light emitted by the enhanced light source enter the inside of the transparent table top and illuminate the inside of the transparent table top;
s6: the image acquisition module is set to be in a working state, the image acquisition module continuously acquires images of the area where the transparent desktop is located below the transparent desktop, and the acquired images are analyzed and processed by the processing control module;
s7: the processing control module continuously monitors the state information of effective bright spots appearing in the operable area in the image acquired by the image acquisition module;
s8: the processing control module matches the state information of the effective bright spots appearing in the operable area with the touch operation event set to determine a corresponding touch operation event;
s9: and the processing control module matches the touch operation event determined in the step S8 with a control instruction in the preset operation logic instruction set to obtain a control instruction corresponding to the current touch operation event, and the communication module transmits the control instruction to the intelligent terminal.
Further, the touch operation event set comprises a click event, a click-slide event, a slide-touch event, and a long-touch event, wherein,
the click event includes: single-finger single click, double-finger single click, three-finger single click, and four-finger single click; two continuous clicks are performed by a single finger, two continuous clicks are performed by two fingers, two continuous clicks are performed by three fingers, and two continuous clicks are performed by four fingers; three continuous clicks are performed by a single finger, three continuous clicks are performed by two fingers, three continuous clicks are performed by three fingers, and three continuous clicks are performed by four fingers; each click event comprises the information of the number of fingers participating in clicking and the information of the number of continuous clicks, and also comprises the position and the time sequence information of each click of each finger;
the click swipe event includes: the single finger slides after two continuous clicks, the double finger slides after two continuous clicks, the three finger slides after two continuous clicks, and the four finger slides after two continuous clicks; each click sliding event comprises the information of the number of fingers participating in click sliding and the information of the number of continuous clicks, and also comprises the information of the position and the time sequence of each click of each finger, the initial position of each finger sliding, the displacement and the time sequence of each finger sliding;
the sliding touch event includes: single-finger sliding, double-finger same-direction sliding, double-finger opposite-direction sliding, double-finger reverse sliding, three-finger same-direction sliding and four-finger same-direction sliding; each sliding touch event comprises the information of the number of fingers participating in sliding touch, and also comprises the information of the initial position, the direction, the displacement and the time sequence of the sliding of each finger;
the long touch event comprises: single-finger long touch, double-finger long touch, three-finger long touch, and four-finger long touch; each long touch event comprises the information of the number of fingers participating in long touch, and also comprises the position and the time sequence information of the long touch of each finger;
the control instructions include: clicking, double clicking, right clicking, dragging, cursor moving, page scrolling up, page scrolling down, middle key pressing, middle key dragging, picture zooming in, picture zooming out, switching to a previous page, switching to a next page, switching to a multitask view, switching to a desktop view, switching to a previous application, switching to a next application, turning up the volume, turning down the volume, switching to a previous song, switching to a next song, playing progress bar forwarding, playing progress bar backing, quick call-out search box, quick screen capture, full screen translation.
Further, the mapping relationship between the touch operation event set and the state information of the effective bright spots comprises
Single finger single click: a single effective bright spot appears in a certain area in the operable area, the effective bright spot disappears in a range which is less than or equal to the first time threshold value after appearing, and no single effective bright spot appears in the area in a zeroth time threshold value range after disappearing;
double finger single click: two adjacent effective bright spots appear in a certain area in the operable area at the same time, the two effective bright spots disappear within a range which is less than or equal to the first time threshold value after appearing, and the area does not have two adjacent effective bright spots appearing in the zero time threshold value range after disappearing at the same time;
three points are single click: three adjacent effective bright spots appear in a certain area in the operable area at the same time, disappear in a range which is less than or equal to the first time threshold value after the three effective bright spots appear, and the area does not have three adjacent effective bright spots in the range of the zeroth time threshold value after the three effective bright spots disappear;
four refers to a single click: four adjacent effective bright spots appear in a certain area in the operable area at the same time, the four effective bright spots disappear within a range which is less than or equal to the first time threshold value after appearing, and the four adjacent effective bright spots do not appear in the area within the range of the zeroth time threshold value after disappearing at the same time;
single finger two clicks in succession: a single effective bright spot appears in a certain area in the operable area, and the effective bright spot completely goes through four processes of appearance-disappearance-second appearance-second disappearance in the area within the range of the second time threshold value or less, and no single effective bright spot appears in the area within the range of the zeroth time threshold value after the second disappearance;
two fingers click twice in succession: two adjacent effective bright spots appear in a certain area in the operable area at the same time, and the two effective bright spots completely go through four processes of appearance-disappearance-second appearance-second disappearance in the area within the range of the second time threshold value or less, and the area does not appear two adjacent effective bright spots in the range of the zeroth time threshold value after the second disappearance;
three fingers click twice in succession: three adjacent effective bright spots appear in a certain area in the operable area at the same time, and the three effective bright spots completely go through four processes of appearance-disappearance-second appearance-second disappearance in the area within the range of the second time threshold value or less, and the three adjacent effective bright spots do not appear in the area within the range of the zeroth time threshold value after the second disappearance at the same time;
four fingers click twice in succession: a certain area in the operable area simultaneously appears four adjacent effective bright spots, and the four effective bright spots completely go through four processes of appearance-disappearance-second appearance-second disappearance in the area within the range of the second time threshold value or less, and no four adjacent effective bright spots simultaneously appear in the area within the range of the zeroth time threshold value after the second disappearance;
single finger three clicks in succession: a single effective bright spot appears in a certain area in the operable area, the effective bright spot completely undergoes six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance in the area within the range of less than or equal to a third time threshold, and no single effective bright spot appears in the area within the range of a zero time threshold after the third disappearance;
two fingers three clicks in succession: two adjacent effective bright spots appear in a certain area in the operable area at the same time, the two effective bright spots completely go through six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance in the area within the range of the third time threshold value which is less than or equal to the third time threshold value, and the two adjacent effective bright spots do not appear in the area at the same time within the range of the zero time threshold value after the third disappearance;
three fingers three consecutive clicks: three adjacent effective bright spots appear in a certain area in the operable area at the same time, the three effective bright spots completely go through six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance in the range of less than or equal to a third time threshold value, and the area does not have three adjacent effective bright spots in the range of a zero time threshold value after the third disappearance;
four fingers three consecutive clicks: a certain area in the operable area simultaneously appears four adjacent effective bright spots, the four effective bright spots completely go through six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance within the range of less than or equal to a third time threshold value, and no four adjacent effective bright spots appear in the area within the range of a zero time threshold value after the third disappearance;
sliding after two continuous clicks with a single finger: a single effective bright spot appears in a certain area in the operable area, the effective bright spot completely goes through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to a fourth time threshold, and the position of the effective bright spot continuously moves after the second appearance;
two fingers slide after two consecutive clicks: two adjacent effective bright spots appear in a certain area in the operable area at the same time, the two effective bright spots completely go through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to a fourth time threshold, and the positions of the two effective bright spots continuously move in the same direction after the two effective bright spots appear for the second time;
three fingers slide after two consecutive clicks: three adjacent effective bright spots appear in a certain area in the operable area at the same time, the three effective bright spots completely go through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to a fourth time threshold, and the positions of the three effective bright spots continuously move in the same direction after the three effective bright spots appear for the second time;
four fingers slide after two consecutive clicks: four adjacent effective bright spots appear in a certain area in the operable area at the same time, the four effective bright spots completely go through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to a fourth time threshold, and the positions of the four effective bright spots continuously move in the same direction after the four effective bright spots appear for the second time;
single-finger sliding: a single effective bright spot appears in a certain area in the operable area, and the position of the effective bright spot continuously moves after the effective bright spot appears;
the two fingers slide in the same direction: two adjacent effective bright spots appear in a certain area in the operable area at the same time, and the positions of the two effective bright spots continuously move in the same direction after appearing;
the two fingers slide towards each other: two effective bright spots appear in the operable area at the same time, and after the two effective bright spots appear, the relative positions of the two effective bright spots generate opposite continuous movement, wherein the opposite continuous movement means that the two effective bright spots are close to each other;
the two fingers slide reversely: two effective bright spots appear in the operable area at the same time, and the relative positions of the two effective bright spots are reversely and continuously moved after the two effective bright spots appear, wherein the reverse and continuous movement means that the two effective bright spots are far away from each other;
the three fingers slide in the same direction: three adjacent effective bright spots appear in a certain area in the operable area at the same time, and the positions of the three effective bright spots continuously move in the same direction after appearing;
the four fingers slide in the same direction: four adjacent effective bright spots appear in a certain area in the operable area at the same time, and the positions of the four effective bright spots continuously move in the same direction after appearing;
single finger long touch: a single effective bright spot appears in a certain area in the operable area, and the position of the effective bright spot does not move or disappear within the range of less than or equal to a fifth time threshold after the effective bright spot appears;
double finger length touch: two effective bright spots appear in a certain area in the operable area at the same time, and the positions of the two effective bright spots do not move or disappear within the range of less than or equal to the fifth time threshold after the two effective bright spots appear;
three-finger long touch: three effective bright spots appear in a certain area in the operable area at the same time, and the positions of the three effective bright spots do not move or disappear within the range of less than or equal to the fifth time threshold after the three effective bright spots appear;
four fingers long touch: and a certain area in the operable area simultaneously appears with four effective bright spots, and the positions of the four effective bright spots do not move or disappear within the range of less than or equal to the fifth time threshold after the four effective bright spots appear.
Preferably, the mapping relationship between the control instruction and the touch operation event in the preset operation logic instruction set is
Clicking: single finger single click;
double clicking: clicking twice continuously by a single finger;
a right key: double fingers click once;
dragging: the single finger slides after two continuous clicks;
cursor movement: sliding by a single finger;
the page scrolls upwards: the two fingers slide forwards in the same direction;
the page scrolls down: the two fingers slide backwards in the same direction;
pressing a middle key: three fingers click twice continuously;
middle key dragging: three fingers slide after two continuous clicks;
and (3) amplifying the picture: the two fingers slide reversely;
and (3) reducing the picture: the two fingers slide in opposite directions;
switching to the previous page: the right fingers slide in the same direction;
switch to the next page: sliding towards the left with the two fingers in the same direction;
switching to a multitask view: the three fingers slide forwards in the same direction;
switching to a desktop view: sliding backwards in the same direction with the three fingers;
switch to the previous application: sliding the three fingers to the right in the same direction;
switch to the next application: sliding towards the left three fingers in the same direction;
turning up the volume: the four fingers slide forwards in the same direction;
turning down the volume: the four fingers slide backwards in the same direction;
switching to the previous track: sliding the four fingers to the right in the same direction;
switching to the next track: sliding towards the left four fingers in the same direction;
the play progress bar advances: four fingers slide to the right after clicking twice continuously;
the playing progress bar retreats: after clicking twice continuously, the four fingers interact leftwards;
quick call search box: double-finger long touch;
quick screen capture: three fingers touch long;
full screen translation: four fingers long touch.
Preferably, the enhanced light source is an infrared light source, and the image acquisition module is a narrow-band infrared camera module corresponding to the wavelength of the infrared light source. The enhancement light source adopts infrared light, which has the advantages that the infrared light is invisible to naked eyes but can be collected and captured by an infrared camera, so that the requirements of the invention on the enhancement light source can be met, and the external environment cannot be polluted by light. The narrow-band infrared camera has the advantage of being free from interference of other ambient light sources.
When the enhanced light source is turned on, if a finger touches the upper surface of the transparent table top, the skin of the finger in the contact area is illuminated, and a remarkable bright spot is formed in an image collected by the image collecting module from the lower part of the transparent table top. It should be noted that the illumination of the finger skin of the contact area refers to the illumination in a physical sense, not the illumination seen by the naked eye, because if the enhancing light source is an invisible infrared light source, the bright spot is not seen by the naked eye, but the bright spot can be captured and photographed by the infrared camera.
Further, the criterion for determining the effective bright spots is as follows:
given two radii R1 and R2, the hot spot is determined to be a valid hot spot when the size of the hot spot cannot be completely surrounded by a small circle of radius R1, while it can be completely surrounded by a large circle of radius R2. Preferably, the radius R1 of the small circle is 2mm, and the radius R2 of the large circle is 10 mm.
Preferably, in the state information of the bright spot, the position of the bright spot refers to a position coordinate of a centroid of the bright spot. Because the bright spot is a pattern composed of many pixels in the image acquired by the image acquisition module, and has both size and shape, but in the actual algorithm development, a coordinate point is needed to describe the specific position of the bright spot pattern having both size and shape, in the visual recognition algorithm, a commonly used method is to select the coordinates of the centroid of the pattern to refer to the current position of the pattern, because the centroid of the pattern is usually considered as the center position of the pattern. By taking the method as a reference, the invention also adopts the position coordinates of the center of mass of the bright spot to indicate the position of the bright spot.
Further, the present invention also includes a keyboard pattern disposed in the operable area on the transparent desktop, the keyboard pattern including a plurality of key patterns, the method including key event detection and interaction, the key event detection and interaction including the steps of:
coordinate coding is carried out on a position area corresponding to each key pattern on the keyboard pattern, and the position area is stored in the storage module after being mapped with key instructions of each key one by one;
the processing control module continuously monitors the state information of effective bright spots appearing in the operable area in the image acquired by the image acquisition module;
when a user carries out keyboard input operation, a certain key pattern on a keyboard pattern on a desktop is clicked by fingers, bright spots are generated at the position of the key pattern in an image collected by the image collection module, the processing control module determines that the position of an effective bright spot appears in the key pattern of the keyboard pattern through coordinate code matching, the effective bright spot disappears within a time range which is less than or equal to a sixth time threshold value after the effective bright spot appears, the effective bright spot is regarded as a key instruction corresponding to the key pattern, and the processing control module transmits the key instruction to the intelligent terminal through the communication module.
Further, the invention also includes a projection module and a transparent reflective film, the projection module is disposed under the transparent desktop, the transparent reflective film is disposed on the transparent desktop, the projection module can project multimedia pictures on the transparent reflective film of the transparent desktop, the method includes projection interaction, and the projection interaction includes the following steps:
connecting an intelligent terminal for providing multimedia pictures, so that the projection module can project the multimedia pictures needing interactive operation;
adjusting projection parameters to enable the multimedia picture projected by the projection module to be clearly imaged on the transparent reflecting film of the transparent desktop, wherein the projection imaging area is positioned in the operable area;
storing the position and size information of the projection imaging area of the projection module in the storage module;
defining the touch operation event in the projection imaging area to obtain a corresponding relation set of the touch operation event in the projection imaging area and a control instruction for the projected multimedia picture, wherein the corresponding relation set is called a preset projection interactive logic instruction set;
the processing control module monitors a touch operation event occurring in the projection imaging area by monitoring state information of effective bright spots in the projection imaging area, matches the touch operation event with a control instruction in the preset projection interaction logic instruction set to obtain a control instruction corresponding to the current touch operation event, and the communication module transmits the control instruction to an intelligent terminal providing a multimedia picture.
In practical tests, it is found that when a multimedia picture is projected on a transparent reflective film of a transparent desktop, and a finger touches the upper surface of the transparent desktop due to light interference of the projected picture, bright spots observed below the transparent desktop become less noticeable, but the problem can be solved by brightening the brightness of the enhanced light source, and when the brightness of the enhanced light source is bright enough, the bright spots become more noticeable and easily recognizable. In addition, due to the constraint effect of the transparent table top on the total internal reflection of the light inside the transparent table top, even if the brightness of the enhanced light source is high, the external environment is not polluted basically. Of course, if the enhanced light source is an infrared light source and the image acquisition module is a narrow-band infrared camera module with a corresponding wavelength, the light of the projection picture basically does not affect the significance of the bright spots, because the light of the projection picture is basically visible light, and the visible light on the narrow-band infrared camera module is cut off and is not sensitive.
The invention has the beneficial effects that:
(1) the invention adopts the reason and the principle of enhancing the light source and has the corresponding beneficial effects that:
the invention aims to realize interaction with an intelligent terminal by clicking, touching, moving and other operations on a desktop, and firstly solves the problem of determining whether a finger or other touch equipment is in contact with the desktop, the position coordinate of a contact point on the desktop and whether the contact point is in continuous movement. In order to solve the problem, the invention adopts a method of image visual identification: the desktop is made of transparent materials, and the upward shooting image acquisition module is arranged below the transparent desktop, so that images of the transparent desktop and objects above the transparent desktop can be shot and acquired through the transparent desktop, and the contact state of fingers or other touch equipment and the transparent desktop is judged through the acquired images;
however, experimental tests show that the contact state is very difficult to identify, for example, when a finger contacts the upper surface of the transparent desktop, in a reverse shooting angle, the difference between two states, that the finger does not contact the transparent desktop and the finger has contacted the transparent desktop, is very small in the acquired image, and sufficient contrast, color difference and brightness difference are lacked, so that the contact state between the finger and the transparent desktop is difficult to determine during image identification processing, and especially when the ambient light is complex, the identification difficulty is further increased. Therefore, the scheme of only using the image acquisition module to shoot the transparent desktop upwards cannot realize accurate and reliable contact state identification;
through continuous research and study of the inventor, the problem of identification of the contact state can be well solved if the enhanced light source is introduced to enable the light emitted by the enhanced light source to illuminate the interior of the transparent desktop. The principle is as follows:
the optical characteristics of the transparent material are utilized, for example, when light is transmitted in a piece of transparent glass, since the glass is an optically dense medium and air is an optically sparse medium, if the incident angle of light emitted from the inside of the glass to the outside air is greater than the critical angle of total reflection, total internal reflection occurs, and the light is bound in the glass and cannot escape, and can only be transmitted along the inside of the glass. However, the inventor has found that if there is another object touching the outer surface of the glass, the condition for total internal reflection is no longer true and light will be transmitted from the touch area to illuminate the object touching the glass, since the propagation medium pair of light in the touch area changes from a "glass-air" medium pair to a "glass-touching object" medium pair. The inventors have experimentally verified that when a light beam is incident into a glass from the side of a flat glass placed horizontally, the light propagates along the inside of the glass without escaping from the upper and lower surfaces of the glass, but when the upper surface of the glass is touched with a finger, the skin of the finger touching the glass is illuminated brightly by the light transmitted from the inside of the glass, and a very significant bright spot is formed. The phenomenon discovered by the inventor is utilized, when the incident direction of the enhanced light source is proper, the light emitted by the enhanced light source is constrained inside the transparent desktop, when a finger clicks, touches, moves and the like on the transparent desktop, as long as the finger contacts the upper surface of the transparent desktop, the skin of the finger contacting the transparent desktop is illuminated to form a very obvious bright spot, and the obvious bright spot is very easy to identify during image processing and is not easy to generate identification misjudgment. In addition, the bright spot features are remarkable, so that the image recognition is not easily interfered by ambient light, and the method has very high recognition reliability and accuracy;
the multi-point touch recognition is easy to carry out by utilizing the principle, when a plurality of bright spots appear at different positions at the same time, the time and the position of each bright spot are recorded, the multi-point touch recognition can be realized, and if the bright spots move continuously, the continuously moving tracks and the continuously moving displacement of the bright spots can be tracked, so that the touch movement recognition is realized. In addition, because the shape of the contact area generated by the contact of the finger tip and the transparent desktop has obvious characteristics, and the contact area is also within a certain range, if the shape or the area of the sampled bright spot obviously does not conform to the shape or the area of the contact area generated by the contact of the finger tip and the transparent desktop, for example, the shape of the bright spot is too long and narrow and irregular, or the area is too large, the bright spot can be judged to be not generated by the contact of the finger tip and the transparent desktop through an effective bright spot judgment criterion, and does not belong to effective interactive information, and the bright spot can be directly filtered. Therefore, when other parts of the hand of the user except the fingertips, such as the palm, the side surfaces of the palm, the wrist and the like, are contacted with the transparent desktop, the normal interactive operation is not interfered;
(2) other beneficial effects of the invention are as follows:
1) the invention provides a method and interaction logic for realizing touch and display interaction on the transparent desktop of the intelligent transparent office desk, and has higher application value;
2) the touch interaction logic is rich, the functions are complete, and multi-point touch is supported;
3) the method can be applied to large-size desktops, and the corresponding cost is not obviously increased along with the increase of the touchable operation area of the desktops;
4) interactive operation on a fully transparent desktop looks cooler and has stronger science and fiction;
5) after a projection interaction function is added, display and interaction similar to a touch screen can be realized on the transparent desktop;
6) if the enhanced light source is an infrared light source and the image acquisition module is a narrow-band infrared camera module with corresponding wavelength, the recognition of bright spots generated by the contact of fingers with the transparent desktop has strong anti-ambient light interference capability and cannot be interfered by light of a projection picture during projection interaction.
It should be noted that the beneficial effects of the present invention are not limited to the above description, and can be understood by combining with the specific technical solutions, and the description of the technical effects and beneficial effects of a specific technical solution is also inserted in the summary of the present invention and the following embodiments.
Drawings
FIG. 1 is a three-dimensional schematic view of an intelligent transparent desk, wherein the direction mark at the upper right corner of the diagram designates all directions mentioned in the present invention, and further, if the direction mark is "top" or "top", the "top" is the same direction as the "up" in the direction mark; if "bottom" or "bottom" is present, the "bottom" is in the same direction as "down" in the orientation indicator.
Fig. 2 is a three-dimensional schematic view of a transparent desktop of an intelligent transparent office desk separated from a supporting part.
Fig. 3 is a schematic diagram of an intelligent transparent desk displaying the projected imaging area.
Detailed Description
The present invention will be further described and illustrated in detail below with reference to examples, embodiments and the accompanying drawings, wherein the examples or embodiments are only a part of the examples or embodiments of the present invention, and not all examples or embodiments. All other embodiments or implementations obtained by a person of ordinary skill in the art based on the embodiments or implementations of the present invention without any creative effort belong to the protection scope of the present invention.
The following description of the examples or embodiments of the invention is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.
The implementation of the invention is based on the physical hardware of the intelligent transparent office table, the intelligent transparent office table is used for realizing touch interaction with an intelligent terminal, and the intelligent terminal can be independent external intelligent equipment such as a desktop computer, a notebook computer, an intelligent mobile phone, a tablet personal computer, an intelligent television, an intelligent watch, an intelligent projector, intelligent glasses with a display function and the like, and can also be intelligent equipment which is integrated in the intelligent transparent office table and has interactive operation requirements.
As shown in fig. 1-3, the intelligent transparent office desk comprises a supporting part, a transparent desktop 3, an enhanced light source 5, an image acquisition module, a processing control module, a storage module, a communication module and a power supply module, the supporting part is used for supporting the transparent desktop 3 and comprises a base 1, a supporting frame 7 and a desktop force bearing frame 4, the tabletop 3 is arranged on the tabletop force bearing frame 4, the enhanced light source 5 is used for illuminating the inside of the transparent tabletop 3, the image acquisition module is arranged below the transparent desktop 3 and can shoot and acquire the image of the area where the transparent desktop 3 is positioned from the lower part of the transparent desktop, the collected images can be analyzed and processed by the processing control module, the storage module is connected with the processing control module, the communication module is connected with the processing control module, and the power supply module is used for providing a power supply.
Transparent desktop implementation
The transparent desktop 3 is made of transparent materials. It should be noted that the transparent material does not need to be transparent in a strict sense, and the color thereof is not limited, and the transparent material can be regarded as the transparent material in the present invention as long as the material has a light transmission capability. Of course, the higher the light transmittance, the closer to the colorless and transparent material, the more suitable it is for making the transparent table top 3. Transparent glass, transparent resin, transparent plastic and other common natural transparent materials and artificial transparent materials which are commonly used in daily life can be used for manufacturing the transparent table top 3.
The material of the transparent desktop 3 is preferably transparent glass. Among common transparent materials, glass has the characteristics of high hardness and high light transmittance, and is not easy to scratch and damage in the using process, so that the durability of the transparent table top 3 can be enhanced by adopting the transparent glass, and the transparent table top is easy to clean. In order to enhance the mechanical property of the transparent desktop 3, toughened transparent glass can be adopted, and if the toughness and strength are further enhanced, laminated transparent toughened glass can be adopted. If the requirements on the smooth degree of the sliding operation of the finger on the transparent desktop 3, the difficulty in sticking the fingerprint, the convenience in cleaning and the like are met, the surface of the transparent glass can be coated with an oil-thinning layer, coated with a film or subjected to other surface treatment.
Embodiments of operable area of desktop
The operable area 306 is an effective area on the transparent desktop 3, where interaction can be performed, and events such as clicking, touching, moving and the like occurring in the area can be responded, otherwise, the events cannot be responded if the area is exceeded. The setting of the operable area 306 mainly considers that the image capturing range of the image capturing module and the shielding of the image capturing module by the obstacle formed by the supporting portion, the desktop area beyond the image capturing range of the image capturing module or where the image capturing cannot be captured due to the shielding of the obstacle should be excluded from the operable area 306. When the acquisition range of the image acquisition module includes the whole transparent desktop 3 and the view field of the image acquisition module is not blocked by the obstacle, the whole transparent desktop 3 can be set as the operable area.
Enhanced light source embodiment
As shown in fig. 1-3, the enhanced light source 5 is used to illuminate the interior of the transparent table top 3. It should be noted that the light emitted by the enhanced light source 5 may be visible light, may also be invisible light, may also be white light, and may also be colored light. The invention does not limit the kind, color, wavelength and frequency of the enhanced light source 5, so that the simple change of the kind, color, wavelength and frequency of the enhanced light source 5 on the basis of the invention should fall into the protection scope of the invention.
Regarding the arrangement of the enhancement light source 5: the purpose of the enhancing light source 5 is to illuminate the entire transparent table top 3 with as much light as possible from the enhancing light source 5 and to minimize light pollution to the environment outside the transparent table top 3, i.e., the light from the enhancing light source 5 should be confined to the inside of the transparent table top 3 as much as possible. Of course, if the enhancement light source 5 is invisible light, the problem of light pollution by the enhancement light source 5 is not considered. In the following, taking the enhancement light source 5 as the visible light as an example, an embodiment of the arrangement of the enhancement light source 5 is provided:
as shown in fig. 2, the transparent table top 3 is thin and includes a top surface 305, a bottom surface 307, a first side 302, a second side 303, a third side 301, and a fourth side 304, and has a rectangular shape with rounded corners in horizontal cross section. The enhanced light source 5 is a strip-shaped light source, the strip-shaped enhanced light source 5 is arranged at the first side surface 302 of the transparent desktop 3 and is tightly attached to the first side surface 302 of the transparent desktop 3, the length direction of the strip-shaped enhanced light source is parallel to the first side surface 302 of the transparent desktop 3, and light emitted by the strip-shaped enhanced light source is perpendicular to the first side surface 302 of the transparent desktop 3 and enters the inside of the transparent desktop 3. The embodiment of the enhanced light source 5 is considered to be relatively better by the inventor and is also a preferred embodiment of the present invention, because the enhanced light source 5 is arranged in a manner that the inside of the transparent table top 3 can be ensured to be illuminated, and the size of the elongated enhanced light source 5 can be controlled to ensure that the light emitted by the enhanced light source 5 can be totally entered into the inside of the transparent table top 3, so that the light pollution to the environment outside the transparent table top 3 is reduced, and the interference of the background light caused by the light pollution during the image processing is also reduced. Furthermore, the second side 303, the third side 301, and the fourth side 304 of the transparent desktop 3 are all processed to be opaque, so that light inside the transparent desktop 3 cannot be emitted through the second side 303, the third side 301, and the fourth side 304, thereby further reducing light pollution to the environment outside the transparent desktop 3.
If the enhancement light source 5 is invisible light, such as infrared light, reference can also be made to the above-described embodiment of the enhancement light source, but since the enhancement light source is invisible light, there is no need to consider the problem of light pollution.
Image acquisition Module implementation
The image acquisition module is arranged below the transparent desktop 3, can shoot and acquire the image of the area of the lower surface of the transparent desktop 3, and the acquired image can be analyzed and processed by the processing control module; the image acquisition module can be a common camera, a special CMOS photosensitive module or other photosensitive components capable of realizing image acquisition. It should be noted that the type of the light sensing module should match the type of the enhanced light source 5, for example, if the light emitted by the enhanced light source 5 is visible light, the image capturing module should at least be capable of sensing and capturing visible light, and if the light emitted by the enhanced light source 5 is infrared light, the image capturing module should at least be capable of sensing and capturing infrared light. The invention does not limit the specific type of the image acquisition module and the adopted image acquisition technology, and the simple change of the type of the image acquisition module and the adopted image acquisition technology on the basis of the invention is all within the protection scope of the invention.
The invention relates to an enhanced light source 5 and an image acquisition module, wherein the enhanced light source 5 is an infrared light source, and the image acquisition module is an infrared camera module. The wavelength of the infrared light source can be selected from common 850nm, 910nm, 940nm, 980nm and the like, and the sampling wavelength of the infrared camera module is matched with the wavelength of the infrared light source. For example, the light emitted by the infrared light source is infrared light of 850nm, the infrared camera module should be capable of photosensitive collecting infrared light of at least 850nm and wavelengths near the infrared light source, and if the interference of the ambient light source is to be filtered, the infrared camera module may also be a narrow-band infrared camera of 850nm, that is, the infrared camera module only collects infrared light of a waveband near 850nm, and is non-photosensitive to visible light and other light of wavelengths far from 850 nm. Similarly, if the infrared light source is an infrared light source with other wavelengths, the infrared camera module may employ a narrow-band infrared camera with a corresponding wavelength in order to filter the interference of the ambient light source. The enhancement light source 5 adopts infrared light, which has the advantages that the infrared light is invisible to naked eyes but can be collected and captured by an infrared camera, so that the requirements of the invention on the enhancement light source can be met, and the external environment cannot be polluted by light. The narrow-band infrared camera has the advantages that the narrow-band infrared camera is not interfered by other light sources, the accuracy of bright spot identification can be improved, and the identification difficulty is reduced. It should be noted that the above conclusions about the infrared light source and the infrared camera module have been verified in experimental tests of the inventor.
Preferably, as shown in fig. 1 to 3, the image capturing module may be a camera 2 mounted at the lower end of a support 7 of the support, and the shooting direction of the camera is toward the area where the lower surface of the transparent table top 3 is located. The specific installation height and installation position of the image acquisition module need to be set according to the actual image sampling requirement and the parameters of the image acquisition module, such as focal length, resolution, aperture and the like. It is necessary to ensure that the image capturing module can capture an upward shot image of the area where the transparent desktop 3 is located, which is clear enough and is not exposed excessively, after being installed and debugged, and the set operable area 306 on the transparent desktop 3 should be completely present in the captured image frame.
Processing control Module implementation
The processing control module at least has the capability of image processing and analysis, can analyze and process the image acquired by the image acquisition module and can transmit a corresponding result to the intelligent terminal needing interaction through the communication module. The processing control module can be a general-purpose chip, such as a Central Processing Unit (CPU), a microprocessor MCU and the like, or a special image processing chip. The invention is not limited to the type of the process control module, and the simple change of the type of the process control module based on the invention is all within the protection scope of the invention.
Storage module implementation
The storage module may be a ROM (read only memory), a RAM (random access memory), a nonvolatile memory such as a hard disk, or the like. The storage module may be a memory independent from the process control module, or may be integrated on the same chip with the process control module. The invention does not limit the type of the memory module, and the type, the packaging mode and the integration mode of the memory module are simply changed on the basis of the invention, which all fall into the protection scope of the invention.
Communication module implementation
The communication module is connected with the processing control module and is used for enabling the intelligent transparent office desk to be in interactive communication with the intelligent terminal. The communication module can be a wireless communication module, such as a bluetooth module, a WiFi module, and the like, and can also be a wired communication module. The communication module can be an independent module independent from the processing control module, and can also be integrated with the processing control module on the same chip. The invention is not limited to the specific type of the communication module and the communication technology adopted by the communication module. The invention is not limited to the above embodiments, and the communication module may be a module with a communication module.
Power supply Module implementation
The power supply module is used for providing power. The power supply module can be a battery or a battery module with electricity storage capacity, and can also supply power to the intelligent transparent office table through an external power supply.
Keyboard pattern implementation mode
The keyboard pattern 6 of the present invention is arranged in the operable area 306 of the transparent desktop 3, and the keyboard pattern 6 can be arranged on the desktop upper surface 305, the desktop lower surface 307, or embedded in the transparent desktop 3 in the production process. The keyboard pattern 6 is preferably a keyboard line frame pattern, i.e. only the border of each key and the corresponding identifier (e.g. letter, number, symbol, etc.) of the key are present in the pattern, while the area inside each key other than the identifier of the key is not filled and remains transparent. The keyboard pattern 6 can be set by printing, etching, engraving, laser engraving and the like when the transparent desktop 3 is produced, or a transparent sticker with the keyboard pattern can be pasted in the operable area 306 on the transparent desktop 3, or a transferable sticker with the keyboard pattern can be pasted in the operable area 306 on the transparent desktop 3, then the base material of the sticker is torn off, and the keyboard line frame pattern which is transferred on the transparent desktop 3 is reserved. In addition, it should be noted that the keyboard pattern according to the present invention may be a virtually projected keyboard pattern, that is, the keyboard pattern 6 is projected in the operable area 306 on the transparent desktop 3, besides the above-mentioned physically visible keyboard pattern 6.
Interactive implementation mode for keyboard input
And carrying out coordinate coding on the position area corresponding to each key pattern on the keyboard pattern 6, mapping the position area with the key instruction of each key one by one, and storing the position area in the storage module. The processing control module continuously monitors the state information of effective bright spots appearing in the operable area 306 in the image acquired by the image acquisition module;
when a user carries out keyboard input operation, a certain key pattern on the keyboard pattern 6 of the transparent desktop 3 is clicked by a finger, bright spots are generated at the position of the key pattern in an image collected by the image collection module, the processing control module determines that the position of an effective bright spot appears in the key pattern of the keyboard pattern 6 through coordinate code matching, the effective bright spot disappears within a time range which is less than or equal to a sixth time threshold value after the effective bright spot appears, the effective bright spot is regarded as triggering a key instruction corresponding to the key pattern, and the processing control module transmits the key instruction to the intelligent terminal through the communication module, namely, one-time keyboard input interaction is realized;
it should be noted that, the sixth time threshold is selected and adjusted according to the actual required key input feedback speed and sensitivity, which is not limited by the present invention. Preferably, the sixth time threshold is 100 milliseconds.
Mouse and touch gesture interactive implementation mode
And performing mouse and touch gesture interactive operation on the transparent desktop 3, wherein the operation logic can refer to the gesture operation of the notebook computer touch pad. Common touch gestures, that is, touch operation events in the present invention, are classified according to the number of fingers participating in touch operation, the number of clicks, whether to slide, whether to touch for a long time, and mainly include four categories: a click event, a click slide event, a slide touch event, a long touch event, wherein,
the click event includes: single-finger single click, double-finger single click, three-finger single click, and four-finger single click; two continuous clicks are performed by a single finger, two continuous clicks are performed by two fingers, two continuous clicks are performed by three fingers, and two continuous clicks are performed by four fingers; three continuous clicks are performed by a single finger, three continuous clicks are performed by two fingers, three continuous clicks are performed by three fingers, and three continuous clicks are performed by four fingers; each click event comprises the information of the number of fingers participating in clicking and the information of the number of continuous clicks, and also comprises the position and the time sequence information of each click of each finger;
the click swipe event includes: the single finger slides after two continuous clicks, the double finger slides after two continuous clicks, the three finger slides after two continuous clicks, and the four finger slides after two continuous clicks; each click sliding event comprises the information of the number of fingers participating in click sliding and the information of the number of continuous clicks, and also comprises the information of the position and the time sequence of each click of each finger, the initial position of each finger sliding, the displacement and the time sequence of each finger sliding;
the sliding touch event includes: single-finger sliding, double-finger same-direction sliding, double-finger opposite-direction sliding, double-finger reverse sliding, three-finger same-direction sliding and four-finger same-direction sliding; each sliding touch event comprises the information of the number of fingers participating in sliding touch, and also comprises the information of the initial position, the direction, the displacement and the time sequence of the sliding of each finger;
the long touch event comprises: single-finger long touch, double-finger long touch, three-finger long touch, and four-finger long touch; each long touch event comprises the information of the number of fingers participating in long touch, and also comprises the position and the time sequence information of the long touch of each finger;
common mouse and touch gesture control commands include: clicking, double clicking, right clicking, dragging, cursor moving, page up scrolling, page down scrolling, middle key pressing, middle key dragging, picture enlarging, picture reducing, switching to the previous page, switching to the next page, switching to a multitask view, switching to a desktop view, switching to the previous application, switching to the next application, turning up the volume, turning down the volume, switching to the previous song, switching to the next song, advancing the play progress bar, backing the play progress bar, quickly calling out a search box, quickly screen capturing and full screen translating;
and performing logic definition on the touch operation events on the transparent desktop 3 to obtain a corresponding relation set of various touch operation events and control instructions, wherein the corresponding relation set is called a preset operation logic instruction set. Storing the preset operation logic instruction set in the storage module;
when the enhanced light source 5 is turned on, if a finger contacts the upper surface of the transparent desktop 3, the skin of the finger in the contact area is illuminated, a significant bright spot is formed in an image collected by the image collection module from the lower part of the transparent desktop 3, different touch operation actions correspond to different bright spot state information, the state information of the bright spot comprises the number of the bright spots and corresponding time sequence data, the position of each bright spot and corresponding time sequence data, the state information of the bright spot and the touch operation action performed on the transparent desktop 3 are correspondingly preset, a touch operation event set mapped by the state information of the bright spot is obtained, and the touch operation event set and the mapping relation are stored in the storage module;
when the user performs mouse or touch gesture interactive operation on the transparent desktop 3, the image acquisition module acquires the image through continuous sampling, single or multiple bright spots generated by a single finger or multiple fingers in the operable area 306 of the transparent desktop 3 and the movement data of the bright spots can be collected, the processing control module analyzes and processes the data of the bright spots and determines the data as effective bright spots, judging which touch operation event the touch operation is carried out belongs to according to the mapping relation between the touch operation event and the state information of the effective bright spots, and determining a control instruction corresponding to the touch operation event according to the preset operation logic instruction set, the control instruction is transmitted to the controlled intelligent terminal through the communication module, and the intelligent terminal receives the control instruction and then carries out corresponding response, namely, the interaction of a mouse or a touch gesture is realized.
Position coordinate implementation of bright spot
The bright spot is a pattern composed of a plurality of pixel points in the image acquired by the image acquisition module, and the pattern has both size and shape, but a coordinate point is required to describe the specific position of the bright spot pattern with both size and shape in the actual algorithm development. In many vision recognition algorithms, one common approach is to select the coordinates of the pattern's centroid to refer to where the pattern is currently located, since the pattern's centroid is usually considered to be the center location of the pattern. By taking this method as a reference, the present invention also uses the position coordinates of the center of mass of the bright spot to describe the position of the bright spot. Regarding the method for calculating the centroid of the pattern or image, since there are many mature algorithms, which belong to the prior art, the present invention is not described in detail.
Implementation of effective hot spot determination criterion
According to the characteristics of the shape and the size of the bright spots generated by the contact of the fingers of the ordinary people and the upper surface of the transparent desktop 3, the invention provides an implementation mode for judging whether the bright spots are effective bright spots, which comprises the following specific steps:
given two radii R1 and R2, the hot spot is determined to be a valid hot spot when the size of the hot spot cannot be completely surrounded by a small circle of radius R1, while it can be completely surrounded by a large circle of radius R2. Preferably, the radius R1 of the small circle is 2mm, and the radius R2 of the large circle is 10 mm.
In practical application, if the bright spot continuously moves, preferably, a dynamic target tracking algorithm may be used to track the position of the bright spot, and it is not necessary to perform effective bright spot determination on each sampling frame of the continuously moving bright spot, so that the time overhead of processing and analyzing by the processing control module may be saved, the feedback speed is increased, and the delay is reduced. The algorithm for tracking the dynamic target is a mature prior art in the field of image recognition and dynamic target tracking, and a plurality of relatively efficient algorithms are available, which are not described herein again.
Implementation manner of mapping relation between touch operation event and state information of effective hot spot
Preferably, as shown in fig. 1 and fig. 2, the present invention provides an embodiment of a mapping set between a touch operation event and the state information of the effective hot spot, where the mapping set includes
Single finger single click: a single effective bright spot appears in a certain area in the operable area 306, the effective bright spot disappears within a range which is less than or equal to the first time threshold after appearing, and no single effective bright spot appears in the area within the range of the zeroth time threshold after disappearing;
double finger single click: two adjacent effective bright spots appear in a certain area in the operable area 306 at the same time, the two effective bright spots disappear within a range which is less than or equal to the first time threshold value after appearing, and no two adjacent effective bright spots appear in the area within the range of the zeroth time threshold value after disappearing at the same time;
three points are single click: three adjacent effective bright spots appear in a certain area in the operable area 306 at the same time, the three effective bright spots disappear within a range which is less than or equal to the first time threshold value after appearing, and the area does not have three adjacent effective bright spots appearing in the range of the zeroth time threshold value after disappearing at the same time;
four refers to a single click: four adjacent effective bright spots appear in a certain area in the operable area 306 at the same time, the four effective bright spots disappear within a range which is less than or equal to the first time threshold value after appearing, and no four adjacent effective bright spots appear in the area within the range of the zeroth time threshold value after disappearing at the same time;
single finger two clicks in succession: a single effective bright spot appears in a certain area in the operable area 306, and the effective bright spot completely goes through four processes of appearance-disappearance-second appearance-second disappearance in the area within a range of less than or equal to a second time threshold, and no single effective bright spot appears in the area within a zero time threshold after the second disappearance;
two fingers click twice in succession: two adjacent effective bright spots appear in a certain area in the operable area 306 at the same time, and the two effective bright spots completely go through four processes of appearance-disappearance-second appearance-second disappearance in the area within the range of the second time threshold value or less, and no two adjacent effective bright spots appear in the area within the range of the zeroth time threshold value after the second disappearance;
three fingers click twice in succession: a certain area in the operable area 306 simultaneously appears three adjacent effective bright spots, and the three effective bright spots completely go through four processes of appearance-disappearance-second appearance-second disappearance within a range of a second time threshold value which is less than or equal to the second time threshold value, and the area does not simultaneously appear three adjacent effective bright spots within a range of a zeroth time threshold value after the second disappearance;
four fingers click twice in succession: a certain area in the operable area 306 simultaneously appears four adjacent effective bright spots, and the four effective bright spots completely go through four processes of appearance-disappearance-second appearance-second disappearance within a range less than or equal to a second time threshold value, and no four adjacent effective bright spots appear in the area within a zero time threshold value range after the second disappearance;
single finger three clicks in succession: a single effective bright spot appears in a certain area in the operable area 306, and the effective bright spot completely undergoes six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance in the area within the range of the time threshold less than or equal to the third time threshold, and no single effective bright spot appears in the area within the range of the time threshold zero after the third disappearance;
two fingers three clicks in succession: a certain area in the operable area 306 simultaneously appears two adjacent effective bright spots, the two effective bright spots completely go through six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance in the area within the range of the third time threshold value, and the area does not simultaneously appear two adjacent effective bright spots again within the range of the zero time threshold value after the third disappearance;
three fingers three consecutive clicks: a certain area in the operable area 306 simultaneously appears three adjacent effective bright spots, the three effective bright spots completely go through six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance within a range of less than or equal to a third time threshold value, and the area does not simultaneously appear three adjacent effective bright spots within a range of a zero time threshold value after the third disappearance;
four fingers three consecutive clicks: a certain area in the operable area 306 simultaneously appears four adjacent effective bright spots, the four effective bright spots completely go through six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance within a range of less than or equal to a third time threshold value, and no four adjacent effective bright spots appear in the area within a range of a zero time threshold value after the third disappearance;
sliding after two continuous clicks with a single finger: a single effective bright spot appears in a certain area in the operable area 306, the effective bright spot completely goes through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to the fourth time threshold, and the position of the effective bright spot continuously moves after the second appearance;
two fingers slide after two consecutive clicks: two adjacent effective bright spots appear in a certain area in the operable area 306 at the same time, the two effective bright spots completely go through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to the fourth time threshold, and the positions of the two effective bright spots continuously move in the same direction after the second appearance;
three fingers slide after two consecutive clicks: a certain area in the operable area 306 simultaneously appears three adjacent effective bright spots, the three effective bright spots completely go through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to the fourth time threshold, and the positions of the three effective bright spots continuously move in the same direction after the second appearance;
four fingers slide after two consecutive clicks: a certain area in the operable area 306 simultaneously appears four adjacent effective bright spots, the four effective bright spots completely go through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to the fourth time threshold, and the positions of the four effective bright spots continuously move in the same direction after the second appearance;
single-finger sliding: a single effective bright spot appears in a certain area in the operable area 306, and the position of the effective bright spot continuously moves after the effective bright spot appears;
the two fingers slide in the same direction: two adjacent effective bright spots appear in a certain area in the operable area 306 at the same time, and the positions of the two effective bright spots continuously move in the same direction after appearing;
the two fingers slide towards each other: two effective bright spots appear in the operable area 306 at the same time, and after the two effective bright spots appear, the relative positions of the two effective bright spots continuously move in opposite directions, that is, the two effective bright spots are close to each other;
the two fingers slide reversely: two effective bright spots appear in the operable area 306 at the same time, and the relative positions of the two effective bright spots are reversely and continuously moved after the two effective bright spots appear, that is, the two effective bright spots are far away from each other;
the three fingers slide in the same direction: three adjacent effective bright spots appear in a certain area in the operable area 306 at the same time, and the positions of the three effective bright spots continuously move in the same direction after appearing;
the four fingers slide in the same direction: four adjacent effective bright spots appear in a certain area in the operable area 306 at the same time, and the positions of the four effective bright spots continuously move in the same direction after appearing;
single finger long touch: a single effective bright spot appears in a certain area in the operable area 306, and the position of the effective bright spot does not move or disappear within a range less than or equal to a fifth time threshold after the effective bright spot appears;
double finger length touch: two effective bright spots appear in a certain area in the operable area 306 at the same time, and the positions of the two effective bright spots do not move or disappear within the range of less than or equal to the fifth time threshold after the two effective bright spots appear;
three-finger long touch: three effective bright spots appear in a certain area in the operable area 306 at the same time, and the positions of the three effective bright spots do not move or disappear within the range of less than or equal to the fifth time threshold after the three effective bright spots appear;
four fingers long touch: four effective bright spots appear in a certain area in the operable area 306 at the same time, and the positions of the four effective bright spots do not move or disappear within a range of less than or equal to a fifth time threshold after the four effective bright spots appear.
It should be noted that, the zeroth time threshold, the first time threshold, the second time threshold, the third time threshold, the fourth time threshold, and the fifth time threshold are all selected and adjusted according to the touch operation feedback speed and sensitivity that are actually required, which is not limited in the present invention. Preferably, the zeroth time is 200 milliseconds, the first time threshold is 100 milliseconds, the second time threshold is 150 milliseconds, the third time threshold is 200 milliseconds, the fourth time threshold is 150 milliseconds, and the fifth time threshold is 400 milliseconds.
Preset operating logic instruction set implementation
Preferably, the present invention provides an embodiment of a preset operation logic instruction set, where a mapping relationship between the control instruction and the touch operation event in the preset operation logic instruction set includes
Clicking: single finger single click;
double clicking: clicking twice continuously by a single finger;
a right key: double fingers click once;
dragging: the single finger slides after two continuous clicks;
cursor movement: sliding by a single finger;
the page scrolls upwards: the two fingers slide forwards in the same direction;
the page scrolls down: the two fingers slide backwards in the same direction;
pressing a middle key: three fingers click twice continuously;
middle key dragging: three fingers slide after two continuous clicks;
and (3) amplifying the picture: the two fingers slide reversely;
and (3) reducing the picture: the two fingers slide in opposite directions;
switching to the previous page: the right fingers slide in the same direction;
switch to the next page: sliding towards the left with the two fingers in the same direction;
switching to a multitask view: the three fingers slide forwards in the same direction;
switching to a desktop view: sliding backwards in the same direction with the three fingers;
switch to the previous application: sliding the three fingers to the right in the same direction;
switch to the next application: sliding towards the left three fingers in the same direction;
turning up the volume: the four fingers slide forwards in the same direction;
turning down the volume: the four fingers slide backwards in the same direction;
switching to the previous track: sliding the four fingers to the right in the same direction;
switching to the next track: sliding towards the left four fingers in the same direction;
the play progress bar advances: four fingers slide to the right after clicking twice continuously;
the playing progress bar retreats: after clicking twice continuously, the four fingers interact leftwards;
quick call search box: double-finger long touch;
quick screen capture: three fingers touch long;
full screen translation: four fingers long touch.
Projection and projection picture interaction implementation mode
Due to the high light transmission and low diffuse reflection of common transparent materials, the projection of the picture on the transparent desktop 3 is realized, and the problem exists that most of the light projected on the transparent desktop 3 directly penetrates through the transparent desktop 3 and is difficult to image on the transparent desktop 3, so that the projection effect is poor, and the picture is unclear.
In order to achieve a good projection effect on the transparent desktop 3, the invention further comprises a projection module 13 and a transparent reflective film, wherein the projection module 13 is arranged below the transparent desktop 3, the transparent reflective film is arranged on the transparent desktop 3, and the projection module 13 can project a multimedia picture on the transparent reflective film of the transparent desktop 3. As shown in fig. 1-3, the projection module 13 includes a projection lens 1301, which is installed at the lower end of the supporting frame 7, the projection direction of which is toward the area where the lower surface of the transparent tabletop 3 is located, and the area where the picture projected onto the transparent reflective film is located is referred to as a projection imaging area 308.
The transparent reflection film can adopt the existing transparent projection film on the market, also called holographic projection film, and the transparent projection film is in a transparent or semitransparent state and has good reflection characteristic. Taking the transparent material as common glass as an example, when a projection device such as a projector is adopted to project a picture on the glass attached with the transparent reflective film, the picture is more clear and brighter than that of the picture directly projected on the common glass, and is relatively close to the effect of screen display.
The transparent reflective film may be disposed on the upper surface of the transparent table top 3 or on the lower surface of the transparent table top 3. The transparent reflective film can be arranged in a manner of film sticking or directly plated on the transparent table top 3 by adopting a film coating process.
The transparent table top 3 becomes a panel similar to a touch screen having both touch interaction capability and screen display function by means of the projection module 13 and the transparent reflective film. If the multimedia picture source projected by the projection module 13 is an interactive picture, by defining the touch interaction logic in the projection imaging area 308, the touch interaction operation with the projected picture can be realized in the projection imaging area 308, which is similar to the operation of a touch mobile phone, a touch tablet computer, and a notebook computer with a touch screen function. The method comprises the following specific steps:
connecting an intelligent terminal for providing multimedia pictures, so that the projection module 13 can project multimedia pictures needing interactive operation;
adjusting projection parameters to enable the multimedia picture projected by the projection module 13 to be clearly imaged on the transparent reflective film of the transparent desktop 3, and the projection imaging area 308 is located in the operable area 306;
storing the position and size information of the projection imaging area 308 of the projection module in the storage module;
defining the touch operation event in the projection imaging area 308 to obtain a corresponding relation set of the touch operation event in the projection imaging area 308 and a control instruction for the projected multimedia picture, wherein the corresponding relation set is called a preset projection interaction logic instruction set;
the processing control module monitors the touch operation event occurring in the projection imaging area 308 by monitoring the state information of the effective bright spots in the projection imaging area 308, matches the touch operation event with the control instruction in the preset projection interaction logic instruction set to obtain a control instruction corresponding to the current touch operation event, and the communication module transmits the control instruction to the intelligent terminal providing the multimedia picture.
Claims (9)
1. The intelligent transparent office table interaction method is used for realizing touch interaction with an intelligent terminal on a transparent table top (3) of the intelligent transparent office table, the intelligent transparent office table comprises the transparent table top (3), an enhanced light source (5), an image acquisition module, a processing control module, a storage module and a communication module, the enhanced light source (5) is used for illuminating the inside of the transparent table top (3), the image acquisition module is arranged below the transparent table top (3) and can shoot and acquire images of the area where the transparent table top (3) is located from the lower part of the transparent table top (3), the acquired images can be analyzed and processed by the processing control module, and the storage module and the communication module are connected with the processing control module
S1: presetting a touch operation action on the transparent desktop (3), when the enhanced light source (5) is turned on, if a finger touches the upper surface of the transparent table top (3), the skin of the finger in the contact area is illuminated, forming obvious bright spots in an image collected by the image collection module from the lower part of the transparent desktop (3), wherein different touch operation actions correspond to different bright spot state information, the state information of the bright spots comprises the number of the bright spots and corresponding time sequence data, the position of each bright spot and corresponding time sequence data, the state information of the bright spots and the touch operation actions performed on the transparent desktop (3) are preset correspondingly to obtain a touch operation event set mapped by the state information of the bright spots, and the touch operation event set and the mapping relation are stored in the storage module;
s2: defining touch operation events on the transparent desktop (3) to obtain a corresponding relation set of various touch operation events and control instructions, wherein the corresponding relation set is called a preset operation logic instruction set, and the preset operation logic instruction set is stored in the storage module;
s3: defining a judgment criterion of effective bright spots, wherein the judgment criterion is used for judging whether the bright spots in the region of the transparent desktop (3) in the image acquired by the image acquisition module are the bright spots generated by the finger tips contacting the transparent desktop (3);
s4: demarcating an operable area (306) on the transparent desktop (3), and storing the position and size information of the operable area (306) in the storage module, wherein the operable area (306) is an effective area on the transparent desktop (3) where interaction can be carried out;
s5: turning on the enhanced light source (5) to make the light emitted by the enhanced light source enter the inside of the transparent table top (3) and illuminate the inside of the transparent table top (3);
s6: the image acquisition module is set to be in a working state, the image acquisition module continuously acquires images of the area where the transparent desktop (3) is located below the transparent desktop (3), and the acquired images are analyzed and processed by the processing control module;
s7: the processing control module continuously monitors the state information of effective bright spots appearing in the operable area (306) in the image acquired by the image acquisition module;
s8: the processing control module matches the state information of the effective bright spots appearing in the operable area (306) with the touch operation event set to determine corresponding touch operation events;
s9: and the processing control module matches the touch operation event determined in the step S8 with a control instruction in the preset operation logic instruction set to obtain a control instruction corresponding to the current touch operation event, and the communication module transmits the control instruction to the intelligent terminal.
2. The intelligent transparent desk interaction method of claim 1, wherein the set of touch manipulation events comprises a click event, a click-slide event, a slide-touch event, a long-touch event, wherein,
the click event includes: single-finger single click, double-finger single click, three-finger single click, and four-finger single click; two continuous clicks are performed by a single finger, two continuous clicks are performed by two fingers, two continuous clicks are performed by three fingers, and two continuous clicks are performed by four fingers; three continuous clicks are performed by a single finger, three continuous clicks are performed by two fingers, three continuous clicks are performed by three fingers, and three continuous clicks are performed by four fingers; each click event comprises the information of the number of fingers participating in clicking and the information of the number of continuous clicks, and also comprises the position and the time sequence information of each click of each finger;
the click swipe event includes: the single finger slides after two continuous clicks, the double finger slides after two continuous clicks, the three finger slides after two continuous clicks, and the four finger slides after two continuous clicks; each click sliding event comprises the information of the number of fingers participating in click sliding and the information of the number of continuous clicks, and also comprises the information of the position and the time sequence of each click of each finger, the initial position of each finger sliding, the displacement and the time sequence of each finger sliding;
the sliding touch event includes: single-finger sliding, double-finger same-direction sliding, double-finger opposite-direction sliding, double-finger reverse sliding, three-finger same-direction sliding and four-finger same-direction sliding; each sliding touch event comprises the information of the number of fingers participating in sliding touch, and also comprises the information of the initial position, the direction, the displacement and the time sequence of the sliding of each finger;
the long touch event comprises: single-finger long touch, double-finger long touch, three-finger long touch, and four-finger long touch; each long touch event comprises the information of the number of fingers participating in long touch, and also comprises the position and the time sequence information of the long touch of each finger;
the control instructions include: clicking, double clicking, right clicking, dragging, cursor moving, page scrolling up, page scrolling down, middle key pressing, middle key dragging, picture zooming in, picture zooming out, switching to a previous page, switching to a next page, switching to a multitask view, switching to a desktop view, switching to a previous application, switching to a next application, turning up the volume, turning down the volume, switching to a previous song, switching to a next song, playing progress bar forwarding, playing progress bar backing, quick call-out search box, quick screen capture, full screen translation.
3. The intelligent transparent desk interaction method of claim 2, wherein the mapping relationship between the touch operation event set and the state information of the effective bright spot comprises
Single finger single click: a single effective bright spot appears in a certain area in the operable area (306), the effective bright spot disappears within a range which is less than or equal to a first time threshold after appearing, and no single effective bright spot appears in the area within a zeroth time threshold after disappearing;
double finger single click: two adjacent effective bright spots appear in a certain area in the operable area (306) at the same time, the two effective bright spots disappear within a range which is less than or equal to the first time threshold after appearing, and the area does not have two adjacent effective bright spots appearing in the range of the zeroth time threshold after disappearing at the same time;
three points are single click: a certain area in the operable area (306) simultaneously appears three adjacent effective bright spots, the three effective bright spots disappear within a range which is less than or equal to a first time threshold value after appearing, and the area does not simultaneously appear three adjacent effective bright spots within a zero time threshold value range after disappearing;
four refers to a single click: a certain area in the operable area (306) simultaneously appears four adjacent effective bright spots, the four effective bright spots disappear within a range which is less than or equal to a first time threshold value after appearing, and the area does not simultaneously appear four adjacent effective bright spots within a zero time threshold value range after disappearing;
single finger two clicks in succession: a single effective bright spot appears in a certain area in the operable area (306), and the effective bright spot completely goes through four processes of appearance-disappearance-second appearance-second disappearance in the area within the range of the second time threshold value or less, and no single effective bright spot appears in the area within the range of the zeroth time threshold value after the second disappearance;
two fingers click twice in succession: two adjacent effective bright spots appear in a certain area in the operable area (306) at the same time, and the two effective bright spots are in a range less than or equal to a second time threshold, the area completely undergoes four processes of appearance-disappearance-second appearance-second disappearance, and the area does not appear two adjacent effective bright spots in the range of a zeroth time threshold after the second disappearance;
three fingers click twice in succession: a certain area in the operable area (306) simultaneously appears three adjacent effective bright spots, and the three effective bright spots are in a range less than or equal to a second time threshold, the area completely undergoes four processes of appearance-disappearance-second appearance-second disappearance, and the area does not simultaneously appear three adjacent effective bright spots in a zero time threshold range after the second disappearance;
four fingers click twice in succession: a certain area in the operable area (306) simultaneously appears four adjacent effective bright spots, and the four effective bright spots are within a range less than or equal to a second time threshold, the area completely undergoes four processes of appearance-disappearance-second appearance-second disappearance, and no more four adjacent effective bright spots simultaneously appear in the area within a zero time threshold after the second disappearance;
single finger three clicks in succession: a certain area in the operable area (306) appears a single effective bright spot, the effective bright spot is in a range less than or equal to a third time threshold, the area completely undergoes six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance, and no single effective bright spot appears in the area within a zero time threshold after the third disappearance;
two fingers three clicks in succession: a certain area in the operable area (306) simultaneously appears two adjacent effective bright spots, the two effective bright spots are in a range less than or equal to a third time threshold, the area completely undergoes six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance, and the area does not simultaneously appear two adjacent effective bright spots in a zero time threshold range after the third disappearance;
three fingers three consecutive clicks: a certain area in the operable area (306) simultaneously appears three adjacent effective bright spots, the three effective bright spots completely go through six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance in the range of less than or equal to a third time threshold, and the area does not simultaneously appear three adjacent effective bright spots in the range of a zero time threshold after the third disappearance;
four fingers three consecutive clicks: a certain area in the operable area (306) simultaneously appears four adjacent effective bright spots, the four effective bright spots completely go through six processes of appearance-disappearance-second appearance-second disappearance-third appearance-third disappearance within the range of less than or equal to a third time threshold, and the area does not simultaneously appear four adjacent effective bright spots within the range of a zero time threshold after the third disappearance;
sliding after two continuous clicks with a single finger: a single effective bright spot appears in a certain area in the operable area (306), the effective bright spot completely goes through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to a fourth time threshold, and the position of the effective bright spot continuously moves after the second appearance;
two fingers slide after two consecutive clicks: two adjacent effective bright spots appear in a certain area in the operable area (306) at the same time, the two effective bright spots completely go through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to a fourth time threshold, and the positions of the two effective bright spots continuously move in the same direction after the second appearance;
three fingers slide after two consecutive clicks: a certain area in the operable area (306) simultaneously appears three adjacent effective bright spots, the three effective bright spots are in the range of less than or equal to a fourth time threshold value, the area completely undergoes three processes of appearance-disappearance-second appearance, and the positions of the three effective bright spots continuously move in the same direction after the second appearance;
four fingers slide after two consecutive clicks: a certain area in the operable area (306) simultaneously appears four adjacent effective bright spots, the four effective bright spots completely go through three processes of appearance-disappearance-second appearance in the area within the range of less than or equal to a fourth time threshold, and the positions of the four effective bright spots continuously move in the same direction after the second appearance;
single-finger sliding: a single effective bright spot appears in a certain area in the operable area (306), and the position of the effective bright spot continuously moves after the effective bright spot appears;
the two fingers slide in the same direction: two adjacent effective bright spots appear in a certain area in the operable area (306) at the same time, and the positions of the two effective bright spots continuously move in the same direction after appearing;
the two fingers slide towards each other: two effective bright spots appear in the operable area (306) at the same time, and the relative positions of the two effective bright spots continuously move oppositely after appearing, namely the two effective bright spots are close to each other;
the two fingers slide reversely: two effective bright spots appear in the operable area (306) at the same time, and the relative positions of the two effective bright spots are reversely and continuously moved after the two effective bright spots appear, namely the two effective bright spots are far away from each other;
the three fingers slide in the same direction: three adjacent effective bright spots appear in a certain area in the operable area (306) at the same time, and the positions of the three effective bright spots continuously move in the same direction after appearing;
the four fingers slide in the same direction: four adjacent effective bright spots appear in a certain area in the operable area (306) at the same time, and the positions of the four effective bright spots continuously move in the same direction after appearing;
single finger long touch: a single effective bright spot appears in a certain area in the operable area (306), and the position of the effective bright spot does not move or disappear within a range of being less than or equal to a fifth time threshold after the effective bright spot appears;
double finger length touch: two effective bright spots appear in a certain area in the operable area (306) at the same time, and the positions of the two effective bright spots do not move or disappear within the range of less than or equal to a fifth time threshold after the two effective bright spots appear;
three-finger long touch: three effective bright spots appear in a certain area in the operable area (306) at the same time, and the positions of the three effective bright spots do not move or disappear within the range of being less than or equal to a fifth time threshold after the three effective bright spots appear;
four fingers long touch: and a certain area in the operable area (306) simultaneously appears with four effective bright spots, and the positions of the four effective bright spots do not move or disappear within the range of less than or equal to the fifth time threshold after appearing.
4. The intelligent transparent desk interaction method as claimed in claim 2, wherein the mapping relationship between the control command and the touch operation event in the preset operation logic command set is
Clicking: single finger single click;
double clicking: clicking twice continuously by a single finger;
a right key: double fingers click once;
dragging: the single finger slides after two continuous clicks;
cursor movement: sliding by a single finger;
the page scrolls upwards: the two fingers slide forwards in the same direction;
the page scrolls down: the two fingers slide backwards in the same direction;
pressing a middle key: three fingers click twice continuously;
middle key dragging: three fingers slide after two continuous clicks;
and (3) amplifying the picture: the two fingers slide reversely;
and (3) reducing the picture: the two fingers slide in opposite directions;
switching to the previous page: the right fingers slide in the same direction;
switch to the next page: sliding towards the left with the two fingers in the same direction;
switching to a multitask view: the three fingers slide forwards in the same direction;
switching to a desktop view: sliding backwards in the same direction with the three fingers;
switch to the previous application: sliding the three fingers to the right in the same direction;
switch to the next application: sliding towards the left three fingers in the same direction;
turning up the volume: the four fingers slide forwards in the same direction;
turning down the volume: the four fingers slide backwards in the same direction;
switching to the previous track: sliding the four fingers to the right in the same direction;
switching to the next track: sliding towards the left four fingers in the same direction;
the play progress bar advances: four fingers slide to the right after clicking twice continuously;
the playing progress bar retreats: after clicking twice continuously, the four fingers interact leftwards;
quick call search box: double-finger long touch;
quick screen capture: three fingers touch long;
full screen translation: four fingers long touch.
5. The intelligent transparent desk interaction method of claim 1, wherein the criterion of effective bright spots is:
given two radii R1 and R2, the hot spot is determined to be a valid hot spot when the size of the hot spot cannot be completely surrounded by a small circle of radius R1, while it can be completely surrounded by a large circle of radius R2.
6. The intelligent transparent desk interaction method of claim 5, wherein the radius R1 of the small circle is 2mm, and the radius R2 of the large circle is 10 mm.
7. The intelligent transparent office table interaction method according to claim 1, wherein in the state information of the bright spot, the position of the bright spot refers to a position coordinate of a center of mass of the bright spot.
8. The intelligent transparent desk interaction method of claim 1, further comprising a keyboard pattern (6) disposed within the operable area (306) on the transparent desktop (3), the keyboard pattern (6) comprising a number of key patterns, the method comprising key event detection and interaction, the key event detection and interaction comprising the steps of:
coordinate coding is carried out on a position area corresponding to each key pattern on the keyboard pattern (6), and the position area is stored in the storage module after being mapped with key instructions of each key one by one;
the processing control module continuously monitors the state information of effective bright spots appearing in the operable area (306) in the image acquired by the image acquisition module;
when a user carries out keyboard input operation, a certain key pattern on a keyboard pattern (6) of a transparent desktop (3) is clicked by fingers, bright spots are generated at the position of the key pattern in an image collected by an image collecting module, the processing control module determines that the position of an effective bright spot appears in the key pattern of the keyboard pattern (6) through coordinate code matching, the effective bright spot disappears within a time range which is less than or equal to a sixth time threshold value after the effective bright spot appears, the effective bright spot is regarded as a key instruction corresponding to the key pattern, and the processing control module transmits the key instruction to the intelligent terminal through the communication module.
9. The intelligent transparent office table interaction method according to claim 1, further comprising a projection module (13), a transparent reflective film, wherein the projection module (13) is disposed under the transparent table top (3), the transparent reflective film is disposed on the transparent table top (3), the projection module (13) can project a multimedia picture on the transparent reflective film of the transparent table top (3), the method comprises a projection interaction, and the projection interaction comprises the following steps:
connecting an intelligent terminal for providing multimedia pictures, and enabling the projection module (13) to project the multimedia pictures needing interactive operation;
adjusting projection parameters to enable the multimedia picture projected by the projection module (13) to be clearly imaged on a transparent reflection film of the transparent desktop (3), wherein the projection imaging area (308) is located in the operable area (306);
storing position and size information of a projection imaging area (308) of the projection module in the storage module;
defining the touch operation event in the projection imaging area (308) to obtain a corresponding relation set of the touch operation event in the projection imaging area (308) and a control instruction for the projected multimedia picture, wherein the corresponding relation set is called a preset projection interaction logic instruction set;
the processing control module monitors the touch operation event occurring in the projection imaging area (308) by monitoring the state information of the effective bright spots in the projection imaging area (308), matches the touch operation event with the control instruction in the preset projection interaction logic instruction set to obtain the control instruction corresponding to the current touch operation event, and the communication module transmits the control instruction to the intelligent terminal providing the multimedia picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111189457.8A CN114138162A (en) | 2021-10-12 | 2021-10-12 | Intelligent transparent office table interaction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111189457.8A CN114138162A (en) | 2021-10-12 | 2021-10-12 | Intelligent transparent office table interaction method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114138162A true CN114138162A (en) | 2022-03-04 |
Family
ID=80394493
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111189457.8A Pending CN114138162A (en) | 2021-10-12 | 2021-10-12 | Intelligent transparent office table interaction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114138162A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115268743A (en) * | 2022-07-29 | 2022-11-01 | 深圳市商汤科技有限公司 | Image processing method, image processing apparatus, electronic device, information input system, and medium |
-
2021
- 2021-10-12 CN CN202111189457.8A patent/CN114138162A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115268743A (en) * | 2022-07-29 | 2022-11-01 | 深圳市商汤科技有限公司 | Image processing method, image processing apparatus, electronic device, information input system, and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9268413B2 (en) | Multi-touch touchscreen incorporating pen tracking | |
| US8842076B2 (en) | Multi-touch touchscreen incorporating pen tracking | |
| KR102335132B1 (en) | Multi-modal gesture based interactive system and method using one single sensing system | |
| US10001845B2 (en) | 3D silhouette sensing system | |
| CA2748881C (en) | Gesture recognition method and interactive input system employing the same | |
| US9317130B2 (en) | Visual feedback by identifying anatomical features of a hand | |
| TWI450159B (en) | Optical touch device, passive touch system and its input detection method | |
| US20100103103A1 (en) | Method And Device for Input Of Information Using Visible Touch Sensors | |
| CN103744542B (en) | Hybrid pointing device | |
| CN102460373A (en) | Surface computer user interaction | |
| CN103995623B (en) | Non-contact type touch screen control device | |
| CN101776971B (en) | Multi-point touch screen device and positioning method | |
| CN114138162A (en) | Intelligent transparent office table interaction method | |
| Maierhöfer et al. | Demonstration of an Infrared Pen as an Input Device for Projected Augmented Reality Tabletops | |
| CN216651666U (en) | Transparent desk of single table leg intelligence | |
| CN114138163A (en) | Intelligent transparent office table | |
| CN114138164A (en) | Transparent touch display system without touch control medium and interaction method thereof | |
| CN114327230A (en) | Transparent desktop handwriting input system and method | |
| TWI536228B (en) | An inductive motion-detective device | |
| JP5118663B2 (en) | Information terminal equipment | |
| Al Sheikh et al. | Design and implementation of an FTIR camera-based multi-touch display | |
| Wang et al. | Large screen multi-touch system integrated with multi-projector |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20220304 |
|
| WD01 | Invention patent application deemed withdrawn after publication |