CN106598422A - Directivity-based control and hybrid control methods, control system and electronic equipment - Google Patents

Directivity-based control and hybrid control methods, control system and electronic equipment Download PDF

Info

Publication number
CN106598422A
CN106598422A CN201610956745.4A CN201610956745A CN106598422A CN 106598422 A CN106598422 A CN 106598422A CN 201610956745 A CN201610956745 A CN 201610956745A CN 106598422 A CN106598422 A CN 106598422A
Authority
CN
China
Prior art keywords
touch
display screen
point
control
sensing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610956745.4A
Other languages
Chinese (zh)
Other versions
CN106598422B (en
Inventor
黄源浩
刘龙
肖振中
许星
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Orbbec Co Ltd
Original Assignee
Shenzhen Orbbec Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Orbbec Co Ltd filed Critical Shenzhen Orbbec Co Ltd
Priority to CN201610956745.4A priority Critical patent/CN106598422B/en
Publication of CN106598422A publication Critical patent/CN106598422A/en
Application granted granted Critical
Publication of CN106598422B publication Critical patent/CN106598422B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range

Abstract

The invention discloses directivity-based control and hybrid control methods, a control system and electronic equipment. The directivity-based control method comprises the following steps of S1, obtaining depth images of a display screen and a control object; S2, obtaining a position on the display screen and directed by the control object through the depth images, so that a directed point on the display screen is obtained; S3, according to indication of the directed point, in combination with a pre-determined control action, identifying and generating a touch instruction; and S4, according to the touch instruction, executing touch operation at the position of the directed point. By means of the directivity-based control method disclosed by the invention, single-hand control on large-screen electronic equipment can be well realized; and precise touch of an object can be realized.

Description

Based on the manipulation of directing, mixing control method and control system and electronic equipment
Technical field
The present invention relates to electronic technology field, more particularly to a kind of based on the control method of directing, mixing manipulation side Method and its control system and electronic equipment.
Background technology
With the functional requirement more and more higher of the popularization and people of mobile Internet to smart machines such as mobile phones, while and The functions such as standby communication, online and video playback are the configurations the most basic of current electronic equipment.At present as equipment such as mobile phones Size has been most likely into the large-size screen monitors epoch, and one-handed performance is out of reach, even so, taking into account larger screen and one-handed performance It is still demand of the current user to mobile phone.
By taking mobile phone as an example, it is a kind of scheme of solution one-handed performance that function key is increased after cell phone back, but undoubtedly can shadow The attractive in appearance of the mobile phone back side is rung, thus this scheme is never easily accepted by a user.Another kind of scheme is to increase volume at the mobile phone back side Outer touch screen, this scheme by overleaf using finger manipulation realize to mobile phone screen cannot one-handed performance region control System.But this scheme is relatively costly, thus cannot become the one-handed performance scheme of main flow.
Meanwhile, touch control operation being carried out currently with depth image, can not realize to area between manipulation object and display screen Overseas display screen carries out touch-control, manipulates object position on a display screen and directly maps corresponding on acquired depth image The coordinate of pixel, that is, it is exactly pixel coordinate mapping of the manipulation object in depth image to manipulate object position on a display screen , it is this directly by pixel coordinate map and obtain showing position on the screen method can not to manipulate object touch-control less than Region carries out touch control operation, so the function of some such as page turnings can only be realized simply, it is impossible to solve the list of large-size screen monitors well Handss manipulate problem, can not carry out accurate touch-control to the object on display screen.
The disclosure of background above technology contents is only used for inventive concept and the technical scheme for aiding in understanding the present invention, and it is not The prior art of present patent application is necessarily belonged to, without tangible proof the applying date of the above in present patent application is being shown In the case of disclosed, above-mentioned background technology should not be taken to evaluate the novelty and creativeness of the application.
The content of the invention
Present invention aim at proposition is a kind of being set based on the manipulation of directing, mixing control method and control system and electronics It is standby, to solve the technical problem that can not well realize singlehanded manipulation, accurate touch-control that above-mentioned prior art is present.
For this purpose, the present invention proposes a kind of directing control method, comprise the following steps:S1:Display screen is obtained with manipulation thing The depth image of body;S2:Position on display screen pointed by manipulation object is obtained by the depth image, display screen is obtained On sensing point;S3:According to the instruction for pointing to point, with reference to predetermined control action, touch command is recognized and generated;S4: Touch control operation is performed at the position for pointing to point according to the touch command.
Preferably, control method of the invention can also have following technical characteristic:
The acquisition pointed to described in step S2 a little includes:Extracting on depth image can describe the feature of the manipulation object Point, the sensing straight line for describing the manipulation object is constructed according to this feature point, obtains the sensing point on display screen.
The point that points to comprises the steps to point to the intersection point that straight line intersects with the display screen:S21:Obtain described The depth information of characteristic point;S22:The characteristic point is obtained in the display screen institute according to the conversion of the depth information of the characteristic point In the spatial positional information of coordinate system, obtained for describing the manipulation object according to the spatial positional information of the characteristic point Point to straight line;S23:The sensing straight line and the display screen intersection point are obtained according to the sensing straight line, the finger on display screen is obtained Xiang Dian, and point display will be pointed on a display screen.
When the manipulation object is contacted with the display screen, the sensing point is the characteristic point of the manipulation object.
The depth image obtained in step S1 includes:S11:Obtain comprising display screen and without the first depth of manipulation object Image;S12:Obtain the second depth image containing display screen and manipulation object;S13:By second depth image and institute The 3rd depth image that the first depth image obtains the manipulation object is stated, is obtained according to the 3rd depth image and is pointed to point.
Meanwhile, the invention allows for a kind of mixing control method, display screen is the touch screen for itself possessing touch controllable function, The touch screen includes Touch Zone and points to area, and the manipulation object is in the Touch Zone according to the tactile of the touch screen itself Control function completes touch-control, and touch control operation is completed by a sensing is pointed to according to above-mentioned control method in the sensing area.
In addition, the invention allows for a kind of control system, for performing above-mentioned touch control method, including image obtains single Unit, processor and display screen;
Described image acquiring unit is used to obtain the depth image and the depth of manipulation object of display screen and manipulation object Information;
The processor includes identification module, conversion module and performing module, and the identification module is used for according to the depth Degree image obtains the position on the display screen pointed by manipulation object, and the predetermined control action for identifying manipulation object; The conversion module is used to be converted into corresponding touch command according to predefined control action;The performing module is used for Touch command is performed at the position for pointing to point complete touch control operation;
The display screen is used to point to the display of point and information.
Preferably, mixing control method of the invention can also have following technical characteristic:
The display screen is the touch screen for itself possessing touch controllable function, and the manipulation object is in the Touch Zone according to institute The touch controllable function for stating touch screen itself completes touch-control, and touch control operation is completed by a sensing is pointed in the sensing area.
The image acquisition unit of the control system is the depth camera based on structure light or TOF (flight time) principle.
Again, the invention allows for a kind of electronic equipment, including the control system being arranged in its bus, by described Control system completes manipulation operation.
The beneficial effect that the present invention is compared with the prior art includes:The present invention is to realize touch control operation using depth image And it is a kind of control method based on directing, the control method obtains the display pointed by manipulation object by depth image Position on screen, obtains the instruction point on display screen, the instruction that user can be put by the instruction, with reference to predetermined control action, Touch control operation is performed at the position for pointing to point, so when user carries out touch-control in the electronic equipment to large-size screen monitors, to difficulty When carrying out touch-control with the object for touching, only need to simply adjust finger position on a display screen and state, such as gradient, away from Height from display screen etc., by finger the object is pointed to, and a sensing point just can be on a display screen obtained, according to the sensing point Indicate, user can accurately know the situation that finger is pointed to, and corresponding touch control operation be performed exactly, compared to prior art For, the present invention not only can realize some simple gesture operations such as page turning, rollback to the display screen that do not possess touch function, Singlehanded touch-control can also well be realized, to it is singlehanded touch less than object corresponding touch-control behaviour can be carried out by way of directing Make, and accurate touch-control can be realized.
In preferred version, the depth image to obtaining carries out acquisition manipulation according to the first depth image and the second depth image 3rd depth image of object, then according to the 3rd depth image obtaining sensing point, so due to obtaining the 3rd depth map As the depth image for only including manipulation object, so when obtaining to sensing point, the calculating of processing procedure can be reduced Amount, improves arithmetic speed, improves the response speed of system.
A kind of mixing control method that the present invention is provided, in the control method based on above-mentioned depth image and directing On the basis of, for display screen possesses touch controllable function in itself, carry out that a kind of not only again utilization referred to using the touch controllable function of itself To the control method of type carry out mix manipulation pattern, this mixing control method can make up due to manipulate object movement with The low problem of precision that machine is caused greatly, provides the user a kind of more preferable experience scheme.
Description of the drawings
Fig. 1 is the structural representation of the control system of the specific embodiment of the invention one;
Fig. 2 is the structural representation of the processor of the specific embodiment of the invention one.
Fig. 3 is the structural representation of the electronic equipment of the specific embodiment of the invention three.
Front schematic view when Fig. 4 is the singlehanded manipulation mobile phone of the specific embodiment of the invention three and four.
Side schematic view when Fig. 5 is the singlehanded manipulation mobile phone of the specific embodiment of the invention three and four.
Fig. 6 is the schematic diagram of the mixing manipulation of the specific embodiment of the invention two and five.
Fig. 7 is the manipulation flow chart one of the specific embodiment of the invention four.
Fig. 8 is the manipulation flowchart 2 of the specific embodiment of the invention four.
Specific embodiment
With reference to specific embodiment and compare accompanying drawing the present invention is described in further detail.It is emphasized that What the description below was merely exemplary, rather than in order to limit the scope of the present invention and its application.
With reference to the following drawings, the embodiment of non-limiting and nonexcludability will be described, wherein identical reference is represented Identical part, unless stated otherwise.
Embodiment one:
The present embodiment proposes a kind of control system, as shown in figure 1, including image acquisition unit 1, processor 2 and showing Screen 3;
Described image acquiring unit 1 is used to obtain the depth image and the depth of manipulation object of display screen 3 and manipulation object Degree information;
As shown in Fig. 2 the processor 2 includes identification module 21, conversion module 22 and performing module 23, the identification mould The position that block 21 is used on the display screen 3 according to pointed by the depth image obtains manipulation object, and identify manipulation thing The predetermined control action of body;The conversion module 22 refers to for being converted into corresponding touch-control according to predefined control action Order;The performing module 23 is used for the execution touch command at the position for pointing to point and completes touch control operation;
The display screen 3 is used to point to the display of point and information.
The system can obtain the depth image of display screen by image acquisition unit, while also can obtain with display screen not The depth image of the manipulation object of contact, the position of the display screen according to pointed by the depth image can identify manipulation object And action, so as to change into corresponding position and instruction on display screen, the manipulation to equipment, this reality so just can be realized In applying example, display screen is the display screen for not touching kinetic energy, such as the display screen of E-book reader.In view of on a display screen Carrying out touch manipulation can affect display effect, such as hand will certainly shield portions when finger page turning is constantly used in e-book Screen, so as to affect reading experience.And can then mitigate this problem using the directivity manipulation of manipulation object, lift making for user With experience.
Here image acquisition unit is the depth camera based on structure light or TOF principles, has generally comprised reception single Unit and transmitter unit composition.
Embodiment two:
The present embodiment proposes a kind of mixing control system, and compared to embodiment one, difference is the display of the present embodiment Shield the touch screen to possess touch function, as shown in fig. 6,1. with sensing area 2. the touch screen includes Touch Zone, the manipulation In the Touch Zone, 1. the interior touch controllable function of itself according to the touch screen completes touch-control to object 14, in the sensing area 2. by referring to Point to point 17 and complete touch control operation.
Specially:The sensing manipulation region that the touch area and one hand that one hand can be reached can not reach, is touching Region still adopt touch manipulation to ensure having more preferable precision, and point to manipulation region then adopt foundation finger sensing with And action is come the manipulation that performs, the present embodiment is particularly well-suited at present because giant-screen causes the mobile phone that one hand cannot manipulate completely Equipment etc. possessing touch function.
Embodiment three:
The present embodiment proposes a kind of electronic equipment, and electronic equipment can be mobile phone, flat board, including for data transfer Bus 4, as shown in figure 3,4 points of the bus is data/address bus, address bus and controlling bus, is respectively intended to transmission data, number According to address and control signal.Be connected with the bus 4 have CPU5, display 6, IMU7 (inertial measuring unit), memorizer 8, Photographic head 10, audio unit 11, network interface 12 and mouse/keyboard 13, for some possess the electronic equipment of touch function For, the shown device 6 of its mouse-keyboard is replaced, the functional realiey such as IMU device is used to positioning, track.Memorizer 8 is used to store Operating system, application program etc., it is also possible to for the ephemeral data during storage running.
Image acquisition unit in the control system of the upper setting of this electronic equipment corresponds to photographic head, and processor is corresponded to CPU, or single processor, display screen corresponds to display.
As shown in figure 4, photographic head 10 is generally secured on electronic equipment.In the present embodiment, take the photograph with existing device As head is compared, there is larger difference in shooting direction.Existing photographic head is generally preposition or rearmounted, and such configuration cannot be obtained The image of equipment itself.The configuration of photographic head 10 in the present embodiment can have various situations, and by means of rotary shaft, i.e., one kind is The purpose shot to oneself is reached after by rotation photographic head is rotated by 90 °;Also it is a kind of be external camera form, will Photographic head is connected entirely through certain fixation and with interfaces such as USB as one with equipment.Those skilled in the art The form that photographic head is arranged on an electronic device can voluntarily be selected according to practical situation, be not limited.
Now, with existing photographic head for, the photographic head of the present embodiment is depth camera, for obtaining target area Depth image.
Depth camera can be the depth camera based on structure light principle or TOF principles, generally comprise receiving unit And transmitter unit composition.
Front schematic view when being singlehanded manipulation mobile phone shown in Fig. 4.Photographic head 10 is configured with the top of mobile phone, photographic head is adopted Collection direction is top-down along mobile phone, can thus obtain the image of mobile phone display screen and finger, is singlehanded behaviour shown in Fig. 5 Side schematic view during control mobile phone.
It can also not possess touch function that the display 6 of electronic equipment can be to possess touch function, when not having During standby touch function, only can be manipulated by directing control method, when possessing touch function, can be in manipulation object energy The display screen area for touching is manipulated, to touch less than region perform directing manipulation.
Example IV:
A kind of directing control method, with reference to shown in Fig. 4-5, is applied to not with display screen in the case of touch function Manipulation, as shown in fig. 7, comprises following steps:
S1:Obtain the depth image of display screen 18 and manipulation object 14;
S2:Position on display screen 18 pointed by manipulation object 14 is obtained by the depth image, display screen is obtained On sensing point 17;
S3:According to the instruction for pointing to point 17, with reference to predetermined control action, touch command is recognized and generated;
S4:Touch control operation is performed at the position for pointing to point 17 according to the touch command.Such as to certain icon Carry out clicking operation.
The acquisition for pointing to point 17 in step s 2 includes:Extracting on depth image can describe the manipulation object Characteristic point 15, the sensing straight line 16 for describing the manipulation object is constructed according to this feature point 15, obtains the sensing point 17 on display screen.
Specifically, as shown in figure 5, the point 17 that points to is to point to the intersection point that straight line 16 intersects with the display screen 18, As shown in figure 8, comprising the steps:
S21:Being extracted according to the depth image can describe the characteristic point 15 of the manipulation object 14, obtain the characteristic point 15 depth information;
S22:The characteristic point 15 is obtained according to the conversion of the depth information of the characteristic point 15 to be located in the display screen 18 The spatial positional information of coordinate system, obtains for describing the manipulation object according to the spatial positional information of the characteristic point 15 Point to straight line 16;
S23:The sensing straight line 16 and the intersection point of the display screen 18 are obtained according to the sensing straight line 16, display screen 18 is obtained On sensing point 17, and point will be pointed to 17 include on display screen 18.
In above-mentioned steps, when the manipulation object 14 is contacted with the display screen 18, the sensing point 17 is institute The characteristic point of manipulation object 14 is stated, now it can be assumed that being touch control operation.It is of course also possible to will regard as with screen contact it is invalid Operation, accurately to calculate manipulation object 14 in the position of display screen 18.
Can be modeled to manipulating object 14 with skeleton modeling or other models when obtaining characteristic point 15, to complete The structure of straight line is pointed to, for a finger, typically using the articulare of finger and finger summit as characteristic point 15.
In existing touching technique, positioning is completed simultaneously with touching, this directing can noncontact (touch is can Lift finger to point to the object of wanted touch-control) in operation can positioning with manipulation while complete, i.e., when the position of finger sensing Put and be immediately performed after determining similar to the acquiescence touch command for selecting, click on, now then need not further identify finger Shape and action.Alternatively, it is also possible to complete step by step, i.e., out position is first calculated, then finger makes corresponding action, such as finger The change of shape, or action clicked on etc. is done to the direction pointed to, respective action is performed after the identification of this these action Corresponding operational order.It is prior art to the shape and action recognition of finger, not describes in detail here.
Certainly some operations without the need for positioning, such as the operation such as page turning, rollback are also not excluded for.For this generic operation, can be with The shape or action for only recognizing finger just can be with.
In the present embodiment, finger is pushed ahead along it or retreated and then thinks to complete one click action, also may be used With by other shapes and mode.The shape of finger needs to preset with the touch command corresponding to action.When recognizing certain When finger shape and action, processor then converts it into corresponding touch command.
Other irrelevant portions are also included in addition to comprising mobile phone display screen 18, finger in the depth image of acquisition, now Then can be limited by the measurement range to depth camera, that is, set certain threshold value, to the depth information more than threshold value Removed, the information only containing mobile phone display screen and finger can be reduced knowledge by the depth image for obtaining in this way Other amount of calculation.In order to further improve system running speed, operand is reduced, image segmentation is utilized when depth image is obtained Obtaining, one of which method is to comprise the steps to method:
S11:Obtain comprising display screen 18 and without the first depth image of manipulation object 14;
S12:Obtain the second depth image containing display screen 18 and manipulation object 14;
S13:It is deep by the 3rd of second depth image and first depth image acquisition manipulation object 14 the Degree image, obtains according to the 3rd depth image and points to point 17.
If manipulation object 14 is finger, the depth map at the finger front end position that above-mentioned steps are obtained by background segment method Picture, can reduce amount of calculation when modeling, and improve calculating speed.
Embodiment five:
In view of the randomness of finger movement, its precision is difficult to reach the degree of touch.Possesses touch thus for some The singlehanded manipulation of functional mobile phone kind equipment, the region that can be reached in one hand still selects touch operation, and cannot arrive in one hand It is a kind of scheme for having and more preferably experiencing that noncontact manipulation is implemented in the region for reaching.So the present embodiment proposes a kind of mixing manipulation side Method, its display screen 18 is the touch screen that itself possesses touch controllable function, as shown in fig. 6,1. the touch screen includes Touch Zone and refer to To area 2., in the Touch Zone, 1. the interior touch controllable function of itself according to the touch screen completes touch-control to the manipulation object 14, On the basis of example IV, 2. touch-control is carried out according to the control method of the directing in the sensing area, by the finger for pointing to point 17 To completing touch control operation.
Hand size, handedness due to different user etc. is different from, thus for Touch Zone 1. with point to area 2. differentiation can be by system automatic identification.I.e.:When finger is contacted with touch display screen 18, then reason touch screen at processor The signal of feedback simultaneously performs corresponding touch command, and when finger is not contacted with touch display screen 18, depth camera is identified This after contact action, does not feed back to processor, and processor is then processed the depth image that depth camera is obtained, and identification is sold Corresponding operational order is performed after the action of the position and finger pointed to so as to realize that noncontact is manipulated.
It would be recognized by those skilled in the art that it is possible that numerous accommodations are made to above description, so embodiment is only For describing one or more particular implementations.
Although having been described above and describing the example embodiment for being counted as the present invention, it will be apparent to those skilled in the art that It can be variously modified and be replaced, without departing from the spirit of the present invention.Furthermore it is possible to make many modifications with by spy Condition of pledging love is fitted to the religious doctrine of the present invention, without departing from invention described herein central concept.So, the present invention is unrestricted May also include belonging to all embodiments and its equivalent of the scope of the invention in specific embodiment disclosed here, but the present invention Thing.

Claims (10)

1. a kind of directing control method, it is characterised in that comprise the following steps:
S1:Obtain the depth image of display screen and manipulation object;
S2:Position on display screen pointed by manipulation object is obtained by the depth image, the sensing on display screen is obtained Point;
S3:According to the instruction for pointing to point, with reference to predetermined control action, touch command is recognized and generated;
S4:Touch control operation is performed at the position for pointing to point according to the touch command.
2. control method as claimed in claim 1, it is characterised in that the acquisition pointed to described in step S2 a little includes:In depth Extracting on degree image can describe the characteristic point of the manipulation object, straight according to the sensing that this feature point construction describes the manipulation object Line, obtains the sensing point on display screen.
3. control method as claimed in claim 2, it is characterised in that the sensing point is to point to straight line and the display screen phase The intersection point of friendship, comprises the steps:
S21:Obtain the depth information of the characteristic point;
S22:Sky of the characteristic point in display screen place coordinate system is obtained according to the conversion of the depth information of the characteristic point Between positional information, according to the spatial positional information of the characteristic point obtain for describe it is described manipulation object sensing straight line;
S23:The sensing straight line and the display screen intersection point are obtained according to the sensing straight line, the sensing point on display screen is obtained, And point display will be pointed on a display screen.
4. the control method as described in any one of claim 1-3, it is characterised in that when the manipulation object and the display screen During contact, the sensing point is the characteristic point of the manipulation object.
5. the control method as described in any one of claim 1-3, it is characterised in that the depth image bag obtained in step S1 Include:S11:Obtain comprising display screen and without the first depth image of manipulation object;S12:Obtain containing display screen and manipulation thing Second depth image of body;S13:The manipulation object is obtained by second depth image and first depth image 3rd depth image, obtains according to the 3rd depth image and points to point.
6. a kind of mixing control method, it is characterised in that display screen is the touch screen for itself possessing touch controllable function, the touch screen Including Touch Zone and sensing area, according to the touch screen, the touch controllable function of itself is completed the manipulation object in the Touch Zone Touch-control, in the sensing area, the control method according to any one of claim 1-5 completes touch control operation by a sensing is pointed to.
7. a kind of control system, it is characterised in that:The touch control method described in any one of 1-5, including figure are required for perform claim As acquiring unit, processor and display screen;
Described image acquiring unit is used to obtain the depth image and the depth information of manipulation object of display screen and manipulation object;
The processor includes identification module, conversion module and performing module, and the identification module is used for according to the depth map The position on display screen as pointed by obtaining manipulation object, and the predetermined control action for identifying manipulation object;It is described Conversion module is used to be converted into corresponding touch command according to predefined control action;The performing module is used for described Touch command is performed at the position for pointing to point complete touch control operation;
The display screen is used to point to the display of point and information.
8. control system as claimed in claim 7, it is characterised in that the display screen is the touch for itself possessing touch controllable function Screen, the touch screen include Touch Zone and point to area, including it is described manipulation object in the Touch Zone according to the touch screen The touch controllable function of itself completes touch-control, and touch control operation is completed by a sensing is pointed in the sensing area.
9. control system as claimed in claim 7, it is characterised in that the image acquisition unit of the control system is based on knot The depth camera of structure light or TOF principles.
10. a kind of electronic equipment, it is characterised in that the bus of the electronic equipment is provided with described in any one of claim 7-9 Control system, touch control operation is completed by the control system.
CN201610956745.4A 2016-10-25 2016-10-25 hybrid control method, control system and electronic equipment Active CN106598422B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610956745.4A CN106598422B (en) 2016-10-25 2016-10-25 hybrid control method, control system and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610956745.4A CN106598422B (en) 2016-10-25 2016-10-25 hybrid control method, control system and electronic equipment

Publications (2)

Publication Number Publication Date
CN106598422A true CN106598422A (en) 2017-04-26
CN106598422B CN106598422B (en) 2019-12-13

Family

ID=58590263

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610956745.4A Active CN106598422B (en) 2016-10-25 2016-10-25 hybrid control method, control system and electronic equipment

Country Status (1)

Country Link
CN (1) CN106598422B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107256100A (en) * 2017-05-23 2017-10-17 珠海市魅族科技有限公司 Method of toch control, device, terminal device and storage medium
CN112363666A (en) * 2020-11-05 2021-02-12 厦门厦华科技有限公司 Window adjusting method and device for electronic whiteboard
CN113095243A (en) * 2021-04-16 2021-07-09 推想医疗科技股份有限公司 Mouse control method and device, computer equipment and medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011136783A1 (en) * 2010-04-29 2011-11-03 Hewlett-Packard Development Company L. P. System and method for providing object information
CN102855066A (en) * 2012-09-26 2013-01-02 东莞宇龙通信科技有限公司 Terminal and terminal control method
CN103135745A (en) * 2011-11-25 2013-06-05 夏普株式会社 Non-touch control method and non-touch control information device and non-touch control system based on depth images
CN104423569A (en) * 2013-09-05 2015-03-18 由田新技股份有限公司 Pointing Direction Detecting Device And Its Method, Program And Computer Readable-medium
CN104978012A (en) * 2014-04-03 2015-10-14 华为技术有限公司 Pointing interactive method, device and system
CN104991684A (en) * 2015-07-23 2015-10-21 京东方科技集团股份有限公司 Touch control device and working method therefor
CN105025121A (en) * 2014-04-26 2015-11-04 刘璐 Solution for settling operation of large-screen mobile phone by one hand

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011136783A1 (en) * 2010-04-29 2011-11-03 Hewlett-Packard Development Company L. P. System and method for providing object information
CN103135745A (en) * 2011-11-25 2013-06-05 夏普株式会社 Non-touch control method and non-touch control information device and non-touch control system based on depth images
CN102855066A (en) * 2012-09-26 2013-01-02 东莞宇龙通信科技有限公司 Terminal and terminal control method
CN104423569A (en) * 2013-09-05 2015-03-18 由田新技股份有限公司 Pointing Direction Detecting Device And Its Method, Program And Computer Readable-medium
CN104978012A (en) * 2014-04-03 2015-10-14 华为技术有限公司 Pointing interactive method, device and system
CN105025121A (en) * 2014-04-26 2015-11-04 刘璐 Solution for settling operation of large-screen mobile phone by one hand
CN104991684A (en) * 2015-07-23 2015-10-21 京东方科技集团股份有限公司 Touch control device and working method therefor

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107256100A (en) * 2017-05-23 2017-10-17 珠海市魅族科技有限公司 Method of toch control, device, terminal device and storage medium
CN112363666A (en) * 2020-11-05 2021-02-12 厦门厦华科技有限公司 Window adjusting method and device for electronic whiteboard
CN113095243A (en) * 2021-04-16 2021-07-09 推想医疗科技股份有限公司 Mouse control method and device, computer equipment and medium

Also Published As

Publication number Publication date
CN106598422B (en) 2019-12-13

Similar Documents

Publication Publication Date Title
CN106104434B (en) User's handedness and orientation are determined using touch panel device
CN104049737B (en) The object control method and apparatus of user equipment
KR101872426B1 (en) Depth-based user interface gesture control
JP5807686B2 (en) Image processing apparatus, image processing method, and program
KR20140130675A (en) Image processing device, and computer program product
US10289214B2 (en) Method and device of controlling virtual mouse and head-mounted displaying device
US9378427B2 (en) Displaying handwritten strokes on a device according to a determined stroke direction matching the present direction of inclination of the device
KR101392936B1 (en) User Customizable Interface System and Implementing Method thereof
CN106598422A (en) Directivity-based control and hybrid control methods, control system and electronic equipment
CN106708255A (en) Interaction control method and system for virtual interface
CN104820584B (en) Construction method and system of 3D gesture interface for hierarchical information natural control
WO2014194148A2 (en) Systems and methods involving gesture based user interaction, user interface and/or other features
CN104516649A (en) Intelligent cell phone operating technology based on motion-sensing technology
US10073612B1 (en) Fixed cursor input interface for a computer aided design application executing on a touch screen device
CN106569716B (en) Single-hand control method and control system
CN106383583A (en) Method and system capable of controlling virtual object to be accurately located and used for air man-machine interaction
US10191553B2 (en) User interaction with information handling systems using physical objects
CN103793053A (en) Gesture projection method and device for mobile terminals
CN112328164B (en) Control method and electronic equipment
CN108829329B (en) Operation object display method and device and readable medium
KR20200065732A (en) Apparatus and method for providing user-defined shape pattern recognition interface
CN105528059B (en) A kind of gesture operation in three-dimensional space method and system
CN114647304A (en) Mixed reality interaction method, device, equipment and storage medium
CN112433624A (en) Inclination angle obtaining method and device and electronic equipment
CN104063077A (en) Touch input method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant