CN102023786A - Man-machine interaction method and structure - Google Patents
Man-machine interaction method and structure Download PDFInfo
- Publication number
- CN102023786A CN102023786A CN2009101765813A CN200910176581A CN102023786A CN 102023786 A CN102023786 A CN 102023786A CN 2009101765813 A CN2009101765813 A CN 2009101765813A CN 200910176581 A CN200910176581 A CN 200910176581A CN 102023786 A CN102023786 A CN 102023786A
- Authority
- CN
- China
- Prior art keywords
- icon
- menu
- touch
- screen
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to a man-machine interaction set suitable for a multipoint touch system.
Description
Technical field
The present invention relates to a cover man-machine interaction mechanism, especially be applicable to the man-machine interaction mechanism of multiple point touching.
Background technology
Existing man-machine interaction mechanism, comprise and be applied to user's operated system and be applied to the structure that the deviser designs program, all be mouse and the design of single-point touches screen, the appearance of multiple point touching, need new man-machine interaction mechanism to match, to bring into play its advantage and reduction difficulty based on the program of multiple point touching mode of operation.
Summary of the invention:
A kind of menu that is applicable to the multiple point touching operation is characterized in that the touch point disappearance of menu of breathing out, and does not have the touch point that menu is being operated on the menu that then menu disappears.A kind of icon is characterized in that can moving before activating, and after activating, to moving of icon, clicks the corresponding function corresponding of pressing operation.A kind of control mode that is used for bimanualness is characterized in that a hand is used for determining operating position, another hand or two hand control operation simultaneously effects.A kind of method based on multiple point touching operating operation 3D object is characterized in that it being controlled based on the base shape of accurate touch image and other preset parameter generation 3D object again, determines the 3D figure and the locus thereof of ultimate demand.A kind of touch keyboard is characterized in that keyboard is divided into a plurality of parts, lays respectively at the screen centerline both sides.The interface of a kind of touch tool and hand co-operate is characterized in that hand is used to control the function of touch tool correspondence.A kind of program architecture is characterized in that generating respective objects according to touching image and touch pressure, transfers to processing such as physical engine, to realize touching the real-time generation of effect.A kind of multiple instruction mechanism is characterized in that allowing the execution priority of self-defined every the instruction of user.A kind of self-defined touch instruction editing machine is characterized in that by touch operation is decomposed into standard operation, and the user can be instructed based on the self-defined touch of standard operation.
Description of drawings:
Fig. 1 is several patterns of menu to Fig. 4
Fig. 5 is that menu appears at and the menucommand of breathing out sends the interface synoptic diagram of diverse location
Fig. 6 is a synoptic diagram at felt pen interface
Fig. 7 is a kind of touch instruction of selected target
Fig. 8 is the touch image of five fingers
Fig. 9 is one and touches image
Figure 10 is people's both hands
Figure 11 is a kind of touch instruction of using menu
Figure 12 is the pattern of several menus
The pattern of several icons of Figure 13
Figure 14 is the pattern of a kind of icon or menu
Figure 15 is the synoptic diagram that 4 fingers are pushed down icon
Figure 16 is the synoptic diagram of a kind of form of bimanualness
A plurality of icons have appearred in Figure 17, finger touch the synoptic diagram of 3 icons wherein
Figure 18 is another synoptic diagram of bimanualness
Figure 19 is the synoptic diagram of with the hands operating according to the object that touches the image generation.
Figure 20 is the interface synoptic diagram that touches the instruction editing machine.
Embodiment:
1, a kind of menu that cooperates the multiple point touching operation
The pattern of menu can adopt but be not limited to Fig. 1, Fig. 2, and Fig. 3, several patterns of Fig. 4,
The exhalation menu can adopt following several method:
1, pass through respective icon
2, menu occurs automatically after selected one or more objects
3, instruct by corresponding touch:
For example one on hand one or more finger keep motionless, other finger slides or knocks, or pushes screen with different big or small strength.For example, among Fig. 8,7,8,9,10th, 4 fingers of people, the 6th, thumb, the 11st, the zone that palm contacts with screen.The user can be with 7,8, and the one or more finger contact screens in 9,10 are motionless, one or more fingers are clicked or slide, or do not frame out, but push the screen menu of breathing out with different pressure wherein, and the touch of exhalation menu is instructed can be very not complicated, the menu because the user will breathe out fast, for example among Fig. 8,6,9,10 keep motionless, and 7,8 slide, or 7,8,9,10 is motionless, and 6 slide the menu of breathing out.This all is people's operation naturally, and is very difficult because people's little finger moves separately.
Further, can be set at and to carry out the above instruction that touches when having only 11 to touch screen.Can avoid like this because prevent that false touch from touching and time-delay is set causes influencing smooth operating experience.
Also can be set at 16 and 11,6 menus of all on selected object or zone, promptly breathing out contact screen the time.
The corresponding touch instruction of exhalation menu can also include but not limited to: in Fig. 9,17,171, point or touch tool for 3 of 172 corresponding users, 17 keep touching with touch-screen, with 171,172 knock screen with certain combination, the menu that different combinations is corresponding different, for example keep under 17 situations about touch touching, 171 click screen once or twice, or 171 slide on screen along any direction, or 17 and screen situation about keep touching under 172 click screen once or twice, or 172 slide on screen along any direction.Or one or more in 17,171,172 remain and the contacting of screen, but push screen with different dynamics, for example the heavy and light weight.
The touch instruction of exhalation menu also can be used as the touch instruction of selected target simultaneously, in other words, target, 7,8 in the process of exhalation menu, have just been selected, 9,10 4 fingers also can add 6 thumbs, 11 palms, for the zone at summit or edge can be considered as a selected district, one or more objects or zone in the selection area.
The function of menu
The operation that can carry out includes, but are not limited to:
1, the slip menu to be demonstrating other icon,
2, the icon on the selected menu, the icon that knocks on the menu starts corresponding function,
3, pin on the menu and control corresponding function with the pressure sensitive function after the respective icon,
4, pin icon it is hauled out menu, with the submenu of the corresponding function that launches the icon correspondence, or be its establishment shortcut,
5, drag other icon it is added menu.
6, menu can be adjusted size according to the quantity of the icon of a less important demonstration and the size of each icon, and the size of menu and the size of each icon can be set up on their own by the user, also can automatic setting.
7, if the icon on the menu is a lot, shows once and do not come that menu can determine the position of icon according to the frequency of utilization of icon, or forms a common list, the preferential icon that shows on the common list under the default situations.
8, can or on the icon of menu, slide by the label on the click menu, or different editions menu is switched in slip on the ad-hoc location of menu.For example among Fig. 3, the 20th, the sliding area of menu, sliding on the Y direction of sliding area upper edge is slip icon in a certain version, icon is shifted to the menu edge until disappearance along the y direction successively, along slip icon on the x axle then is that justifying is switched, all icons all are switched, and for example original demonstration is the icon that is used for " checking ", have just become the icon that is used to regulate font after switching.Sliding area might not be in the same place with the chart viewing area, also can separate, so that bimanualness
9, menu can be set special sliding area, is considered as regulating the pairing function of icon and will slide on corresponding icon, for example increases or reduce volume, brightness, the width of figure, the height of 3D figure etc.As Fig. 6, menu can the setup menu control zone, in order to the option of setup menu to be provided, for example, 1, peg menu, make the finger of all operations menu leave menu after menu can not disappear yet, 2, increase the width of menu, show more icon simultaneously.If not special the setting requires not peg menu, menu will disappear after menu is left in the touch point of exhalation menu or all touch points disappear after leaving menu automatically, can be the operating experience that the user provides smoothness like this.
10, the icon that the icon of menu is convenient to operating position near the user can become bigger than the icon of other position, so that user's operation.
11, menu might not appear at the position of the operation generation of exhalation menu.The position that position that the operation of exhalation menu takes place and menu occur is positioned at the two ends of screen centerline, is convenient to bimanualness like this.
Fig. 1 is several patterns that menu can adopt to Fig. 4:
Fig. 1 is a basic menu, and 17 have represented the finger or the touch objects of one or more menu that is used to breathe out, the 29th, a plurality of options of menu.Be applicable to simple operations, slide on icon along x or y axle that finger is corresponding respectively to switch new icon, or regulate the pairing function of icon, or icon is hauled out one of three kinds of functions of menu creation shortcut.Push icon and also can regulate the pairing option of icon, for example firmly pin the icon that increases brightness
Among Fig. 2, the 24th, sliding area, the 27th, menu option, the 25th, menu control district.
The center of circle with menu on 27 is to slide in the center of circle, can be in order to regulate the pairing function of option, for example brightness
, can be set at clockwise direction for increasing brightness, intend clockwise for reducing brightness.
Is that icon is hauled out menu with the establishment shortcut along radial direction to the outer slip icon of menu, otherwise is to add icon on menu.
The 25th, the menu control district, be used for Control-Menu, for example peg menu and make menu after finger leaves, also can not disappear or make the picture mark position on the menu adjustable, for example, after icon on being set at menu allows to move, can drag icon and change the position of icon on menu.
Among Figure 12, the 203rd, option, by the touch point of sliding clockwise or counterclockwise in 203 upper edges, for example finger starts corresponding function or increase and reduces relevant parameter, for example brightness, figure height etc.Relevant parameter is regulated in the touch point or startup wants to use function by sliding up and down in 204 upper edge y directions.Horizontally slip the touch point without any effect, slided between different options in the touch point and do not cause maloperation.For example the user is with hand-held when holding screen, the easy contact screen of time standby that finger moves.
Behind the exhalation menu, the finger of exhalation menu continues to operate on the menu of breathing out with extra finger by motionless on screen.Menu might not appear at the position of the operation generation of exhalation menu.The position that menu occurs can be set by the user, be fixed in a certain zone of screen, for example on the left side of screen, also can be software basis user's hand and operation tool at that time, for example the residing position of writing pencil and contact image and user's custom is set in real time, for example, after the user is right-handed with default, near the finger that takes place to touch with screen that the control area just is positioned at the operation tool left end all the time.Or when the operation of exhalation menu occurred in screen one side, menu appeared at the opposite side of screen centerline, especially the idle finger of opposite side near.For example hold the equipment both sides when seeing a film when user's both hands, the mode of clicking based on multiple pressure with the thumb of a hand is breathed out after the menu, with just can the slide sliding area of menu of the thumb of another hand, thumb is placed on the chart of corresponding volume, the thumb that slides is then regulated volume.System also can determine the position that menu occurs according to the quantity of each zone finger on the screen.About system can differentiate by gravity sensor.Certainly, menu can appear at the position of the operation generation of exhalation menu.
Pattern for example shown in Figure 5, the 171st, one or more finger of left hand, the 172nd, one or more finger of the right hand.
The 202nd, the control zone of menu 201 or sliding area.The user is by 171 exhalation menus 201, controls 201,201 function and the patterns that can adopt the menu of similar Fig. 2 with 172, also can adopt the menu of the described pattern of similar Figure 12.202 can adopt also and can not adopt, and its function is the icon in amplification 201 and the switching 201.Can be set at along respectively corresponding these two functions of x axle y axle slip.
The pattern of Fig. 5 is well suited for holding function screen under the situation of equipment at both hands, for example when seeing a film, can adjust brightness, contrast etc. with 201.
Figure 11 is a kind of touch instruction of breathing out and using menu, the 173rd, one or more also neat finger together, use 173 knock screen or with greater than the pressure of certain numerical value by in motionless a period of time of screen, or push after the screen with different weights, the exhalation menu, 173 keep motionless, use 174.
Menu can disappear automatically in following situation:
Situation A: menu disappears when the touch point of the finger exhalation menu of exhalation menu disappears;
Situation B: or menu disappears when not having the touch point on the touch point while of the exhalation menu menu;
Situation C: menu disappears after menu has been operated stipulated number.
2, touch tool and operation interface
Writing pencil is write on screen.In another zone of screen, the pairing function of finger control writing pencil.
For example, hand of user is held writing pencil and is write on screen, when writing pencil and screen come in contact, suitable position on the screen, this position can be set by the user, be fixed in a certain zone of screen, for example on the left side of screen, also can be software sets in real time according to user's hand and the residing position of pen and contact image and user's custom at that time, for example, after the user is right-handed with default, near the finger that takes place to touch with screen that the control area just is positioned at the operation tool left end all the time.About system can differentiate by gravity sensor.
The control area shows the menu of one or more alternative icons simultaneously.The pattern of menu can have multiple pattern.
For example, adopt pattern as Fig. 1, the 71st, control zone, the 72nd, touch screen operating tool.Show possible alternative icon in 71, function corresponding is provided when the user touches, for example: check, annotations and comments, color is revised in deletion, inserts etc.Menu area can slide, when user's thumb slides on menu, icon on the menu just slides to respective direction, conceal old icon, show that the icon that makes new advances can be imagined as whole menu spheroid or the annulus of a centre of sphere below screen, the menu of demonstration partly is its part.Icon on the menu and picture mark position can artificially be edited, also can be by system according to frequency of utilization auto arrangement position, perhaps generating a common list, is the icon that shows common list under the situation of acquiescence, and other icon need slide and just show.Icon on the menu can be that writing pencil switches to corresponding function after clicking, and also can be to pin not loose one's grip to switch to corresponding function, promptly switches to other preset state after loosing one's grip.Icon on the menu also can have pressure sensitive, for example controls the button of writing pencil person's handwriting shade, thickness and just can control color, thickness by pressure sensitive, and for example pressure is greatly dark, thick.In order to prevent in the slip menu, maloperation to occur, independently sliding area can be set, for example Fig. 1 is the slide district of menu.The control zone also can adopt alternate manners such as exhalation formula icon that option is provided.Even can directly adopt and touch instruction and do not adopt any way to provide guiding the user.The pattern as Figure 72 1 also can be adopted in 72 control zones, slides along the direction of arrow and divides in addition corresponding " shearing " and " pasting ".
The icon menu of control zone is dynamic change.According to the touch track and the touch location of writing pencil, pressure, pairing function is switched not menu on the same group.For example writing pencil is made as the menu that will occur the function that to use in the new correspondence annotations and comments after the annotations and comments.
The control zone can be applied to finger equally to the control mode of special operating tool, and for example, when finger was selected the objects such as literal of a panel region, the control zone corresponding menu option can occur equally, and the rest may be inferred for other function.
Touch tool can be by being similar to the mode of optical mouse, by contrasting the image that CCD scans at a high speed, determine the translational speed of oneself, because screen self can be luminous, therefore writing pencil does not need as mouse luminously voluntarily, watches content on the screen to avoid screen reflecting to influence the user.
3, split type keyboard
Split type keyboard.Keyboard is positioned at the edge of screen, make user's hand can be when holding equipment operation keyboard, especially both hands operation keyboard simultaneously.For example scheme on " split type keyboard ", shift and numerical key are on the both sides of screen centerline, and the user can with the hands press shift and numerical key simultaneously when both hands hold equipment.Split type keyboard is not limited to the pattern of computer keyboard, can set any button, even sliding area can be set.
4, the free style icon
After the user carries out a certain operation, demonstrate a plurality of icons on the screen, in order to strengthen visual effect, the mode that icon occurs can be from a certain zone, fly out on for example chosen operand, also can be to appear in one's mind out in a certain zone gradually.After icon was activated, the user can pass through mobile icon all around, pushes, knocks, controls corresponding function by different pressure.Icon can be to allow mobile also can fixing on screen.Icon can be considered as protruding from screen surface, and the touch point can promote icon by hookup target edge and move, just as the glass blocks on the mobile glass plate.The user can promote a plurality of icons simultaneously or shift one side onto hindering the icon of operating to gather with hand.
A plurality of free style icons have appearred in Figure 17 on the operation interface, finger touch the synoptic diagram of 3 icons wherein.Wherein, the 42nd, icon, the 17th, finger, the 43rd, the icon that is touched.
User's icon of can breathing out by the following method
1, by using specific icon
2, on certain zone or object, use corresponding touch instruction, the touch instruction of the menu of for example breathing out can be used to the icon of breathing out equally.
3, a selected zone or object carry out corresponding operating in the another location, and for example touch instruction or click other icon, or selected simultaneously another object, the exhalation respective icon.
Icon occurs automatically after 4 selected one or more objects
5. carry out corresponding operating after selecting one or more objects, for example touch is instructed or one or more is pointed contact screen and remains to the icon appearance.
Or other method exhalation icon.
The pattern of icon: as shown in figure 13, wherein 17 is touch points that any one or more finger or other touch objects form, 30,31,32,33,34,35,36,37th, the prompting user moves the pairing function of icon to all directions, and is adjusted the parameter numerical value of this moment.They are can be with user's operation real-time change.30 to 37 color also can change, and can represent the pressure pointed or the numerical value of a certain parameter qualitatively by color.
The user will wish that the icon that uses activates, and use multiple click then, and all around mode such as mobile icon is used corresponding function.This class icon is used for adjusting the size of a certain parameter more, selected another zone handle icon time for example, and icon can be in order to adjust the color of selection area.A selected 2D or 3D figure in handle icon, icon can be adjusted its shape, and height on the X/Y/Z axle is with X/Y/Z axle or other object angulation etc.
Activating icon is exactly the function of bringing into use the icon correspondence.Icon can not be moved after being activated, and mobile icon and clickable icon apply different pressures etc. to icon and all will be regarded as function operations to the icon correspondence.
The pattern of icon
When the unactivated icon of finger contact, icon can
1, become big
2, show the appearance after being activated
3, show the details of relevant icon function,
Icon is passable after being activated
1, become big
2, demonstrate more suggestion content, indicative sign appears in the front end of pushing down the finger of icon, informs that the user is preceding at x, y, the function of mobile icon correspondence on the positive negative direction of z axle.
3, the icon division also may split into a plurality of icons.The user has activated an icon, and the finger that contacts with screen of user the subgraph mark all occurred 2,3 times afterwards, can regulate corresponding functions by moveable finger 2,3.
Icon also can carry out further dividing after some operation after being activated, for example, after icon is activated, demonstrated the prompting that icon is further operated, or icon just is in state of activation when occurring, the user makes its further division by respective icon is operated.
Icon also can just be in state of activation when occurring.
The example person of being to use has selected a zone, and 5 fingers below or near the icon of knowing clearly and having activated accordingly of all occurring afterwards, the icon of this moment also can be regarded as a kind of form of special menu.
For example, the user has activated " demonstration " icon, the adjusting saturation degree just occurred, brightness under user and the several fingers that screen contacts afterwards, colour cast, four icons of contrast, the prompting user moves down icon on the colour cast icon, and the user moves down after the colour cast icon, icon under three fingers has just become red partially, green partially, yellow partially, for adjusting.
The activation of icon
1, push down if can be set at the icon touched point, just activate icon, mobile icon can only promote icon by hookup target edge.
2, special Petting Area and operational zone can be set on icon, touching Petting Area is mobile icon, the touch operation district brings into use icon.
3, can set special operational zone, after moving into the operational zone, icon activates icon
4, can activate icon with specific touch instruction.Especially the thumb with the hand of selecting icon activates icon, " thumb "
Activate by Figure recognition, for example, the palm of the hand at the finger place by identifying selected icon has been pushed down screen and has been activated icon.
5, can activate icon with the mode of multiple click.
6, can apply different pressure to icon during with handle icon at mobile icon, for example when using icon, on icon, apply bigger pressure, to distinguish mutually with the operation of mobile icon.
7, can shown in " activation icon ", when two or more icons are pushed down by finger, activate the icon of being pushed down.In this case, need a special activation graph target icon that is used for, so that activate icon when only needing to use an icon.
8, can set a special icon, when it is pushed down by finger with other icon, activate the icon of being pushed down.
After the user does not need to continue to use icon, icon is disappeared by suitable operation.
1, for example, if the icon of breathing out is that after icon was used up, and the touch point of handle icon disappears, icon promptly disappeared automatically in order to parameters such as adjustment screen intensitys.
2, can use specific touch instruction, figure " elimination icon " is the touch instruction the adopted touch image to screen, both part palm contact screens of user, traversing palm is holded up palm simultaneously gradually, until using the palmar side contact screen, just as scanning out screen to icon.Also can be directly just traversing on screen with palmar side, make the icon disappearance or switch the new icon of a screen with this.For preventing maloperation, can be set at and just be considered as this when the movement velocity of having only palm is higher than a certain value and touch instruction.
3, if icon is to be used to operate selected zone, when the zone is cancelled when selected, icon promptly disappears automatically.
4, also can set a special zone, icon enters this zone and promptly disappears.
5, a kind of menu or icon style: as shown in figure 14, it has a plurality of options 231,232,233, allows a plurality of fingers to operate simultaneously, and in operation, it can similarly be the belt of a softness along with the slip distortion of finger.The user can pass through specific operation, for example several icons that usually use is together put into a specially special icon container, they can together be moved form a menu or icon.Can rename and define outward appearance to it.If it is a free style icon, then when being activated, it should become big according to user's finger spacing, for example adopt pattern shown in Figure 14, to adapt to a plurality of fingers with operation simultaneously.
6, selected and activation manipulation
1, as shown in Figure 7, be the touch instruction of object in an a kind of selected zone or the zone, 131 and 141 are not to use respectively the side of two palms of person.Concrete operations can be that two palms carry out one and reduce central area between the two, and are selected or shear the target area.Can be equipped with the corresponding dynamic effect, for example after moving a segment distance, central area between the two keeps static a period of time at palm, excite corresponding operating, afterwards, the highlighted marking of the position at place was gradient to the static position of palm when palm began to move, and the operation of expression system validation excites.
2, in conjunction with Fig. 8, Figure 10 introduces the mode of another kind of selection area, and 7,8,9,10th, people's 4 follows finger, the 6th, thumb, the 11st, people's the centre of the palm.By 6,7, any in 8,9,10,11 can be selected object in a zone or the certain zone for edge or edge vertices more than three or three.Wherein, thumb 6 and palm 11 can be set to crucial touch point, promptly have only when 11 1 or the thumb 6 of thumb 6 or palm and palm 11 simultaneously touchscreen system think that just the user prepares to select operation, system can carry out corresponding prompt, for example highlighted chosen or zone or object that can be chosen.Also can touch the zone that image covers and select zone or object by touch objects.
Can also further operate afterwards, include but not limited to
1, thumb 6 slides along direction shown in the arrow or opposite direction,
The maintenance of 2 palms contacts with touch-screen, pushes touch-screen with the pressure of different weights, perhaps knocks touch-screen.
3,7, finger or 8, one or two in the finger simultaneously along direction shown in the arrow or slide in the other direction or along the x axle just/negative direction, the y axle just/negative direction slides.
4,7, finger or 8, one or two in the finger carry out following operation
1, knock screen,
2, under situation about not frameing out, pushing screen with different pressures, for example 7,8 simultaneously successively with heavy, and light, heavy pressure is pushed screen.
5, keep the touch point of the selected operation that is useful on not frame out.Center with selection area is the center rotation.
These operations can be used for
1: confirm selectedly, be used in selected touch point and still keep selected to zone or object after leaving.
2: shear selected object
3: exhalation menu or exhalation formula icon etc.
4, on selected object, carry out the operation that has configured.For example on selected zone, paste.In recreation, the unit in the selection area is carried out supply etc.
If finger has been selected object, for example Figure 15 has exhalation formula icon under the finger, perhaps just at actions menu, has perhaps selected a plurality of operands, can take following operation
1, thumb 6 slides along direction shown in the arrow or opposite direction,
The maintenance of 2 palms contacts with touch-screen, pushes touch-screen with the pressure of different weights, perhaps knocks touch-screen.
3,7, finger or 8, one or two in the finger simultaneously along direction shown in the arrow or slide in the other direction or along the x axle just/negative direction, the y axle just/negative direction slides.
4,7, finger or 8, one or two in the finger carry out following operation
1, knock screen,
2, under situation about not frameing out, pushing screen with different pressures, for example 7,8 simultaneously successively with heavy, and light, heavy pressure is pushed screen.
5, keep the touch point of the selected operation that is useful on not frame out.Center with selection area is the center rotation.
These operations can
1, activation/releasing activation manipulation object for example activates free icon and makes it to enter operable state/enter untapped state/disappearance.
2, switch to other menu.
3, peg object, make it after finger leaves, also can not disappear.
7, bimanualness
Comprise three kinds of situations
1, one hand is used for selected object or zone, and the another hand is used for selected object or zone are operated.
For example among Figure 16, the 6th, thumb, 7,8,9,10th, finger, the 11st, palm has been selected a plurality of objects 51,52,53.1,2,3,4th, finger, the 5th, thumb, the 12nd, palm.38,39,40,41st, the free style icon also can use menu or pattern shown in Figure 14 provides option.The user can be by the same mode of actuator-activated free style icon, and by all around moving, different pressures is pushed screen, knocks the mode of screen and operates chosen object.
2, two hands are adjusted a parameter with finger then simultaneously with the together selected one or more object of a plurality of fingers.
3, one fixed objects of picking, two hands are operated this object simultaneously.For example Figure 18 points 6 by 16, thumb, the projection of 11 palms control 3D object 61 on plane, touch-screen place or curved surface is by 16, one of color/thickness of 11,6 pressure control 61 is by 15,6,11 push screen, on screen, slide and wait other parameter of control 61, for example its coordinate figure on the z axial coordinate, thickness is with angle of coordinate axis or a certain object of reference etc.Control the locus and the shape of figure 61 by two hands with this.Can also further control 61 pairing functions etc., for example 61 if a brush, 15 can control the concentration of the color that it brushes out, 16 can control the concentration of the color that brushes out on its diverse location, for example 16 certain root finger increase screen pressure, and the color density that then corresponding with it part brushes out increases.
The application example of a bimanualness is in recreation, with shrouding the selected logistics car of gesture, a plurality of options appear down in the finger of contact screen, selected their pairing options of one or more root finger with wherein for example knock screen with them, have selected additional ammunition, and maintenance, chosen function choosing-item has become slider bar, can move forward and backward finger with selected degree of replenishing ammunition and maintenance degree, for example is to replenish the ammunition that meets the eye on every side mark 70% or 80%.As figure " recreation ".Afterwards with the selected target that needs to replenish ammunition and maintenance, if selected a slice target can be implemented the operation that configures with shrouding gesture.Three kinds of delivery modes are arranged.
A kind of person of being to use touches the logistics car all the time, and this moment, the every other object that touched by the user all can be implemented operation, after the user does not touch the logistics car, the object that is touched just can be not again by supply.A kind of is only to be set at thereafter first to be touched object to carry out supply.Can also be on the logistics car, to finish those can set after setting and preserve or system preserves automatically, after select other object after, just can in menu, find this option, the logistics car will carry out supply to it afterwards.
This process also can promptly keep after the selected object the selected again logistics car of the touch of object conversely.Can set earlier chosen object wants initiatively near the chosen object in back.
8, a kind of control is according to touching figure, the especially method of 3D figure that image generates
Determine the profile of object by touching image, by the attitude and the position of finger control object.
For example the user is placed on a hand on the touch-screen, with this state of determining virtual palm, be that fork is pointed or five are closed up, or Qu Qiyi only points.If system knows the general shape that touches object in advance, for example learn that by setting the user can be placed on a hand on the screen in advance, then can judge the roughly attitude of hand according to the touch image on the plane, appearance for example shown in Figure 19 is exactly user's left index finger that risen in the wrong, because 4, forefinger and 11, the touch image between the palm is discontinuous.The user can also be by pushing that double swerve is represented to continue.For example, the user can keep palmar side to contact with touch-screen with the palmar side contact screen, firmly pushes touch-screen, on the plane at palm place, swing palm fast, in swing process, allow the edge short time of palmar side because be swung away from touch-screen.This action is controlled virtual palm along the incision of palmar side direction.
Among Figure 19, the finger 16 of user by the right hand, thumb 6, and whole palm and palm the attitude in space of x axle angulation control system according to the touch image generation of left hand.Serves as the angle of axle rotation by the virtual palm of the Spin Control of the right hand on screen with the y axle with respect to user's left hand, controlling it with thumb serves as the angle of axle rotation with respect to user's left hand with the x axle, controls its angle along the z axle rotation on plane, normal to screen place with other 4 fingers.
9, touch the instruction editing machine
By the movement decomposition with hand is the series of standards action, the user can be defined voluntarily touch instruction.
For example a kind of sorting technique
To touch instruction and be divided into a plurality of time periods, and define the action in each time period respectively, one touches instruction and can be made of a plurality of time periods.
Each hand is divided into a plurality of parts
As Figure 10 is people's both hands, is example with the left hand
11, palm lower edge, 13, whole palm, 151, little finger, 152, the third finger, 153, middle finger, 154, forefinger, 155 thumbs
1, little finger finger tip, 2, nameless finger tip, 3 middle fingertips, 4 forefinger finger tips, 5 thumb finger tips.
131, the palm middle part.
System is according to Figure recognition and each touch point various piece of the method identification palm of relative position each other.
Each part all has standard operation: the standard operation of finger and finger tip comprises: have neither part nor lot in touch, slide along x axle or y, slide to contiguous finger, knock screen and number of times, push screen and weight order, the duration, the original place is reversed, and slides (to describe with x axle y axle angulation) along self-defined direction, slides on screen along particular track, track comprises: circle, ellipse, triangle, trapezoidal, parallelogram, five jiaos of types, serrate.And self-defined track, the user can be self-defined with drafting instrument, uses test pattern and broken line, straight line, and curve is formed new track.
Palm, palm lower edge, palm middle part, standard operation comprise: have neither part nor lot in touch, slide, knock screen and number of times along x axle or y, push screen and weight order, the duration, the original place is reversed, slide (to describe with x axle y axle angulation) along self-defined direction, slide on screen along particular track, track comprises: circle, ellipse, triangle, trapezoidal, parallelogram, five jiaos of types, serrate.And self-defined track, the user can be self-defined with drafting instrument, uses test pattern and broken line, straight line, and curve is formed new track.
The user can adopt the self-defined touch instruction of the form of coding
Can adopt this kind form
Which what action of part of time period hand
Some touches instruction and does not limit the finger of carrying out touch order, and can adopt this description this moment: certain finger or palm, the palm lower edge, the palm middle left, to the right, which root finger upwards.
The various piece of exercises and hand can be given respective code, as each several part on hand with capitalization, thumb is represented with A, action is code with the lowercase, track is code with the numeral, is a as sliding, and elliptical orbit is 2 clockwise, then sliding clockwise along elliptical orbit is (a2), and in addition bracket is to prevent confusion.
Also can make the user self-defined in the mode that menu is selected.Directly perceived so more and be convenient to operation.As Figure 20, wherein 71 is menu, and 72 is button.
Instruct owing to adopted standardization, user can use keyboard to send some simple touches, realized touching the seamless connection of instruction and keyboard operation.
Can take identification mode equally, allow the user under the guiding of system, finish and touch instruction.
Time period according to system suggestion, the user implements gesture step by step.So that system identification.
10, multiple attack instruction.
After an object is assigned an order, when assigning new instruction, can set the priority of new instruction with respect to existing instruction.For example, after tank of order moves in recreation, open fire to a certain target to its order again, at this moment just can set the priority of opening fire with respect to moving.
Order it to evade a certain zone afterwards again, just can set and evade this operation of a certain zone with respect to the priority of opening fire and moving this moment.
11, touch engine
Touch image according to the user, especially the touch image of user's hand, put on the pressure of screen, and other preset parameter, generate one corresponding with it, can be by physical engine, the object that special efficacy engine etc. are handled is transferred to physical engine with this touch effect with the user, special efficacy engine etc. is handled, and generates in real time.。System should built-in a series of default parameterss, as hand and other touch tool be not endowed other parameter the time use.The virtual object that generates does not need to show, and it only generates for touch information being converted to physical engine and the treatable information of other engine.For example, object the moving among a small circle on the z axle that the pressure size of touch generates in order to decision is perhaps to other object applied pressure.For example in recreation, the user selectes a navvy, then the physics material of system's object that will generate according to user's finger touch image is set at navvy corresponding physical material, determine its size and excavating depth according to touch pressure, determine the contact area on it and ground according to touching image to the ground applied pressure.At this moment, the user press...withes one's finger and presses ground in the recreation, or with other object, as the tank collision,, set in advance and need not the Game Developer just can produce the effect that navvy dug, alleviated Game Developer's workload greatly.
Claims (9)
1. a menu that is applicable to the multiple point touching operation is characterized in that the touch point disappearance of menu of breathing out, and does not have the touch point that menu is being operated on the menu, and then menu disappears.
2. an icon is characterized in that can moving before activating, and after activating, to moving of icon, clicks the corresponding function corresponding of pressing operation.
3. a control mode that is used for bimanualness is characterized in that a hand is used for determining operating position, another hand or two hand control operation simultaneously effects.
4. the method based on multiple point touching operating operation 3D object is characterized in that it being controlled based on the base shape of accurate touch image and other preset parameter generation 3D object again, determines the 3D figure and the locus thereof of ultimate demand.
5. a touch keyboard is characterized in that keyboard is divided into a plurality of parts, lays respectively at the screen centerline both sides.
6. the interface of touch tool and hand co-operate is characterized in that hand is used to control the function of touch tool correspondence.
7. a program architecture is characterized in that generating respective objects according to touching image and touch pressure, transfers to processing such as physical engine, to realize touching the real-time generation of effect.
8. a multiple instruction mechanism is characterized in that allowing the execution priority of self-defined every the instruction of user.
9. a self-defined touch instruction editing machine is characterized in that by touch operation is decomposed into standard operation, and the user can be instructed based on the self-defined touch of standard operation.
Priority Applications (9)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009101765813A CN102023786A (en) | 2009-09-23 | 2009-09-23 | Man-machine interaction method and structure |
| JP2012530118A JP2013505505A (en) | 2009-09-23 | 2010-09-22 | GUI (Graphical User Interface) structure method and method in touch operation environment |
| KR1020127010155A KR20120085783A (en) | 2009-09-23 | 2010-09-22 | Method and interface for man-machine interaction |
| US13/497,560 US20120182296A1 (en) | 2009-09-23 | 2010-09-22 | Method and interface for man-machine interaction |
| CA2775007A CA2775007A1 (en) | 2009-09-23 | 2010-09-22 | Method and interface for man-machine interaction |
| EP10818417A EP2482175A1 (en) | 2009-09-23 | 2010-09-22 | Method and interface for man-machine interaction |
| PCT/CN2010/077239 WO2011035723A1 (en) | 2009-09-23 | 2010-09-22 | Method and interface for man-machine interaction |
| AU2010297695A AU2010297695A1 (en) | 2009-09-23 | 2010-09-22 | Method and interface for man-machine interaction |
| CN2010800421918A CN102812426A (en) | 2009-09-23 | 2010-09-22 | Method And Interface For Man-machine Interaction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009101765813A CN102023786A (en) | 2009-09-23 | 2009-09-23 | Man-machine interaction method and structure |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102023786A true CN102023786A (en) | 2011-04-20 |
Family
ID=43865132
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009101765813A Pending CN102023786A (en) | 2009-09-23 | 2009-09-23 | Man-machine interaction method and structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102023786A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013021409A (en) * | 2011-07-07 | 2013-01-31 | Olympus Imaging Corp | Imaging apparatus, imaging method, and program |
| CN103106006A (en) * | 2013-03-01 | 2013-05-15 | 广东欧珀移动通信有限公司 | Method of sorting desktop icons fast and device |
| CN103365400A (en) * | 2012-03-26 | 2013-10-23 | 联想(北京)有限公司 | Method and device for feedback of gesture recognizing operations |
| CN103858078A (en) * | 2011-10-03 | 2014-06-11 | 古野电气株式会社 | Device having touch panel, radar device, plotter device, marine network system, symbol identification method and symbol identification program |
| CN104461324A (en) * | 2013-09-16 | 2015-03-25 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN104571849A (en) * | 2013-10-18 | 2015-04-29 | Lg电子株式会社 | Wearable device and method for controlling the same |
| CN104781777A (en) * | 2012-09-13 | 2015-07-15 | 谷歌公司 | Providing radial menus with touchscreens |
| CN104881110A (en) * | 2014-02-28 | 2015-09-02 | 三星麦迪森株式会社 | Apparatus And Method Of Processing A Medical Image By Using A Plurality Of Input Units |
| US9261989B2 (en) | 2012-09-13 | 2016-02-16 | Google Inc. | Interacting with radial menus for touchscreens |
| WO2016101160A1 (en) * | 2014-12-24 | 2016-06-30 | Intel Corporation | User interface for liquid container |
| CN106126049A (en) * | 2016-06-22 | 2016-11-16 | 宇龙计算机通信科技(深圳)有限公司 | Menu-operating method and system |
| CN107274174A (en) * | 2017-05-18 | 2017-10-20 | 青岛海信移动通信技术股份有限公司 | The auth method and mobile terminal of a kind of mobile payment |
| CN107577404A (en) * | 2013-03-21 | 2018-01-12 | 联想(北京)有限公司 | Information processing method, device and electronic equipment |
| CN110716683A (en) * | 2019-09-29 | 2020-01-21 | 北京金山安全软件有限公司 | Generation method, device and equipment of collision object |
-
2009
- 2009-09-23 CN CN2009101765813A patent/CN102023786A/en active Pending
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9075459B2 (en) | 2011-07-07 | 2015-07-07 | Olympus Corporation | Imaging apparatus, imaging method, and computer-readable storage medium providing a touch panel display user interface |
| JP2013021409A (en) * | 2011-07-07 | 2013-01-31 | Olympus Imaging Corp | Imaging apparatus, imaging method, and program |
| US9678657B2 (en) | 2011-07-07 | 2017-06-13 | Olympus Corporation | Imaging apparatus, imaging method, and computer-readable storage medium providing a touch panel display user interface |
| CN103858078A (en) * | 2011-10-03 | 2014-06-11 | 古野电气株式会社 | Device having touch panel, radar device, plotter device, marine network system, symbol identification method and symbol identification program |
| US9891728B2 (en) | 2011-10-03 | 2018-02-13 | Furuno Electric Co., Ltd. | Device having touch panel, radar apparatus, plotter apparatus, ship network system, symbol specifying method and symbol specifying program |
| CN103365400A (en) * | 2012-03-26 | 2013-10-23 | 联想(北京)有限公司 | Method and device for feedback of gesture recognizing operations |
| CN104781777B (en) * | 2012-09-13 | 2019-04-16 | 谷歌有限责任公司 | Radial menu is provided using touch screen |
| CN104781777A (en) * | 2012-09-13 | 2015-07-15 | 谷歌公司 | Providing radial menus with touchscreens |
| US9261989B2 (en) | 2012-09-13 | 2016-02-16 | Google Inc. | Interacting with radial menus for touchscreens |
| CN103106006B (en) * | 2013-03-01 | 2016-08-24 | 广东欧珀移动通信有限公司 | A kind of method and device of fast finishing desktop icons |
| CN103106006A (en) * | 2013-03-01 | 2013-05-15 | 广东欧珀移动通信有限公司 | Method of sorting desktop icons fast and device |
| CN107577404A (en) * | 2013-03-21 | 2018-01-12 | 联想(北京)有限公司 | Information processing method, device and electronic equipment |
| CN104461324B (en) * | 2013-09-16 | 2017-12-29 | 联想(北京)有限公司 | A kind of information processing method and electronic equipment |
| CN104461324A (en) * | 2013-09-16 | 2015-03-25 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN104571849B (en) * | 2013-10-18 | 2017-12-26 | Lg电子株式会社 | Wearable device and its control method |
| CN104571849A (en) * | 2013-10-18 | 2015-04-29 | Lg电子株式会社 | Wearable device and method for controlling the same |
| CN104881110A (en) * | 2014-02-28 | 2015-09-02 | 三星麦迪森株式会社 | Apparatus And Method Of Processing A Medical Image By Using A Plurality Of Input Units |
| WO2016101160A1 (en) * | 2014-12-24 | 2016-06-30 | Intel Corporation | User interface for liquid container |
| US10120563B2 (en) | 2014-12-24 | 2018-11-06 | Intel Corporation | User interface for liquid container |
| CN106126049A (en) * | 2016-06-22 | 2016-11-16 | 宇龙计算机通信科技(深圳)有限公司 | Menu-operating method and system |
| WO2017219531A1 (en) * | 2016-06-22 | 2017-12-28 | 宇龙计算机通信科技(深圳)有限公司 | Menu operation method and system |
| CN106126049B (en) * | 2016-06-22 | 2019-06-11 | 宇龙计算机通信科技(深圳)有限公司 | Menu-operating method and system |
| CN107274174A (en) * | 2017-05-18 | 2017-10-20 | 青岛海信移动通信技术股份有限公司 | The auth method and mobile terminal of a kind of mobile payment |
| CN110716683A (en) * | 2019-09-29 | 2020-01-21 | 北京金山安全软件有限公司 | Generation method, device and equipment of collision object |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102023786A (en) | Man-machine interaction method and structure | |
| US9223471B2 (en) | Touch screen control | |
| KR101036217B1 (en) | User interface for touchscreen device | |
| Kurtenbach et al. | The design of a GUI paradigm based on tablets, two-hands, and transparency | |
| EP2256614B1 (en) | Display control apparatus, display control method, and computer program | |
| JP4975634B2 (en) | Method and device for controlling and entering data | |
| Xia et al. | Object-oriented drawing | |
| Ni et al. | Design and evaluation of freehand menu selection interfaces using tilt and pinch gestures | |
| US20120182296A1 (en) | Method and interface for man-machine interaction | |
| US20110209087A1 (en) | Method and device for controlling an inputting data | |
| Vogel et al. | Conté: multimodal input inspired by an artist's crayon | |
| EP1674976A3 (en) | Improving touch screen accuracy | |
| Matulic et al. | Pen and touch gestural environment for document editing on interactive tabletops | |
| CN103809875A (en) | Human-computer interaction method and human-computer interaction interface | |
| CN101636711A (en) | Gesturing with a multipoint sensing device | |
| CN101581994B (en) | Device and method for inputting characters on a touch screen in a terminal | |
| JP2010514020A (en) | Human interaction device, electronic device, and human interaction method | |
| CN105431810A (en) | Multi-touch virtual mouse | |
| JP5374564B2 (en) | Drawing apparatus, drawing control method, and drawing control program | |
| US20210247849A1 (en) | Input device, signal processing unit thereto, and method to control the input device | |
| Matulic et al. | Empirical evaluation of uni-and bimodal pen and touch interaction properties on digital tabletops | |
| Scheibel et al. | Virtual stick in caret positioning on touch screens | |
| Fuccella et al. | Touchtap: A gestural technique to edit text on multi-touch capable mobile devices | |
| TW200807284A (en) | Programmable touch system | |
| CN106873763B (en) | Virtual reality equipment and information input method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20110420 |