WO2018186011A1 - Information processing device, information processing method, and program - Google Patents

Information processing device, information processing method, and program Download PDF

Info

Publication number
WO2018186011A1
WO2018186011A1 PCT/JP2018/003081 JP2018003081W WO2018186011A1 WO 2018186011 A1 WO2018186011 A1 WO 2018186011A1 JP 2018003081 W JP2018003081 W JP 2018003081W WO 2018186011 A1 WO2018186011 A1 WO 2018186011A1
Authority
WO
WIPO (PCT)
Prior art keywords
operation
twist
information processing
change
processing apparatus
Prior art date
Application number
PCT/JP2018/003081
Other languages
French (fr)
Japanese (ja)
Inventor
津田 崇基
Original Assignee
ソニー株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to JP2017073904 priority Critical
Priority to JP2017-073904 priority
Application filed by ソニー株式会社 filed Critical ソニー株式会社
Publication of WO2018186011A1 publication Critical patent/WO2018186011A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object or an image, setting a parameter value or selecting a range
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

[Problem] To provide an information processing device, information processing method, and program which are capable of achieving a more intuitive gesture control. [Solution] Provided is an information processing device comprising: an assessment unit which, on the basis of information on operations by an operating body on an operation input screen, assesses as a twist operation a twisting action by the operating body on the operation input screen; and a display control unit which, in response to the assessed twist operation, carries out a prescribed display control corresponding to the application.

Description

The information processing apparatus, information processing method, and program

The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Currently, as an operation gesture on the touch panel, the Tap, Swipe, the Drag, etc. Long press is generally used.

Here, for example, in Patent Document 1, near an information processing apparatus capable of it is disclosed by actual operation the relation of the operation of the image and the user to be displayed on the display unit.

JP 2010-33158 JP

However, with the evolution of a touch panel device, will be able to obtain more detailed touch information, it is required further realize intuitive widely used can gesture operation.

In the present disclosure, an information processing apparatus capable of realizing a more intuitive gesture operation is proposed an information processing method, and a program.

According to the present disclosure, based on the operation information by the operation tool on the operation input screen, a determination unit for twisting operation by the operation tool with respect to the operation input screen as the twisting operation, in response to the determined twist operation, the application and a display control unit that performs predetermined display control corresponding to propose an information processing apparatus.

According to the present disclosure, processor, based on the operation information by the operation tool on the operation input screen, and determining the twisting operation by the operation tool with respect to the operation input screen as the twist operation, depending on the said determined twist operation includes performing the predetermined display control corresponding to an application, and proposes an information processing method.

According to the present disclosure, a computer, on the basis of the operation information by the operation tool on the operation input screen, a determination unit for twisting operation by the operation tool with respect to the operation input screen as the twist operation, depending on the determined twist operation Te proposes a display control unit that performs predetermined display control corresponding to an application, a program to function as a.

According to the present disclosure described above, it is possible to realize a more intuitive gesture operation.

Incidentally, the above effect is not necessarily restrictive, with the above effects, or instead of the above effects, any effects shown herein, or other effects that may be grasped from the description, it may be achieved.

Is a diagram illustrating an example of a configuration of an information processing apparatus according to an embodiment of the present disclosure. Is a diagram illustrating the operation process of the information processing apparatus according to this embodiment. Is a diagram illustrating definitions of various information used for gesture determination according to the present embodiment. Is a diagram for describing the determination condition of the Tap Operation according to this embodiment. Is a diagram for describing determination condition Long press operation according to this embodiment. It is a diagram for describing determination of Swipe (Flick) or Drag operating conditions according to the present embodiment. Is a diagram illustrating an overview of Power Press Operation according to this embodiment. Is a diagram for describing determination condition Power Press Operation according to this embodiment. Is a diagram illustrating an overview of Twist operation according to this embodiment. Is a diagram for describing determination condition Twist operation according to this embodiment. It is a schematic of a Spiral operation according to this embodiment. Is a diagram for describing the determination condition of Spiral operation according to this embodiment. Is a diagram for describing determination of Power Swipe (Flick) / Power Drag Operation of the present embodiment. It is a diagram for describing operation judgment condition based on the moving amount and the elapsed time according to the present embodiment. Is a diagram for describing Swipe operations and Twist operation in the slider bar according to the present embodiment. Is a diagram for describing Swipe operations and Twist operation in alphabetical index bar according to the present embodiment. Is a diagram for describing Swipe operations and Twist operation in view point and line-of-sight movement according to this embodiment. It is a diagram illustrating a shutter operation by Power press operation according to this embodiment. Is a diagram illustrating an example of a Spiral operations and Twist operation in the camera application according to this embodiment. Is a diagram illustrating an example of a Power press operation and Spiral operation in the Contacts list application according to the present embodiment. It is a diagram illustrating an example of Twist operation in the Contacts list application according to the present embodiment. Is a diagram illustrating an example of Twist operation in the text input according to the present embodiment. Is a diagram illustrating an example of Twist operation in notification card display according to the present embodiment. Is a diagram illustrating an example of Twist operation in file browser according to the present embodiment.

Reference will now be described in detail preferred embodiments of the present disclosure. In the specification and the drawings, components having substantially the same function and structure are a repeated explanation thereof by referring to the figures.

Further, description will be made in the following order.
1. 1-1 of the information processing apparatus 1 according to an embodiment of the present disclosure. Configuration 1-2. Operation process 2. Gesture determination 2-1. Tap Operation 2-2. Long press operation 2-3. Swipe (Flick) or Drag operation 2-4. Power Press Operation 2-5. Twist operation 2-6. Spiral operation 2-7. Power Swipe (Flick) / Power Drag operation 2-8. Supplement 3. Operation in the actual application Example 3-1. Slider bar 3-2. Alphabetical index bar
3-3. Operation of the view point and line-of-sight movement 3-4. The camera application 3-5. Image viewer
3-6. Contacts list
3-7. Text input 3-8. Display of the notification card 3-9. File Browser 4. Summary

<< 1. Overview of information processing apparatus 1 according to an embodiment of the present disclosure >>
In the present embodiment, as the gesture in the operation input to the operation input screen such as a touch panel, the Tap, Swipe, the Drag, in addition to the existing gestures such Long press, Power Press, Twist, Spiral, and Power Swipe, and Power the Drag designations It makes it possible to recognize a predetermined gesture.

Although details will be described later, for example, "Power Press" operation is an intact operation of pressing the ball of a finger after the finger touches the previous to the screen, can be recognized based on the change in contact area. Thus, without adding a new (physical) configuration, it is possible to realize a UI operation of pushing on an existing touch panel.

Further, "Twist" operation, rolling from the touch state of the finger to the right or left (twist) that is, to enable analog control. It may implement a UI operation for performing parameter adjustment in accordance with the twist condition.

Further, "Spiral" operation, the status "Power Press" described above, and an operation to rotate clockwise or counterclockwise a fingertip as a supporting point, it is possible to realize a UI operation such as dialing.

Further, "Power Swipe, Power Drag" operation, the status "Power Press" described above, and an operation for Swipe or Drag finger, it is possible to use the UI, such as moving an object.

Thus, in the present embodiment, it is possible to provide a more intuitive and widely gesture operation that can be utilized.

<1-1. Configuration>
First, the gesture operation according to this embodiment the configuration of recognizing the information processing apparatus 1 will be described with reference to FIG. Figure 1 is a diagram showing an example of configuration information processing apparatus 1 according to this embodiment.

As shown in FIG. 1, the information processing apparatus 1 includes a control unit 10, an operation input unit 11, display unit 12 and a storage unit 13,.

Control unit 10 functions as an arithmetic processing device and a control device, and controls the overall operation of the information processing apparatus 1 according to various programs. Control unit 10, for example, CPU (Central Processing Unit), is realized by an electronic circuit such as a microprocessor. The control unit 10, ROM (Read Only Memory) for storing programs used and operation parameters, and may include a RAM (Random Access Memory) that temporarily stores parameters that appropriately change.

The control unit 10 according to the present embodiment also functions as a determination unit 101 and the display control unit 102.

Determination unit 101, based on the touch information detected by the operation input unit 11, performs the gesture determination of a user on the touch panel. Although described later for specific determination method, the determination unit 101 according to the present embodiment, the Tap, Swipe, the Drag, in addition to the existing gestures such Long press, and Power Press, Twist, Spiral, Power Swipe, and Power the Drag it is possible to determine the predetermined gesture referred. For example decision unit 101, the operation input screen change in the contact area by the operation of the (touch panel) (also referred to as a ground plane), change in the center position of the contact area (also referred to as center of gravity), and altered contact area with the pre-change based on the relationship among the center positions of, we are possible to perform multiple gesture determination.

The display control unit 102 controls the display on the display unit 12, or performs display control according to the determined gesture by the determination unit 101, or performs display control according to the instructions of an application running on the information processing apparatus 1 .

The operation input unit 11 accepts an operation instruction from the user, and outputs the operation content to the control unit 10. The operation input unit 11, a touch sensor provided integrally with the display unit 12 may be a pressure sensor or proximity sensor. Here is realized by the operation input unit 11 as an example touch panel will be described an example of performing the gesture determination using the touch information detected, operation input information process of gesture determination to be described later is possible, using the touch is not limited to the information may be, for example, a pressure sensor value or proximity sensor values.

Display unit 12 is a display device for outputting various operation input screens. The display unit 12 is, for example, a liquid crystal display (LCD: Liquid Crystal Display), or may be a display device such as an organic EL (Electroluminescence) display.

Storage unit 13 is realized by a ROM for storing programs, calculation parameters and the like used for the processing of the control unit 10 (Read Only Memory), and RAM for temporarily storing the parameters that appropriately change (Random Access Memory).

Above, it was specifically described configuration of the information processing apparatus 1 according to this embodiment. The information processing apparatus 1, for example a smartphone, a cellular phone terminal, a tablet terminal, wearable devices (e.g. smart band, smart watch), may be implemented by a PC (personal computer) or the like. It should be noted that the configuration of the information processing apparatus 1 is not limited to the example shown in FIG. 1, for example, it may be realized by an information processing system comprising a client terminal and a server. In this case, for example, the operation input information detected by the client terminal side of the operation input unit 11 is transmitted to the server on the network, the server side, gesture determination is made, the display control signal is client based on further determined gesture It is transmitted to the terminal.

<1-2. Operation process>
Next, the operation process of the information processing apparatus according to this embodiment with reference to FIG. 2 will be described.

Figure 2 is a flowchart showing the operation process of the information processing apparatus according to this embodiment. As shown in FIG. 2, first, the display control unit 102 of the information processing apparatus 1 displays on the display unit 12 a predetermined user interface (ie the operation input screen) (step S103), the operation information input by the user to the user interface (here, the touch information) acquires from the operation input unit 11 (step S106).

Next, the determination unit 101 of the information processing apparatus 1, based on the touch information, the gesture determination (step S109). Although described in detail later gesture determination, for example, the determination unit 101 as a touch information to obtain the position of the ground surface of the operating body on the operation input screen (e.g., user's finger), the shape or the change of area, etc., performing a gesture judgment.

Then, the display control unit 102 of the information processing apparatus 1 performs a predetermined display control according to the determined gesture (step S112).

This completes the description of the operation process according to the present embodiment. Subsequently, specifically described with reference to the drawings various gesture determination according to the present embodiment.

<< 2. The gesture determination >>
Subsequently, specifically described with reference to the drawings Details of the gesture determination according to the present embodiment.

First, referring to FIG. 3, illustrating the definition of the various information used for gesture determination according to the present embodiment. These information are those that can be acquired by the operation input unit 11 is realized by a touch panel or the like.

As shown in FIG. 3, in this embodiment, the operation input unit 11, Sn to the first point of placing the finger (Start point), the area (area) in contact with the finger pad An, area a code such Cn applied to; center (centroid point of gravity). Further, although not shown in FIG. 3, when the Sn (Start point) is outside the An (area), the most Sn point close to (Start point) (approximate point; Proximity point) in the An in , to give the sign of Pn. Depending on whether the approximation point is in any position within the area, it is possible to recognize whether the finger has come to move from either direction.

In the touch panel operation, be subtly from the touch location (contrary to the intention of the user) or moving a finger at the time of the operation is assumed. For this reason, it is difficult occurs that the "position of the location you release the location and finger you put the finger is exactly the same," but, taking into account the occurrence of a deviation of the delicate position, these are treated as an error. In the description of the herein below, such errors in but no mention strictly for consideration in the description such as "position (or shape) are the same", the position including the "error (deviation) (or shape ) is assumed to include the meaning that the same ".

<2-1. Tap Operation>
First, it will be described with reference to FIG. 4 the determination of the Tap operations. Figure 4 is a diagram for explaining the determination condition of the Tap Operation according to this embodiment.

Determination unit 101, the finger is not more than approximately 1 second time is, for example, to release from being placed on the touch panel (operation input section 11), and, as shown in FIG. 4, the location of the center of gravity C1 placing the finger and, if the center of gravity C2 of place you release your finger is the same, it is determined that the Tap operation. It may be added on condition that further includes the starting point S1 to the area A2.

<2-2. Long press operation>
Will now be described with reference to FIG. 5 the determination of Long press operation. Figure 5 is a diagram for explaining the determination condition of Long press operation according to this embodiment.

Determination unit 101, finger continues placed touch panel (operation input section 11), for example, approximately one second or more, and, as shown in FIG. 5, the centroid C3 of locations spaced fingers, continues placing the finger If the center of gravity C4 of the place where the same, it is determined that the Long press operation. In the case it may be added to the condition further there is no change in the area of ​​the area A3 and the area A4.

<2-3. Swipe (Flick) or Drag operation>
Next, will be described with reference to FIG. 6 determines Swipe (Flick) or Drag operation. Figure 6 is a diagram for explaining the determination condition Swipe (Flick) or Drag Operation of the present embodiment.

Determination unit 101, a finger monitors the change in position and shape and area of ​​the ground plane from being placed on the touch panel, one of the following in the "condition a and Conditions b-1" or "conditions a and condition b-2" If either met determines that Swipe (Flick) or Drag operation.

Condition a: it moves the finger within approximately one second, for example, from at a finger.
Condition b-1: the shape of the ground plane A5, A6 in the mobile before and after the movement (or moving) in the finger is the same (this condition is useful if capable of shape determination of the ground surface).
Condition b-2: It area of ​​the ground plane A5, A6 in before and after the movement of the finger (or moving) is the same (this condition is useful when you can not shape determination of the ground surface).

Still, the center of gravity C6 is away from C5, it may be added on condition that the approximate point P6 is away from the starting point S5.

Further, if the condition is satisfied, further Swipe (quick sweep intends operation), Flick (drag while sweeping operation), and Drag is which of (moving operation), the moving speed (momentum movement of the finger) in response it may be determined.

Note that when moving the finger to wait at least one second, may be determined as Long press operation.

<2-4. Power Press Operation>
Next, it will be described with reference to FIGS. 7 and 8 for the determination of the Power Press operation. Figure 7 is a diagram illustrating the outline of the Power Press Operation according to this embodiment.

As shown in FIG. 7, the Power Press operation, the finger 2 (especially fingertip) and left on the touch panel (operation input section 11), further operation of pressing flat using finger 2 into contact with the belly of the finger 2 is there. The operation of such a two-step, without adding a physical configuration, it is possible to realize a UI operation of pushing on an existing touch panel.

Figure 8 is a diagram for explaining the determination condition of Power Press Operation according to this embodiment. Determination unit 101, a finger monitors the change in position and shape and area of ​​the ground plane from being placed on the touch panel, the following "conditions c-1 and the condition d and conditions e" or "condition c-2 and the condition d if you meet any of the conditions e "and determines that the Power Press operation.

Conditions c-1: the shape of the ground surface A is changed characteristically (oval extended) it (this condition is useful if capable of shape determination of the ground surface).
Condition c-2: it increased, for example, about 150% area of ​​the ground surface A (this condition is useful when you can not shape determination of the ground surface).
Condition d: the center of gravity C7 when placing the finger included in the ground contact surface A8 when the finger pressed.
Condition e: the center of gravity C7 when you put the finger, is different from that the location of the center-of-gravity point C8 at the time of pressing the finger.

Also, further, it may be added on condition that the center of gravity C8 is away from C7.

Incidentally, I guess the position of the operator by the movement direction of the center of gravity C, together in easy direction read text (the direction of the screen in a direction to match the posture of the user) to be also a display control such rotation.

<2-5. Twist operation>
It will now be described with reference to FIGS. 9 and 10 for determination of the Twist operation. Figure 9 is a diagram illustrating the outline of Twist operation according to this embodiment.

As shown in FIG. 9, the Twist operation, the finger 2 (especially fingertip) and left on the touch panel (operation input section 11), in the direction of right or left rolling the fingers 2 on the touch panel (twist) is operated . More specifically, in the finger 2 from the state touches the touch panel by rolling the finger 2 on the right or left, side of the finger 2 is in a state to touch the touch panel (the ball of the finger 2 is perpendicular state with respect to the touch panel) it is an operation. The operation of such a two-step, without adding a physical configuration, on an existing touch panel, it is possible to realize a UI operation for performing parameter adjustment in accordance with the twist condition.

Figure 10 is a diagram for explaining the determination condition of Twist operation according to this embodiment. Determination unit 101, a finger monitors the change in position and shape and area of ​​the ground plane from being placed on the touch panel, the following "conditions f-1 and Condition g and Conditions h" or "condition f-2 and condition g if you meet any of the conditions h "and determines that the Twist operation.

Conditions f-1: the shape of the ground surface A have (has elongated) characteristically altered it (this condition is useful if capable of shape determination of the ground surface).
Conditions f-2: it has reduced the area of ​​the ground surface A, for example, more than approximately 80% (this condition is useful when you can not shape determination of the ground surface).
Conditions g: that the center of gravity point C9 when you put the finger is not included in the ground plane A10 after twisting.
Conditions h: the center-of-gravity point C9 before twisting the fingers, centroid C10 finger one roll (eg 1 cm) after twisting (middle of twist) to a distance of less. (Note, further, may be a condition that the center of gravity C10 is far more fingers one roll in the lateral direction (within ± 45 °) with respect to the center of gravity point C9.)

<2-6. Spiral operation>
Next, it will be described with reference to FIGS. 11 and 12 for determination of Spiral operation. Figure 11 is a diagram illustrating an overview of Spiral operation according to this embodiment.

As shown in FIG. 11, the Spiral operation fulcrum finger 2 and left on the touch panel (operation input section 11), the Power Press Operation (see FIGS. 7 and 8), further a finger 2 on the touch panel an operation of rotating horizontally to the right or left direction. The operation of such a three-step, without adding a physical configuration, on an existing touch panel, it is possible to realize a UI operation such as dialing.

Figure 12 is a diagram for explaining the determination condition of Spiral operation according to this embodiment. Determination unit 101 finger is placed on the touch panel as shown in FIG. 12 left, and then after the Power Press operation is performed as shown in FIG. 12 central (Power Press operation determination conditions are as described above) , when continuing to monitor changes in position and shape and area of ​​the ground surface, satisfying the "condition i and condition j and conditions k and conditions l" below (the state shown in FIG. 12 right), it is determined that Spiral operation .

Conditions i: Power Press the center of gravity of the operation is continuously detect the transition of the center-of-gravity point from the center of gravity point C11 immediately after that took place, when pressed against the center-of-gravity point C11 and the finger immediately after placing the finger (Power Press Operation) a line L1 connecting the point C12, to compare the line connecting the current center of gravity C13 and the center of gravity point C11, it is in a state of rotating the centroid C11 as the starting point.
Conditions j: that the area of ​​the ground plane A12 before rotating, the area of ​​the ground plane A13 after rotating (or rotating middle) is the same.
Conditions k: the center of gravity C11 when placed finger is included in the ground contact surface A13 after rotating (or rotating middle).
Conditions l: the distance between the front of the center of gravity C12 rotate with the center of gravity C11 when placed finger, the center of gravity C13 after (or rotating middle) obtained by rotating the center of gravity C11 when placed finger that the distance is the same.

<2-7. Power Swipe (Flick) / Power Drag operation>
It will now be described with reference to FIG. 13 for the determination of the Power Swipe (Flick) / Power Drag operation. Power Swipe (Flick) / Power Drag operation is a Power Press Operation (FIGS. 7 and 8 refer) state move the finger from the (Swipe, Flick, or the Drag,) is so operated.

Figure 13 is a diagram for explaining the determination condition of Power Swipe (Flick) / Power Drag Operation of the present embodiment. Determination unit 101, is placed on the finger touch panel as shown in FIG. 13 left, then Power Press Operation As shown in FIG. 13 central (Power Press operation determination conditions are as described above) immediately after is performed determination of continuing to monitor the change in position and shape and area of ​​the ground surface, if it meets the "conditions m and conditions n" below (the state shown in FIG. 13 right), Power Swipe (Flick) / Power Drag operation and to.

Conditions m: the center of gravity C15 before moving the finger, in the region outside of the current ground contact surface A16.
Condition n: that the area of ​​the ground plane A15 before moving, the area of ​​the ground plane A16 after moving (or moving the middle) is the same.

If the condition is satisfied, further Swipe (quick sweep intends operation), Flick (drag while sweeping operation), and Drag which one of (moving operation), according to the moving speed (momentum movement of the finger) It may be determined.

Further, the center of gravity C16 after movement, may be added as a further condition that is away from the center of gravity C15 before the movement.

The determination unit 101, the approximate point P16 of the ground surface A16 of the moved finger 2 from a direction located may determine that it has moved.

<2-8. Supplement>
Above, it was specifically described gesture determination according to the present embodiment. Below, do some supplementary explanation.

(2-8-1. If you do not have to obtain more detailed information)
If more detailed information such as the shape and area of ​​the ground surface A can not be obtained from the touch information, for example, it may be to determine the difference between Twist operations and Swipe / the Drag operation based on the movement amount and the elapsed time (moving time) . Hereinafter will be described with reference to FIG. 14.

Figure 14 is a diagram for describing operation judgment condition based on the elapsed time and the movement amount according to the present embodiment. As shown in FIG. 14, the threshold value Th of determining distance so as to change over time. Thus, so Twist operation can in a wide range from by moving slowly, for example a finger.

That, Swipe / the Drag operation, as the elapsed time (moving time) is short, if the distance operation determination is shortened, but it is possible Swipe / the Drag operation as when flicking ever, you move slow, it is possible to recognize the Swipe / the Drag operations by moving a distance greater than the corresponding threshold Th.

Further, Twist operation, as the elapsed time (moving time) is long, the distance operation determination becomes long, by moving slowly, a wider Twist operation becomes possible (however, the threshold value Th corresponding to the moving speed and the moving distance below).

(2-8-2. Continuous gesture operation)
Further, in the present embodiment, it is possible to continuously perform each gesture operation described above.

For example, after holding the Power Press state a certain time, or immediately, it is possible to perform the Twist operation or Spiral operation.

Further, after maintaining a Twist state a certain time, or immediately, it is possible to perform Spiral operation or Swipe / the Drag operation.

Further, after maintaining the state of the Spiral certain time or immediately, it is possible to perform the Twist operation or Swipe / the Drag operation.

Further, Power the Drag operated from Power Press operations can be executed in succession with further Spiral operation.

(2-8-3. Adjustment process)
· The curved portion if the screen adjustment operation input screen of shape and location according to the threshold of using Curved display or, in the area of ​​the edge of the screen, that adjusts the threshold value Th as described above, to facilitate a more Twist operation possible it is. On the other hand, in areas such as central flat portion and screen may be adjusted threshold value Th to facilitate a more Swipe operation or Drag Operation in the reverse direction.

Depending on the physical movement and tilt changes in-terminal movement and when the threshold of the adjustment operation by the inclination of the change of the terminal (information processing apparatus 1) may adjust the threshold value Th as described above. Movement and tilt information processing apparatus 1, an acceleration sensor provided in the information processing apparatus 1 can be detected by the gyro sensor or a geomagnetic sensor or the like, (all not shown).

For example, under the assumption that when Swipe operated in the right direction is also a case where the information processing apparatus 1 itself slightly moves to the right, if the information processing apparatus 1 is moved in the right direction by the acceleration sensor, Swipe operations and Drag more determined easily so as threshold operation may be adjusted.

Also, under the assumption that the information processing apparatus 1 when performing the Twist operation in the right direction would be twisted together also hand holding the information processing apparatus 1 is slightly inclined, the information processing apparatus 1 is the right direction by the acceleration sensor If inclined may be adjusted thresholds to be more easily determined Twist operation in the right direction.

<< 3. Operation example in the actual application >>
Subsequently, specifically described with reference to the drawings exemplary operation in the actual application of gestures becomes possible to determine newly by the present embodiment.

Swipe operations performed in the horizontal direction in the touch UI is could only so far one system parameter control, in the present embodiment, it is possible to parameter control of another system by using a Twist operation. In addition, by using the Spiral operation, it is possible to further parameter control of another system.

Operation of each of these strains, may be the independent ones (example of a camera application will be described with reference to FIG. 19), associated ones (such as the example of the slider bar to be described with reference to FIG. 15) even to be , a series of ones (example Alphabetical index bar, which will be described with reference to FIG. 16) may be used.

Although this embodiment uses the index finger to the operating body as an example, this embodiment is not limited to this, continuous analog operations such as Volume adjustment and angle adjustment is dynamic operation is also possible by the thumb . Further, non-continuous operations such as character selection and mode switching can be finely operated by the index finger.

<3-1. Slider bar>
Figure 15 is a diagram illustrating a Swipe operations and Twist operation in the slider bar. As shown in FIG. 15, the case of performing when the slider bar 121 volume control on the display unit 12 (touch panel display that integrates an operation input section 11) is displayed, the Swipe operation, for example, a finger 2 15 to allow volume adjustment as shown in the left, the case of performing the Twist operation to allow balancing of the LR as shown in FIG. 15 right. When Twist operation is determined, the display control unit 102 displays the balance adjustment screen 123 of LR, it is possible to adjust the balance between the left and right in response to the rotation of the lateral direction. For example it may also be in accordance with the twist degree to the right moves the knob of the bar of the balance adjustment screen 123 in the right direction, while maintaining the twisted state, the right knob bar balance adjustment screen 123 it may be moving in the direction.

Thus, according to the present embodiment, it continued without releasing the finger 2 from the touch panel can increase the operating parameters controlled, also, it is possible to more intuitive parameter adjustment.

<3-2. Alphabetical index bar>
Figure 16 is a diagram illustrating a Swipe operations and Twist operation in alphabetical index bar according to the present embodiment.

As shown in FIG. 16, the case of performing when the alphabet index bar 125 of the address book on the display unit 12 (touch panel display that integrates an operation input section 11) is displayed, the Swipe operation, for example, a finger 2 Figure 16 to allow the index search alphabetically as shown in the left, the case of performing the Twist operation allows scrolling in the index as shown in FIG. 16 right.

Thus, according to this embodiment, it is possible to continue without releasing the finger 2 from the touch panel increases the operating parameters controlled.

<3-3. Viewpoint and operation of the line-of-sight movement>
Figure 17 is a diagram illustrating a Swipe operations and Twist operation in view point and line-of-sight movement according to this embodiment. When the example is the view point and line-of-sight movement capable viewing screen 17 is displayed, as shown in the left 17, and allows movement location (viewpoint) by Swipe operation, as shown in FIG. 17 right , the angle of view (sight) by Twist operation may allow movement.

Thus, according to this embodiment, it is possible to perform the moving operation of the clear view point and line-of-sight by Swipe operations and Twist operation.

<3-4. Camera application>
Next, the operation UI when the camera application with reference to FIGS. 18 and 19 will be described.

When the through screen by the camera application is displayed on the display unit 12, for example, focus lock by Tap operation, shutter Power press operation, regulating orifice in Twist operation, brightness adjustment for the image after shooting in Spiral operation, Swipe Operation in may be to allow continuous shooting at scene select, and Power Drag operation.

Figure 18 is a diagram for explaining a shutter operation by Power press operation. Figure 18 as shown in the left, place a finger 2 on the through screen in the display unit 12 (touch panel display that integrates an operation input section 11), in this state is performed focus lock is judged Tap operation, further drawing 18 it is possible to release the shutter by Power press operation of pressing the belly of the finger 2 as shown in the right.

Figure 19 is a diagram illustrating an example of a Spiral operations and Twist operation in the camera application. As shown in FIG. 19, for example with brightness adjustment screen 127 in accordance with Spiral operation is displayed, the brightness adjustment screen 127, the movement such as the dial operation to rotate clockwise or counterclockwise as the fingertip fulcrum it is possible to perform brightness adjustment by. Further, the aperture adjustment screen 129 in accordance with the Twist operation is displayed, it is possible to perform aperture adjustment according to twist condition.

Thus, both to allow the analog control, it is possible to more intuitively parameter control.

<3-5. Image viewer>
Next, a description will be given of an operation example in the Image viewer.

Image by viewer application when an image is displayed on the display unit 12, for example, to allow information display ON / OFF by Tap operation, may be caused to overlay the details in Power press operation.

Also, to enable the filter switchable Twist operation (on canceling the overlay display of detailed information) in Spiral operation may be performed zoom in / out.

Also, to display the next / previous image in Swipe operation, it is possible to perform the switching of the details in Power the Drag operation.

<3-6. Contacts list>
Next, a description will be given of an operation example in the Contacts list (contact list).

When the contact list screen by the Contacts list application is displayed on the display unit 12, for example, selected by the Tap operation, the contents Preview (e-mail address, address, etc.) by the Power press operation to display, the group registered by the Twist operation carried out, e-mail address in Spiral operation, function is selected and executed, such as address (mail creation, phone call) may be. It will be specifically described with reference to FIGS. 20 and 21.

Figure 20 is a diagram illustrating an example of a Power press operation and Spiral operation in the Contacts list application. As shown in FIG. 20 left, when the contact list screen is displayed on the display unit 12, when the intact Power press operation by selecting a contact by Tap operation, contents Preview screen 130 is displayed.

Then, as shown in FIG. 20 right, when the Spiral operation while contents Preview screen 130 is displayed on the display unit 12, it is possible to select the execution of the detailed information displayed in the content Preview screen 130. Incidentally selection cursor sequentially moves according to the rotation degree of the Spiral operation.

Figure 21 is a diagram illustrating an example of Twist operation in the Contacts list application. As shown in FIG. 21, when the Twist operation while contents Preview screen 130 is displayed by selecting a contact, the group registration screen 132 is displayed. The user is able to group registration by moving the cursor to any group. Incidentally selection cursor sequentially moves in response to twisting degree of Twist operation.

<3-7. Text input>
Next, display the software keyboard is the operation example when subjected to text input will be described.

In this case, for example, to move the focus between characters button Swipe operation, also, it is possible to perform character type switching on the character button Twist operation. It will be specifically described with reference to FIG. 22.

Figure 22 is a diagram illustrating an example of Twist operation in the text input. As shown in FIG. 22, when the Twist operation on any character button, switching of the plurality of character types corresponding to the character button is possible. Character type switching of is switched according to the twist state of e.g. Twist operation.

Thus, according to this embodiment, to move the focus between characters button Swipe operation, it can be switched character type by Twist operation continues without releasing the finger 2 from the touch panel. In addition, (not find the character type of mercenary) after the Twist operation Again when I wanted to move to other characters button, the focus of inter-character button to directly proceeds to Swipe operation without lifting a finger 2 from the touch panel again moved it is possible to.

<3-8. Display of the notification card>
Subsequently, the smart phone and tablet devices like, will be described operation example on the display of a notification card displayed on the home screen or lock screen.

To the display notification card, open the notification card by, for example, Tap operation, to display the context menu in the Power press operation, to switch the display limited by the Twist operation, (after having to cancel the context menu display) in Spiral operation a notification card of the expand / Collapse (expansion, folding).

Also, do Dismiss (deletion of the notification card) in Swipe operation, it may be to allow the item selection in the context menu in the Power Drag operation. Here, with reference to FIG. 23 illustrating an example of Twist operation in notification card display.

Figure 23 is a diagram illustrating an example of Twist operation in notification card display according to the present embodiment. As shown in FIG. 23 left, for example, when the notification card 136a ~ 136 b is displayed, when the Twist operation for any notification card 136a with a finger 2, as shown in FIG. 23 right, the display restriction switching screen 137a is displayed. In selective screen 137a of the display limit, it is possible to switch the display limits according to the type of notification card selected. In the example shown in FIG. 23, a notification card of the received mail, the selective screen 137a, as a display restriction notification card, select one of the "From only", "Title Only" "From the title" "to body" it is possible to. Selection cursor switching screen 137a is moved in accordance with the twist degree of finger 2 for Twist operation.

<3-9. File Browser>
Next, a description will be given of an operation example at the time when the file browser is displayed.

When being displayed folder list of documents, such as by the file browser application, for example, select a folder in the Tap operation, to display the context menu in the Power press operation, to be able to change the status (such as the importance) in the Twist operation it may be.

Also, (after having to cancel the context menu display) in Spiral operation and selects the archive destination (destination), it may be to allow the item selection in the context menu in the Power Drag operation.

Hereinafter, it will be described with particular reference to Figure 24 for the case of Twist operation. Figure 24 is a diagram illustrating an example of Twist operation in file browser.

As shown in FIG. 24, when the folder list of documents or the like on the display section 12 is displayed, when the Twist operation for any folder display with a finger 2, the mark 140 (e.g. severity indicating the status it is possible to give a star mark) shown. Note the grantable status one without being limited, it is possible to switch to any mark in accordance with the twist degree, for example, a finger 2.

<< 4. Summary >>
As described above, the information processing apparatus 1 according to an embodiment of the present disclosure, it is possible to realize a more intuitive gesture operation.

Having described in detail preferred embodiments of the present disclosure with reference to the accompanying drawings, the present technology is not limited to such an example. It would be appreciated by those skilled in the art of the present disclosure, within the scope of the technical idea described in the claims, it is intended to cover various modifications, combinations, these for it is also understood to belong to the technical scope of the present disclosure.

For example, CPU incorporated in the information processing apparatus 1 described above, ROM, and hardware such as RAM, a computer program for exhibiting the function of the information processing apparatus 1 can also be created. The computer-readable storage medium storing the computer program is also provided.

The effects described herein are not limiting be those that only illustrative or exemplary. In other words, the technology according to the present disclosure, together with the above effects, or instead of the above effects, can exhibit the apparent other effects to those skilled in the art from the description herein.

The present technology may also be configured as below.
(1)
Based on the operation information by the operation tool on the operation input screen, a determination unit for twisting operation by the operation tool with respect to the operation input screen as the twist operation,
Depending on the determined twist operation, a display control unit that performs predetermined display control corresponding to the application,
Comprising the information processing apparatus.
(2)
The determination unit, based on the operation information by the operation tool on the operation input screen, the tap operation by the operating body on the operation input screen, long pressing operation, swipe, and further determine the flick operation or drag operation,
Wherein the display control unit, the determined tap operation, long pressing operation, in accordance with the swipe operation, flick operation, or drag operation, performs the twisting operation with different display controller, an information processing apparatus according to (1).
(3)
The display control unit may display the parameter adjustment screen in accordance with the twist operation, the adjustment of the parameters in accordance with the twisting degree of the twisted operation, the information processing apparatus according to (1) or (2).
(4)
The display controller,
Scroll Search to view the index in response to the swipe operation,
Wherein when switched from swiping the twist operation, the display control to scroll through the list in the corresponding index information processing apparatus according to (1) or (2).
(5)
The display controller,
It performs a viewpoint movement of the display screen in accordance with the swipe operation,
Controlling to perform eye movement of the display screen in response to twisting operation,
The information processing apparatus according to (1) or (2).
(6)
The display controller,
It performs a focus movement between characters button on the software keyboard in response to the swipe operation,
Performing a character type switching control of the corresponding character button in response to the twist operation,
The information processing apparatus according to (1) or (2).
(7)
The determination unit,
The change of the ground surface by the operation body to the operation input screen, on the basis of the change in the gravity center point of the ground plane, the change of the ground plane, and the positional relationship of the ground plane and the change before the center of gravity of the changed when a predetermined condition is satisfied, determines that the twist operation, the (1) the information processing apparatus according to any one of - (6).
(8)
The determination unit, the shape of the ground surface is changed into a predetermined shape, prior to the ground surface after the change is a change not include the center of gravity point and the change before the center of gravity and the change after the center of gravity of the distance of a given If within the range, determines that the twist operation, the information processing apparatus according to (7).
(9)
The determination unit, the area of ​​the ground surface is changed to a predetermined ratio, before the ground surface after the change is a change not include the center of gravity point and the change before the center of gravity and the change after the center of gravity of the distance of a given If within the range, determines that the twist operation, the information processing apparatus according to (7).
(10)
Processor,
Based on the operation information by the operation tool on the operation input screen, and determining the twisting operation by the operation tool with respect to the operation input screen as the twist operation,
And performing the said in response to the determined twist operation, predetermined display control corresponding to the application,
Including, information processing method.
(11)
The computer,
Based on the operation information by the operation tool on the operation input screen, a determination unit for twisting operation by the operation tool with respect to the operation input screen as the twist operation,
Depending on the determined twist operation, a display control unit that performs predetermined display control corresponding to the application,
Function as the cause, program.

1 the information processing apparatus 10 control unit 11 operation input unit 12 display unit 13 storage unit 101 determination unit 102 display control unit

Claims (11)

  1. Based on the operation information by the operation tool on the operation input screen, a determination unit for twisting operation by the operation tool with respect to the operation input screen as the twist operation,
    Depending on the determined twist operation, a display control unit that performs predetermined display control corresponding to the application,
    Comprising the information processing apparatus.
  2. The determination unit, based on the operation information by the operation tool on the operation input screen, the tap operation by the operating body on the operation input screen, long pressing operation, swipe, and further determine the flick operation or drag operation,
    Wherein the display control unit, the determined tap operation, long pressing operation, in accordance with the swipe operation, flick operation, or drag operation, performs the twisting operation with different display control apparatus according to claim 1.
  3. The display control unit may display the parameter adjustment screen in accordance with the twist operation, the adjustment of the parameters in accordance with the twisting degree of the twisted operation, the information processing apparatus according to claim 1.
  4. The display controller,
    Scroll Search to view the index in response to the swipe operation,
    Wherein when switched from swiping the twist operation, the display control to scroll through the list in the corresponding index information processing apparatus according to claim 1.
  5. The display controller,
    It performs a viewpoint movement of the display screen in accordance with the swipe operation,
    Controlling to perform eye movement of the display screen in response to twisting operation,
    The information processing apparatus according to claim 1.
  6. The display controller,
    It performs a focus movement between characters button on the software keyboard in response to the swipe operation,
    Performing a character type switching control of the corresponding character button in response to the twist operation,
    The information processing apparatus according to claim 1.
  7. The determination unit,
    The change of the ground surface by the operation body to the operation input screen, on the basis of the change in the gravity center point of the ground plane, the change of the ground plane, and the positional relationship of the ground plane and the change before the center of gravity of the changed when a predetermined condition is satisfied, it determines that the twist operation, the information processing apparatus according to claim 1.
  8. The determination unit, the shape of the ground surface is changed into a predetermined shape, prior to the ground surface after the change is a change not include the center of gravity point and the change before the center of gravity and the change after the center of gravity of the distance of a given If within the range, it determines that the twist operation, the information processing apparatus according to claim 7.
  9. The determination unit, the area of ​​the ground surface is changed to a predetermined ratio, before the ground surface after the change is a change not include the center of gravity point and the change before the center of gravity and the change after the center of gravity of the distance of a given If within the range, it determines that the twist operation, the information processing apparatus according to claim 7.
  10. Processor,
    Based on the operation information by the operation tool on the operation input screen, and determining the twisting operation by the operation tool with respect to the operation input screen as the twist operation,
    And performing the said in response to the determined twist operation, predetermined display control corresponding to the application,
    Including, information processing method.
  11. The computer,
    Based on the operation information by the operation tool on the operation input screen, a determination unit for twisting operation by the operation tool with respect to the operation input screen as the twist operation,
    Depending on the determined twist operation, a display control unit that performs predetermined display control corresponding to the application,
    Function as the cause, program.
PCT/JP2018/003081 2017-04-03 2018-01-31 Information processing device, information processing method, and program WO2018186011A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2017073904 2017-04-03
JP2017-073904 2017-04-03

Publications (1)

Publication Number Publication Date
WO2018186011A1 true WO2018186011A1 (en) 2018-10-11

Family

ID=63712473

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/003081 WO2018186011A1 (en) 2017-04-03 2018-01-31 Information processing device, information processing method, and program

Country Status (1)

Country Link
WO (1) WO2018186011A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010503130A (en) * 2006-09-11 2010-01-28 アップル インコーポレイテッド Media player with an image-based browsing
JP2011014045A (en) * 2009-07-03 2011-01-20 Sony Corp Map information display device, map information display method, and program
US20120302291A1 (en) * 2011-05-24 2012-11-29 Microsoft Corporation User character input interface with modifier support
JP2013003962A (en) * 2011-06-20 2013-01-07 Konica Minolta Business Technologies Inc Information browsing-device, display control program, and display control method
US20130318482A1 (en) * 2012-05-24 2013-11-28 International Business Machines Corporation Gestural control for quantitative inputs

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010503130A (en) * 2006-09-11 2010-01-28 アップル インコーポレイテッド Media player with an image-based browsing
JP2011014045A (en) * 2009-07-03 2011-01-20 Sony Corp Map information display device, map information display method, and program
US20120302291A1 (en) * 2011-05-24 2012-11-29 Microsoft Corporation User character input interface with modifier support
JP2013003962A (en) * 2011-06-20 2013-01-07 Konica Minolta Business Technologies Inc Information browsing-device, display control program, and display control method
US20130318482A1 (en) * 2012-05-24 2013-11-28 International Business Machines Corporation Gestural control for quantitative inputs

Similar Documents

Publication Publication Date Title
JP5510185B2 (en) The information processing apparatus, a program and a display control method
US10275066B2 (en) Input apparatus, input method and program
US9619038B2 (en) Electronic device and method of displaying a cover image and an application image from a low power condition
US9471145B2 (en) Electronic device and method of displaying information in response to a gesture
EP2577438B1 (en) User interface with z-axis interaction
US9690476B2 (en) Electronic device and method of displaying information in response to a gesture
US8009146B2 (en) Method, apparatus and computer program product for facilitating data entry via a touchscreen
US20100037183A1 (en) Display Apparatus, Display Method, and Program
EP2293176B1 (en) Information display apparatus, method, and program comprising tilt detection
US8555184B2 (en) Method of interacting with a scrollable area on a portable electronic device
US20100245275A1 (en) User interface apparatus and mobile terminal apparatus
US9015641B2 (en) Electronic device and method of providing visual notification of a received communication
US8539375B1 (en) Method and apparatus for providing a user interface on a device enabling selection of operations to be performed in relation to content
US20040100479A1 (en) Portable information terminal, display control device, display control method, and computer readable program therefor
US9733752B2 (en) Mobile terminal and control method thereof
EP2169528A2 (en) Method of operating a user interface
US20130227413A1 (en) Method and Apparatus for Providing a Contextual User Interface on a Device
US9170645B2 (en) Method and apparatus for processing input in mobile terminal
US20060123359A1 (en) Portable electronic device having user interactive visual interface
US20130141371A1 (en) Electronic device and method of displaying information in response to a gesture
US8775966B2 (en) Electronic device and method with dual mode rear TouchPad
JP4609543B2 (en) Information processing apparatus and information processing method
US8739053B2 (en) Electronic device capable of transferring object between two display units and controlling method thereof
EP2068235A2 (en) Input device, display device, input method, display method, and program
US9274705B2 (en) Mobile terminal and method for controlling the same

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18781727

Country of ref document: EP

Kind code of ref document: A1