JP2015038771A - User operation control program, mobile device, and user operation control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mobile terminal device with a touch-panel display unit, configured to enable a user holding the device with one hand to touch the whole surface of the touch panel.SOLUTION: A mobile terminal device includes: a detection step of detecting a change in inclination of the terminal; a determination step of determining a direction of inclination of the mobile information terminal from a detection result of the detection step; a display control step of moving a display position of a screen including an operator with an operation on a touch panel, which is to be displayed in a display unit, on the basis of the content determined in the determination step; and an input control step which receives the operation to the operator moved in the display-control step.

Description

  The present invention relates to a user operation control program, a mobile device, and a user operation control method suitable for a portable information terminal device such as a smartphone provided with a touch panel.

  In recent years, portable information terminals equipped with touch panels such as smartphones and tablet terminals are widely used. In this type of device, a large portion of the front surface of the thin plate-shaped device housing is used as a display unit and a touch input unit, so that a keyboard dedicated to input is not required and a display called pinch out / pinch in, for example, is displayed. More intuitive operations such as a multi-touch operation for instructing enlargement / reduction of an image are possible.

  While a portable information terminal equipped with such a touch panel can be operated intuitively, for example, in an environment where it is operated with one hand while being caught in a strap in a train, the gripping and touch operation of the device main body is performed. An area that cannot be operated even on the touch panel may be generated depending on the way of gripping.

  In addition to the pinch-out / pinch-in operation described above, a technique is conceivable in which an area traced with a finger in a displayed character string is selected and an enlarged image of the character string can be displayed in a separate window. (For example, Patent Document 1)

JP 2004-152217 A

  The technique described in the above patent document is also based on the premise that the touch panel is operated with the fingers of the other hand after the device is held with one hand. Therefore, in an environment in which operation is performed with one hand while being caught on a strap in the train as described above, there is a high possibility that an area that cannot be operated on the touch panel is generated.

  The present invention has been made in view of the above circumstances, and the object of the present invention is user operation control capable of easily realizing a touch operation over the entire surface of the touch panel while holding the device with one hand. A program, a portable device, and a user operation control method are provided.

  According to one embodiment of the present invention, a detection step of detecting a direction in which the computer is tilted on a computer, and a first operator or a second operator displayed on the display unit in accordance with the direction detected by the detection step A display control step for executing a display control step for moving the display position, wherein the display control step displays the first operator when the direction of tilt is detected by the detection step. If the second operation element is displayed by moving the display position of the first operation element according to the first form corresponding to the inclination direction, the first operation corresponding to the inclination direction is performed. It is a user operation control program for moving the display position of the second operator in a second form different from the form.

  According to the present invention, it is possible to easily realize a touch operation over the entire surface of the touch panel while holding the apparatus with one hand.

The block diagram which shows the function structure of the electronic circuit of the smart phone which concerns on one Embodiment of this invention. The flowchart which shows the processing content of the display content on a display screen provided with the touchscreen which concerns on the embodiment. The figure which illustrates the 1st transition state of the display screen which concerns on the embodiment. The figure which illustrates the 2nd transition state of the display screen which concerns on the same embodiment. The figure which illustrates the 3rd transition state of the display screen which concerns on the same embodiment.

Hereinafter, an embodiment when the present invention is applied to a smartphone will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a functional configuration of an electronic circuit of the smartphone 10 according to the present embodiment. In this figure, the CPU 11 controls the overall operation of the smartphone 10, and the main memory 12, solid state drive (SSD) 13, 3G & 4G communication unit 14, wireless LAN communication unit is connected to the CPU 11 via the system bus SB. 15, an external memory card 16, a display unit 17, a touch panel input unit 18, a key input unit 19, an audio processing unit 20, an image processing unit 21, and a triaxial acceleration sensor 28 are connected.

  The CPU 11 reads out an operation program (OS and application program that operates on the OS) and standard data stored in the solid state drive 13, develops and stores them in the main memory 12, and executes the program Thus, the overall operation of the smartphone 10 is controlled and controlled.

  The main memory 12 is constituted by an SRAM, for example, and functions as a work memory for the CPU 11. The solid state drive 13 is configured by a non-volatile memory such as a flash memory, and stores various contents such as image data and music data as a storage medium of the smartphone 10 in addition to the above-described operation program and various fixed data.

  The 3G & 4G communication unit 14 is based on a third generation mobile communication system compliant with the IMT-2000 standard and a fourth generation mobile communication system compliant with the IMT-Advanced standard. It is a communication unit that operates in a dual mode to transmit and receive data.

  The wireless LAN communication unit 15 transmits / receives data to / from the nearest access point (not shown) via the antenna 23 based on, for example, the IEEE 802.11a / b / g / n standard.

  The display unit 17 includes a TFT color liquid crystal panel with a backlight and its driving unit, and displays various images.

  The touch panel input unit 18 is configured integrally with the display unit 17 using a transparent electrode, and generates and outputs two-dimensional position coordinate information corresponding to a user's touch operation.

  The key input unit 19 includes a number of key switches including a power key and a shutter key for a camera function provided on the housing surface of the smartphone 10 and a key sampling circuit thereof.

  The audio processing unit 20 converts the digital audio data supplied via the system bus SB into an analog signal and outputs it from the speaker 24, while sampling an analog audio signal input from the microphone 25, converts it into digital data, and outputs the digital data.

  The image processing unit 21 is connected to the solid-state image sensor via the optical lens system 26 according to an optical image focused on the imaging surface of the solid-state image sensor 27 configured by, for example, a CCD (Charge Coupled Device). The image signal output from 27 is converted into digital data, and file data in which the data amount is compressed by a preset file format, for example, JPEG (Joint Photographic Experts Group) or the like is generated and output in the case of a still image.

  The three-axis acceleration sensor 28 detects an inclination angle of the housing of the smartphone 10 in the three-dimensional space. The CPU 11 monitors the output of the three-axis acceleration sensor 28, so that the attitude angle at which the smartphone 10 is used with respect to the gravitational acceleration and a change in inclination due to a temporary external force, for example, one of the housings of the smartphone 10 When there is a posture change that lifts the edge of the head in a short period, the direction and degree can be detected and determined.

Next, the operation in the above embodiment will be described.
The operation shown below is executed after the CPU 11 reads the operation program stored in the solid state drive 13 as described above, develops it on the main memory 12, and stores it. The operation program stored in the solid state drive 13 is not limited to the one stored in the solid state drive 13 at the time of factory shipment of the smartphone 10, but the user of the smartphone 10 may use the antenna 22 and the 3G & 4G communication unit 14, or A program input from the outside via the antenna 23 and the wireless LAN communication unit 15 is included.

  FIG. 2 shows the processing contents related to basic image display control executed by the CPU 11 on the image displayed on the display unit 17 when the user using the smartphone 10 executes any application program. It is shown.

  Initially, the CPU 11 displays an image corresponding to the application program and executes a necessary operation (step S101), detects an output from the triaxial acceleration sensor 28, and analyzes a change on the time axis. (Step S102).

  Based on the analysis result, the CPU 11 determines whether or not the housing of the smartphone 10 has tilted at an angle equal to or greater than a certain value in the short term, depending on whether or not there has been a certain change within a short time (step S103).

  If it is determined that there is no inclination at an angle greater than a certain value, the process returns to step S101 and the operation of the application program is continued as before.

  In the process in which the user continues an arbitrary application program while repeatedly executing the processes of steps S101 to S103, for example, when the user performs an operation that lifts one end of the housing of the smartphone 10 in a short period of time, The CPU 11 determines the operation in step S103, and first determines whether or not to execute display control based on the first mode (referred to as “case 1” in the figure) according to the setting of the application program being executed at that time. Is determined (step S104).

In the present embodiment, regarding display control for moving at least a part of a display image within the screen, the following three states, that is, <first form (case 1)>: move according to the contents set by the application program When the displayed button is operated within a certain period of time, the screen moves to a display screen corresponding to the button operation, and the immediate movement display state is canceled.
<Second Mode (Case 2)>: An operation for a scroll bar that has been moved and displayed is received, and the movement display state is canceled when no operation is performed for a certain period of time.
<Third Form (Case 3)>: The entire display screen is moved in a direction in which the smartphone 10 is tilted.
It is assumed that control is executed by selecting one of the three methods.

  If it is determined in step S104 that display control based on the first form is to be executed based on the setting of the application program being executed, the CPU 11 moves a part of the screen including the operation buttons in accordance with the direction of the inclination. The displayed image is displayed on the display unit 17 (step S105).

FIG. 3 is a diagram illustrating display state transitions based on the first embodiment.
FIG. 3A illustrates a display screen that is displayed on the display unit 117 immediately before it is determined in step S103 that it has tilted at an angle of a certain value or more in the short term. In this display screen, a case is considered in which the user holds, for example, the lower right side of the smartphone 10 with the right hand so as to support the back surface, and operates a button or the like displayed on the touch panel input unit 18 with the thumb of the right hand.

  In this case, the user's right thumb holding the smartphone 10 naturally does not reach the upper left side of the screen, and the buttons and the like located on the upper left side cannot be operated in the holding state as it is.

  Here, it is assumed that the user performs an operation of pulling a button or the like toward the front (an operation of dragging the button or the like to the hand) by lifting the upper left side of the housing of the smartphone 10. The CPU 11 sets a diagonal line connecting the upper right corner and the lower left corner of the screen of the display unit 17 from the inclination direction based on the user's operation based on the detection from the triaxial acceleration sensor 28, and the upper left corner across the diagonal line. As shown in FIG. 3B, the upper left image is rotated and moved to the lower right side as shown in FIG. 3B. Display the image.

  With the display state of FIG. 3B, it is possible to selectively touch and operate some of the buttons originally on the upper left side on the lower right side.

  The CPU 11 counts the time when there is no button touch operation while executing the moving display of the closed partial image performed in the tilt direction as shown in FIG. It is determined whether or not a state where there is no time has reached a certain time, for example, 2 [seconds] (step S106).

  If it is determined that the state where there is no operation has not reached the predetermined time, the CPU 11 further determines whether or not a touch operation on any button is performed by the touch panel input unit 18 (step S107).

  Here again, when determining that there is no touch operation on the button, the CPU 11 returns to the processing from step S105 again.

  In the process of repeatedly executing the processes in steps S105 to S107, when a touch operation is performed on any button by the touch panel input unit 18, when the operation is determined in step S107, the CPU 11 performs a process corresponding to the touched button. Is executed (step S108), the state of partial movement display so far is canceled (step S109), and the processing from step S101 is performed again to continue the application program from the state corresponding to the button operation. Return.

  Further, in the process of repeatedly executing steps S105 to 107, if it is determined that the predetermined time has elapsed in step S106 when the predetermined time has elapsed without any touch operation on any button by the touch panel input unit 18, the CPU 11 Skips the process in step S108 as it is, cancels the partial moving display state until then to return to the display state before detecting the tilt in the previous step S103 (step S109), and again The process returns to step S101.

  Further, when the inclination of a certain value or more is detected in the step S103 and it is determined in the subsequent step S104 that the display control based on the first form is not executed, the CPU 11 next performs the display control based on the second form. Is determined based on the setting of the application program being executed (step S110).

  If it is determined in step S110 that the display control based on the second form is to be executed, the CPU 11 displays an image in which the position of the scroll bar in the display screen is moved in accordance with the tilt direction. Is displayed (step S111).

FIG. 4 is a diagram illustrating display state transitions based on the second mode.
FIG. 4A illustrates a display screen displayed on the display unit 117 immediately before it is determined in step S103 that it has tilted at an angle equal to or greater than a certain value in the short term. In this display screen, a case is considered in which the user holds the lower left side of the smartphone 10 with the left hand as shown in the figure, and operates the buttons displayed on the touch panel input unit 18 with the left hand thumb.

  In this case, the left thumb of the user holding the smartphone 10 naturally does not reach the right side of the screen, and the scroll bar SRB located on the right side cannot be operated in the holding state as it is.

  Here, it is assumed that the user performs an operation of pulling the scroll bar SRB to the near side by raising the right side of the housing of the smartphone 10. The CPU 11 generates a moving image in which the scroll bar SRB located on the right side of the screen of the display unit 17 is moved to the left side of the screen as it is from the tilt direction based on the user's operation by detection from the three-axis acceleration sensor 28. As shown in FIG. 4 (B).

  With the display state of FIG. 4B, the scroll bar SRB originally on the right side can be touched and operated on the left side.

  The CPU 11 counts the time when there is no sliding operation on the scroll bar SRB while executing the moving display of the image of the closed scroll bar SRB performed in the tilt direction as shown in FIG. 4B in step S111. By doing so, it is determined whether or not the state of no operation has reached a certain time, for example, 2 [seconds] (step S112).

  If it is determined that the state where there is no operation has not reached the predetermined time, the CPU 11 further determines whether or not a slide operation is performed on the scroll bar SRB by the touch panel input unit 18 (step S113).

  If it is determined here that there is no operation on the scroll bar SRB, the CPU 11 returns to the process from step S112 again.

  When the touch panel input unit 18 performs a sliding operation on the scroll bar SRB in the process of repeatedly executing the processes of steps S112 and S113, the CPU 11 determines that the operation has been performed in step S113 and the CPU 11 performs the sliding operation of the scroll bar SRB. After the display image is scrolled according to the amount of, the count value in the state where the scroll bar SRB is not operated is reset (step S114), and the process returns to the process from step S112 again.

  In the process of repeatedly executing Steps S112 to S113, if a certain time has passed without any slide operation on the scroll bar SRB, the CPU 11 determines that the certain time has passed in Step S112. The display state in which the scroll bar SRB is moved is canceled (step S115), and the process returns to step S101.

  If it is determined in step S110 that the display control based on the second mode is not executed, the CPU 11 performs the remaining display control based on the third mode in accordance with the direction of the inclination. Is displayed on the display unit 17 (step S116), and the process returns to step S101.

FIG. 5 is a diagram illustrating display state transitions based on the third mode.
Here, it is assumed that a part of an image spread by the application program, for example, a part of a map image as shown in FIG. In the case where the range of the frame FR1 indicated by the solid line in the drawing is displayed on the display unit 17 at that time, the user pulls up one of the housings of the smartphone 10 to draw the image that the user desires to browse forward. It is assumed that such an operation has been performed.

  The CPU 11 detects a map in a range adjacent to the map image displayed on the display unit 17 at that time from the tilt direction (any one of DR1 to DR4) based on the user's operation by detection from the triaxial acceleration sensor 28. An image is generated and displayed again on the display unit 17 as a moving image.

  As described above, by executing display control based on the configuration according to the settings in the application program being executed at that time, even if the user is in a somewhat inconvenient operating environment, the user is stressed. And the application program can be continued.

  As described above in detail, according to this embodiment, the user can easily touch the entire surface of the touch panel while holding the device with one hand.

  In the above embodiment, the movement display state is canceled when an operation is performed by the user or when there is no operation for a certain period of time depending on the display form of the moving image. There is no need to perform an operation, and the original display state can be restored very naturally.

  Further, as shown in FIG. 3 above, the user can operate half of the screen before moving display by rotating half of the screen of the display unit 17 and moving it to the opposite side of the original screen. If it is a state, the whole surface of the display part 17 can be operated by the said movement display, without changing the holding | grip state of the smart phone 10. FIG.

  In the above embodiment, the case where a button or a slide bar is used as an operator operated on the screen has been described. However, the present invention is not limited to this, and the same applies to other operators such as icons operated using a touch panel. Of course, it is applicable to.

  Moreover, although the said embodiment demonstrated the case where this invention was applied to a smart phone, this invention is not restricted to this, What is various portable information terminals provided with the touch panel, such as a mobile type personal computer and a tablet terminal. Any of them can be applied.

In addition, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention in the implementation stage. Further, the functions executed in the above-described embodiments may be combined as appropriate as possible. The above-described embodiment includes various stages, and various inventions can be extracted by an appropriate combination of a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, if the effect is obtained, a configuration from which the constituent requirements are deleted can be extracted as an invention.
The invention described in the scope of the original claims of the original application of the present application will be appended below.
[1]
A user operation control program to be executed by a mobile terminal device having a touch panel display unit,
A detecting step for detecting a change in inclination of the portable information terminal;
A determination step of determining a direction of inclination given to the portable information terminal from a detection result in the detection step;
Based on the content determined in the determination step, a display control step for moving a display position of a screen including an operator with an operation on the touch panel, which is displayed on the display unit,
A user operation control program, comprising: an input control step for receiving an operation on an operator whose display position is moved in the display control step.
[2]
The user operation control program according to [1], wherein the display control step moves a display position of a part of a screen including an operator accompanied by an operation on the touch panel.
[3]
The display control step is one of the screens when an operation on the operation element is accepted in the input control step and when at least one of a preset time has passed without any operation on the operation element. The user operation control program according to [1] or [2], wherein the display state is released by moving the display position of the unit.
[4]
The display control step moves one half of the screen including the operation element with the operation on the touch panel displayed on the display unit to the display position of the other half based on the content determined in the determination step. The user operation control program according to any one of [1] to [3].
[5]
A mobile terminal device including a touch panel display unit,
Detecting means for detecting a change in the inclination of the terminal housing;
Determination means for determining the direction of inclination given to the portable information terminal from the detection result of the detection means;
Based on the content determined by the determination means, a display control step of moving the display position of a screen including an operator with an operation on the touch panel, which is displayed on the display unit,
A portable terminal device comprising: input control means for receiving an operation on an operator whose display position has been moved by the display control means.
[6]
[5] The portable terminal device according to [5], wherein the display control step moves a display position of a part of a screen including an operator accompanied by an operation on the touch panel.

  DESCRIPTION OF SYMBOLS 10 ... Smartphone, 11 ... CPU, 12 ... Main memory, 13 ... Solid state drive (SSD), 14 ... 3G & 4G communication part, 15 ... Wireless LAN communication part, 16 ... External memory card, 17 ... Display part, 18 ... Touch-panel input , 19 ... Key input unit, 20 ... Audio processing unit, 21 ... Image processing unit, 22, 23 ... Antenna, 24 ... Speaker, 25 ... Microphone, 26 ... Optical lens system, 27 ... Solid-state imaging device, 28 ... 3-axis Acceleration sensor, SB ... system bus, SRB ... scroll bar.

Claims (9)

On the computer,
A detecting step for detecting a direction in which the computer is tilted;
A user operation control program for executing a display control step for moving the display position of the first operator or the second operator displayed on the display unit according to the direction detected by the detection step,
In the display control step, when the first operation element is displayed when the inclination direction is detected in the detection step, the display position of the first operation element is determined according to the first form corresponding to the inclination direction. When the second operation element is displayed, the user operation control is performed to move the display position of the second operation element according to the second form different from the first form according to the direction of the inclination. program. An input control step for receiving an operation on the first operator or the second operator whose display position has been moved in the display control step;
2. The user operation according to claim 1, wherein the display control step returns the display position of the second operator to the original position before the movement when the operation to the second operator is not received by the input control step for a predetermined time. Control program.   The display control step moves the display position of the first operator by rotating the image including the first operation displayed on the display unit as the first form, and the second form as the second form. The user operation control program according to claim 1, wherein the display position of the second operation element is moved in the opposite side direction of the screen of the display unit. A portable device,
Detecting means for detecting a direction in which the portable device is tilted;
Display control means for moving the display position of the first operating element or the second operating element displayed on the display unit according to the direction detected by the detecting means,
When the first operation element is displayed when the display control means detects the direction of the inclination by the detection means, the display position of the first operation element according to the first form corresponding to the direction of the inclination. When the second operation element is displayed, the portable device moves the display position of the second operation element according to a second form different from the first form according to the tilt direction. An input control means for receiving an operation on the first operation element or the second operation element whose display position is moved by the display control means;
The portable device according to claim 4, wherein the display control means returns the display position of the second operation element to the original position before the movement when the input control means has not received the second operation element for a predetermined time.   The display control means moves the display position of the first operator by rotating the image including the first operation displayed on the display unit as the first form, and the second form as the second form. The portable device according to claim 4, wherein the display position of the second operator is moved in the opposite side direction of the screen of the display unit. A user operation control method executed on a portable device,
A detecting step for detecting a direction in which the portable device is tilted;
A display control step of moving the display position of the first operating element or the second operating element displayed on the display unit according to the direction detected by the detecting step,
In the display control step, when the first operation element is displayed when the inclination direction is detected in the detection step, the display position of the first operation element is determined according to the first form corresponding to the inclination direction. When the second operation element is displayed, the user operation control is performed to move the display position of the second operation element according to the second form different from the first form according to the direction of the inclination. Method. An input control step for receiving an operation on the first operator or the second operator whose display position has been moved in the display control step;
8. The user according to claim 7, wherein the display control step returns the display position of the second operator to the original position before the movement when the input control step does not accept an operation for the second operator for a predetermined time. Operation control method.   The display control step moves the display position of the first operator by rotating the image including the first operation displayed on the display unit as the first form, and the second form as the second form. The user operation control method according to claim 7, wherein the display position of the second operation element is moved in the opposite side direction of the screen of the display unit.

Images