JP2018060539A - User interface program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a user interface program which facilitates operation at a first-person view point and a third-person view point.SOLUTION: A rotation amount from one or more designated reference attitudes around an axis is calculated based on an attitude of a device, and while using one or more numerical values as input values and using an existing input method together therewith, a plurality of input values are used simultaneously.SELECTED DRAWING: Figure 10

Description

The present invention relates to a method of performing a plurality of inputs simultaneously and independently together with an existing input method by a function of handling posture information acquired by an electronic device as a stick input.

In a device having a touch panel such as a smartphone or a tablet, input by touch is a main input means.
In the case of input with one hand, if the user holds the general way as shown in FIG. 1, the input is made with one thumb (2).
When holding the device with one hand and typing with the other hand (3), up to six with the thumb (2) of the holding hand (1) and the five fingers of the other hand (3) Will be input.
As shown in Figure 2, hold the device with both hands and perform touch input with two thumbs (11 and 12) .In the holding method used mainly in game operations, two inputs with both thumbs (11 and 12) Become.

In the field of games, virtual buttons and analog sticks are arranged on the screen, and by operating them by touch input according to the method described above, it is possible to operate elements such as characters and menus in the game (for example, Patent Document 1
And Patent Document 2).

In the way of holding in FIG. 2 that is mainly used for game operations, when there are actions such as movement, camera operation, attack and jump, as in the first person viewpoint game and third person viewpoint game, move with the virtual joystick in the screen, There is a method in which two game operations such as attack and jump can be operated by touch input using virtual buttons, and the viewpoint operation is linked to the posture of the device.

JP2015225397 JP 2012168931 JP

In the case of touch input according to the holding method shown in FIG. 1 as used in Patent Document 1, only one input can be handled at the same time. It is possible to handle 6 inputs by adding inputs with 5 fingers of a hand that does not hold the device, but the hand that does not hold the device covers the screen and it becomes difficult to see the screen, making operation difficult .

The touch input by the way of holding in FIG. 2 as used in Patent Document 2 can handle only two inputs at the same time, for example, movement, viewpoint operation, attack, jump, etc. like the first person viewpoint game and the third person viewpoint game. When it is necessary to operate actions individually at the same time, existing methods that can handle only two inputs simultaneously cannot be operated satisfactorily. When an operation with an index finger is performed, it becomes difficult to hold the device and the operation becomes difficult.

In order to solve the above problem, when the viewpoint operation is performed by a method of interlocking with the posture of the device, the operator needs to change the posture of the body for the viewpoint operation, which makes other operations difficult.

Therefore, an object of the present invention is to provide a user interface program that is easy to operate, in particular, movement from a first-person viewpoint or third-person viewpoint, viewpoint aiming, and action in a device having a touch panel.

An aspect of the present invention is a program executed in a computer having a means for detecting a touch input and an inclination calculating means for sequentially calculating an inclination of an input device capable of posture operation,
A slope acquisition step for converting the slope into one or more numerical values;
A conversion step for converting the numerical value obtained in the acquisition step into a stick input signal;
A stick input step using the stick input signal obtained in the conversion step on an arbitrary program;
And a touch input step for performing an arbitrary process in response to the touch input.

Here, the stick input is to input a direction, and is equivalent to an input by an analog stick or a cross key of a real device.

With the above user interface program, when two or more operations are performed, an operation using a touch input and an operation using a stick input obtained in a stick input step can be performed independently and simultaneously.

For example, in the way shown in Fig. 1, when operating the device with one hand, touch input is performed with two fingers, so a total of two operations can be performed simultaneously with one operation based on stick input and one operation based on touch input. Do it independently.
In the way of holding in FIG. 2, since touch input is performed with two fingers, a total of three operations are performed simultaneously with one operation by stick input and two operations based on touch input.

Using the input according to the present invention, operating the viewpoint or aim or both, moving, and other functions, moving, viewpoint, aiming operation, etc. attack and shooting, jump, open menu, pause Operate the functions arbitrarily set as required, such as

A function that assigns one or more numerical values obtained from the posture information to a movement operation, a function that installs zero or more buttons in the touch input area, and performs an operation arbitrarily determined according to the state of each button And performs an arbitrarily determined operation according to an input by touching in zero or more set areas.

In the input system for the first-person view and third-person view, as shown in FIG. 7 or FIG. 0 or more are installed, and the operation is performed according to the state of each button, and the operation arbitrarily determined according to the input by touching in the 0 or more set areas is performed. If necessary, an arbitrarily determined operation is performed using input values based on the posture information (41) in FIG. 7 and the posture information (51) in FIG.

In the input system for the first-person view and third-person view, as shown in Fig. 7 or Fig. 8, the input by posture information (for example, 40 42 in Fig. 7 and 50 52 in Fig. 8) is assigned to the movement, and the button in the touch area 0 or more are installed, and the operation determined arbitrarily according to the state of each button is performed. Also, the viewpoint operation or aiming operation or both are performed by touch input in one or more set areas, and the operation arbitrarily determined according to the input by touch in zero or more set areas . If necessary, an arbitrarily determined operation such as a viewpoint operation is performed using the input values based on the posture information 41 in FIG. 7 and the posture information (51) in FIG.

Instead of touch input in the present invention, it is possible to handle more input operations by adding touch input devices, button input devices, stick input devices, microphones, and camera inputs in any number of combinations. And

If necessary, a function for performing an operation for enabling confirmation of a current input value by numerical input based on posture information is added.

By way of example, a means of confirmation is provided by any combination of the methods described below.
(1) Depending on the input value, the screen can be visually confirmed by adding numerical display, bar display, arrow direction display, etc.
(2) The sound can be confirmed by hearing by changing the sound type and volume according to the input value.
(3) Tactile confirmation can be made by changing the strength and manner of vibration by the vibration function of the device according to the input value.
(4) By changing the force sense given by a device having a function of giving a force sense according to the input value, it is possible to confirm by the force sense.

As a method of obtaining a numerical value input from posture information acquired by an electronic device, one or more rotation amounts from a specified reference posture around a specified axis are calculated, and one or more numerical values are handled as input values.

One or more numerical values determined by calculating the amount of rotation from one or more arbitrarily specified reference postures around an arbitrarily specified axis from the posture information acquired by the electronic device are treated as input values and necessary Accordingly, a range of rotation amount that invalidates the input value is added.

Regarding the rotation amount range in which the input value is invalid, the rotation amount corrected with reference to the rotation amount when the input value is out of the range is treated as an input.

Regarding the method of determining numerical input based on posture information acquired by electronic devices, the numerical value is determined according to the amount of change from the arbitrarily specified reference posture, and a range of rotation amount that treats the numerical value as invalid is added if necessary. To do.

Regarding the method of determining numerical input based on posture information acquired by an electronic device, the numerical value is determined according to the amount of change from the arbitrarily specified reference posture, and if necessary, there is a range of rotation amounts that can be treated as invalid. Then, a numerical value is determined in accordance with the amount of change based on the posture when it goes out of range.

Here, the electronic device is a device having a function of detecting the posture of the device, such as an acceleration sensor, a gyro sensor, a camera, or an inertial measurement device.

As an example of how to specify the axis and handle numeric input, for the Cartesian coordinate x-axis, y-axis, and z-axis, if the gravity direction is the y-axis and the direction the device is facing is the z-axis, The numerical input obtained by rotating the device on the z axis is treated as a joystick input.
Treat device rotation on the y-axis as a numeric input if necessary.

As an example of the method of determining numerical input, as shown in Fig. 3, the rotation amount (25) x from the reference value (20) and the rotation amount threshold1 (21) required to enable the joystick input for a certain axis And threshold2 (22), maximum rotation amounts max1 (23) and max2 (24), the value when the numeric input is disabled is 0, and the input value of one axis is joy.
The value of joy when the rotation amount x becomes threshold 1 is the minimum value in the positive direction, and the value of joy when the rotation amount x is max 1 is the maximum value in the positive direction.
The joy value when the rotation amount x becomes threshold2 is the negative maximum value, and the joy value when max2 is max2 is the negative minimum value.
When x is in the range from threshold1 to max1, the value of joy is changed from the minimum value in the positive direction to the maximum value in the positive direction according to the value of x. When x is greater than max1, the value of joy is the maximum value in the positive direction.
When x is in the range from threshold2 to max2, the value of joy is changed from the maximum value in the negative direction to the minimum value in the negative direction according to the value of x. When x is less than max2, the value of joy is the minimum value in the negative / negative direction.
When x is in the range of threshold1 to threshold2, numerical input is invalidated, and the value of joy is set to 0 here.

By setting threshold1 (21) and threshold2 (22), it functions to ignore the rotation at an arbitrarily determined rotation amount, and makes it easy to make the input value an input invalid state.
By setting max1 (23) and max2 (24), it is possible to set the rotation amount at which the input value is maximized.
Here, the reference value (20), threshold1 (21), threshold2 (22), max1 (23), and max2 (24) can be arbitrarily set as parameter values as necessary.

For example, with respect to the reference value (20), a button or the like is provided, and the attitude of the device when turned on is handled as the reference value. This allows the user to easily determine a reference value for calculating the amount of rotation of the device according to his / her preference.

As described above, it is possible to designate a rotation amount range in which numerical input is invalidated as necessary. In addition, it is possible to determine a numerical value input based on the rotation amount based on the posture of the device when it goes out of the range of the rotation amount that invalidates the numerical value input.

According to the present invention, it is possible to provide a user interface program that facilitates movement, viewpoint aiming, and action operation from a first-person viewpoint or a third-person viewpoint, particularly in a device mainly having a touch panel.

For example, in a first-person shooter game where it is necessary to operate the movement, viewpoint operation, and shooting action on a tablet device individually at the same time, only two touch inputs can be operated at the same time, and one input is assigned to movement. Shooting actions are performed at the same time, and it is difficult to operate the viewpoint change and the shooting independently independently of the movement operation. The user interface program that performs the touch operation according to the present invention and the operation using one or more numerical values obtained from the posture information simultaneously and independently enables a plurality of input values to be handled at the same time. Is solved.

Example of holding vertically Example of how to hold Explanatory diagram of numerical input decision method based on posture information Block diagram showing the hardware configuration of the tablet device User interface program module Numerical decision flowchart Examples of input systems according to the invention Examples of input systems according to the invention Example of device rotation axis Illustration of character movement Touch area and button layout example Configuration example with stick, button, camera and microphone

As an example for explanation, according to the present invention, an embodiment of a user interface program of a third-percent character game in a tablet device (60) having an inclination calculating means and a touch display is shown.

FIG. 4 is a block diagram showing a hardware configuration example of the tablet device (60). The tablet device (60) in this embodiment is a general computer that includes a CPU (100), a main memory (101), an auxiliary memory (102), a display unit (106), and an input unit (103). Among these, the user interface program according to the present embodiment is stored in the auxiliary storage (102). The display unit (106) includes a touch display. The input unit (103) includes a touch input unit (105) and a tilt calculation unit (104) of the touch display. The program executed by the tablet device (60) can detect that there has been some operation on the touch display by the touch input means (105). The program executed on the tablet device (60) can detect the tilt of the tablet device (60) by the tilt calculation means (104). Here, the touch input means (105) and the inclination calculation means (104) are realized by hardware, software, or a combination thereof.

FIG. 5 shows an example of a module of the user interface program of the present embodiment.
The acquisition step (115) of the stick input unit (114) acquires the inclination of the device from the inclination calculation means (111) as a numerical value handled in the conversion step.
In the conversion step (116), the numerical value received from the acquisition step (115) is converted into a stick input signal and transferred to the stick input step (117).
In the stick input step (117), the stick input signal received from the conversion step (116) is transferred as a stick input to an arbitrary program (113) executed on the device.
The arbitrary program (113) that has received the stick input performs some processing according to the stick input. At this time, there may be a plurality of arbitrary programs (113).
In the UI display step (119) of the touch input unit (118), a virtual input device that can be operated by touch input is displayed on the display unit (106).
In the UI operation step (120), the operation of the virtual input device displayed in the UI display step (119) is detected to determine the UI input value. For example, whether or not the button is pressed, the amount of movement of the virtual stick, and the like. Further, the display of the virtual input device is changed according to the operation of the virtual input device.
In the touch input step (121), the touch input obtained from the touch input means (112) and the UI input value of the virtual input device determined in the UI operation step (120) are arbitrary programs (113) executed on the device. To hand.
Any program that has received these inputs (113) performs some processing in accordance with the input. At this time, there may be a plurality of arbitrary programs (113).

The program according to the present invention may be implemented as a part of an arbitrary program (113).

In the example of the embodiment shown here, in a third person viewpoint game (third person character game, TPS) that can operate a character capable of moving forward and backward, left and right, jumping and attacking, and operating the game viewpoint, It is a user interface program that operates based on a posture in which character movement is acquired by a gyro sensor.

Explain the movement of characters.

As shown in Fig. 9, the device rotation axis is assigned to the right direction of the device as x-axis (61), the device height direction as z- axis ( 63 ), and the device thickness (front) direction as y- axis ( 62 ). Suppose that there are three orthogonal coordinate axes consisting of xyz axes. This is an example, and the rotation axis of the device can be arbitrarily set according to the preference of the user of the device. For example, it is possible to take the z axis so that it is in the opposite direction to gravity. Further, the method of taking the axes is not limited to the orthogonal coordinates, and for example, the number of axes may be other than 3, or the axes may not be orthogonal.

The numerical input based on the rotation of each axis is determined as follows.
As shown in Fig. 3, the rotation amount (25) x from the reference value (20), rotation amounts threshold1 (21) and threshold2 (22) required to enable joystick input, and maximum rotation for a certain axis When the amount is max1 (23) and max2 (24), and the value when the numeric input is disabled is 0, the input value joy of one axis is
if (x> max1) x = max1
if (x <max2) x = max2
if (x> threshold1) {
joy = (x-threshold1) / (max1-threshold1) * 100.0
}
if (x <threshold2) {
joy = (x-threshold2) / (max2-threshold2) * 100.0
}
if (x <threshold1 &&x> threshold2) {
joy = 0.0;
}
Decide like this. A flowchart is shown in FIG. Here, the value of joy is a value in the range of -100 to 100, but it can be changed to any range as necessary.

By setting threshold1 (21) and threshold2 (22), it functions to ignore the rotation at an arbitrarily determined rotation amount, and makes it easy to make the input value an input invalid state.
By setting max1 (23) and max2 (24), it is possible to set the rotation amount at which the input value is maximized.
Here, the reference value (20), threshold1 (21), threshold2 (22), max1 (23), and max2 (24) can be set freely as needed as parameter values that can be set individually for each rotation axis. It is.

For example, for the reference value (20), a reset button (80) or the like is provided, and the attitude of the device when the button is pressed is handled as the reference value. This allows the user to easily determine the reference posture for calculating the amount of rotation of the device according to his / her preference.
These processes are performed in the acquisition step (115) of the stick input unit (114).

Numerical input based on the rotation of the x-axis (61) and the y-axis (62) obtained as described above will be referred to as joy_x and joy_y, respectively.

The character is moved based on the values of joy_x and joy_y. More specifically, as shown in Fig. 10 (a), when the joy_x value moves in the front-rear direction as viewed from the viewpoint, and the device is rotated in a direction that tilts the device toward the back, the character is It moves in the direction (left side of FIG. 10 (a)), and moves in the front direction when rotated in the opposite direction (right side of FIG. 10 (a)).
As shown in Fig. 10 (b), depending on the value of joy_y, the character moves left and right as viewed from the viewpoint, and when the device is rotated in a direction that tilts to the right, the character moves right as viewed from the viewpoint (Fig. 10 (b) right side), it moves to the left when rotated in the direction tilted to the left (left side of FIG. 10 (b)).
The process related to the movement here is an example, and the movement of the character may be determined by another method based on the values of joy_x and joy_y.

If necessary, means for indicating the moving direction of the character by the above method is provided. In this embodiment, an arrow (70) indicating the moving direction is displayed around the character. The direction of the arrow indicates the moving direction, and the size of the arrow indicates the moving speed. The moving direction of the character shown at this time is preferably the target value before correction determined by the present invention, not the moving direction corrected as a result of obstacles or terrain in the game.

Alternatively, stick input is generated based on the values of joy_x and joy_y. As a specific example, the value of joy_x corresponds to the vertical movement of the stick, and the value of joy_y corresponds to the horizontal movement of the stick.
These processes are performed in the conversion step (116) of the stick input unit (114).

At this time, the character performs a moving operation by a stick input including a stick input obtained from the stick input step (117) of the stick input unit (114) and a stick input by an existing stick input means.
Operations other than movement may be performed by stick input based on the values of joy_x and joy_y.

If necessary, means for indicating the stick input direction by the above method is provided. In this embodiment, an arrow (70) indicating the stick input direction is displayed around the character. The direction of the arrow indicates the stick input direction, and the size of the arrow indicates the amount corresponding to how much the stick is moved in the stick input. In the present embodiment, when a vector having joy_x and joy_y as components is considered, the direction of the arrow indicates the direction of the vector, and the size of the arrow indicates the size of the vector.

The viewpoint is operated based on the touch input in the viewpoint operation area (81) designated in the touch display.

In the present embodiment, the viewpoint is changed based on the sliding direction and amount of the touch input performed in the viewpoint operation area (81). As an example, the viewpoint is rotated around an arbitrary vertical axis in the game space according to a horizontal slide, and is rotated around an arbitrary vertical axis in the game space according to a vertical slide. May be. A more detailed example is an axis that penetrates a player in the game space with respect to the axis.

For character attacks, an attack button (82) is arranged in the touch display, and when the button is pressed, the character performs an attack action corresponding to the button.

With regard to character jumping, in this embodiment, a touch gesture area (83) is set in the touch display, and when a double tap is performed within the area, the character performs a jump action.

FIG. 12 shows a configuration example in which a stick (91), buttons (92, 93, 94, 95, 96, 97), a camera (98), and a microphone (99) are added. The viewpoint, attack, jump, and reset button (80) are operated by the touch display using the stick (91), buttons (92, 93, 94, 95, 96, 97), camera (98), and microphone (99). More specifically, the viewpoint is operated by operating the stick (91), and the attack or jump action or the reset button (80 corresponding to the pressed button is operated by operating the buttons (92, 93, 94, 95, 96, 97). ). For example, the camera (98) recognizes the facial expression of the operator and performs an action corresponding to the facial expression. For example, the voice of the operator is recognized by the microphone (99), and an action corresponding to the voice is performed.

If necessary, a function for performing an operation for enabling confirmation of a current input value by numerical input based on posture information is added.

By way of example, a means of confirmation is provided by any combination of the methods described below.
(1) Depending on the input value, a numerical display, a bar display, and an arrow display (70) are added to the screen for visual confirmation.
(2) The sound can be confirmed by hearing by changing the sound type and volume according to the input value.
(3) Tactile confirmation can be made by changing the strength and manner of vibration by the vibration function of the device according to the input value.
(4) By changing the force sense given by a device having a function of giving a force sense according to the input value, it is possible to confirm by the force sense.

1 Hand holding the device
2 thumb
3 Hand not holding the device
11 Thumb of left hand
12 Thumb of right hand
Reference value for 20 rotations
21 Threshold for enabling input
22 Threshold for enabling input
23 Maximum value of rotation
Maximum value of 24 revolutions
25 Rotation amount
40 axis of rotation
41 Rotation axis
42 Rotating shaft
43 Example of touch area
44 Example of how to set the touch area
45 Button layout example
46 Button layout example
47 Button layout example
50 axis of rotation
51 Rotation axis
52 Rotation axis
53 Example of touch area
54 Button layout example
55 Button layout example
60 tablet devices
61 x-axis
62 y-axis
63 z-axis
70 Display showing moving direction
80 Reference posture reset button
81 Viewpoint operation area
82 button
83 Touch gesture area
91 stick
92 button
93 button
94 button
95 button
96 button
97 button
98 camera
99 microphone
100 CPU
101 Main memory
102 Auxiliary memory
103 Input section
104 Inclination calculation means
105 Touch input means
106 Display
110 Program configuration example according to the present invention
111 Inclination calculation module
112 Touch input module
113 Any program that uses input by a user interface program according to the present invention
114 Stick input section
115 Acquisition steps
116 Conversion steps
117 Stick input step
118 Touch input section
119 UI display step
120 UI operation steps
121 Touch input step

Claims (9)

A program that is executed in a computer having an inclination calculating unit that sequentially calculates an inclination of an input device capable of posture operation;
An inclination obtaining step for converting the inclination obtained by the inclination calculating means into a numerical value corresponding to one or more rotation axes;
In the inclination acquisition step, one or more numerical values obtained from the inclination are determined based on a rotation amount from an arbitrarily designated reference posture of the input device, and the rotation amount invalidates the numerical value as necessary. A user interface program characterized in that the numerical value is used on an arbitrary program. A program that is executed in a computer having an inclination calculating unit that sequentially calculates an inclination of an input device capable of posture operation;
An inclination obtaining step for converting the inclination obtained by the inclination calculating means into a numerical value corresponding to one or more rotation axes;
In the inclination acquisition step, one or more numerical values obtained from the inclination are determined based on a rotation amount from an arbitrarily designated reference posture of the input device, and the rotation amount invalidates the numerical value as necessary. As for the rotation amount outside the range of the rotation amount that invalidates the input, a process according to the rotation amount that invalidates the input is performed and used on an arbitrary program
A user interface program characterized by that. A program that is executed in a computer having an inclination calculating unit that sequentially calculates an inclination of an input device capable of posture operation;
An inclination obtaining step for converting the inclination obtained by the inclination calculating means into a numerical value corresponding to one or more rotation axes;
A conversion step of converting the numerical value obtained in the inclination acquisition step into a stick input signal;
A stick input step for using the stick input signal obtained in the conversion step as a stick input on an arbitrary program;
The stick input is to input a direction,
In the inclination acquisition step, one or more numerical values obtained from the inclination are determined based on a rotation amount from an arbitrarily designated reference posture of the input device, and the rotation amount invalidates the numerical value as necessary. A user interface program that performs a process of applying a correction according to a rotation amount that invalidates the input for a rotation amount outside the rotation amount range that invalidates the input. A program that is executed in a computer having an inclination calculating unit that sequentially calculates an inclination of an input device capable of operating a posture, a unit that detects touch input, and a display as necessary.

An inclination obtaining step for converting the inclination obtained by the inclination calculating means into a numerical value corresponding to one or more rotation axes;
A conversion step of converting the numerical value obtained in the inclination acquisition step into a stick input signal;
A stick input step for using the stick input signal obtained in the conversion step as a stick input on an arbitrary program;
The stick input is to input a direction,
In the inclination acquisition step, one or more numerical values obtained from the inclination are determined based on a rotation amount from an arbitrarily designated reference posture of the input device, and the rotation amount invalidates the numerical value as necessary. For the rotation amount outside the range of the rotation amount that invalidates the input, a process according to the rotation amount that invalidates the input is performed,
In the case of having the display, a UI display step for displaying a virtual input device operable by touch input on the display;
A UI operation step of detecting an operation of the virtual input device displayed in the UI display step and determining a UI input value;
And a touch input step of using the touch input obtained by the means for detecting the touch input and the UI input value determined in the UI operation step when the display is provided on the arbitrary program. A user interface program characterized by that. The user interface program according to any one of claims 3 and 4,
A user interface program comprising means for moving an operation target object having moving means in a real or virtual world, wherein the moving operation is performed according to the stick input available in the stick input step. 6. The user interface program according to claim 5, wherein the movement of the operation target object and the operation of the viewpoint or aim or both, and any other function as necessary are performed from a first person viewpoint or a third person viewpoint. And
The movement operation of the operation target object is a movement operation according to claim 5,
The operation of the viewpoint includes changing the observation direction of the observation means for the operator to observe the space where the operation target object exists,

The arbitrary function is a function of at least one arbitrary program,
The operation of the viewpoint, the aim, or both, and the operation of the arbitrary function as necessary are performed according to the touch input and / or UI input value that can be used in the touch input step. A user interface program characterized by The movement operation according to any one of claims 5 and 6 is such that the operation target object moves in a direction associated with a corresponding input direction in accordance with the stick input available in the stick input step. A user interface program that is to control. A computer that executes the user interface program according to any one of claims 1 to 7,
Input means for detecting the input of the button input device, input means for detecting the input of the stick input device, microphone input means, camera input means,
A user interface program that is executed by a computer having input means added in any number of combinations,
The user interface program further comprising a step of determining an input value according to an input obtained by the input means and using the input value on the arbitrary program. 9. The user interface program according to claim 3, wherein the input state of the stick input that can be used in the stick input step is confirmed by any combination of visual, auditory, tactile, and force senses. A user interface program with the ability to

Images