JP2008015887A - User interface device and user interface method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily perform menu selection and command execution in an apparatus or the like. <P>SOLUTION: This user interface device is provided with: a first inputting means for detecting a standing position of a user; a second inputting means for detecting an operation by the user's hand or a finger; an input determining means for determining positions and input states detected by the first inputting means and the second inputting means; a timer means for executing temporal waiting processing; a storing means holding a layout table, a state transition table and a function table; a state transition updating means for reading data from the storing means on the basis of processing of the input determining means to update a state transition; a displaying means for displaying an update result of the state transition updating means; and an outputting means for executing a command of the function table when data of the state transition table and the function table satisfy a predetermined condition in the processing of the input determining means. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a user interface device and a user interface that use both hands and feet to enable operation and input of dynamically changing items so that menu selection and command execution in devices can be easily performed. Regarding the method.

  The development of user interfaces that manipulate information using physical gestures of users has been actively conducted, centering on game machines and systems that apply virtual reality technology.

For example, Patent Literature 1 and Patent Literature 2 disclose an invention in which a game operation is performed at a foot step. Patent Documents 3 and 4 disclose an invention of an input interface device for movement in virtual reality that detects the direction and magnitude of weight movement.
JP 2004-24903 A “Game Device, Information Storage Medium, Game Distribution Device, and Game Distribution Method” JP 2005-143714 A “Game Device Control Method and Game Device” Japanese Patent Laid-Open No. 11-305907 “Virtual Space Control Method, Virtual Space Control Device, and Recording Medium Recording Virtual Space Control Program” Japanese Unexamined Patent Application Publication No. 2004-102800 “Body Direction Detection Device, Body Direction Detection Method, and Virtual Environment Simulated Experience System”

  However, in these prior arts, the foot switch and the visual operation target are uniquely determined, and dynamic operations such as state transition cannot be operated. In addition, there is no mechanism to operate limbs at the same time.

  Therefore, in these prior arts, input by foot can be used as a mere input switch, but complicated menu selection in which a plurality of items are in a tree shape cannot be performed.

  In addition, users cannot use their hands and feet simultaneously, and there is no user interface for operating the same information with their hands and feet, so items can be selected and input efficiently. There wasn't.

  The present invention has been proposed in view of the above-mentioned conventional problems, and the object of the present invention is to use both hands and feet to easily perform menu selection and command execution in devices. It is an object of the present invention to provide a user interface device and a user interface method that enable input by operating items that change dynamically.

  In order to solve the above problems, in the present invention, as described in claim 1, the first input means for detecting the standing position of the user and the operation by the user's hand or finger are detected. Second input means, input determination means for determining the position and input state detected by the first input means and the second input means, timer means for executing a temporal standby process, and the first input means A storage table that stores a layout table that stores the arrangement of each position that can be input by the input unit, a state transition table that stores state transition data, and a function table that stores commands in the transitioned state, and the above input State transition update means for reading data from the storage means based on processing of the determination means and updating the state transition, display means for displaying the update result of the state transition update means on a display device, and the input determination Data of the state transition table and the function table is summarized as a user interface device and an output means for executing the command of the function table when a predetermined condition is satisfied in the processing stage.

  In addition, as described in claim 2, in the user interface device according to claim 1, the first input unit includes a time measuring unit that measures a time interval, and is equal to or greater than a predetermined value within a predetermined time. It is possible to detect an input state such as a click or a double click by the number of times exceeding the value of.

  In addition, as described in claim 3, in the user interface device according to claim 1, the first input means includes a matrix weight scale and a pressure sensor for measuring the position and weight shift of the user. It can be configured.

  According to a fourth aspect of the present invention, in the user interface device according to the first aspect, the display unit is configured such that the selection target is the first input unit or the second input unit according to an operation by the first input unit or the second input unit. The display object can be moved so as to be displayed in the center of the display device.

  Further, according to a fifth aspect of the present invention, in the user interface device according to the first aspect, the display means enlarges the relationship between the functions of the state transition table compared to the unrelated one. And highlighting such as changing colors.

  Moreover, as described in claims 6 to 10, it can be configured as a user interface method.

  In the user interface device and the user interface method of the present invention, a user can operate an object on a display device that changes dynamically by using both hands and feet for the object. Menu selection and command execution can be performed easily. In this case, since it is not always necessary to use the hand, for example, the device can be operated with the foot even when the baggage is held in the hand. In addition, according to the present invention, an operation corresponding to a click operation or a double click operation realized by a hand or a finger can be realized at the time of input by a foot.

  Hereinafter, preferred embodiments of the present invention will be described.

  FIG. 1 is a diagram showing an example of the appearance of an embodiment in which the user interface device of the present invention is applied to a copying machine.

  In FIG. 1, the user interface device includes a foot input device 2 including a scale, a pressure sensor, and the like disposed at the foot of the user U, a display device 3 including a touch panel display, and a manual input device 4. Become. For example, the display device 3 and the manual input device 4 are operation panels of the device 1 such as a copying machine. In addition, the foot input device 2 arranges the sensors in a matrix so that the user U can easily move with his / her foot. For example, it can be configured as a matrix of 3 horizontal rows and 2 vertical columns.

  FIG. 2 is a diagram showing an example of an external appearance in which the user interface device of the present invention is applied to another device. (A) is applied to a cash register (cash register) as the device 1, and (b) is a device 1. Applied to the advertising display. It will be apparent that it can be applied to other devices.

  FIG. 3 is a diagram illustrating an example of the foot input device 2, the display device 3, and the manual input device 4.

  In FIG. 3, on the display device 3, objects A to F like the “state transition diagram” are displayed, and the user U operates the transition between the objects by hands and feet. Select the menu associated with those objects and execute commands.

  For example, the object currently selected by the user U can be identified by changing the background color or changing the display state of the outer frame.

  The objects A to F on the display device 3 exist in a form corresponding to the matrix arrangement of the sensors W0 to W5 of the foot input device 2. For example, if the user U currently selects the object B of the display device 3 and is on the sensor W1 of the foot input device 2, when the user U moves in the direction “a”, that is, the sensor W0, On the display device 3, the selected state moves from the object B to the object A. At this time, the object A is displayed just in the center of the display device 3.

  The user U can perform the same operation using the manual input device 4. That is, when the object A is at the position of the object B, the selected state moves from the object B to the object A by pressing the object A with a hand.

  That is, the user U can execute complicated menus and command systems that change dynamically using hands and feet simultaneously. At this time, since it is not always necessary to use the hand, for example, the device 1 is operated with the foot even when a magazine or the like is pressed against the copy surface of the copying machine or when the baggage is held in the hand. it can.

  Of course, there is an existing input device that makes foot input correspond to general key input just like a foot switch for assisting people with disabilities, and a visual interface that makes state transitions for input such as pen touch. Although it is known, there has been no user interface that can operate the limbs at the same time, can operate the same operation interface with both hands and feet, and can also operate each hand and feet separately. .

  FIG. 4 is a diagram illustrating an operation example of the sensor W constituting the foot input device 2.

  In FIG. 4, when the user U puts his / her foot on one sensor W of the foot input device 2 arranged in a matrix, the sensor W which is a scale or a pressure sensor detects that the weight of the user U is applied. To do.

  FIG. 5 is a diagram illustrating a state in which a click is detected by a load change.

  In FIG. 5, the foot input device 2 measures a change in applied load on the time series. When the measured value exceeds a specified value P, it is determined that an input such as a so-called mouse click on a personal computer has been performed.

  FIG. 6 is a diagram showing a state in which a double click is detected by a load change.

In FIG. 6, the foot input device 2 measures a change in load applied in time series, but in particular, by having a time measuring function (means), the prescribed value P was exceeded twice within a time t _interval . In this case, it is determined that a double click operation has been performed. The number of times of detection is not limited to once or twice, and more times may be detected.

  As shown in FIG. 7, the foot input device 2 regards the entire input surface as one input device, detects the inclination of the input surface with an inclination sensor, and determines the current user state based on the value of the inclination. It can also be realized by the method of doing.

  FIG. 8 is a diagram illustrating an example of a matrix configuration of the sensors W of the foot input device 2.

  FIG. 8A shows a case where the sensors W are configured in a 3 × 2 matrix, and FIG. 8B shows a case where the sensors W are configured in a 6 × 4 matrix. The number of matrices is not limited to these, and can be further increased or decreased. Also, the aspect ratio can be changed.

  FIG. 9 is a diagram illustrating an example of another recognition method of the foot input device 2.

  FIG. 9A shows a system in which LED light is projected by the light emitting / receiving device 5 provided in the device 1 to recognize the reflected light and detect the standing position of the user U. (B) is a method of recognizing the standing position of the user U in the left-right depth direction from the image by a plurality of cameras 6 provided in the device 1. In (c), the ultrasonic wave is transmitted / received by a plurality of ultrasonic transmission / reception devices 7 provided in the device 1 and the distance to the user U is measured from the time difference between them to determine the standing position of the user U. This is a detection method. Various other methods can be used.

  FIG. 10 is a combination of the technique using the weight scale and pressure sensor shown in FIG. 1 and the technique shown in FIG. That is, the light emitting / receiving device 5, the camera 6, or the ultrasonic transmission / reception device 7 recognizes that the user U approaches the device 1 and starts up the display unit of the device 1, or the user U uses the foot input device 2. You can recognize when you are not riding.

  FIG. 11 is a diagram illustrating an example of an additional function of the foot input device 2, which is devised so that the user U can easily identify the position of the foot input device 2. FIG. 11A is an example in which the sensor W of the foot input device 2 is provided with an actuator that causes a change in light or sound when selected, and FIG. 11B is provided with a display for displaying the state on the selected sensor W. This is an example. By doing in this way, when the user U gets on the predetermined sensor W of the foot input device 2, it can be easily determined whether or not the user U is properly recognized.

  FIG. 12 is a diagram showing an example of functional blocks of the user interface device of the present invention.

  In FIG. 12, the user interface device includes a first input unit 11 (corresponding to the foot input device 2 in FIG. 1) for detecting input by a foot composed of a scale, a pressure sensor, etc., and a hand composed of a touch panel. And a second input unit 12 (corresponding to the manual input device 4 in FIG. 1) for detecting input by a finger, an input determination unit 13 for determining these inputs, and a timer unit 14 for executing temporal standby processing I have.

  In addition, a layout table T1 that stores the arrangement of each position that can be input by the first input unit 11, a state transition table T2 that stores the name of a state transition object and a related tree, and a function table that stores commands in a transitioned state The storage unit 15 that holds T3, the state transition update unit 16 that reads data from the storage unit 15 based on the processing of the input determination unit 13 and updates the state transition, and the update result of the state transition update unit 16 are displayed on the display device 3 And the output unit 18 that executes the command of the function table T3 to the device 1 when the data of the state transition table T2 and the function table T3 satisfy a predetermined condition in the processing of the input determination unit 13. And.

  By configuring in this way, it is possible to realize simultaneous operation by hand and foot, and furthermore, it is possible to perform the same user interface operation on the display device 3 for the hand and foot.

  FIG. 13 is a diagram showing an example of the layout table T1.

  FIG. 13A shows an example of the layout table T1, FIG. 13B shows an array of sensors W of the foot input device 2, and FIG. 13C shows state transitions between the sensors W.

  When the sensors W of the foot input device 2 arranged in a matrix form on the floor as shown in (b) are numbered as 0 to 5, respectively, the state transitions are sensors adjacent to each other as shown in (c). You can move to W. As a condition for the state transition at this time, it is assumed that each sensor W transitions when a specified load value P is exceeded.

  In the layout table T1, as shown in (a), the sensor W currently selected by the user U and the candidate of the sensor W that can be changed next are stored in association with each other.

  FIG. 14 is a diagram illustrating an example of the state transition table T2.

  FIG. 14A shows an example of the state transition table T2, and FIG. 14B shows an example of a corresponding function tree.

  For example, when the function of the copying machine is taken as an example, it can be expressed as a tree of functions including the initial state “copy start” of S0 in which the function to copy is selected, as shown in FIG. If each function is numbered as 0 to 9 in the state transition table T2, as shown in (a), state transition candidates for the states to be advanced from the respective states are associated with each other and held. .

  FIG. 15 is a diagram illustrating an example of the function table T3.

  In FIG. 15, in the function table T3, the numbering, the function name, the condition, and the command are stored in association with each other for the function states as described in FIG. For example, for S4, the function name is “+1”, the command is executed as a condition when clicked, and the command is “increase the number of copies”. Recorded at T3.

  FIG. 16 is a flowchart showing an overall processing example of the user interface apparatus of the present invention. Details of each defined process listed here will be described later.

  In FIG. 16, when the operation process in the device 1 is started by installing and starting the device 1 so that the device 1 can be used by the user (step S101), first, initialization of interrupt processing by a timer is executed, and the variable 0 is substituted for F (step S102).

  Next, timer interruption is started (step S103). The change (weight change) of the movement of the user on the specific foot input device 2 is measured by the timer process.

  Next, initial state processing (step S104), sensor identification processing (step S105), layout display processing (step S106), and command execution processing (step S107) are sequentially executed.

  Then, it is determined whether or not the value of the variable F has been updated to “1” by the timer process (step S108). If F is not “1”, the process returns to the sensor identification process (step S105) and the process is repeated. When “1” is “1”, the interruption of the timer is terminated (step S109), and all the processes are terminated (step S110).

  FIG. 17 is a flowchart showing an example of the initial state process (step 104 in FIG. 16). In the timer of FIG. 16, whether or not the foot input device 2 stays for a certain period of time, FIG. 17 shows processing when the user actually takes the foot input device 2.

  In FIG. 17, when the processing is started (step S201), first, the object j of the selected state transition table T2 is represented as Sj, and S0 is selected as Sj, that is, the first object in the state transition table T2 is selected. The state is set (step S202).

  Next, it is determined whether or not a foot input is detected by any of the sensors W of the foot input device 2 (step S203). If not detected, the detection process is repeated.

  If a foot input is detected, the initial state process is terminated (step S204).

  FIG. 18 is a flowchart showing an example of the sensor identification process (step 105 in FIG. 16). This process is a process of detecting which specific foot input device 2 the user is actually operating after the immediately preceding initial state process.

  In FIG. 18, when the process is started (step S301), first, 0 is substituted for variable i as initialization (step S302).

  Next, when the sensor W for the number i of the foot input device 2 is represented as Wi, it is determined whether or not a load is applied to Wi (step S303).

  If not, i = i + 1 is set (step S304), and it is determined whether i exceeds the total number of sensors MAX (step S305). Return to. If exceeded, the process returns to initialization of the variable i (step S302).

  When a change in body weight is detected in Wi, the weight value is assumed to be Gi, and it is determined whether the Gi exceeds a specified value P (step S306). P is, for example, 30 kg.

  If not, the detection is repeated with i = i + 1 sequentially (steps S304 and S305).

  If it exceeds, it is determined that Wi has been clicked or double-clicked by user U, and user U has selected an object at a position corresponding to Wi as an object on display device 3 (step In step S307, the process ends (step S308).

  FIG. 19 is a flowchart showing an example of the layout display process (step 106 in FIG. 16). This is a process of updating the display state corresponding to the specific foot input device 2 detected by the process of FIG.

  In FIG. 19, when the process is started (step S401), it is assumed that the user U is on the sensor Wi, and the current state Sj is displayed on the object at the corresponding position on the display device 3 (step S402). The object displaying the current state Sj is displayed so as to be in the center of the screen.

  Next, referring to the layout table T1, the next candidate corresponding to the current Wi is searched, and if there is a next candidate for Wi, a list thereof is acquired (steps S403 to S405). If there is no next candidate, the process ends (step S410).

  Next, with reference to the state transition table T2, the next candidate corresponding to the current Sj is searched, and if there is a next candidate, the list is acquired (steps S406 to S408). If there is no next candidate, the process ends (step S410).

  Next, the next candidate list of Sj is sequentially displayed in the next candidate list of Wi (step S409). At this time, as shown in the function tree of FIG. 14B, the state transition is preferentially displayed so that the upper functions are not updated as a display.

  FIG. 20 is a diagram illustrating an example of the relationship between the sensor W and the state S of the foot input device 2 in the layout display process described above.

  20A, 20 </ b> B, and 20 </ b> C, the state S of the function tree shown in FIG. 14B is shown on the matrix of each sensor W of the foot input device 2.

  For example, as shown in FIG. 20A, if the user U starts an operation from the sensor W0, “copy start” which is a function of the state S0 is assigned thereto.

  In this state, when the user U selects “number of copies” which is a function of the state S1, the user U moves to the sensor W1. Then, as shown in (b), since the user U is on the sensor W1 and selects S1 as a function, S4 in the function tree shown in FIG. In order to select S5 and S0 which is the previous state, functions are assigned to the surroundings of the sensor W1, such as S4, S5 and S0, to the sensors W2, W4 and W0, respectively.

  In addition, as shown in FIG. 20 (c), for functions such as S2 and S3 that do not change state directly with S1, those that are not functionally inconsistent are assigned to vacant sensors such as sensor W3 and sensor W5. It is also possible to assign other functions without going back up the function tree. By doing so, for example, even when the user U selects the state of S4, the state of S2 is directly selected without going back to the state of S4 → S1 → S0. It becomes possible to improve convenience.

  FIG. 21 supplementarily shows the state of FIG. 20A and FIG. 20B in which the sensor number is assigned above the head of the function tree. FIG. 21A shows the state of FIG. 20A, and FIG. 21B shows the state of FIG. As the state transitions in this way, the function of the state that can be selected (transitioned) next is sequentially assigned around the position of the sensor W where the user U is currently standing.

  FIG. 22 is a diagram showing an example of command execution processing (step 107 in FIG. 16).

  In FIG. 22, when the process is started (step S501), it is determined whether or not the function Sj currently selected by the user U satisfies a condition such as a click or double click of Wi in the function table T3 (step S501). S502).

  If the condition is met, the command Sj corresponding to the condition is executed (step S503), and the process is terminated (step S504).

  In the case of a copying machine, for example, when Sj is S0, if a double-click operation is performed, the command “execute copy” is executed. Executing the copy means sending the copy execution control information to the copying machine through the output unit 18 (FIG. 12). Since a well-known technique can be used for a hardware control method for devices other than the configuration of the present invention such as a copying machine, detailed description thereof is omitted here.

  FIG. 23 is a flowchart showing an example of timer interrupt processing. This timer interrupt process is executed as a process in parallel with the process flow described so far.

  In FIG. 23, when the process is started (step S601), the number of interrupts is checked (step S602), and it is determined whether the specified value is exceeded (step S603).

  If the specified value is not exceeded, the interrupt count is incremented (step S604), F = 0 is substituted (step S605), and the process returns to the standby state (step S607).

  If the specified value is exceeded, F = 1 is substituted (step S606), and the process returns to the standby state (step S607).

  In this case, if the timer generates an event every second, for example, the specified value = 3 is set.

  FIG. 24 is a diagram showing an operation example reflecting the function of the copying machine.

  For example, as shown in FIG. 24A, when the user U first places on the sensor W0 of the foot input device 2, an initial screen is displayed on the display device 3. On the upper part of the display device 3, the parameters of the currently selected copier are displayed as a menu.

  When the user U moves to the sensor W1, a click operation corresponding to selecting the state of the object “number of copies” from the state of the object “copy start” on the display device 3 is executed. On the display device 3, as shown in FIG. 24B, the object “number of copies” is displayed as if it has been moved to the center on the display device 3, and the user U is in a state of operating the “number of copies”. The display of the display part “Number of copies:” at the top of the display device 3 is updated. At this time, an object not related to the object related to the selected object is displayed so that it can be visually distinguished.

  Next, when the user U moves from the sensor W1 to the sensor W2, the object “+1” is selected on the display device 3 (FIG. 24C). Here, on the display device 3, “+1” is added to the “number of copies” whose state has just changed, that is, an operation of increasing the number of copies from 1 to 2 is performed, and the display on the upper portion of the display device 3 is also displayed as “number of copies: 2 The display is updated to "Part".

  Next, when the user U moves from the sensor W2 to the sensor W4, the display state changes so that the object “color” is displayed at the center of the display device 3 on the display device (FIG. 25A). Here, what has been the operation for “number of copies” is changed to “color” operation, and the display state of “color:” is updated from “number of copies:” at the top of the display device 3.

  Next, when the user U moves from the sensor W4 to the sensor W3, the display is updated so that the “color” is also in the center on the display device 3 (FIG. 25B). Here, on the display device 3, an operation of changing the “color” whose state has just changed to “color”, that is, changing the color from monochrome to color is executed. The display device 3 is updated to “color” so that the update can be seen also in the display on the upper part of the display device 3.

  Next, when the user U moves from the sensor W3 to the sensor W0, the user U returns to the position of the first object “copy start” (FIG. 25C). Here, when the user U performs a double-click operation, that is, an operation that jumps on the sensor W0, “copy start”, that is, copying of the copying machine is executed. At this time, the copying is executed in a state where the parameters of the copying machine “number of copies: 2” and “color: color” selected in the previous operations are selected.

  FIG. 26 is a diagram showing an example of display when there are a plurality of candidates that cannot be displayed. That is, an example of processing in the case where the number of candidates that can change in the state transition table T2 exceeds the number of objects on the layout is shown.

  For example, it is assumed that there are many transition candidates such as D, E, F, and G before D.

  In such a case, it is displayed on the display device 3 that there are a plurality of candidates such as “D˜” in the state 1.

  When the user U selects “D˜”, as the state 2 (a), the movement candidate “D” is displayed in the center, and other movement candidates are displayed in the surroundings. When executing, double-click. Alternatively, as in state 2 (b), D, E, F, and G are displayed in the surroundings, and each command is executed when transition is made as a click operation.

  FIG. 27 is a diagram showing an example of the operation when hand input and foot input are used together.

  The object displayed on the display device 3 may be operated through the hand input device 4 in addition to the foot input device 2 as described above. In this case, the user U may remain at a predetermined position of the foot input device 2, and in this state, the user U selects an object of the display device 3 through the manual input device 4.

  In this case, for example, as shown in (a), when the adjacent object B is selected from the state of A, the display is updated on the display device 3 so that B becomes the center as shown in (b). Since the user U is located at the sensor at the upper left end of the foot input device 2, the object that should transition to the left side of B cannot be displayed on the display device 3. In this case, the state transition destination that is higher in the function tree is preferentially displayed on the screen.

  As a result, the state transition can be displayed so that it can be returned to the previous state, and the menu changes dynamically according to the state transition when the state transition of the menu sequence is operated and the sensor of the foot input device 2 is walked. In addition, in combination with the manual input device 4 such as a touch panel device, the state transition can be operated by operating the menu with the hand at the same time.

  The present invention has been described above by the preferred embodiments of the present invention. While the invention has been described with reference to specific embodiments, various modifications and changes may be made to the embodiments without departing from the broad spirit and scope of the invention as defined in the claims. Obviously you can. In other words, the present invention should not be construed as being limited by the details of the specific examples and the accompanying drawings.

1 is a diagram illustrating an external appearance example of an embodiment in which a user interface device of the present invention is applied to a copying machine. It is a figure which shows the example of an external appearance which applied the user interface apparatus of this invention to other apparatuses. It is a figure which shows the example of a foot input device, a display apparatus, and a manual input device. It is a figure which shows the operation example of the sensor which comprises a leg | foot input apparatus. It is a figure which shows a mode that a click is detected by a load change. It is a figure which shows a mode that a double click is detected by a load change. It is a figure which shows the other structural example of the sensor which comprises a leg | foot input apparatus. It is a figure which shows the example of the matrix structure of the sensor of a foot input device. It is FIG. (1) which shows the example of the other recognition method of a foot input device. It is FIG. (2) which shows the example of the other recognition method of a foot input device. It is a figure which shows the example of the additional function of a foot input device. It is a figure which shows the example of the functional block of the user interface apparatus of this invention. It is a figure which shows the example of a layout table. It is a figure which shows the example of a state transition table. It is a figure which shows the example of a function table. It is a flowchart which shows the example of a whole process of the user interface apparatus of this invention. It is a flowchart which shows the example of an initial state process. It is a flowchart which shows the example of a sensor identification process. It is a flowchart which shows the example of a layout display process. It is FIG. (The 1) which shows the example of correlation with the sensor of a foot input device, and a state. It is FIG. (2) which shows the example of the relationship between the sensor of a leg input device, and a state. It is a figure which shows the example of a command execution process. It is a flowchart which shows the example of a timer interruption process. FIG. 6 is a diagram (part 1) illustrating an operation example reflecting the function of the copier. FIG. 10 is a second diagram illustrating an operation example reflecting the function of the copier. It is a figure which shows the example of a display when there exist several candidates which cannot be displayed. It is a figure which shows the example of operation | movement at the time of using hand input and foot input together.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Apparatus 2 Foot input device 3 Display apparatus 4 Hand input device 5 Light emission and light-receiving device 6 Camera 7 Ultrasonic transmitter / receiver 11 First input part 12 Second input part 13 Input determination part 14 Timer part 15 Storage part 16 State transition update part 17 Display unit 18 Output unit T1 Layout table T2 State transition table T3 Function table W Sensor U User

Claims (10)

A first input means for detecting a user's standing position;
A second input means for detecting a user's hand or finger operation;
Input determining means for determining the position and input state detected by the first input means and the second input means;
A timer means for executing a temporal standby process;
Storage means for holding a layout table for storing the arrangement of each position that can be input by the first input means, a state transition table for storing state transition data, and a function table for storing commands in the transitioned state; ,
A state transition update unit that reads data from the storage unit based on the processing of the input determination unit and updates the state transition;
Display means for displaying the update result of the state transition update means on the display device;
A user interface apparatus comprising: output means for executing a command of the function table when data of the state transition table and the function table satisfy a predetermined condition in the processing of the input determination means. The user interface device according to claim 1,
The first input means has time measuring means for measuring a time interval,
A user interface device that detects an input state such as a click or a double click by a number of times exceeding a specified value within a predetermined time. The user interface device according to claim 1,
The user interface device according to claim 1, wherein the first input means comprises a matrix scale and a pressure sensor for measuring a user's position and weight shift. The user interface device according to claim 1,
The user interface device, wherein the display means moves a display object so that a selection target is displayed in the center of the display device in response to an operation by the first input means or the second input means. The user interface device according to claim 1,
The user interface device according to claim 1, wherein the display means performs highlighting such that a relationship between functions of the state transition table is increased or a color is changed as compared with an unrelated one. A first input step for detecting the user's standing position;
A second input process for detecting an operation by a user's hand or finger;
An input determination step of determining the detected position and input state of the first input step and the second input step;
A timer process for executing a time standby process;
A storage means for holding a layout table for storing the arrangement of each position that can be input in the first input step, a state transition table for storing state transition data, and a function table for storing commands in the transitioned state; ,
A state transition update step of reading data from the storage means based on the processing of the input determination step and updating the state transition;
A display process for displaying the update result of the state transition update process on a display device;
A user interface method comprising: an output step of executing a command of the function table when data of the state transition table and the function table satisfy a predetermined condition in the processing of the input determination step. The user interface method according to claim 6, wherein
The first input step has a timing step for measuring a time interval,
A user interface method, wherein an input state such as a click or a double click is detected by a number of times exceeding a specified value within a predetermined time. The user interface method according to claim 6, wherein
The user interface method according to claim 1, wherein the first input step inputs a signal from a matrix weight scale or a pressure sensor that measures the position and weight shift of the user. The user interface method according to claim 6, wherein
In the user interface method, the display step moves the display object so that the selection target is displayed in the center of the display device according to the operation in the first input step or the second input step. The user interface method according to claim 6, wherein
The user interface method according to claim 1, wherein the display step performs highlighting such that the relationship between the functions of the state transition table is made larger or the color is changed as compared with those not related.

Images