JP2012181806A - Input device, input method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an input device which allows contents displayed on a screen to be changed through an intuitive operation of a user.SOLUTION: An input device 1 comprises: a first display part 4 which consists of a first touch panel 42 for receiving a touch operation of a user and a first display panel 41 for displaying an image; a second display part 5 which is arranged adjacent to the first display part 4, and which consists of a second touch panel 52 for receiving the touch operation of the user and a second display panel 51 for displaying an image; and a control part 2 for controlling display. The control part 2 detects touch-panel input operations from the first touch panel 42 and the second touch panel 52; identifies displayed applications from the first display panel 41 and the second display panel 51; and controls, on the basis of the detected input operations and the identified applications, the display images of the applications.

Description

  The present invention relates to an input device, an input method, and a program.

  In devices such as mobile terminals, PHS, and personal computers, a touch panel is mounted on the display screen, and a fingertip touches (touches) the touch panel, or a finger moves (drags) on the touch panel, or two or more fingers. A method of performing an operation simply and intuitively by simultaneously touching (multi-touch) or dragging is known. For example, in Patent Literature 1, a touch panel is mounted on each of two display screens, and control is performed so that the contents displayed on the two display screens are switched when the user performs a predetermined button operation. Is disclosed.

JP 2003-162354 A

  However, in the configuration disclosed in Patent Document 1, a touch panel is mounted on each of the two display screens, but the touch panel is not used in order to replace the contents displayed on the two display screens. It is necessary to press a predetermined button. Since the touch panel is not used when switching the contents of the display screen, the operation is not intuitive, and some users may not know which button to press. Therefore, there is a need for a new method that allows the display content displayed on the screen to be switched by using the touch panel and the user's intuitive operation.

  The present invention has been made in view of the above problems, and provides an input device, an input method, and a program capable of switching display contents displayed on a screen by a user's intuitive operation. Objective.

In order to achieve the above object, an input device according to a first aspect of the present invention includes:
A first display unit including a first touch panel that receives a touch operation from a user and a first display panel that displays an image;
A second display unit that includes a second touch panel that receives a touch operation from the user and a second display panel that displays an image, and is disposed adjacent to the first display unit;
A detection unit for detecting an input operation to the touch panel from the first touch panel and the second touch panel;
A specifying unit for specifying a displayed application from the first display panel and the second display panel;
A control unit that controls a display image of the application based on the detected input operation and the identified application;
It is characterized by that.

The input method according to the second aspect of the present invention is:
A first display unit including a first touch panel that receives a touch operation from a user and a first display panel that displays an image; a second touch panel that receives a touch operation from the user; and a second that displays an image. A display panel, and a second display unit arranged adjacent to the first display unit, and an input method in an input device comprising:
An input operation on the touch panel is detected from the first touch panel and the second touch panel,
The displayed application is identified from the first display panel and the second display panel,
When an application displayed across the first display panel and the second display panel is specified, the detected input operation is sent to the application,
When an application displayed on each of the first display panel and the second display panel is specified, the display image position of the specified application is changed based on the detected input operation. Prompt the user to
It is characterized by that.

The program according to the third aspect of the present invention is:
Computer
A first display unit including a first touch panel that receives a touch operation from a user and a first display panel that displays an image;
A second display unit that includes a second touch panel that receives a touch operation from the user and a second display panel that displays an image, and is disposed adjacent to the first display unit;
A detection unit for detecting an input operation to the touch panel from the first touch panel and the second touch panel;
A specifying unit for specifying a displayed application from the first display panel and the second display panel;
Based on the detected input operation and the identified application, function as a control unit that controls a display image of the application,
It is characterized by that.

  According to the present invention, the display content displayed on the screen can be switched by a user's intuitive operation.

It is a figure which shows the structure of the input device which concerns on Embodiment 1 of this invention. It is a schematic external view of an input device. It is a figure which shows the coordinate example of a touchscreen. It is a flowchart for demonstrating drag operation processing. It is the figure which showed the example of touch event information. It is a figure for demonstrating the operation method which changes the display size of an application image. It is a figure for demonstrating the operation method which changes the display size of an application image. It is a figure for demonstrating the operation method which changes the display size of an application image. It is a flowchart for demonstrating a screen change process. It is the figure which showed the example of a screen in 1 application display mode. It is the figure which showed the example of a screen in 2 application display modes. It is the figure which showed the example of the application operation screen. It is a flowchart for demonstrating an image movement process. It is a figure for demonstrating the operation method which changes the display position of an application image. It is a figure for demonstrating the operation method which changes the display position of an application image. It is a figure for demonstrating the operation method which changes the display position of an application image. It is a figure for demonstrating the operation method which changes the display position of an application image. 10 is a flowchart for explaining a screen change process according to the second embodiment. 10 is a flowchart for explaining a screen change process according to the third embodiment.

(Embodiment 1)
Embodiments of the present invention will be described below. The present invention can be applied to information processing apparatuses such as various computers, PDAs (Personal Digital Assistants), portable terminals, electronic book terminals, and in-vehicle PNDs (Portable Navigation Devices). In other words, the embodiments described below are for illustrative purposes and do not limit the scope of the present invention. Therefore, those skilled in the art can employ embodiments in which each or all of these elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.

  FIG. 1 is a block diagram showing a configuration of an input device 1 according to Embodiment 1 of the present invention. FIG. 2 is a schematic external view of the input device 1. Hereinafter, description will be given with reference to these drawings.

  As illustrated in FIG. 1, the input device 1 includes a control unit 2, a storage unit 3, a first display unit 4 including a first display panel 41 and a first touch panel 42, a second display panel 51, and a first display unit 51. The second display unit 5 includes two touch panels 52, a communication unit 6, an operation unit 7, an audio output unit 8, an audio input unit 9, a timer unit 10, and a bus 11.

  The control unit 2 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The control unit 2 performs the entire operation of the input device 1 according to an operation program stored in the storage unit 3. Control. By executing the operation program stored in the storage unit 3, the control unit 2 functions as the detection unit 21, the identification unit 22, and the like.

  The detection unit 21 detects, for example, an event type (touch state) by detecting a position pointed (touched) by a user's finger or touch pen on the first touch panel 42 and the second touch panel 52. Here, the event type (touch state) includes, for example, “touch start” indicating a state in which the user touches the touch panel, “touching” in which the touch state continues, and the touch is released (terminated). There is a “touch end” indicating the state. The detection unit 21 detects touch (contact), drag (movement), drop (release), multi-touch (two or more simultaneous contacts), and the like by detecting the event type.

  The specifying unit 22 specifies (detects) an application displayed on the first display panel 41 and the second display panel 51 to specify the current display mode. Here, the application indicates a unit of software (program) such as an Internet browser, a mailer, a clock, and a calculator. The display mode indicates the state of applications displayed on the first display panel 41 and the second display panel 51, such as a 1-app display mode and a 2-app display mode. The 1-application display mode is a mode in which one application is displayed using both the first display panel 41 and the second display panel 51. The two-application display mode is a mode in which one application is displayed on each of the first display panel 41 and the second display panel 51. For example, when the application A is displayed across both the first display panel 41 and the second display panel 51, the specifying unit 22 specifies the 1-app display mode, and the first display panel 41 When application B is displayed and application C is displayed on the second display panel 51, it is specified as the two-application display mode.

  The storage unit 3 is composed of, for example, a nonvolatile semiconductor memory such as a ROM or a flash memory, or a volatile semiconductor memory such as a RAM, and is necessary for executing each process in addition to an operation program (application) executed by the control unit 2. Data and data generated by execution of each process are stored. In addition, the storage unit 3 stores a touch event information history 31 indicating an event type (touch state) detected by the detection unit 21 and a display mode flag 32 for identifying the display mode specified by the specifying unit 22.

  The first display unit 4 includes a first display panel 41 including an LCD (Liquid Crystal Display) or an organic EL panel and a driver circuit, a pressure-sensitive touch sensor panel (touch screen), and the like. 1 touch panel 42 and displays an arbitrary application or image as appropriate. The first display panel 41 and the first touch panel 42 are arranged so as to overlap each other, and the first display unit 4 receives an instruction input from the user by detecting pressure from the user's finger or touch pen.

  The first touch panel 42 includes a panel unit touched by a user and a touch panel control unit (not shown). When the first touch panel 42 detects a user touch, the touch panel control unit outputs touch event information. Here, the touch event information is information including coordinates indicating the touched position, time, event type, panel number, number of touches, and the like. Details of each will be described later. The touch detection electrodes are arranged in a matrix on the panel unit, and the first touch panel 42 can also detect multi-touch by sequentially scanning the change in capacitance.

  As shown in FIG. 3, the first touch panel 42 stores coordinates for representing the horizontal and vertical directions of the touched positions. For example, when the size of the first touch panel 42 is a half VGA (Video Graphics Array) size, the horizontal size is 320 pixels and the vertical direction is 480 pixels. When the upper left point (corner) of the first touch panel 42 is the origin (0, 0), the upper right point coordinate (319, 0), the lower left point coordinate (0, 479), the lower right point Coordinates (319, 479).

  The second display unit 5 has the same function and configuration as the first display unit 4. As shown in FIG. 2, the second display unit 5 is disposed adjacent to the first display unit 4.

  The communication unit 6 includes, for example, an antenna used for wireless access, a wireless communication circuit for communication, and the like, and communicates with the base station and other input devices 1 via the antenna according to a predetermined communication protocol. Wireless communication with Data communication is realized by the communication unit 6 performing communication via a cellular mobile communication system such as CDMA (Code Division Multiple Access). The communication unit 6 transmits and receives various data and the like via a cable or the like in accordance with a method (standard) such as USB (Universal Serial Bus).

  The operation unit 7 includes various keys and buttons for receiving an operation instruction from the user, receives an operation from the user, and inputs an operation signal corresponding to the operation from the user to the control unit 2.

  The audio output unit 8 includes a call speaker 81, a speaker 82, a DAC (Digital Analog Converter), and the like. For example, the audio output unit 8 performs DA conversion on the audio signal of the content reproduced by the control unit 2 and emits sound through the speaker 82. The call speaker 81 outputs voice and music, and outputs a received voice. The speaker 82 outputs voice and music, and outputs a notification sound at the time of incoming call.

  The voice input unit 9 includes a call microphone 91, an ADC (Analog Digital Converter), and the like. The voice input unit 9 collects a voice signal during a call and supplies the collected voice signal to the communication unit 7. The call microphone 91 receives voices uttered by the user and surrounding voices.

  The timer unit 10 is composed of, for example, an RTC (Real Time Clock) or the like, and functions as a timer, a calendar, or a clock by counting time (including year / month / day / hour / minute / second and day of the week). When the timer unit 10 generates an interrupt at a constant time interval, the control unit 2 can set a time interval for generating the interrupt.

  The bus 11 transmits data between the units.

  The input device 1 is a foldable device as shown in FIG. 2, for example, and includes a first housing 101 and a second housing 102. The first casing 101 includes the first display unit 4, the operation unit 7, a call speaker 81, a speaker 82, a call microphone 91, and the like. Further, the second housing 102 includes the second display unit 5 and the like. The first casing 101 and the second casing 102 are coupled by a hinge 103, and can be opened or folded. In the open state, the input device 1 can display different applications on the first display unit 4 (first display panel 41) and the second display unit 5 (second display panel 51). The user's operation input can be received from each touch panel (the first touch panel 42 and the second touch panel 52).

  Next, the operation of the input device 1 will be described. FIG. 4 is a flowchart for explaining a drag operation process for realizing a drag operation straddling two touch panels. Hereinafter, description will be given with reference to the drawings.

  The input device 1 prompts the user to perform an operation input, and accepts the operation input from the user, thereby starting the drag operation process.

  First, the control part 2 controls the detection part 21, detects that the 1st touch panel 42 or the 2nd touch panel 52 was touched (step S101), and acquires touch event information (EVn) (step). S102). Here, touch event information (EVn) output from the first touch panel 42 and the second touch panel 52 is stored in the storage unit 3 as the touch event information history 31.

  As shown in FIG. 5, the touch event information history 31 includes an event number (event No.) 301, a time 302, a panel number (panel No.) 303, an X coordinate 304, a Y coordinate 305, an event for the first touch event. The type 306, the panel number (panel number) 307, the X coordinate 308, the Y coordinate 309, the event type 310, the number of touches 311 and the like for the second touch event are described. Here, time 302 is the time when touch event information (EVn) is output. The touch event information (EVn) is output not only when the finger is touched on the touch panel, but also when the finger is dragging or leaving the touch panel. The panel number 303 is a number for distinguishing from which touch panel the information is output. For example, the panel number of the first touch panel 42 is 1 and the panel number of the second touch panel 52 is 2. The event types 306 and 310 indicate the operation state of the user, and are distinguished from “touch start”, “touching”, and “touch end”, for example. The X coordinates 304 and 308 and the Y coordinates 305 and 309 are positions where a touch is detected. The number of touches 311 is the number of times the touch panel is touched at the same time. When the number of touches 311 is “2”, two touch events of touch event 1 and touch event 2 are described in the touch event information history 31. That is, the same number of touch events as the value indicated by the touch count 311 are described in the touch event information history 31. The number of touches 311 describes the total number of touches output from the first touch panel 42 and the second touch panel 52. The maximum number of touches 311 increases to the sum of the maximum number of touch events that can be detected by the touch panel control unit of the first touch panel 42 and the touch panel control unit of the second touch panel 52, and is about 10, for example.

  Note that the touch event information (EVn) acquired in step S102 may be temporarily stored in the storage unit 3 until the next touch event information (EVn + 1) can be acquired. May not be described. Therefore, the input device 1 may not include the touch event information history 31.

  Next, the control unit 2 determines whether or not the event types 306 and 310 included in the touch event information (EVn) are “touch end” (step S103).

  When the event types 306 and 310 are not “touch end” (step S103; No), the control unit 2 notifies the application of touch event information (EVn) (step S112), and ends this process.

  On the other hand, when the event types 306 and 310 are “touch end” (step S103; Yes), the control unit 2 controls the timer unit 10 to start (start) the timer (step S104). It enters a state of waiting for the touch event information (EVn + 1) to be output. Then, the control unit 2 controls the timer unit 10 to determine whether or not the timer has expired before acquiring the next touch event information (EVn + 1) (step S105).

  When the timer has expired (step S105; Yes), the control unit 2 notifies the touch event information (EVn) of “touch end” to the application (step S112), and ends this process.

  On the other hand, when the timer has not expired (step S105; No), the control unit 2 controls the detection unit 21 to determine whether or not the next touch event information (EVn + 1) has been acquired (step S106). . This is because, even if a “touch end” event occurs due to a drag operation across the touch panel, the application is not immediately notified, and the next touch event information (EVn + 1) is obtained, thereby obtaining a series of drag operations. This is to determine whether or not.

  When the next touch event information (EVn + 1) has not been acquired before the timer expires (step S106; No), the control unit 2 controls the timer unit 10 again to determine whether or not the timer has expired. Determination is made (step S105).

  On the other hand, when the next touch event information (EVn + 1) is acquired before the timer expires (step S106; Yes), the control unit 2 controls the detection unit 21 to display the next touch event information (EVn + 1) panel. It is determined whether the numbers 303 and 307 are different from the panel numbers 303 and 307 of the touch event information (EVn) (step S107).

  When the panel number of the next touch event information (EVn + 1) and the panel number of the touch event information (EVn) are not different, that is, when they are the same (step S107; No), the control unit 2 determines the touch event information (EVn). Is directly notified to the application (step S112), and this process is terminated.

On the other hand, when the panel number of the next touch event information (EVn + 1) is different from the panel number of the touch event information (EVn) (step S107; Yes), the control unit 2 controls the detection unit 21 to perform the next touch. A distance d between the coordinates of the event information (EVn + 1) and the coordinates of the touch event information (EVn) is calculated (step S108). Here, the method for calculating the distance d is as follows. When the respective coordinates are (X1, Y1) and (X2, Y2), the distance d = {(X2-X1) ² + (Y2-Y1) ² } ⁽¹ A formula for obtaining the distance between two points such as ^{/ 2)} can be used. However, as shown in FIG. 3, since the origin (0, 0) of the first touch panel 42 and the second touch panel 52 is different, the points on the first touch panel 42 and the second touch panel 52 are different. Even if the point is physically close to the position (distance), the coordinates are greatly different. Therefore, the X component (X direction) of the first touch panel 42 arranged on the left side X-max (here) is added to the X component of the coordinates touched on the second touch panel 52 arranged on the right side when viewed from the front. Then add 319). That is, the new X coordinate X1 ′ = X1 + X_max = X1 + 319 of the touch event information (EVn) is set. Thus, by converting the coordinates, the control unit 2 calculates the distance d between the coordinates (X1 + 319, Y1) and the coordinates (X2, Y2) by the above formula. When performing a drag operation straddling two touch panels, the user's finger moves away from one touch panel and moves to another touch panel, so that it is determined that the drag operation has ended. However, by determining the panel number and converting the coordinates, even if the drag operation straddles two touch panels, it can be processed as a continuous series of drag operations.

  Next, the control unit 2 determines whether or not the calculated distance d is smaller than the threshold value D1 (step S109). Here, the threshold value D1 is a value that varies depending on the size of the touch panel, the main body device, a hinge that couples the display unit, and the like, and an arbitrary value can be set.

  When the distance d is smaller than the threshold value D1 (step S109; Yes), the control unit 2 converts the event types 306 and 310 of the touch event information (EVn) and the touch event information (EVn + 1) to “touching” (step) S110), each touch event information is notified to the application (step S111). Then, this process ends.

  Here, an operation method for changing the display size of an application image by performing a multi-touch operation when one application is displayed across the first display unit 4 and the second display unit 5 will be described. To do. When a multi-touch operation as illustrated in FIGS. 6A to 6C is performed, the touch event information history 31 includes a value as illustrated in FIG. Here, event number (event No.) 1 is an event output when touched, and event number 7 is an event output when a finger is released from the touch panel. When dragging across two touch panels, first, the finger f1 is released from the first touch panel 42 at a point near the right end of the first touch panel 42 (event number 4), and the left end of the second touch panel 52 is immediately A finger f1 touches a nearby point (event number 5). At this time, since the event type of event number 4 is “touch end”, the drag operation ends in the conventional input device. The input device 1 according to the present invention does not notify the application of “touch end” when the touch event information of event number 4 is output, and waits for the touch event information of the next event number 5 before notifying. At that time, the touch event information of event number 4 and the touch event information of event number 5 are output from different touch panels, the coordinates of the two touch event information are close, and the difference in time when the event occurred Is small, it is determined that this is a continuous series of drag operations. Then, both the touch event information of the event number 4 and the event type of the touch event information of the event number 5 in the touch event 1 are changed to “touching” and notified to the application. As a result, the drag operation can be continued without ending.

  On the other hand, when the distance d is not smaller than the threshold value D1, that is, when it is larger (step S109; No), the control unit 2 determines the event types 306 and 310 of the touch event information (EVn) and the touch event information (EVn + 1). The touch event information is notified to the application without changing (Step S111). Then, this process ends.

  Note that it is possible to accurately determine whether the drag operation is finished or continued by using the coordinate distance d between the touch event information (EVn) and the touch event information (EVn + 1), but this is not always necessary. Regardless of the distance between the coordinates of the two touch event information, if the touch event occurs within a predetermined time, it may be determined that the drag operation is continued. In that case, the process of step S108 and the process of step S109 may be omitted.

  With the above processing, the drag operation across the two touch panels can be performed, so that the convenience of the operation can be improved.

  Next, screen change processing for changing the screen on which an application is displayed by multi-touch operation will be described with reference to FIG.

  The input device 1 prompts the user to perform an operation input, and accepts the operation input from the user, thereby starting the screen change process.

  First, when the control unit 2 controls the detection unit 21 to detect a touch, the control unit 2 acquires touch event information (EVn) (step S201).

  Next, the control unit 2 determines whether or not the acquired touch event information (EVn) is multi-touch (step S202). Specifically, the control unit 2 determines the number of touches 311 in the touch event information history 31 shown in FIG. If the number of touches 311 is 2 or more, multi-touch is performed. If the number of touches is 1, the control unit 2 also acquires touch event information of the other touch panel. If the touch event information (EVn) of the event type “touching” or “touch start” can be acquired, the control unit 2 determines that multi-touch has occurred.

  When it is not multi-touch (step S202; No), the control part 2 notifies the acquired touch event information (EVn) to the application (step S212), and ends this process.

  On the other hand, in the case of multi-touch (step S202; Yes), the control unit 2 detects the touch panel number from the touch event information (EVn) (step S203). It is detected whether or not the touch extends over two screens (step S204). Specifically, the control unit 2 determines whether the panel numbers 303 and 307 of the touch event information history 31 are different.

  When the multi-touch does not extend over two screens (step S204; No), the control unit 2 notifies the acquired touch event information (EVn) to the application (step S212), and ends this process.

  On the other hand, when multi-touch extends over two screens (step S204; Yes), the control unit 2 controls the specifying unit 22 to acquire a display mode (step S205), and the display mode is “2 apps”. It is determined whether or not the “display mode” is set (step S206). FIG. 8A is a screen example of “1 app display mode”, and FIG. 8B is a screen example of “2 app display mode”. As shown in FIG. 8A, when one application is displayed across two screens, the control unit 2 determines that the current display mode is “one-app display mode”, and as shown in FIG. 8B. In addition, when one application is displayed on each screen, it is determined that the current display mode is the “2-app display mode”. Then, the control unit 2 stores the current display mode in the storage unit 3 as the display mode flag 32.

  When the display mode is not “two-app display mode” (step S206; No), the control unit 2 notifies the acquired touch event information (EVn) to the application (step S212), and ends this process.

  On the other hand, when the display mode is “2-app display mode” (step S206; Yes), the control unit 2 acquires the event types 306 and 310 (step S207), and the event types 306 and 310 indicate “touch start”. Is determined (step S208).

  If the event type is not “touch start” (step S208; other than “touch start”), the control unit 2 performs an error process (step S213) because the process cannot be performed originally, and ends this process. Here, the error process is an arbitrary process such as a process for generating a warning sound or a warning display for the user, or a process for returning to a predetermined step.

  On the other hand, when the event type is “touch start” (step S208; “touch start”), the control unit 2 controls the first display unit 4 and the second display unit 5 to display an image indicating an application. A transition is made to an “application operation screen” prompting the user to perform an operation (step S209). Here, the “application operation screen” is a screen for allowing the user to recognize that the arrangement of the application image (screen) can be changed, and is, for example, a screen as shown in FIG. For example, the control unit 2 changes the value of the variable on the program indicating the system state to a value (for example, 1) indicating the “application operation screen”.

  Next, the control unit 2 holds the panel numbers 303 and 307 and the touch positions 304, 305, 308, and 309 (X coordinate, Y coordinate) of the touch event information as the start point of the “application operation screen” (step SS210). . Then, as illustrated in FIG. 9, the control unit 2 draws an “application operation screen” in which the screen on which the application is displayed is displayed slightly smaller than the normal state (step S211), and the current state is “ The user recognizes that it is an “application operation screen”, and the process is terminated.

  Through the above processing, it is possible to shift to the “application operation screen” for changing the arrangement position of the application image using the multi-touch operation. Further, by changing the screen, the user can easily recognize that the current state is a state in which the arrangement position of the application image can be changed.

  Next, image movement processing for moving an application image when a drag operation across two touch panels is detected while the “application operation screen” is displayed will be described with reference to FIG.

  The input device 1 prompts the user to perform an operation input, and accepts the operation input from the user, thereby starting the image moving process. Here, an operation will be described in which only one multitouched finger is dragged and moved, and the other finger is not moved.

  First, when the control unit 2 controls the detection unit 21 to detect a touch, the control unit 2 acquires touch event information (EVn) (step S301).

  Next, the control unit 2 determines whether or not the acquired touch event information (EVn) is multi-touch (step S302). A method for determining whether or not multi-touch is performed is the same as the process in step S202.

  If it is not multi-touch (step S302; No), the control unit 2 performs an error process (step S307) and ends this process because it is a process that is not possible.

  On the other hand, when it is multi-touch (step S302; Yes), the control part 2 detects a touch position (X coordinate 304,308 and Y coordinate 305,309) from touch event information (EVn) (step S303). ).

  Next, the control unit 2 acquires the event types 306 and 310 from the touch event information (EVn), and determines which of the event types 306 and 310 is (step S304).

  When the event type is “touching” (step S304; “touching”), the control unit 2 controls the first display unit 4 and the second display unit 5 to draw an “application operation screen”. (Step S305), and this process is terminated. Specifically, the control unit 2 calculates the difference between the start point of the “application operation screen” held in step S210 and the touch position detected in step S303, and the application is the same as the difference. Draw images with different display positions. For example, as illustrated in FIG. 11C described later, the control unit 2 displays the application image 401 while shifting the display position.

  If the event type is “touch start” (step S304; “touch start”), the control unit 2 performs an error process (step S307) because the process cannot be performed originally, and the process ends.

  When the event type is “touch end” (step S304; “touch end”), the control unit 2 finishes drawing the “application operation screen” and transitions the screen to the normal display state (step S306). .

  After the screen transitions to the normal display state, the control unit 2 acquires the start point of the “application operation screen” (step S308), and acquires the panel numbers 303 and 307 of the start point (step S309). The control unit 2 also acquires the panel numbers 303 and 307 of the touch event information acquired in step S301 (step S309).

  Then, the control unit 2 compares the panel numbers 303 and 307 of the start point of the “application operation screen” with the panel numbers 303 and 307 of the touch event information acquired in step S301, so that the panel numbers 303 and 307 are compared. Are different from each other (step S310). In this step, the control unit 2 determines whether or not the user has dragged the application image to move the application image across the two display units.

  When the panel numbers are different (Step S310; Yes), the control unit 2 controls the first display unit 4 and the second display unit 5 to change the display position of the screen on which the application image is displayed (Step S310). S311), the process is terminated.

  On the other hand, when the panel numbers are the same, that is, when they are not different (step S310; No), the control unit 2 controls the first display unit 4 and the second display unit 5 to display the application image. The display position of the displayed screen is returned to the original (step S312), and this process is terminated.

  Here, in the case of the “2-app display mode”, a method of changing the display position of the application image will be described with reference to FIGS. As shown in FIG. 11A, a search application image 401 is displayed on the first display unit 4, and a clock application image 501 is displayed on the second display unit 5. In this state, when multi-touch is started with the fingers f1 and f2, as shown in FIG. 11B, an “application operation screen” in which the sizes of the two application images 401 and 501 are slightly reduced is displayed. For example, a plain background image is displayed as the background of the application images 401 and 501. In this state, when the finger f1 is moved and dragged on the touch panel, the search application image 401 is moved and displayed according to the position of the finger f1. Then, when the finger f1 moves to the second display unit 5, the search application image 401 is divided into the first display unit 4 and the second display unit 5 and displayed as shown in FIG. 11C. Also, a television application image 402 is displayed at the place (position) where the search application image 401 was displayed. This is displayed in this way when the television application has been started up before the search application is started. If the finger f1 is then released from the touch panel (second display unit 5) (steps S304 to S310), the search application image 401 is displayed on the second display unit 5 as shown in FIG. 11D (step S311). The TV application image 402 is displayed on the first display unit 4. Here, as shown in FIG. 11B, when the finger f1 is released from the touch panel (first display unit 4) without straddling the touch panel, the search application image 401 is displayed on the second display unit 5. First, as shown in FIG. 11A, the state displayed on the first display unit 4 continues (step S312).

  With the above processing, the display position of the application image can be changed only by the multi-touch operation, so that the operability is improved. Moreover, the display content displayed on the screen can be switched by a user's intuitive operation.

  In this flowchart, the process using only the touch event information when only one finger is not moved has been described, but the process using the touch event information when two fingers are moved is similarly illustrated in the flowchart of FIG. Will be described below.

  First, the control unit 2 acquires touch event information (step S301), and determines whether or not it is multi-touch (step S302). Here, the control unit 2 acquires two points of touch event information (step S301). Then, it is determined as multi-touch (step S302; Yes). Then, the control unit 2 acquires the touch positions of the two touch events (Step S303).

  Next, when the control unit 2 determines the event types 306 and 310 from the touch event information (step S304), when both of the event types 306 and 310 of the two touch event information are “touching” ( Step S304: “During touch”), “App operation screen” is drawn (Step S305). If either one of the event types 306 and 310 is “touch start” (step S304; “touch start”), the control unit 2 performs an error process (step S307) and ends this process. To do. When one of the event types 306 and 310 is “touch end” (step S304; “touch end”), the control unit 2 shifts the screen to a normal display state (step S306).

  Thereafter, the control unit 2 acquires two start points of the “application operation screen” (step S308), and acquires the respective panel numbers 303 and 307 (step S309). The control unit 2 also acquires two panel numbers 303 and 307 of the touch event information acquired in step S301. Then, the control unit 2 compares the panel numbers 303 and 307 at the two start points of the “application operation screen” with the panel numbers 303 and 307 of the two touch event information, thereby obtaining a panel. It is determined whether the numbers 303 and 307 are different (step S310).

  If the touch panel numbers are different (step S310; Yes), the display position of the application image is changed (step S311). Otherwise (step S310; No), the display position of the application image is restored (step S312). ), This process is terminated.

  When two pieces of touch event information are acquired in step S301, it is necessary to sequentially determine which of the “app operation screens” each drag event is dragged from. is there. For example, first, it is assumed that the start points of the “application operation screen” are the points A0 and B0, and the touch positions of the next acquired touch event information are the points A1 and B1, respectively. Here, the control unit 2 compares the distance D1 between the point A0 and the point A1 with the distance D2 between the point B0 and the point A1. If the distance D1 is shorter than the distance D2, the control unit 2 determines that the point A1 is a point dragged from the point A0, and determines that the point B1 is a point dragged from the point B0. Next, when the points A2 and B2 are acquired as the touch positions of the touch event information, the control unit 2 obtains the coordinates of the points A1 and B1 acquired previously, the coordinates of the points A2 and B2, Are compared to determine from which point the operation is performed. By doing in this way, it can be judged from which point the newly acquired plurality of pieces of touch event information are drag operations, and the plurality of drag operations can be controlled.

(Embodiment 2)
In the input device 1 according to the first embodiment, if a user accidentally touches another screen while operating one screen, there is a possibility that the application image may be unintentionally moved. It was. Thus, in the input device 1 according to the second embodiment, a case will be described in which a transition to the “application operation screen” is performed only when the user intentionally operates in order to further improve operability. The screen change process according to the present embodiment will be described below with reference to FIG. Note that the description of the same configuration and operation as those of the input device 1 according to Embodiment 1 will be omitted as appropriate.

  The processing from step S201 to S213 shown in FIG. 12 is the same as the processing from step S201 to S213 shown in FIG. In this process, after the process of step S208, the time difference of touch is determined.

  In step S208, when the event types 306 and 310 are “touch start” (step S208; “touch start”), the control unit 2 acquires the time 302 from the two touch event information (EVn), respectively, The difference between the times 302 is calculated, and it is determined whether or not the difference is smaller than a predetermined time difference dT (step S401). When the user performs multi-touch, the control unit 2 acquires two touch event information (EVn) in the process of step S201. Specifically, the control unit 2 determines the time T1 when the “touch start” of the first touch event and the “touch” of the second touch event as the difference between the times 302 of the two touch event information (EVn). The difference from the time “T2” when “start” is calculated. Then, the control unit 2 determines whether or not | T2−T1 | <dT holds.

  The predetermined time difference dT is an allowable range (allowable time) of a difference in touch time that occurs when the user tries to touch two points at the same time, and is typically about 0.5 seconds. In the case of multi-touch of 3 points or more, the time of “touch start” of all touch events can be acquired, and the time difference can be obtained by selecting two points having the largest time difference.

  If the touch time difference is equal to or greater than dT (step S401; No), it is determined that the user's operation is not intended for transition to the “app operation screen”, and the control unit 2 does not perform the state transition and acquires it. The touch event information (EVn) is notified to the application (step S212), and this process ends.

  On the other hand, when the touch time difference is less than dT (step S401; Yes), the control unit 2 determines that the user's operation is intended to transition to the “app operation screen”, and the control unit 2 performs the first display unit. 4 and the second display unit 5 are controlled to transit to the “application operation screen” (step S209).

  With the above processing, it is possible to prevent the screen from being changed due to a user's erroneous operation or the like, so that the convenience of operation can be improved.

(Embodiment 3)
In the first embodiment, the method for changing the size of the application image in the “1 app display mode” and the method for changing the display position of the application image in the “2 app display mode” have been described. In the third embodiment, a method of switching between changing the size of the application image and changing the display position of the application image according to the position (distance) of the finger that the user performs multi-touch will be described. The screen change process according to the present embodiment will be described below with reference to FIG. Note that the description of the same configuration and operation as those of the input device 1 according to Embodiment 1 will be omitted as appropriate.

  The processes in steps S201 to S203 and steps S205 to S213 shown in FIG. 13 are the same as the processes in steps S201 to S203 and steps S205 to S213 shown in FIG. In this process, after it is determined in step S204 that multi-touch does not extend over two screens (step S204; No), the control unit 2 controls the detection unit 21 so that the user's drag operation is performed. It is determined whether or not the drag operation straddles the screen (display unit) (step S501). Specifically, the control unit 2 performs the process of step S109 shown in FIG. 4, and when “Yes” is determined, the control unit 2 determines that the user's drag operation is a drag operation across the screen. If the user performs a drag operation across the screen, the user's two fingers are touching the screen, and therefore, “No” is determined in the process of step S204.

  When the user operation is not an operation across the screen (Step S501; No), the control unit 2 notifies the application of the acquired touch event information (EVn) without performing the state transition of the display mode (Step S212). The process ends.

  On the other hand, when the user operation is an operation across the screen (step S501; Yes), the control unit 2 determines whether or not the display mode is the “2-app display mode” (step S502).

  When the display mode is the “2-app display mode” (step S502; Yes), the control unit 2 draws the “app operation screen” (step S211), and the current state is the “app operation screen”. Is recognized by the user, and the process is terminated.

  On the other hand, when the display mode is not “two-app display mode” (step S502; No), the control unit 2 controls the first display unit 4 and the second display unit 5 to change the display mode to “two-app display”. The mode is changed to “mode” (step S503), and a transition is made to the “application operation screen” (step S209).

  Here, a method of changing the display position of the application image after changing the size of the application image will be described with reference to FIGS. 6A, 6B, 11C, and 11D. As shown in FIG. 6A, in a state where the application image extends over the first display unit 4 and the second display unit 5, that is, in the case of the one application display mode (step S206; No), the finger f1 is moved to the right. When it is slid in the direction of the second display unit 5, the search application image is displayed so as to become gradually smaller as shown in FIG. 6B (step S212). When the finger f1 is slid rightward and the finger f1 comes from the first display unit 4 to the second display unit 5 across the screen as shown in FIG. 11C (step S501; Yes). When the display mode is “1 app display mode” (step S502; No), the screen shifts to the “app operation screen” (step S209). Then, when the finger f1 is released from the touch panel (second display unit 5), the search application image 401 can be displayed on the second display unit 5 as shown in FIG. 11D, and the display position of the application image is changed. can do.

  With the above processing, the size change of the application image and the display position change of the application image can be performed by one operation, so that the operability can be improved.

  In addition, this invention is not limited to said embodiment, A various deformation | transformation and application are possible.

  The communication method (method) performed by the communication unit 6 is arbitrary. For example, any communication system such as IEEE (the Institute of Electrical and Electronic Engineers) 802.11b, IEEE802.11g, IEEE802.11n standard wireless LAN (Local Area Network), infrared communication, RFID, Bluetooth (registered trademark), etc. Thus, communication can be performed.

  Although the case where there are two display units, the first display unit 4 and the second display unit 5, has been described, the number of display units is arbitrary. For example, even when the number of display units is one, when a predetermined boundary line is provided in one display unit and the user's drag operation is a drag operation that crosses the boundary line, the user's drag The operation can also be a drag operation that straddles two display portions.

  Although the number of pixels of the display panel is the same as the number of pixels of the touch panel, they may be different. In that case, since the coordinate system of the touch event information is different from the coordinate system of the display panel, it may be converted. For example, when the number of pixels on the touch panel is ten times the number of pixels on the display panel, the coordinates of the touch event information may be corrected to 1/10 to match the coordinate system of the display panel. Then, by notifying the application of the corrected coordinates, the application can display various data at a position that matches the coordinate system of the display panel. By performing such control, processing can be performed with any number of touch panels.

  In addition, the hardware configuration and the flowchart described above are merely examples, and can be arbitrarily changed and modified.

  The central part for performing the processing of the input device 1 can be realized by using a normal computer system without depending on a dedicated system. For example, a computer program for executing the above operation is stored and distributed in a computer-readable recording medium (flexible disk, CD-ROM, DVD-ROM, etc.), and the computer program is installed in the computer. Thus, the input device 1 that executes the above-described processing may be configured. Alternatively, the input device 1 may be configured by storing the computer program in a storage device included in a server device on a communication network such as the Internet and downloading it by a normal computer system.

  Further, when the functions of the input device 1 are realized by sharing an OS (operating system) and an application program, or by cooperation between the OS and the application program, only the application program portion is stored in a recording medium or a storage device. May be.

  It is also possible to superimpose a computer program on a carrier wave and distribute it via a communication network. For example, the computer program may be posted on a bulletin board (BBS) on a communication network, and the computer program may be distributed via the network. The computer program may be started and executed in the same manner as other application programs under the control of the OS, so that the above-described processing may be executed.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
A first display unit including a first touch panel that receives a touch operation from a user and a first display panel that displays an image;
A second display unit that includes a second touch panel that receives a touch operation from the user and a second display panel that displays an image, and is disposed adjacent to the first display unit;
A detection unit for detecting an input operation to the touch panel from the first touch panel and the second touch panel;
A specifying unit for specifying a displayed application from the first display panel and the second display panel;
A control unit that controls a display image of the application based on the detected input operation and the identified application;
An input device characterized by that.

(Appendix 2)
When an input operation is detected from the first touch panel and the second touch panel,
The controller is
When an application displayed across the first display panel and the second display panel is specified, the detected input operation is sent to the application,
When an application displayed on each of the first display panel and the second display panel is specified, the image position of the specified application is changed based on the detected input operation. Prompt the user to
The input device according to Supplementary Note 1, wherein:

(Appendix 3)
When the control unit detects the touch operation across the first touch panel and the second touch panel when prompting the user to change the image position of the application, the control unit detects the specified application. Control to change the image position,
The input device according to Supplementary Note 2, wherein

(Appendix 4)
The control unit is configured to perform a touch operation across the first touch panel and the second touch panel when an application displayed across the first display panel and the second display panel is specified. Is detected, control to change the image size of the identified application,
The input device according to Supplementary Note 2, wherein

(Appendix 5)
When the detection unit detects an input operation from the second touch panel within a predetermined time after detecting the input operation from the first touch panel, the detection unit detects that the touch operation from the user is simultaneous.
The input device according to any one of supplementary notes 1 to 4, wherein

(Appendix 6)
A first display unit including a first touch panel that receives a touch operation from a user and a first display panel that displays an image; a second touch panel that receives a touch operation from the user; and a second that displays an image. A display panel, and a second display unit arranged adjacent to the first display unit, and an input method in an input device comprising:
An input operation on the touch panel is detected from the first touch panel and the second touch panel,
The displayed application is identified from the first display panel and the second display panel,
When an application displayed across the first display panel and the second display panel is specified, the detected input operation is sent to the application,
When an application displayed on each of the first display panel and the second display panel is specified, the display image position of the specified application is changed based on the detected input operation. Prompt the user to
An input method characterized by that.

(Appendix 7)
Computer
A first display unit including a first touch panel that receives a touch operation from a user and a first display panel that displays an image;
A second display unit that includes a second touch panel that receives a touch operation from the user and a second display panel that displays an image, and is disposed adjacent to the first display unit;
A detection unit for detecting an input operation to the touch panel from the first touch panel and the second touch panel;
A specifying unit for specifying a displayed application from the first display panel and the second display panel;
Based on the detected input operation and the identified application, function as a control unit that controls a display image of the application,
A program characterized by that.

DESCRIPTION OF SYMBOLS 1 Input device 2 Control part 21 Detection part 22 Identification part 3 Memory | storage part 4 1st display part 41 1st display panel 42 1st touch panel 5 2nd display part 51 2nd display panel 52 2nd touch panel 6 Communication unit 7 Operation unit 8 Audio output unit 81 Speaker for call 82 Speaker 9 Audio input unit 91 Microphone for call 10 Timer unit 11 Bus 101 First housing 102 Second housing 103 Hinge

Claims (7)

A first display unit including a first touch panel that receives a touch operation from a user and a first display panel that displays an image;
A second display unit that includes a second touch panel that receives a touch operation from the user and a second display panel that displays an image, and is disposed adjacent to the first display unit;
A detection unit for detecting an input operation to the touch panel from the first touch panel and the second touch panel;
A specifying unit for specifying a displayed application from the first display panel and the second display panel;
A control unit that controls a display image of the application based on the detected input operation and the identified application;
An input device characterized by that. When an input operation is detected from the first touch panel and the second touch panel,
The controller is
When an application displayed across the first display panel and the second display panel is specified, the detected input operation is sent to the application,
When an application displayed on each of the first display panel and the second display panel is specified, the image position of the specified application is changed based on the detected input operation. Prompt the user to
The input device according to claim 1. When the control unit detects the touch operation across the first touch panel and the second touch panel when prompting the user to change the image position of the application, the control unit detects the specified application. Control to change the image position,
The input device according to claim 2. The control unit is configured to perform a touch operation across the first touch panel and the second touch panel when an application displayed across the first display panel and the second display panel is specified. Is detected, control to change the image size of the identified application,
The input device according to claim 2. When the detection unit detects an input operation from the second touch panel within a predetermined time after detecting the input operation from the first touch panel, the detection unit detects that the touch operation from the user is simultaneous.
The input device according to claim 1, wherein the input device is an input device. A first display unit including a first touch panel that receives a touch operation from a user and a first display panel that displays an image; a second touch panel that receives a touch operation from the user; and a second that displays an image. A display panel, and a second display unit arranged adjacent to the first display unit, and an input method in an input device comprising:
An input operation on the touch panel is detected from the first touch panel and the second touch panel,
The displayed application is identified from the first display panel and the second display panel,
When an application displayed across the first display panel and the second display panel is specified, the detected input operation is sent to the application,
When an application displayed on each of the first display panel and the second display panel is specified, the display image position of the specified application is changed based on the detected input operation. Prompt the user to
An input method characterized by that. Computer
A first display unit including a first touch panel that receives a touch operation from a user and a first display panel that displays an image;
A second display unit that includes a second touch panel that receives a touch operation from the user and a second display panel that displays an image, and is disposed adjacent to the first display unit;
A detection unit for detecting an input operation to the touch panel from the first touch panel and the second touch panel;
A specifying unit for specifying a displayed application from the first display panel and the second display panel;
Based on the detected input operation and the identified application, function as a control unit that controls a display image of the application,
A program characterized by that.

Images