US20130093719A1 - Information processing apparatus - Google Patents
Information processing apparatus Download PDFInfo
- Publication number
- US20130093719A1 US20130093719A1 US13/588,300 US201213588300A US2013093719A1 US 20130093719 A1 US20130093719 A1 US 20130093719A1 US 201213588300 A US201213588300 A US 201213588300A US 2013093719 A1 US2013093719 A1 US 2013093719A1
- Authority
- US
- United States
- Prior art keywords
- contactless
- controller
- application
- touch panel
- operations
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0416—Control or interface arrangements specially adapted for digitisers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/044—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
- G06F2203/04108—Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
Definitions
- the present disclosure relates to an information processing apparatus applied to an electronic device provided with an operable unit that detects an operation mode on the basis of changes in the capacitance of an operable surface, such as a touch panel, for example.
- the present disclosure relates to an information processing apparatus applied to an electronic device such as a mobile phone, a PHS phone (PHS: Personal Handyphone System), a PDA (PDA: Personal Digital Assistant), a digital camera, a digital video camera, a portable game console, or a notebook computer, for example.
- PHS Personal Handyphone System
- PDA Personal Digital Assistant
- a digital camera a digital video camera
- portable game console a portable game console
- notebook computer for example.
- FIG. 10 illustrates a function block diagram of such a controller realized as a result of a controller acting on the basis of the input processing program.
- the controller functions as a two-dimensional coordinate converter 101 which detects a direct touch operation position on a touch panel 100 and outputs two-dimensional coordinate information (X coordinate information and Y coordinate information) corresponding to the direct touch operation position.
- the controller functions as an application execution controller 103 that controls execution of information processing which, from among the various information processing of a currently activated application program, corresponds to a direct touch operation state, etc. detected when the controller functions as the operation detector 104 of the window manager 102 .
- FIG. 11 illustrates a flowchart of a controller's input processing action based on the input processing program.
- the touch panel 100 is configured to identify an operation position by detecting changes in the capacitance between a user's finger and the conductive film 151 of the two-dimensional coordinate converter 101 , as illustrated in FIG. 12A .
- a change in capacitance values occurs centered about the direct touch operation position on the conductive film 151 , as illustrated in FIG. 12B .
- the capacitance values change such that the approximate center of the direct touch operation position on the conductive film 151 becomes the peak value of the capacitance values, with the capacitance values gradually becoming smaller away from the center of the direct touch operation position.
- step S 101 the controller, by functioning as the two-dimensional coordinate converter 101 , compares the capacitance values of the individual sensors to a direct touch threshold level as illustrated in FIG. 12B , and determines whether or not there exist sensors among all sensors on the touch panel 100 having capacitance values equal to or greater than the direct touch threshold level.
- the controller advances the process to step S 104 and determines whether or not to continue detecting the capacitance values of the respective sensors. Then, in the case where it is determined not to continue detecting the capacitance values of the respective sensors, the controller ends the process of the flowchart in FIG. 11 . Meanwhile, in the case where it is determined to continue detecting the capacitance values of the respective sensors, the controller returns the process to step S 100 at the timing for conducting the next detection of the capacitance values of the respective sensors and acquires the capacitances value of the respective sensors again.
- step S 101 determines that there do exist sensors having capacitance values equal to or greater than the direct touch threshold level
- the controller advances the process to step S 102 , identifies the sensors having capacitance values equal to or greater than the direct touch threshold level, and advances the process to step S 103 .
- step S 103 the controller creates two-dimensional coordinate information (i.e., respective X-axis and Y-axis coordinate information) corresponding to the placement positions on the touch panel 100 of the identified sensors having capacitance values equal to or greater than the direct touch threshold level, controls storage of the two-dimensional coordinate information to a register, and advances the process to step S 104 .
- two-dimensional coordinate information i.e., respective X-axis and Y-axis coordinate information
- the controller is configured to control storage of two-dimensional coordinate information thus created to a register every time the capacitance values of respective sensors are acquired from the touch panel 100 and the two-dimensional coordinate information is created. For this reason, respective sets of two-dimensional coordinate information in accordance with direct touch operations on the touch panel 100 by the user are successively stored in the register. Consequently, direct touch operation history information (i.e., the respective sets of two-dimensional coordinate information) corresponding to direct touch operations on the touch panel 100 by the user is stored in the register.
- the controller When functioning as the operation detector 104 of the window manager 102 , the controller detects the user's direct touch operation mode, like that illustrated in FIG. 12C for example, on the basis of the direct touch history given by the respective sets of two-dimensional coordinate information stored in the register.
- the controller controls execution of information processing which, from among the various information processing of a currently activated application program, corresponds to the detected direct touch operation mode and the two-dimensional coordinate information used when detecting the direct touch operation mode.
- the controller By determining to continue detecting capacitance values in step S 104 and returning the process to step S 100 , the controller acquires the capacitance values of the respective sensors in the touch panel 100 at given time intervals, and conducts processing for creating the two-dimensional coordinate information, detecting the direct touch operation mode, and executing an application program based on the detected operation mode, etc., on the basis of the acquired capacitance values.
- the present inventor has recognized the need for an information processing apparatus in which information processing in a given application program is operable not only by direct touch operations, but also by contactless operations.
- the disclosure is directed to an information processing apparatus comprising: a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel; and a controller that determines whether an application activated at the information processing apparatus supports contactless operations; controls the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and controls the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.
- the disclosure is directed to an information processing method performed by an information processing apparatus including a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel, the method comprising: determining whether an application activated at the information processing apparatus supports contactless operations; controlling the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and controlling the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.
- the disclosure is directed to a non-transitory computer-readable medium including computer program instructions, which when executed by an information processing apparatus including a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel, cause the information processing apparatus to perform a method, the method comprising: determining whether an application activated at the information processing apparatus supports contactless operations; controlling the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and controlling the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.
- new operation modes can be provided in which information processing in a given application program is operable by contactless operations.
- FIG. 1 is a block diagram of a mobile phone according to an embodiment implementing the present disclosure.
- FIG. 2 is a function block diagram during input processing by a controller in a mobile phone according to an embodiment.
- FIG. 3 is a flowchart illustrating input processing action in a mobile phone according to an embodiment.
- FIGS. 4A to 4C are diagrams for explaining a direct touch threshold level and a contactless threshold level provided in a mobile phone according to an embodiment.
- FIG. 5 is a diagram illustrating exemplary capacitance values of respective sensors during a direct touch operation in a mobile phone according to an embodiment.
- FIG. 6 is a diagram illustrating exemplary capacitance values of respective sensors during an contactless operation in a mobile phone according to an embodiment.
- FIG. 7 is a flowchart for explaining action for creating two-dimensional coordinate information for a contactless operation in a mobile phone according to an embodiment.
- FIGS. 8A to 8C are diagrams illustrating an exemplary transition pattern across capacitance maps during a contactless operation in a mobile phone according to an embodiment.
- FIG. 9 is a timing chart for explaining information processing of a telephony application conducted by a direct touch operation and a contactless operation in a mobile phone according to an embodiment.
- FIG. 10 is a function block diagram during input processing by a controller in a conventional mobile phone.
- the present disclosure can be applied to a mobile phone as one example.
- the mobile phone also includes a light emitter 7 (LED: Light Emitting Diode) for notifying the user of incoming/outgoing signals, etc. with light, a first camera unit 8 a for shooting a still image or video of a desired subject, and a second camera unit 8 b for shooting the user, etc. of the mobile phone during video telephony, for example.
- LED Light Emitting Diode
- the mobile phone also includes an operation history register 16 that stores a capacitance map which indicates the capacitance values of all sensors in the display unit 5 forming the touch panel.
- the operation history register 16 includes a storage area for direct touch operations and a storage area for contactless operations. It is configured such that direct touch operation positions and direct touch operation-induced capacitance values corresponding to direct touch operations by the user are successively stored in the storage area for direct touch operations in the operation history register 16 . It is also configured such that capacitance values corresponding to contactless operations by the user are successively stored in the storage area for contactless operations in the operation history register 16 .
- an application program that allows contactless operations includes a contactless operation mode file in which contactless operation mode data corresponding to various processes in the application program is stored as a file.
- the contactless operation mode data is data expressing transition patterns on the capacitance map for respective contactless operation modes corresponding to respective processes in the application program. Furthermore, in the contactless operation mode file, contactless operation mode data for the respective contactless operation modes is formed as a file.
- an application program that allows contactless operations includes a contactless operation mode file, in which the capacitance map transition patterns for each of the contactless operations corresponding to respective processes in that application program are stored in a file as the contactless operation mode data.
- the controller 15 reads out a contactless operation mode file attached to the application program at given timings, such as when activating the application program, for example, and loads it into the contactless operation mode file memory 17 .
- the controller 15 compares a capacitance map transition pattern detected during a contactless operation to the capacitance map transition patterns expressed by the respective contactless operation mode data in the contactless operation mode file for that application program which was loaded into the contactless operation mode file memory 17 , and detects the contactless operation mode. It is configured such that the controller 15 then controls execution of a process corresponding to the detected contactless operation mode on the basis of the activated application program.
- the mobile phone in this embodiment includes threshold level memory 18 in which the direct touch threshold level and the contactless threshold level are respectively stored.
- the controller 15 is provided with a first control mode that controls execution of processes in an application program according to direct touch operations, and a second control mode that controls execution of processes in an application program according to direct touch operations and contactless operations.
- the controller 15 switches to the first control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in the threshold level memory 18 .
- the controller 15 switches to the second control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in the threshold level memory 18 , while also controlling execution of processes in the activated application program according to contactless operations detected on the basis of the contactless threshold level stored in the threshold level memory 18 .
- an input processing program for conducting information processing of direct touch operations and contactless operations on the display unit 5 forming the touch panel is stored in the memory 14 .
- a camera control program for controlling the shooting of still images or video with the respective camera units 8 a and 8 b .
- the camera control program is provided with a viewer program for displaying shot still images on the display unit 5 , etc.
- the viewer program is provided with functions for changing the display magnification by enlarging or reducing the displayed image, and facial recognition functions for detecting facial images of subjects (persons) appearing in still images.
- the camera control program is also provided with a video playback program for displaying shot videos on the display unit 5 , etc. Also, the video playback program is provided with playback speed modification functions for controlling changes to the video playback speed.
- Also stored in the memory 14 are an email management program for controlling the creation and transmitting/receiving of email, and a scheduler management program for managing a scheduler in which the user's schedule is registered.
- Also stored in the memory 14 are a Web browsing program for viewing Web pages by transmitting/receiving information by accessing a server provided on a given network such as a communication network or the Internet, a contacts list management program for managing a contacts list that registers personal information such as the names, addresses, telephone numbers, email addresses, and facial photos of friends and acquaintances (i.e., the contacts list is a personal information registration area), and a music player program for playing back music data.
- a Web browsing program for viewing Web pages by transmitting/receiving information by accessing a server provided on a given network such as a communication network or the Internet
- a contacts list management program for managing a contacts list that registers personal information such as the names, addresses, telephone numbers, email addresses, and facial photos of friends and acquaintances (i.e., the contacts list is a personal information registration area)
- a music player program for playing back music data.
- Also stored in the memory 14 is a lock/unlock application program for realizing an unlocked mode that receives direct touch operations and contactless operations on the display unit 5 and executes actions according to the operations, as well as a locked mode that cancels direct touch operations and contactless operations on the display unit 5 .
- a scheduler in which the user's desired schedule is registered (i.e., a schedule data registration area), and a contacts list in which information such as the user names, still images (facial images, etc.), addresses, telephone numbers, email addresses, and birthdates of the user's friends and acquaintances is registered (i.e., a personal information registration area for respective users).
- Also stored in the memory 14 are music data played back by the music player program, still image data and video data played back by the viewer program and video playback program in the camera control program, transmitted/received email data, and a history of transmitted/received telephone calls and emails.
- a projected capacitive touch panel is provided as the touch panel of the display unit 5 .
- the projected capacitive touch panel includes a resistive film with an electrode layer underneath, as well as a substrate layer provided with a control IC (control integrated circuit).
- a control IC control integrated circuit
- many electrode patterns forming a mosaic consisting of two layers (horizontal and vertical) are arranged on a glass, plastic, or other substrate using transparent electrodes such as indium tin oxide (ITO).
- ITO indium tin oxide
- the projected capacitive touch panel identifies an operation position by detecting changes in electrode capacitance due to a direct touch operation or a contactless operation from two (horizontal and vertical) electrode lines. By respectively providing many electrode lines in the horizontal and vertical directions, multipoint detection of direct touch operations becomes possible.
- a projected capacitive touch panel is provided as the display unit 5 in this example, what is called a surface capacitive touch panel may also be provided instead of the projected capacitive touch panel.
- the controller 15 of the mobile phone By acting on the basis of an input processing program stored in the memory 14 , the controller 15 of the mobile phone detects a “direct touch operation” made by causing a finger or other operating element (conductive member) to contact the operable surface of the display unit 5 which forms a projected capacitive touch panel.
- the mobile phone is also configured to detect a “contactless operation” in which a finger or other conductive member is moved over the operable surface of the display unit 5 such that the distance between the operable surface of the display unit 5 and the finger is short enough to cause at least a given change in the capacitance of the display unit 5 , but without the finger directly touching the operable surface.
- the controller 15 is also provided with a “first control mode” that controls execution of processes in an application program according to direct touch operations, and a “second control mode” that controls execution of processes in an application program according to both direct touch operations and contactless operations.
- the controller 15 switches to the first control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in the threshold level memory 18 .
- the controller 15 switches to the second control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in the threshold level memory 18 , while also controlling execution of processes in the activated application program according to contactless operations detected on the basis of the contactless threshold level stored in the threshold level memory 18 .
- FIG. 2 illustrates function blocks of the controller 15 which are realized by the controller 15 acting on the basis of the input processing program.
- the controller 15 When acting on the basis of the input processing program, the controller 15 switches to the first control mode in the case where a contactless operation mode file discussed earlier is not attached to the activated application program.
- the controller 15 Upon switching to the first control mode, the controller 15 functions as the read controller 19 illustrated in FIG. 2 , reads out the capacitance values of all capacitive sensors in the display unit 5 , and compares the respective capacitance values to the direct touch threshold level stored in the threshold level memory 18 . In so doing, the controller 15 creates two-dimensional coordinate information for the display unit 5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation).
- the controller 15 upon switching to the first control mode, the controller 15 functions as a touch driver 21 , and stores two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in the operation history register 16 .
- the controller 15 upon switching to the first control mode, the controller 15 functions as an operation detector 26 of a window manager 23 , and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when the controller 15 functioned as the touch driver 21 , as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in the operation history register 16 .
- a direct touch operation mode direct touch operation pattern
- a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when the controller 15 functioned as the touch driver 21 , as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in the operation history register 16 .
- the controller 15 functions as an application execution controller 24 , and controls execution of a process from among respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position.
- the controller 15 switches to the second control mode in the case where a contactless operation mode file discussed earlier is attached to the activated application program.
- the controller 15 Upon switching to the second control mode, the controller 15 functions as a contactless operation enabler 25 of the window manager 23 , and reads the contactless operation mode file attached to the activated application program. Then, the controller 15 functions as a contactless operation detector 20 , and loads the contactless operation mode file read above into the contactless operation mode file memory 17 .
- the controller 15 upon switching to the second control mode, the controller 15 functions as the read controller 19 , reads out the capacitance values of all capacitive sensors in the display unit 5 , and by comparing the respective capacitance values to the direct touch threshold level and the contactless threshold level stored in the threshold level memory 18 , determines whether a direct touch operation or a contactless operation has been performed by the user.
- the controller 15 functions as the read controller 19 to create two-dimensional coordinate information for the display unit 5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation) as discussed earlier, on the basis of the capacitance values for all capacitive sensors that were read out from the display unit 5 as well as the direct touch threshold level stored in the threshold level memory 18 .
- the controller 15 functions as the touch driver 21 , and stores two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in the operation history register 16 .
- the controller 15 functions as the operation detector 26 of the window manager 23 , and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when the controller 15 functioned as the touch driver 21 , as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in the operation history register 16 .
- a direct touch operation mode direct touch operation pattern
- a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when the controller 15 functioned as the touch driver 21 , as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in the operation history register 16 .
- the controller 15 functions as the application execution controller 24 , and controls execution of a process from among the respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position.
- the controller 15 functions as the read controller 19 and thereby determines that a contactless operation has been performed by the user, the controller 15 functions as the contactless operation detector 20 , creates a capacitance map indicating the capacitance values of all capacitive sensors that have been read out from the display unit 5 , and controls storage of it in the storage area for contactless operations in the operation history register 16 .
- the controller 15 functions as the contactless operation detector 20 , and detects the contactless operation mode (contactless operation pattern) of the contactless operation performed by the user on the basis of a plurality of capacitance maps stored in the operation history register 16 . Then, the controller 15 functions as the contactless operation detector 20 , and on the basis of the contactless operation file stored in the contactless operation mode file memory 17 , creates two-dimensional coordinate information corresponding to the process from among respective processes in the activated application program for the contactless operation pattern that has been detected on the basis of the plurality of capacitance maps.
- the controller 15 functions as the operation detector 26 of the window manager 23 , and identifies the contactless operation mode (contactless operation pattern) and contactless operation position on the basis of the created two-dimensional coordinate information.
- the controller 15 functions as the application execution controller 24 , and controls execution of a process from among respective processes in the currently active application program that corresponds to the identified contactless operation mode (contactless operation pattern) and contactless operation position.
- FIG. 3 illustrates a flowchart of input processing action by the controller 15 based on the input processing program.
- the controller 15 starts the process illustrated by the flowchart in FIG. 3 on the basis of an input processing program stored in the memory 14 .
- step S 1 the controller 15 functions as the application execution controller 24 and determines whether or not the activated application program has a contactless operation mode file, thereby determining whether or not the activated application program is able to support contactless operations.
- the controller 15 switches to the first control mode and executes the processing from step S 11 to step S 16 .
- the controller 15 switches to the second control mode and executes the processing from step S 2 to step S 10 as well as the processing from step S 31 to step S 34 in the flowchart in FIG. 7 .
- step S 11 acquires the capacitance values of all capacitive sensors in the display unit 5 by functioning as the read controller 19 , and advances the process to step S 12 .
- step S 12 the controller 15 compares the acquired capacitance values to a direct touch threshold level indicated by the broken line in FIG. 4A which is stored in the threshold level memory 18 . In so doing, the controller 15 determines whether or not there exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level. Then, the controller 15 advances the process to step S 13 in the case where it is determined that there do exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level, and advances the process to step S 16 in the case where it is determined that there do not exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level.
- the capacitance values of all capacitive sensors in the display unit 5 are acquired at given time intervals while an application program is being executed. For this reason, if the controller 15 advances the process to step S 16 due to determining that there do not exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level, the controller 15 functions as the read controller 19 and determines whether or not an application program is currently running. In so doing, the controller 15 determines whether or not to continue detecting capacitance values.
- the controller 15 ends all processing in the flowchart illustrated in FIG. 3 .
- the controller 15 advances the process to step S 13 due to determining that there do exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level, the controller 15 functions as the read controller 19 and compares the acquire capacitance values to the direct touch threshold level stored in the threshold level memory 18 . In so doing, the controller 15 creates two-dimensional coordinate information for the display unit 5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation), and advances the process to step S 14 .
- the controller 15 functions as the touch driver 21 and stores the two-dimensional coordinate information corresponding to a direct touch operation in the storage area for direct touch operations in the operation history register 16 .
- the controller 15 functions as the operation detector 26 of the window manager 23 and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when the controller 15 functioned as the touch driver 21 , as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in the operation history register 16 .
- the controller 15 then advances the process to step S 15 .
- step S 15 the controller 15 functions as the application execution controller 24 , controls execution of a process from among respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position, and advances the process to step S 16 .
- step S 16 the controller 15 functions as the read controller 19 as discussed above and determines whether or not an application program is currently running. In so doing, the controller 15 determines whether or not to continue detecting capacitance values.
- the controller 15 ends all processing in the flowchart illustrated in FIG. 3 .
- step S 2 the controller 15 advances the process to step S 2 and improves the sensitivity of the capacitive sensors in the display unit 5 by raising the output gain of the capacitive sensors by a given amount, for example.
- the controller 15 then advances the process to step S 3 .
- the controller 15 attempts to improve the sensitivity of the capacitive sensors by raising the output gain of the capacitive sensors by a given amount as discussed above, for example.
- step S 3 the controller 15 functions as the application execution controller 24 and reads the contactless operation mode file attached to the activated application program. Then, the controller 15 functions as the contactless operation enabler 25 and the contactless operation detector 20 , loads the contactless operation mode file thus read into the contactless operation mode file memory 17 , and advances the process to step S 4 .
- the contactless operation mode file is a file of contactless operation mode data expressing capacitance map transition patterns for individual contactless operation modes used to specify execution of a desired process in the currently activated application program.
- step S 4 the controller 15 functions as the read controller 19 and thereby acquires the capacitance values of all capacitive sensors in the display unit 5 as a capacitance map, and then advances the process to step S 5 .
- step S 5 the controller 15 compares the capacitance values in the acquired capacitance map to the direct touch threshold level indicated by the broken line in FIG. 4A which is stored in the threshold level memory 18 . In so doing, the controller 15 determines whether or not there exist capacitance values from among the acquired capacitance values that are equal to or greater than the direct touch threshold level.
- the controller 15 upon advancing the process to step S 6 , the controller 15 functions as the read controller 19 and compares the capacitance values in the acquired capacitance map to the direct touch threshold level stored in the threshold level memory 18 . In so doing, the controller 15 creates two-dimensional coordinate information for the display unit 5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation), and advances the process to step S 7 .
- the controller 15 functions as the touch driver 21 and stores the two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in the operation history register 16 .
- the controller 15 functions as the operation detector 26 of the window manager 23 and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when the controller 15 functioned as the touch driver 21 , as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in the operation history register 16 .
- the controller 15 then advances the process to step S 8 .
- step S 8 the controller 15 functions as the application execution controller 24 , controls execution of a process from among respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position, and advances the process to step S 9 .
- step S 9 the controller 15 functions as the read controller 19 as discussed above and determines whether or not an application program is currently running. In so doing, the controller 15 determines whether or not to continue detecting capacitance values.
- the controller 15 ends all processing in the flowchart illustrated in FIG. 3 .
- the controller 15 returns the process to step S 4 and once again acquires the capacitance values of all capacitive sensors in the display unit. While an application program is running, the controller 15 controls repeated execution of action for acquiring the capacitance values of all capacitive sensors in this way at given time intervals.
- step S 5 a lack of capacitance values equal to or greater than the direct touch threshold level among the capacitance values in the acquired capacitance map means there is a possibility that a contactless operation has been performed by the user. For this reason, the controller 15 advances the process to step S 10 in the case where the existence of capacitance values equal to or greater than the direct touch threshold level is not detected.
- step S 10 the controller 15 functions as the read controller 19 and compares the acquired capacitance values to the contactless threshold level indicated by the chain line in FIG. 4B which is stored in the threshold level memory 18 . In so doing, the controller 15 determines whether or not there exist capacitance values among the acquired capacitance values which are equal to or greater than the contactless threshold level.
- the controller 15 discards the acquired capacitance values by treating them as noise, returns the process to step S 4 via step S 9 , and once again acquires the capacitance values of all capacitive sensors in the display unit 5 .
- the controller 15 advances the process to step S 31 in FIG. 7 .
- step S 5 and step S 10 will be once again explained in detail.
- FIG. 5 Exemplary capacitance values for all capacitive sensors detected during a direct touch operation are illustrated in FIG. 5 .
- a direct touch operation only the capacitance value of the sensor where the direct touch operation is centered and the capacitance values of the immediately adjacent sensors are detected as large capacitance values, as indicated by being enclosed in bold lines in FIG. 5 .
- FIG. 6 Exemplary capacitance values for all capacitive sensors detected during a contactless operation are illustrated in FIG. 6 .
- capacitance values that have changed to intermediate values over a wider range than during a direct touch operation are detected, as indicated by being enclosed in bold lines in FIG. 6 .
- a direct touch threshold level indicated by the broken line in FIGS. 4A and 4B there are a total of two threshold levels: a direct touch threshold level indicated by the broken line in FIGS. 4A and 4B , and a contactless threshold level indicated by the chain line in FIGS. 4A and 4B , which is set at a lower level than the level of the direct touch threshold level.
- step S 5 the controller 15 compares the capacitance values of respective sensors in the acquired capacitance map to the direct touch threshold level indicated by the broken line in FIGS. 4A and 4B . Then, the controller 15 determines whether or not there exist capacitance values among the capacitance values from all sensors in the display unit 5 which are equal to or greater than the direct touch threshold level.
- the controller 15 advances the process to step S 10 and respectively compares the capacitance values of respective sensors in the acquired capacitance map to the contactless threshold level indicated by the chain line in FIGS. 4A and 4B . Then, in this step S 10 , the controller 15 determines whether or not there exist capacitance values among the respective capacitance values detected by all sensors in the display unit 5 which are less than the direct touch threshold level but also equal to or greater than the contactless threshold level.
- the controller 15 controls the storage of the acquired capacitance map in the storage area for contactless operations in the operation history register 16 and advances the process to step S 31 of the flowchart in FIG. 7 hereinafter described.
- step S 10 When the controller 15 advances the process to step S 31 of the flowchart in FIG. 7 due to determining in step S 10 that there exist capacitance values among the capacitance values in the capacitance map acquired from the display unit 5 which are lower than the direct touch threshold level indicated by the broken line in FIG. 4B but also equal to or greater than the direct touch threshold level indicated by the chain line in FIG. 4B , the controller 15 next functions as the contactless operation detector 20 and determines whether or not a given number or more capacitance maps are being stored in the storage area for contactless operations in the operation history register 16 such that a contactless operation pattern) can be detected.
- the controller 15 then advances the process to step S 32 in the case where it is determined that the given number or more capacitance maps are being stored, while advancing the process to step S 9 of the flowchart in FIG. 3 in the case where it is determined that the given number or more capacitance maps are not being stored.
- capacitance maps are acquired from the display unit 5 at given time intervals while an application program is running, as discussed above. For this reason, upon advancing the process to step S 9 of the flowchart in FIG. 3 due to determining that the given number or more capacitance maps are not being stored, the controller 15 determines whether or not to continue detecting capacitance maps by determining whether or not an application program is currently running.
- the controller 15 ends all processing in the respective flowcharts illustrated in FIGS. 3 and 7 .
- the controller 15 determines to continue detecting capacitance maps, the controller 15 returns the process to step S 4 and from the display unit 5 acquires a capacitance map expressing the capacitance values of all capacitive sensors in the display unit 5 as discussed earlier. Then, if there exist capacitance values among the capacitance values in the acquired capacitance map which are lower than the direct touch threshold level but also equal to or greater than the contactless threshold level, the controller 15 controls storage of the capacitance map in the storage area for direct contactless operations in the operation history register 16 .
- the controller 15 returns the process to step S 4 via step S 9 of the flowchart in FIG. 3 , and once again acquires a capacitance map and controls its storage in the storage area for contactless operations in the operation history register 16 .
- capacitance maps corresponding to a contactless operation are successively stored in the storage area for contactless operations in the operation history register 16 .
- the controller 15 is configured to load contactless operation mode data, which indicates capacitance map transition patterns for individual contactless operation modes of contactless operations used by the application program, into the contactless operation mode file memory 17 as the contactless operation mode file, as discussed earlier.
- the controller 15 upon advancing the process to step S 32 due to determining that a number of capacitance maps enabling detection of a contactless operation mode are being stored in the storage area for contactless operations in the operation history register 16 , the controller 15 functions as the contactless operation detector 20 to specify for each capacitance map the placement positions on the display unit 5 of the capacitive sensors that detected capacitance values which are lower than the direct touch threshold level but also equal to or greater than the contactless threshold level, on the basis of the respective capacitance maps stored in the operation history register 16 . Then, the controller 15 detects shifts in the capacitive sensors specified for the individual capacitance maps as a capacitance map transition pattern corresponding to a contactless operation.
- FIGS. 8A to 8C An exemplary transition pattern across respective capacitance maps during a contactless operation is illustrated in FIGS. 8A to 8C .
- the example illustrated in FIGS. 8A to 8C is an example for the case where a contactless operation is made on the display unit 5 of a mobile phone from left to right along the short axis of the display unit 5 .
- the positions of the capacitive sensors that detect capacitance values equal to or greater than the contactless threshold level move from left to right along the short axis of the display unit 5 , as indicated by being respectively enclosed in bold lines in FIGS. 8A to 8C .
- step S 32 the controller 15 detects the shift in the positions of the capacitive sensors that detect capacitance values equal to or greater than the contactless threshold level due to such a contactless operation as a capacitance map transition pattern corresponding to a contactless operation.
- the controller 15 advances the process to step S 33 and functions as the contactless operation detector 20 to compare the capacitance map transition pattern detected with respect to a contactless operation in step S 32 to capacitance map transition patterns expressed by contactless operation mode data in a contactless operation mode file loaded into the contactless operation mode file memory 17 , and specify a contactless operation mode (contactless operation gesture). Then, the controller 15 detects two-dimensional coordinate information corresponding to the specified contactless operation.
- contactless operation mode data in a contactless operation mode file is made up of data expressing contactless operation modes (gestures) corresponding to capacitance map transition patterns such as “right-to-left”, “left-to-right”, “top-to-bottom”, and “bottom-to-top”, as well as data expressing two-dimensional coordinates corresponding to contactless operation modes.
- step S 33 the controller 15 detects from the contactless operation mode file memory 17 the contactless operation mode data of a transition pattern that matches the capacitance map transition pattern detected with respect to a contactless operation.
- the contactless operation mode data is made up of data expressing contactless operation modes corresponding to capacitance map transition patterns, and data expressing two-dimensional coordinates corresponding to the contactless operation modes for specifying processes of the application program.
- the controller 15 detects a contactless operation mode and two-dimensional coordinates corresponding to this contactless operation mode.
- step S 34 the controller 15 advances the process to step S 34 and functions as the operation detector 26 to recognize a contactless operation mode and two-dimensional coordinate information corresponding to this contactless operation mode. Then, in step S 34 , the controller 15 functions as the application execution controller 24 to control execution of a process in an application program on the basis of the recognized contactless operation mode and two-dimensional coordinate information corresponding to this contactless operation mode.
- the controller 15 Upon controlling execution of a process in an application program corresponding to a contactless operation in this way, the controller 15 advances the process to step S 9 in FIG. 3 and determines whether or not to continue detecting capacitance maps by determining whether or not an application program is currently running, as discussed earlier.
- step S 4 the controller 15 returns the process to step S 4 and acquires from the display unit 5 a capacitance map expressing the capacitance values of all sensors on the display unit 5 , as discussed earlier.
- controller 15 determines that acquisition of the next capacitance map has terminated, such when an operation for ending the currently active application program is given, for example, the controller 15 ends all processing in the respective flowcharts illustrated in FIGS. 3 and 7 .
- the mobile phone in this embodiment is provided with a display unit 5 which forms a capacitive touch panel, and in addition, a direct touch threshold level as well as a contactless threshold level are set. Furthermore, two-dimensional coordinate information expressing a direct touch operation position is created on the basis of the capacitance values of the respective sensors in the display unit 5 during a direct touch operation and the direct touch threshold level, and the execution of information processing corresponding to the created two-dimensional coordinate information expressing a direct touch operation position from among various information processing in the currently active application program is controlled.
- two-dimensional coordinate information expressing a contactless operation mode is created on the basis of the capacitance values of the respective sensors in the display unit 5 during a contactless operation and the contactless threshold level, and the execution of information processing corresponding to the created two-dimensional coordinate information expressing a contactless operation mode from among various information processing in the currently active application program is controlled.
- a mobile phone of this embodiment is configured to switch to what is called a hands-free telephony mode if a contactless operation is conducted so as to horizontally swipe a finger across the display unit 5 as illustrated in FIG. 9 in the case where an incoming telephone call or an incoming video phone call is received while the mobile phone is in a locked mode that cancels most operations.
- the controller 15 functions as the contactless operation enabler 25 to load a contactless operation mode file of contactless operation mode data expressing contactless operation modes (gestures) of contactless operations and two-dimensional coordinates corresponding to the contactless operation modes used by a telephone application program into the contactless operation mode file memory 17 .
- controller 15 acquires the capacitance maps discussed earlier from the display unit 5 and controls their successive storage in the operation history register 16 .
- the controller 15 compares the transition pattern of the capacitance maps stored in the operation history register 16 to the capacitance map transition patterns for individual contactless operation modes expressed by the respective contactless operation mode files loaded into the contactless operation mode file memory 17 . Then, the controller 15 detects contactless operation mode data expressing a transition pattern matching the transition pattern of the respective capacitance maps acquired from the operation history register 16 from among the capacitance map transition patterns for individual contactless operation modes expressed by the respective contactless operation mode data stored in the contactless operation mode file memory 17 .
- the contactless operation mode data expresses a contactless operation pattern corresponding to a transition pattern of the respective capacitance maps, and two-dimensional coordinate information corresponding to this contactless operation pattern.
- the controller 15 functions as the operation detector 26 of the window manager 23 to detect a contactless operation pattern and two-dimensional coordinate information corresponding to this contactless operation pattern on the basis of the detected contactless operation mode data.
- the detected contactless operation pattern is for a contactless operation instructing the switch to a hands-free telephony mode in the case of a contactless operation pattern wherein a finger is moved so as to horizontally swipe across the display unit 5 .
- the controller 15 functions as the application execution controller 24 to switch to a hands-free telephony mode on the basis of a telephone application program, and controls driving of the microphone unit 4 which picks up telephone transmitter audio and the second speaker unit 3 b , which is an external speaker unit for obtaining the acoustic output of telephone receiver audio, while also controlling communication by the communication circuit 2 so as to transmit and receive telephony audio obtained thereby.
- the controller 15 switches to a hands-free telephony mode on the basis of a video phone application program, and controls driving of the microphone unit 4 which picks up telephone transmitter audio, the second speaker unit 3 b , which is an external speaker unit for obtaining the acoustic output of telephone receiver audio, the second camera unit 8 b for obtaining transmitter images, and the display unit 5 for displaying receiver images, while also controlling communication by the communication circuit 2 so as to transmit and receive telephony audio and images obtained thereby.
- telephone or video phone communication in what is called a hands-free telephony mode can be made possible simply by conducting a contactless operation so as to horizontally swipe a finger across the display unit 5 .
- the controller 15 controls driving of the microphone unit 4 which picks up telephone transmitter audio, the second speaker unit 3 b , which is an external speaker unit for obtaining the acoustic output of telephone receiver audio, the second camera unit 8 b for obtaining transmitter images, and the display unit 5 for displaying receiver images, while also controlling communication by the communication circuit 2 so as to transmit and receive telephony audio and images obtained thereby.
- the controller 15 activates an email management program stored in the memory 14 and notifies the user of the receipt of the email.
- a contactless operation that horizontally swipes a finger across the display unit 5 becomes a contactless operation instructing display of the body text of the received email.
- a contactless operation that moves a finger up or down along the long axis of the display unit 5 becomes a contactless operation instructing scrolling of the body text of the displayed email.
- the controller 15 switches to a locked mode that cancels most key operations on the basis of a lock/unlock application program stored in the memory 14 .
- a contactless operation that horizontally swipes a finger across the display unit 5 becomes a contactless operation instructing the mobile phone to accept a key operation and switch from the locked mode to an unlocked mode which conducts information processing corresponding to the key operation.
- a contactless operation that horizontally swipes a finger across the display unit 5 becomes a contactless operation instructing the mobile phone to switch from the unlocked mode to a locked mode which cancels most key operations.
- a direct touch operation that slides an unlock slider on the display unit 5 in an unlock direction becomes a direct touch operation instructing the mobile phone to switch from the locked mode to the unlocked mode.
- a direct touch operation that slides an unlock slider on the display unit 5 in a lock direction becomes a direct touch operation instructing the mobile phone to switch from the unlocked mode to the locked mode.
- the controller 15 During the locked mode, if the controller 15 detects a contactless operation that horizontally swipes a finger across the display unit 5 on the basis of the created two-dimensional coordinate information while functioning as the window manager 23 , the controller 15 switches to the unlocked mode in accordance with the contactless operation, accepts a key operation by the user, and controls execution of information processing corresponding to the key operation.
- the controller 15 detects a contactless operation that horizontally swipes a finger across the display unit 5 on the basis of the created two-dimensional coordinate information while functioning as the window manager 23 , the controller 15 switches to the locked mode and cancels most key operations.
- the mobile phone can be switched to an unlocked mode simply by conducting a contactless operation that horizontally swipes a finger across the display unit 5
- the mobile phone can be switched to a locked mode simply by conducting a contactless operation that horizontally swipes a finger across the display unit 5 .
- the controller 15 detects a direct touch operation that slides an unlock slider on the display unit 5 in an unlock direction on the basis of the created two-dimensional coordinate information while functioning as the window manager 23 , the controller 15 switches to an unlocked mode in accordance with the direct touch operation, accepts a key operation by the user, and controls execution of information processing corresponding to the key operation.
- the controller 15 detects a direct touch operation that slides an unlock slider on the display unit 5 in a lock direction on the basis of the created two-dimensional coordinate information while functioning as the window manager 23 , the controller 15 switches to a locked mode and cancels most key operations.
- the mobile phone can be switched to an unlocked mode simply by conducting a direct touch operation that slides an unlock slider on the display unit 5 in an unlock direction
- the mobile phone can be switched to a locked mode simply by conducting a direct touch operation that slides an unlock slider on the display unit 5 in a lock direction.
- a contactless operation that causes a finger to revolve in a clockwise direction over the display unit 5 becomes a contactless operation instructing the mobile phone to enlarge the web page being displayed.
- a contactless operation that moves a finger over the display unit 5 upwards along the long axis of the display unit 5 becomes a contactless operation instructing the mobile phone to scroll up the web page being displayed.
- a contactless operation that moves a finger over the display unit 5 downwards along the long axis of the display unit 5 becomes a contactless operation instructing the mobile phone to scroll down the web page being displayed.
- the controller 15 detects a contactless operation that causes a finger over the display unit 5 to revolve in a clockwise direction on the basis of the created two-dimensional coordinate information while functioning as the window manager 23 , the controller 15 enlarges the web page being displayed at gradually larger display magnifications in accordance with the contactless operation.
- the controller 15 detects a contactless operation that causes a finger over the display unit 5 to revolve in a counter-clockwise direction on the basis of the created two-dimensional coordinate information while functioning as the window manager 23 , the controller 15 reduces the web page being displayed at gradually smaller display magnifications in accordance with the contactless operation.
- the controller 15 detects a contactless operation that moves a finger over the display unit 5 upwards along the long axis of the display unit 5 on the basis of the created two-dimensional coordinate information while functioning as the window manager 23 , the controller 15 scrolls up the web page being displayed in accordance with the contactless operation.
- the controller 15 detects a contactless operation that moves a finger over the display unit 5 downwards along the long axis of the display unit 5 on the basis of the created two-dimensional coordinate information while functioning as the window manager 23 , the controller 15 scrolls down the web page being displayed in accordance with the contactless operation.
- the web page when a web page is being displayed, the web page can be enlarged or reduced by conducting a contactless operation that causes a finger over the display unit 5 to revolve in a clockwise or counter-clockwise direction.
- the web page can be scrolled up or scrolled down by conducting a contactless operation that moves a finger over the display unit 5 upwards or downwards along the long axis of the display unit 5 .
- the mobile phone is configured to detect both contactless operations and direct touch operations, but the mobile phone may also detect only contactless operations and control execution of information processing in application program on the basis of detected contactless operations. Even in such cases, the same advantages as discussed above can be obtained.
- the present disclosure was an example wherein the present disclosure was applied to a mobile phone, the present disclosure may also be applied to other electronic devices besides a mobile phone, such as a PHS phone (PHS: Personal Handyphone System), a PDA (PDA: Personal Digital Assistant), a digital camera, a digital video camera, a portable game console, or a notebook computer, for example.
- PHS phone Personal Handyphone System
- PDA Personal Digital Assistant
- digital camera digital camera
- digital video camera digital video camera
- portable game console a portable game console
- notebook computer for example.
- the same advantages as those of the foregoing embodiment can still be obtained in any case.
Abstract
Description
- The present application claims the benefit of the earlier filing date of U.S. Provisional Patent Application Ser. No. 61/547,783 filed on Oct. 17, 2011, the entire contents of which is incorporated herein by reference.
- 1. Field of the Disclosure
- The present disclosure relates to an information processing apparatus applied to an electronic device provided with an operable unit that detects an operation mode on the basis of changes in the capacitance of an operable surface, such as a touch panel, for example.
- Specifically, the present disclosure relates to an information processing apparatus applied to an electronic device such as a mobile phone, a PHS phone (PHS: Personal Handyphone System), a PDA (PDA: Personal Digital Assistant), a digital camera, a digital video camera, a portable game console, or a notebook computer, for example.
- 2. Description of Related Art
- At present, mobile phones provided with capacitive touch panels are known. In such mobile phones, a controller detects a direct touch operation via the touch panel on the basis of an input processing program. Then, the controller controls execution of a process corresponding to the detected direct touch operation from among various information processes in a given application program.
-
FIG. 10 illustrates a function block diagram of such a controller realized as a result of a controller acting on the basis of the input processing program. - As illustrated in
FIG. 10 , in the case of acting on the basis of the input processing program, the controller functions as a two-dimensional coordinate converter 101 which detects a direct touch operation position on atouch panel 100 and outputs two-dimensional coordinate information (X coordinate information and Y coordinate information) corresponding to the direct touch operation position. - Herein, in the case of functioning as the two-
dimensional coordinate converter 101, the controller may also create two-dimensional coordinate information corresponding to a direct touch operation position on the basis of output from thetouch panel 100, which is hardware (HW) as discussed above. Alternatively, before functioning as the two-dimensional coordinate converter 101, the controller may function as firmware (FW) which detects two-dimensional coordinate information corresponding to a direct touch operation position on the basis of output from the hardware (HW)touch panel 100, and subsequently function as the two-dimensional coordinate converter 101 and create two-dimensional coordinate information corresponding to the direct touch operation position. - Also, in the case of acting on the basis of the input processing program, the controller functions as an
operation detector 104 in awindow manager 102 which detects a direct touch operation position and a direct touch operation state (direct touch operation mode) on the basis of the two-dimensional coordinate information created when the controller functions as the two-dimensional coordinate converter 101. - Then, the controller functions as an
application execution controller 103 that controls execution of information processing which, from among the various information processing of a currently activated application program, corresponds to a direct touch operation state, etc. detected when the controller functions as theoperation detector 104 of thewindow manager 102. -
FIG. 11 illustrates a flowchart of a controller's input processing action based on the input processing program. Thetouch panel 100 is configured to identify an operation position by detecting changes in the capacitance between a user's finger and theconductive film 151 of the two-dimensional coordinate converter 101, as illustrated inFIG. 12A . - In step S100 of the flowchart in
FIG. 11 , the controller acquires (scans) the capacitance values of all sensors on thetouch panel 100 by functioning as the two-dimensional coordinate converter 101. Also, in step S100 the controller determines whether or not capacitance values have been detected for all sensors on thetouch panel 100, and advances the process to step S101 when capacitance values have been detected for all sensors. - If the user performs a direct touch operation with his or her finger on the
conductive film 151 forming the operable surface of thetouch panel 100, a change in capacitance values occurs centered about the direct touch operation position on theconductive film 151, as illustrated inFIG. 12B . In other words, in the case where a direct touch operation is conducted on thetouch panel 100, the capacitance values change such that the approximate center of the direct touch operation position on theconductive film 151 becomes the peak value of the capacitance values, with the capacitance values gradually becoming smaller away from the center of the direct touch operation position. - In step S101 the controller, by functioning as the two-
dimensional coordinate converter 101, compares the capacitance values of the individual sensors to a direct touch threshold level as illustrated inFIG. 12B , and determines whether or not there exist sensors among all sensors on thetouch panel 100 having capacitance values equal to or greater than the direct touch threshold level. - In the case where it is determined that sensors having capacitance values equal to or greater than the direct touch threshold level do not exist, the controller advances the process to step S104 and determines whether or not to continue detecting the capacitance values of the respective sensors. Then, in the case where it is determined not to continue detecting the capacitance values of the respective sensors, the controller ends the process of the flowchart in
FIG. 11 . Meanwhile, in the case where it is determined to continue detecting the capacitance values of the respective sensors, the controller returns the process to step S100 at the timing for conducting the next detection of the capacitance values of the respective sensors and acquires the capacitances value of the respective sensors again. - In contrast, in the case where it is determined in step S101 that there do exist sensors having capacitance values equal to or greater than the direct touch threshold level, the controller advances the process to step S102, identifies the sensors having capacitance values equal to or greater than the direct touch threshold level, and advances the process to step S103.
- Then, in step S103 the controller creates two-dimensional coordinate information (i.e., respective X-axis and Y-axis coordinate information) corresponding to the placement positions on the
touch panel 100 of the identified sensors having capacitance values equal to or greater than the direct touch threshold level, controls storage of the two-dimensional coordinate information to a register, and advances the process to step S104. - The controller is configured to control storage of two-dimensional coordinate information thus created to a register every time the capacitance values of respective sensors are acquired from the
touch panel 100 and the two-dimensional coordinate information is created. For this reason, respective sets of two-dimensional coordinate information in accordance with direct touch operations on thetouch panel 100 by the user are successively stored in the register. Consequently, direct touch operation history information (i.e., the respective sets of two-dimensional coordinate information) corresponding to direct touch operations on thetouch panel 100 by the user is stored in the register. - When functioning as the
operation detector 104 of thewindow manager 102, the controller detects the user's direct touch operation mode, like that illustrated inFIG. 12C for example, on the basis of the direct touch history given by the respective sets of two-dimensional coordinate information stored in the register. - Then, when functioning as the
application execution controller 103, the controller controls execution of information processing which, from among the various information processing of a currently activated application program, corresponds to the detected direct touch operation mode and the two-dimensional coordinate information used when detecting the direct touch operation mode. - By determining to continue detecting capacitance values in step S104 and returning the process to step S100, the controller acquires the capacitance values of the respective sensors in the
touch panel 100 at given time intervals, and conducts processing for creating the two-dimensional coordinate information, detecting the direct touch operation mode, and executing an application program based on the detected operation mode, etc., on the basis of the acquired capacitance values. - By performing a direct touch operation on the
touch panel 100 with an operating object such as a finger as discussed above, information processing operations based on the currently activated application program become possible. - However, the present inventor has recognized the need for an information processing apparatus in which information processing in a given application program is operable not only by direct touch operations, but also by contactless operations.
- According to one exemplary embodiment, the disclosure is directed to an information processing apparatus comprising: a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel; and a controller that determines whether an application activated at the information processing apparatus supports contactless operations; controls the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and controls the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.
- According to another exemplary embodiment, the disclosure is directed to an information processing method performed by an information processing apparatus including a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel, the method comprising: determining whether an application activated at the information processing apparatus supports contactless operations; controlling the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and controlling the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.
- According to another exemplary embodiment, the disclosure is directed to a non-transitory computer-readable medium including computer program instructions, which when executed by an information processing apparatus including a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel, cause the information processing apparatus to perform a method, the method comprising: determining whether an application activated at the information processing apparatus supports contactless operations; controlling the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and controlling the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.
- According to an embodiment of the present disclosure, new operation modes can be provided in which information processing in a given application program is operable by contactless operations.
-
FIG. 1 is a block diagram of a mobile phone according to an embodiment implementing the present disclosure. -
FIG. 2 is a function block diagram during input processing by a controller in a mobile phone according to an embodiment. -
FIG. 3 is a flowchart illustrating input processing action in a mobile phone according to an embodiment. -
FIGS. 4A to 4C are diagrams for explaining a direct touch threshold level and a contactless threshold level provided in a mobile phone according to an embodiment. -
FIG. 5 is a diagram illustrating exemplary capacitance values of respective sensors during a direct touch operation in a mobile phone according to an embodiment. -
FIG. 6 is a diagram illustrating exemplary capacitance values of respective sensors during an contactless operation in a mobile phone according to an embodiment. -
FIG. 7 is a flowchart for explaining action for creating two-dimensional coordinate information for a contactless operation in a mobile phone according to an embodiment. -
FIGS. 8A to 8C are diagrams illustrating an exemplary transition pattern across capacitance maps during a contactless operation in a mobile phone according to an embodiment. -
FIG. 9 is a timing chart for explaining information processing of a telephony application conducted by a direct touch operation and a contactless operation in a mobile phone according to an embodiment. -
FIG. 10 is a function block diagram during input processing by a controller in a conventional mobile phone. -
FIG. 11 is a flowchart illustrating input processing action in a conventional mobile phone. -
FIGS. 12A to 12C are diagrams for explaining a direct touch threshold level provided in a conventional mobile phone. - The present disclosure can be applied to a mobile phone as one example.
-
FIG. 1 is a block diagram of a mobile phone according to an embodiment of the present disclosure. As illustrated inFIG. 1 , a mobile phone according to an embodiment includes anantenna 1 and acommunication circuit 2 which conduct audio telephony, video telephony, and wireless communication such as email and Web data (Web: World Wide Web) with a base station. - Also, the mobile phone includes a
first speaker unit 3 a (internal speaker unit) for obtaining acoustic output such as telephone receiver audio at a volume hearable when the mobile phone is brought close to the ear, as well as asecond speaker unit 3 b (external speaker unit) for obtaining acoustic output such as telephone receiver audio in cases where the mobile phone is used at a position distanced from the user's ear, such as during hands-free telephony or during video telephony. - The mobile phone also includes a
microphone unit 4 for picking up telephone transmitter audio, etc., adisplay unit 5 forming what is called a touch panel for performing direct touch operations and contactless operations thereon, and a plurality ofhardware keys 6 physically provided on the chassis of the mobile phone. - The mobile phone also includes a light emitter 7 (LED: Light Emitting Diode) for notifying the user of incoming/outgoing signals, etc. with light, a
first camera unit 8 a for shooting a still image or video of a desired subject, and asecond camera unit 8 b for shooting the user, etc. of the mobile phone during video telephony, for example. - The mobile phone also includes a
vibration unit 9 for notifying the user of incoming/outgoing signals, etc. by causing the chassis of the mobile phone to vibrate, and atimer 10 that keeps the current time. - The mobile phone also includes an
acceleration sensor 11 for detecting shake operations, etc. imparted to the chassis of the mobile phone, as well as a GPS antenna 12 (GPS: Global Positioning System) and aGPS unit 13 for detecting the present location of the mobile phone and the shooting location of still images or videos shot primarily with thefirst camera unit 8 a. - The mobile phone also includes
memory 14 storing a communication program for conducting the wireless communication processing via a base station and various application programs in addition to various data handled by these various application programs, and acontroller 15 that controls overall action of the mobile phone. - The mobile phone also includes an operation history register 16 that stores a capacitance map which indicates the capacitance values of all sensors in the
display unit 5 forming the touch panel. - The
operation history register 16 includes a storage area for direct touch operations and a storage area for contactless operations. It is configured such that direct touch operation positions and direct touch operation-induced capacitance values corresponding to direct touch operations by the user are successively stored in the storage area for direct touch operations in theoperation history register 16. It is also configured such that capacitance values corresponding to contactless operations by the user are successively stored in the storage area for contactless operations in theoperation history register 16. - The mobile phone also includes contactless operation
mode file memory 17 storing contactless operation mode files for detecting contactless operation modes (contactless operation patterns) for respective contactless operations. - In the case of the mobile phone in this embodiment, an application program that allows contactless operations includes a contactless operation mode file in which contactless operation mode data corresponding to various processes in the application program is stored as a file.
- In other words, the contactless operation mode data is data expressing transition patterns on the capacitance map for respective contactless operation modes corresponding to respective processes in the application program. Furthermore, in the contactless operation mode file, contactless operation mode data for the respective contactless operation modes is formed as a file.
- Stated differently, in the case of the mobile phone in this embodiment, an application program that allows contactless operations includes a contactless operation mode file, in which the capacitance map transition patterns for each of the contactless operations corresponding to respective processes in that application program are stored in a file as the contactless operation mode data.
- The
controller 15 reads out a contactless operation mode file attached to the application program at given timings, such as when activating the application program, for example, and loads it into the contactless operationmode file memory 17. - The
controller 15 compares a capacitance map transition pattern detected during a contactless operation to the capacitance map transition patterns expressed by the respective contactless operation mode data in the contactless operation mode file for that application program which was loaded into the contactless operationmode file memory 17, and detects the contactless operation mode. It is configured such that thecontroller 15 then controls execution of a process corresponding to the detected contactless operation mode on the basis of the activated application program. - Next, the mobile phone in this embodiment includes
threshold level memory 18 in which the direct touch threshold level and the contactless threshold level are respectively stored. - The
controller 15 is provided with a first control mode that controls execution of processes in an application program according to direct touch operations, and a second control mode that controls execution of processes in an application program according to direct touch operations and contactless operations. - Additionally, in the case where it is determined that the activated application program does not support contactless operations, the
controller 15 switches to the first control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in thethreshold level memory 18. - Meanwhile, in the case where it is determined that the activated application program does support contactless operations, the
controller 15 switches to the second control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in thethreshold level memory 18, while also controlling execution of processes in the activated application program according to contactless operations detected on the basis of the contactless threshold level stored in thethreshold level memory 18. - Next, in addition to the communication program, an input processing program for conducting information processing of direct touch operations and contactless operations on the
display unit 5 forming the touch panel (input processing) is stored in thememory 14. - Also stored in the
memory 14 is a camera control program for controlling the shooting of still images or video with therespective camera units display unit 5, etc. The viewer program is provided with functions for changing the display magnification by enlarging or reducing the displayed image, and facial recognition functions for detecting facial images of subjects (persons) appearing in still images. - The camera control program is also provided with a video playback program for displaying shot videos on the
display unit 5, etc. Also, the video playback program is provided with playback speed modification functions for controlling changes to the video playback speed. - Also stored in the
memory 14 are an email management program for controlling the creation and transmitting/receiving of email, and a scheduler management program for managing a scheduler in which the user's schedule is registered. - Also stored in the
memory 14 are a Web browsing program for viewing Web pages by transmitting/receiving information by accessing a server provided on a given network such as a communication network or the Internet, a contacts list management program for managing a contacts list that registers personal information such as the names, addresses, telephone numbers, email addresses, and facial photos of friends and acquaintances (i.e., the contacts list is a personal information registration area), and a music player program for playing back music data. - Also stored in the
memory 14 is a lock/unlock application program for realizing an unlocked mode that receives direct touch operations and contactless operations on thedisplay unit 5 and executes actions according to the operations, as well as a locked mode that cancels direct touch operations and contactless operations on thedisplay unit 5. - Also stored in the
memory 14 is a scheduler in which the user's desired schedule is registered (i.e., a schedule data registration area), and a contacts list in which information such as the user names, still images (facial images, etc.), addresses, telephone numbers, email addresses, and birthdates of the user's friends and acquaintances is registered (i.e., a personal information registration area for respective users). - Also stored in the
memory 14 are music data played back by the music player program, still image data and video data played back by the viewer program and video playback program in the camera control program, transmitted/received email data, and a history of transmitted/received telephone calls and emails. - A projected capacitive touch panel is provided as the touch panel of the
display unit 5. - The projected capacitive touch panel includes a resistive film with an electrode layer underneath, as well as a substrate layer provided with a control IC (control integrated circuit). In the electrode layer underneath the resistive film, many electrode patterns forming a mosaic consisting of two layers (horizontal and vertical) are arranged on a glass, plastic, or other substrate using transparent electrodes such as indium tin oxide (ITO).
- The projected capacitive touch panel identifies an operation position by detecting changes in electrode capacitance due to a direct touch operation or a contactless operation from two (horizontal and vertical) electrode lines. By respectively providing many electrode lines in the horizontal and vertical directions, multipoint detection of direct touch operations becomes possible.
- Although a projected capacitive touch panel is provided as the
display unit 5 in this example, what is called a surface capacitive touch panel may also be provided instead of the projected capacitive touch panel. - By acting on the basis of an input processing program stored in the
memory 14, thecontroller 15 of the mobile phone detects a “direct touch operation” made by causing a finger or other operating element (conductive member) to contact the operable surface of thedisplay unit 5 which forms a projected capacitive touch panel. - The mobile phone is also configured to detect a “contactless operation” in which a finger or other conductive member is moved over the operable surface of the
display unit 5 such that the distance between the operable surface of thedisplay unit 5 and the finger is short enough to cause at least a given change in the capacitance of thedisplay unit 5, but without the finger directly touching the operable surface. - The
controller 15 is also provided with a “first control mode” that controls execution of processes in an application program according to direct touch operations, and a “second control mode” that controls execution of processes in an application program according to both direct touch operations and contactless operations. - Additionally, in the case where it is determined that the activated application program does not support contactless operations, the
controller 15 switches to the first control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in thethreshold level memory 18. - Meanwhile, in the case where it is determined that the activated application program does support contactless operations, the
controller 15 switches to the second control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in thethreshold level memory 18, while also controlling execution of processes in the activated application program according to contactless operations detected on the basis of the contactless threshold level stored in thethreshold level memory 18. -
FIG. 2 illustrates function blocks of thecontroller 15 which are realized by thecontroller 15 acting on the basis of the input processing program. - When acting on the basis of the input processing program, the
controller 15 switches to the first control mode in the case where a contactless operation mode file discussed earlier is not attached to the activated application program. - Upon switching to the first control mode, the
controller 15 functions as theread controller 19 illustrated inFIG. 2 , reads out the capacitance values of all capacitive sensors in thedisplay unit 5, and compares the respective capacitance values to the direct touch threshold level stored in thethreshold level memory 18. In so doing, thecontroller 15 creates two-dimensional coordinate information for thedisplay unit 5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation). - Also, upon switching to the first control mode, the
controller 15 functions as atouch driver 21, and stores two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in theoperation history register 16. - Also, upon switching to the first control mode, the
controller 15 functions as anoperation detector 26 of awindow manager 23, and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when thecontroller 15 functioned as thetouch driver 21, as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in theoperation history register 16. - Additionally, in the first control mode, the
controller 15 functions as anapplication execution controller 24, and controls execution of a process from among respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position. - In contrast, when acting on the basis of the input processing program, the
controller 15 switches to the second control mode in the case where a contactless operation mode file discussed earlier is attached to the activated application program. - Upon switching to the second control mode, the
controller 15 functions as acontactless operation enabler 25 of thewindow manager 23, and reads the contactless operation mode file attached to the activated application program. Then, thecontroller 15 functions as acontactless operation detector 20, and loads the contactless operation mode file read above into the contactless operationmode file memory 17. - Also, upon switching to the second control mode, the
controller 15 functions as theread controller 19, reads out the capacitance values of all capacitive sensors in thedisplay unit 5, and by comparing the respective capacitance values to the direct touch threshold level and the contactless threshold level stored in thethreshold level memory 18, determines whether a direct touch operation or a contactless operation has been performed by the user. Then, in the case where it is determined that a direct touch operation has been performed by the user, thecontroller 15 functions as theread controller 19 to create two-dimensional coordinate information for thedisplay unit 5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation) as discussed earlier, on the basis of the capacitance values for all capacitive sensors that were read out from thedisplay unit 5 as well as the direct touch threshold level stored in thethreshold level memory 18. - Also, in the second control mode, the
controller 15 functions as thetouch driver 21, and stores two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in theoperation history register 16. - Also, in the second control mode, the
controller 15 functions as theoperation detector 26 of thewindow manager 23, and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when thecontroller 15 functioned as thetouch driver 21, as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in theoperation history register 16. - Additionally, in the second control mode, the
controller 15 functions as theapplication execution controller 24, and controls execution of a process from among the respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position. - Also, in the second control mode, in the case where the
controller 15 functions as theread controller 19 and thereby determines that a contactless operation has been performed by the user, thecontroller 15 functions as thecontactless operation detector 20, creates a capacitance map indicating the capacitance values of all capacitive sensors that have been read out from thedisplay unit 5, and controls storage of it in the storage area for contactless operations in theoperation history register 16. - Also, in the second control mode, the
controller 15 functions as thecontactless operation detector 20, and detects the contactless operation mode (contactless operation pattern) of the contactless operation performed by the user on the basis of a plurality of capacitance maps stored in theoperation history register 16. Then, thecontroller 15 functions as thecontactless operation detector 20, and on the basis of the contactless operation file stored in the contactless operationmode file memory 17, creates two-dimensional coordinate information corresponding to the process from among respective processes in the activated application program for the contactless operation pattern that has been detected on the basis of the plurality of capacitance maps. - Also, in the second control mode, the
controller 15 functions as theoperation detector 26 of thewindow manager 23, and identifies the contactless operation mode (contactless operation pattern) and contactless operation position on the basis of the created two-dimensional coordinate information. - Then, in the second control mode, the
controller 15 functions as theapplication execution controller 24, and controls execution of a process from among respective processes in the currently active application program that corresponds to the identified contactless operation mode (contactless operation pattern) and contactless operation position. -
FIG. 3 illustrates a flowchart of input processing action by thecontroller 15 based on the input processing program. When an application program is activated, thecontroller 15 starts the process illustrated by the flowchart inFIG. 3 on the basis of an input processing program stored in thememory 14. - Among the application programs executed in the mobile phone in this embodiment, application programs able to support contactless operations keep information indicating contactless operation modes (gestures) corresponding to respective information processing as a file. For this reason, when such an application program is activated, in step S1 the
controller 15 functions as theapplication execution controller 24 and determines whether or not the activated application program has a contactless operation mode file, thereby determining whether or not the activated application program is able to support contactless operations. - In the case where a contactless operation mode file is not attached to the activated application program, the
controller 15 switches to the first control mode and executes the processing from step S11 to step S16. - In contrast, in the case where a contactless operation mode file is attached to the activated application program, the
controller 15 switches to the second control mode and executes the processing from step S2 to step S10 as well as the processing from step S31 to step S34 in the flowchart inFIG. 7 . - First, if the
controller 15 switches to the first control mode because a contactless operation mode file is not attached to the activated application program, thecontroller 15 advances the process to step S11, acquires the capacitance values of all capacitive sensors in thedisplay unit 5 by functioning as theread controller 19, and advances the process to step S12. - In step S12, the
controller 15 compares the acquired capacitance values to a direct touch threshold level indicated by the broken line inFIG. 4A which is stored in thethreshold level memory 18. In so doing, thecontroller 15 determines whether or not there exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level. Then, thecontroller 15 advances the process to step S13 in the case where it is determined that there do exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level, and advances the process to step S16 in the case where it is determined that there do not exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level. - In the case of the mobile phone in this embodiment, the capacitance values of all capacitive sensors in the
display unit 5 are acquired at given time intervals while an application program is being executed. For this reason, if thecontroller 15 advances the process to step S16 due to determining that there do not exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level, thecontroller 15 functions as theread controller 19 and determines whether or not an application program is currently running. In so doing, thecontroller 15 determines whether or not to continue detecting capacitance values. - Then, in cases where it is determined that acquisition of capacitance values has been terminated, such as when an operation for ending the currently active application is performed, for example, the
controller 15 ends all processing in the flowchart illustrated inFIG. 3 . - In contrast, if the
controller 15 advances the process to step S13 due to determining that there do exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level, thecontroller 15 functions as theread controller 19 and compares the acquire capacitance values to the direct touch threshold level stored in thethreshold level memory 18. In so doing, thecontroller 15 creates two-dimensional coordinate information for thedisplay unit 5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation), and advances the process to step S14. - If two-dimensional coordinate information corresponding to a direct touch operation is created, the
controller 15 functions as thetouch driver 21 and stores the two-dimensional coordinate information corresponding to a direct touch operation in the storage area for direct touch operations in theoperation history register 16. - Next, upon advancing the process to step S14, the
controller 15 functions as theoperation detector 26 of thewindow manager 23 and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when thecontroller 15 functioned as thetouch driver 21, as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in theoperation history register 16. Thecontroller 15 then advances the process to step S15. - In step S15, the
controller 15 functions as theapplication execution controller 24, controls execution of a process from among respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position, and advances the process to step S16. - In step S16, the
controller 15 functions as theread controller 19 as discussed above and determines whether or not an application program is currently running. In so doing, thecontroller 15 determines whether or not to continue detecting capacitance values. - Then, in cases where it is determined that acquisition of capacitance values has been terminated, such as when an operation for ending the currently active application is performed, for example, the
controller 15 ends all processing in the flowchart illustrated inFIG. 3 . - Next, if the
controller 15 switches to the second control mode because a contactless operation mode file is attached to the activated application program, thecontroller 15 advances the process to step S2 and improves the sensitivity of the capacitive sensors in thedisplay unit 5 by raising the output gain of the capacitive sensors by a given amount, for example. Thecontroller 15 then advances the process to step S3. - In other words, since contactless operations are conducted in a state where an operating object is not made to contact the
display unit 5, capacitance values detected by the capacitive sensors will be detected at lower values than those during direct touch operations. For this reason, upon switching to the second control mode, thecontroller 15 attempts to improve the sensitivity of the capacitive sensors by raising the output gain of the capacitive sensors by a given amount as discussed above, for example. - Thus, changes in capacitance values corresponding to contactless operations can be detected with high sensitivity, making it possible to accurately detect the contactless operation position, etc.
- Next, in step S3, the
controller 15 functions as theapplication execution controller 24 and reads the contactless operation mode file attached to the activated application program. Then, thecontroller 15 functions as thecontactless operation enabler 25 and thecontactless operation detector 20, loads the contactless operation mode file thus read into the contactless operationmode file memory 17, and advances the process to step S4. - The contactless operation mode file is a file of contactless operation mode data expressing capacitance map transition patterns for individual contactless operation modes used to specify execution of a desired process in the currently activated application program.
- For this reason, by loading the contactless operation mode file into the contactless operation
mode file memory 17, capacitance map transition patterns for individual contactless operation modes used to specify execution of a desired process in the currently activated application program become held in the contactless operationmode file memory 17. - In step S4, the
controller 15 functions as theread controller 19 and thereby acquires the capacitance values of all capacitive sensors in thedisplay unit 5 as a capacitance map, and then advances the process to step S5. - In step S5, the
controller 15 compares the capacitance values in the acquired capacitance map to the direct touch threshold level indicated by the broken line inFIG. 4A which is stored in thethreshold level memory 18. In so doing, thecontroller 15 determines whether or not there exist capacitance values from among the acquired capacitance values that are equal to or greater than the direct touch threshold level. - The existence of capacitance values equal to or greater than the direct touch threshold level among the capacitance values in the acquired capacitance map means that a direct touch operation has been performed by the user. For this reason, the
controller 15 advances the process to step S6 and conducts processing for direct touch operations. - In other words, upon advancing the process to step S6, the
controller 15 functions as theread controller 19 and compares the capacitance values in the acquired capacitance map to the direct touch threshold level stored in thethreshold level memory 18. In so doing, thecontroller 15 creates two-dimensional coordinate information for thedisplay unit 5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation), and advances the process to step S7. - The
controller 15 functions as thetouch driver 21 and stores the two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in theoperation history register 16. - Next, upon advancing the process to step S7, the
controller 15 functions as theoperation detector 26 of thewindow manager 23 and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when thecontroller 15 functioned as thetouch driver 21, as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in theoperation history register 16. Thecontroller 15 then advances the process to step S8. - In step S8, the
controller 15 functions as theapplication execution controller 24, controls execution of a process from among respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position, and advances the process to step S9. - In step S9, the
controller 15 functions as theread controller 19 as discussed above and determines whether or not an application program is currently running. In so doing, thecontroller 15 determines whether or not to continue detecting capacitance values. - Then, in cases where it is determined that acquisition of capacitance values has been terminated, such as when an operation for ending the currently active application is performed, for example, the
controller 15 ends all processing in the flowchart illustrated inFIG. 3 . - Meanwhile, in cases where it is determined to continue detecting capacitance values, such as when an application program is currently running, the
controller 15 returns the process to step S4 and once again acquires the capacitance values of all capacitive sensors in the display unit. While an application program is running, thecontroller 15 controls repeated execution of action for acquiring the capacitance values of all capacitive sensors in this way at given time intervals. - Meanwhile, in step S5, a lack of capacitance values equal to or greater than the direct touch threshold level among the capacitance values in the acquired capacitance map means there is a possibility that a contactless operation has been performed by the user. For this reason, the
controller 15 advances the process to step S10 in the case where the existence of capacitance values equal to or greater than the direct touch threshold level is not detected. - In step S10, the
controller 15 functions as theread controller 19 and compares the acquired capacitance values to the contactless threshold level indicated by the chain line inFIG. 4B which is stored in thethreshold level memory 18. In so doing, thecontroller 15 determines whether or not there exist capacitance values among the acquired capacitance values which are equal to or greater than the contactless threshold level. - In the case where capacitance values equal to or greater than the contactless threshold level does not exist, the
controller 15 discards the acquired capacitance values by treating them as noise, returns the process to step S4 via step S9, and once again acquires the capacitance values of all capacitive sensors in thedisplay unit 5. - In contrast, the existence of capacitance values equal to or greater than the contactless threshold level means that a contactless operation like that illustrated in
FIG. 4C , for example, has been performed by the user. For this reason, thecontroller 15 advances the process to step S31 inFIG. 7 . - At this point, the respective processing in step S5 and step S10 will be once again explained in detail.
- Exemplary capacitance values for all capacitive sensors detected during a direct touch operation are illustrated in
FIG. 5 . During a direct touch operation, only the capacitance value of the sensor where the direct touch operation is centered and the capacitance values of the immediately adjacent sensors are detected as large capacitance values, as indicated by being enclosed in bold lines inFIG. 5 . - Exemplary capacitance values for all capacitive sensors detected during a contactless operation are illustrated in
FIG. 6 . During a contactless operation, capacitance values that have changed to intermediate values over a wider range than during a direct touch operation are detected, as indicated by being enclosed in bold lines inFIG. 6 . - In the case of the mobile phone in this embodiment, there are a total of two threshold levels: a direct touch threshold level indicated by the broken line in
FIGS. 4A and 4B , and a contactless threshold level indicated by the chain line inFIGS. 4A and 4B , which is set at a lower level than the level of the direct touch threshold level. - Upon acquiring all capacitance values from the
display unit 5, in step S5 thecontroller 15 compares the capacitance values of respective sensors in the acquired capacitance map to the direct touch threshold level indicated by the broken line inFIGS. 4A and 4B . Then, thecontroller 15 determines whether or not there exist capacitance values among the capacitance values from all sensors in thedisplay unit 5 which are equal to or greater than the direct touch threshold level. - The case where capacitance values equal to or greater than the direct touch threshold level do exist as illustrated in
FIG. 4A means that a direct touch operation is being conducted on thedisplay unit 5 by the user. For this reason, thecontroller 15 controls the storage of two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in theoperation history register 16. - In contrast, in the case where is it determined that capacitance values equal to or greater than the direct touch threshold level do not exist among the capacitance values from all sensors in the
display unit 5, thecontroller 15 advances the process to step S10 and respectively compares the capacitance values of respective sensors in the acquired capacitance map to the contactless threshold level indicated by the chain line inFIGS. 4A and 4B . Then, in this step S10, thecontroller 15 determines whether or not there exist capacitance values among the respective capacitance values detected by all sensors in thedisplay unit 5 which are less than the direct touch threshold level but also equal to or greater than the contactless threshold level. - The existence of sensors having capacitance values which are lower than the direct touch threshold level but also equal to or greater than the contactless threshold level means that a contactless operation is being conducted on the
display unit 5 by the user. For this reason, thecontroller 15 controls the storage of the acquired capacitance map in the storage area for contactless operations in theoperation history register 16 and advances the process to step S31 of the flowchart inFIG. 7 hereinafter described. - When the
controller 15 advances the process to step S31 of the flowchart inFIG. 7 due to determining in step S10 that there exist capacitance values among the capacitance values in the capacitance map acquired from thedisplay unit 5 which are lower than the direct touch threshold level indicated by the broken line inFIG. 4B but also equal to or greater than the direct touch threshold level indicated by the chain line inFIG. 4B , thecontroller 15 next functions as thecontactless operation detector 20 and determines whether or not a given number or more capacitance maps are being stored in the storage area for contactless operations in the operation history register 16 such that a contactless operation pattern) can be detected. - The
controller 15 then advances the process to step S32 in the case where it is determined that the given number or more capacitance maps are being stored, while advancing the process to step S9 of the flowchart inFIG. 3 in the case where it is determined that the given number or more capacitance maps are not being stored. - In the case of the mobile phone in this embodiment, capacitance maps are acquired from the
display unit 5 at given time intervals while an application program is running, as discussed above. For this reason, upon advancing the process to step S9 of the flowchart inFIG. 3 due to determining that the given number or more capacitance maps are not being stored, thecontroller 15 determines whether or not to continue detecting capacitance maps by determining whether or not an application program is currently running. - Then, in the case where it is determined that acquisition of the next capacitance map has terminated, such when an operation for ending the currently active application program is given, for example, the
controller 15 ends all processing in the respective flowcharts illustrated inFIGS. 3 and 7 . - In contrast, if the
controller 15 determines to continue detecting capacitance maps, thecontroller 15 returns the process to step S4 and from thedisplay unit 5 acquires a capacitance map expressing the capacitance values of all capacitive sensors in thedisplay unit 5 as discussed earlier. Then, if there exist capacitance values among the capacitance values in the acquired capacitance map which are lower than the direct touch threshold level but also equal to or greater than the contactless threshold level, thecontroller 15 controls storage of the capacitance map in the storage area for direct contactless operations in theoperation history register 16. - In this way, in the case where a number of capacitance maps enabling detection of a contactless operation pattern (contactless operation mode) or more are not being stored in the operation history register 16 while an application program is running, the
controller 15 returns the process to step S4 via step S9 of the flowchart inFIG. 3 , and once again acquires a capacitance map and controls its storage in the storage area for contactless operations in theoperation history register 16. In so doing, capacitance maps corresponding to a contactless operation are successively stored in the storage area for contactless operations in theoperation history register 16. - At this point, in the case where the activated application program does support contactless operations, the
controller 15 is configured to load contactless operation mode data, which indicates capacitance map transition patterns for individual contactless operation modes of contactless operations used by the application program, into the contactless operationmode file memory 17 as the contactless operation mode file, as discussed earlier. - For this reason, upon advancing the process to step S32 due to determining that a number of capacitance maps enabling detection of a contactless operation mode are being stored in the storage area for contactless operations in the
operation history register 16, thecontroller 15 functions as thecontactless operation detector 20 to specify for each capacitance map the placement positions on thedisplay unit 5 of the capacitive sensors that detected capacitance values which are lower than the direct touch threshold level but also equal to or greater than the contactless threshold level, on the basis of the respective capacitance maps stored in theoperation history register 16. Then, thecontroller 15 detects shifts in the capacitive sensors specified for the individual capacitance maps as a capacitance map transition pattern corresponding to a contactless operation. - An exemplary transition pattern across respective capacitance maps during a contactless operation is illustrated in
FIGS. 8A to 8C . The example illustrated inFIGS. 8A to 8C is an example for the case where a contactless operation is made on thedisplay unit 5 of a mobile phone from left to right along the short axis of thedisplay unit 5. - In the case where such a contactless operation is conducted, the positions of the capacitive sensors that detect capacitance values equal to or greater than the contactless threshold level move from left to right along the short axis of the
display unit 5, as indicated by being respectively enclosed in bold lines inFIGS. 8A to 8C . - In step S32, the
controller 15 detects the shift in the positions of the capacitive sensors that detect capacitance values equal to or greater than the contactless threshold level due to such a contactless operation as a capacitance map transition pattern corresponding to a contactless operation. - Next, the
controller 15 advances the process to step S33 and functions as thecontactless operation detector 20 to compare the capacitance map transition pattern detected with respect to a contactless operation in step S32 to capacitance map transition patterns expressed by contactless operation mode data in a contactless operation mode file loaded into the contactless operationmode file memory 17, and specify a contactless operation mode (contactless operation gesture). Then, thecontroller 15 detects two-dimensional coordinate information corresponding to the specified contactless operation. - In other words, during a direct touch operation, capacitance values change over an extremely tight area on the
display unit 5, as illustrated inFIG. 5 . For this reason, two-dimensional coordinates on thedisplay unit 5 corresponding to a direct touch operation can also be accurately detected. - In contrast, during a contactless operation, capacitance values change over a wider area than during a direct touch operation, as illustrated in
FIG. 6 andFIGS. 8A to 8C . For this reason, it is difficult to accurately detect two-dimensional coordinates on thedisplay unit 5 corresponding to a contactless operation. - For this reason, in the case of the mobile phone in this embodiment, contactless operation mode data in a contactless operation mode file is made up of data expressing contactless operation modes (gestures) corresponding to capacitance map transition patterns such as “right-to-left”, “left-to-right”, “top-to-bottom”, and “bottom-to-top”, as well as data expressing two-dimensional coordinates corresponding to contactless operation modes.
- Stated differently, since it is difficult to accurately detect two-dimensional coordinates on the
display unit 5 corresponding to a contactless operation position, individual capacitance map transition patterns and individual contactless operation modes are associated with each other in advance, while in addition, two-dimensional coordinates for specifying processes in an application program are respectively associated with the individual contactless operation modes. The result is taken to be the contactless operation mode data. - In step S33, the
controller 15 detects from the contactless operationmode file memory 17 the contactless operation mode data of a transition pattern that matches the capacitance map transition pattern detected with respect to a contactless operation. As discussed above, the contactless operation mode data is made up of data expressing contactless operation modes corresponding to capacitance map transition patterns, and data expressing two-dimensional coordinates corresponding to the contactless operation modes for specifying processes of the application program. - For this reason, by detecting from the contactless operation
mode file memory 17 the contactless operation mode data of a transition pattern that matches a capacitance map transition pattern detected with respect to a contactless operation, thecontroller 15 detects a contactless operation mode and two-dimensional coordinates corresponding to this contactless operation mode. - Next, the
controller 15 advances the process to step S34 and functions as theoperation detector 26 to recognize a contactless operation mode and two-dimensional coordinate information corresponding to this contactless operation mode. Then, in step S34, thecontroller 15 functions as theapplication execution controller 24 to control execution of a process in an application program on the basis of the recognized contactless operation mode and two-dimensional coordinate information corresponding to this contactless operation mode. - In so doing, execution of the process that corresponds to the user's contactless operation from among the various processes in the currently active application program is controlled by the
controller 15 on the basis of the currently active application program. - Upon controlling execution of a process in an application program corresponding to a contactless operation in this way, the
controller 15 advances the process to step S9 inFIG. 3 and determines whether or not to continue detecting capacitance maps by determining whether or not an application program is currently running, as discussed earlier. - Then, if the
controller 15 determines to continue detecting capacitance maps, thecontroller 15 returns the process to step S4 and acquires from thedisplay unit 5 a capacitance map expressing the capacitance values of all sensors on thedisplay unit 5, as discussed earlier. - In contrast, if the
controller 15 determines that acquisition of the next capacitance map has terminated, such when an operation for ending the currently active application program is given, for example, thecontroller 15 ends all processing in the respective flowcharts illustrated inFIGS. 3 and 7 . - As the foregoing explanation demonstrates, the mobile phone in this embodiment is provided with a
display unit 5 which forms a capacitive touch panel, and in addition, a direct touch threshold level as well as a contactless threshold level are set. Furthermore, two-dimensional coordinate information expressing a direct touch operation position is created on the basis of the capacitance values of the respective sensors in thedisplay unit 5 during a direct touch operation and the direct touch threshold level, and the execution of information processing corresponding to the created two-dimensional coordinate information expressing a direct touch operation position from among various information processing in the currently active application program is controlled. - Also, two-dimensional coordinate information expressing a contactless operation mode is created on the basis of the capacitance values of the respective sensors in the
display unit 5 during a contactless operation and the contactless threshold level, and the execution of information processing corresponding to the created two-dimensional coordinate information expressing a contactless operation mode from among various information processing in the currently active application program is controlled. - In so doing, information processing based on a given application program can be made operable by contactless operations, and new operational modes for application programs can be provided.
- Hereinafter, exemplary information processing action by various application programs in a mobile phone of this embodiment will be described.
- First, a mobile phone of this embodiment is configured to switch to what is called a hands-free telephony mode if a contactless operation is conducted so as to horizontally swipe a finger across the
display unit 5 as illustrated inFIG. 9 in the case where an incoming telephone call or an incoming video phone call is received while the mobile phone is in a locked mode that cancels most operations. - Specifically, if an incoming telephone call or an incoming video phone call is received while in the locked mode, the
controller 15 functions as thecontactless operation enabler 25 to load a contactless operation mode file of contactless operation mode data expressing contactless operation modes (gestures) of contactless operations and two-dimensional coordinates corresponding to the contactless operation modes used by a telephone application program into the contactless operationmode file memory 17. - Also, the
controller 15 acquires the capacitance maps discussed earlier from thedisplay unit 5 and controls their successive storage in theoperation history register 16. - Next, the
controller 15 compares the transition pattern of the capacitance maps stored in the operation history register 16 to the capacitance map transition patterns for individual contactless operation modes expressed by the respective contactless operation mode files loaded into the contactless operationmode file memory 17. Then, thecontroller 15 detects contactless operation mode data expressing a transition pattern matching the transition pattern of the respective capacitance maps acquired from the operation history register 16 from among the capacitance map transition patterns for individual contactless operation modes expressed by the respective contactless operation mode data stored in the contactless operationmode file memory 17. - The contactless operation mode data expresses a contactless operation pattern corresponding to a transition pattern of the respective capacitance maps, and two-dimensional coordinate information corresponding to this contactless operation pattern. The
controller 15 functions as theoperation detector 26 of thewindow manager 23 to detect a contactless operation pattern and two-dimensional coordinate information corresponding to this contactless operation pattern on the basis of the detected contactless operation mode data. - The detected contactless operation pattern is for a contactless operation instructing the switch to a hands-free telephony mode in the case of a contactless operation pattern wherein a finger is moved so as to horizontally swipe across the
display unit 5. Thus, thecontroller 15 functions as theapplication execution controller 24 to switch to a hands-free telephony mode on the basis of a telephone application program, and controls driving of themicrophone unit 4 which picks up telephone transmitter audio and thesecond speaker unit 3 b, which is an external speaker unit for obtaining the acoustic output of telephone receiver audio, while also controlling communication by thecommunication circuit 2 so as to transmit and receive telephony audio obtained thereby. - Similarly, in the case of detecting the above-discussed contactless operation wherein a finger is moved so as to horizontally swipe across the
display unit 5 when an incoming video phone call is received, thecontroller 15 switches to a hands-free telephony mode on the basis of a video phone application program, and controls driving of themicrophone unit 4 which picks up telephone transmitter audio, thesecond speaker unit 3 b, which is an external speaker unit for obtaining the acoustic output of telephone receiver audio, thesecond camera unit 8 b for obtaining transmitter images, and thedisplay unit 5 for displaying receiver images, while also controlling communication by thecommunication circuit 2 so as to transmit and receive telephony audio and images obtained thereby. - Thus, when an incoming telephone call or an incoming video phone call is received, telephone or video phone communication in what is called a hands-free telephony mode can be made possible simply by conducting a contactless operation so as to horizontally swipe a finger across the
display unit 5. - Next, a mobile phone of this embodiment is configured such that if a direct touch operation is conducted so as to slide an unlock slider displayed on the
display unit 5 in an unlock direction as illustrated inFIG. 9 in the case where an incoming telephone call is received while in the locked mode, the mobile phone recognizes that an off-hook operation has been performed, cancels the locked mode, and switches to a normal telephony mode that conducts telephony using thefirst speaker unit 3 a and themicrophone unit 4. - Specifically, if the
controller 15 detects a direct touch operation that slides the unlock slider in the unlock direction on the basis of a transition pattern of respective capacitance maps stored in the operation history register 16 as discussed earlier in the case where an incoming telephone call is received while in the locked mode, thecontroller 15 controls the driving of thefirst speaker unit 3 a, which is an internal speaker unit, and themicrophone unit 4, and switches to a normal telephony mode. - Thus, if a direct touch operation that slides an unlock slider in the unlock direction is conducted when an incoming telephone call is received, the lock is released and telephony can be conducted in a normal telephony state.
- Next, a mobile phone of this embodiment is configured such that if a direct touch operation is conducted so as to slide an unlock slider displayed on the
display unit 5 in an unlock direction as illustrated inFIG. 9 in the case where an incoming video phone call is received while in the locked mode, the mobile phone cancels the locked mode and switches to a hands-free telephony mode. - Specifically, if the
controller 15 detects a direct touch operation that slides an unlock slider displayed on thedisplay unit 5 in an unlock direction on the basis of a transition pattern of respective capacitance maps stored in the operation history register 16 as discussed earlier in the case where an incoming video phone call is received while in the locked mode, thecontroller 15 cancels the locked mode while also switching to a hands-free telephony mode on the basis of a video phone application program. - Then, the
controller 15 controls driving of themicrophone unit 4 which picks up telephone transmitter audio, thesecond speaker unit 3 b, which is an external speaker unit for obtaining the acoustic output of telephone receiver audio, thesecond camera unit 8 b for obtaining transmitter images, and thedisplay unit 5 for displaying receiver images, while also controlling communication by thecommunication circuit 2 so as to transmit and receive telephony audio and images obtained thereby. - Thus, when an incoming video phone call is received, video phone communication in what is called a hands-free telephony mode can be made possible simply by conducting a direct touch operation that slides an unlock slider displayed on the
display unit 5 in an unlock direction. - Next, if an email is received, the
controller 15 activates an email management program stored in thememory 14 and notifies the user of the receipt of the email. - In this example, when the email management program is activated, a contactless operation that horizontally swipes a finger across the
display unit 5 becomes a contactless operation instructing display of the body text of the received email. - For this reason, when the
controller 15 detects a contactless operation that horizontally swipes a finger across thedisplay unit 5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager 23, thecontroller 15 controls display of the body text of the received email on thedisplay unit 5 in accordance with the contactless operation. - Also, in this example, when the email management program is activated, a contactless operation that moves a finger up or down along the long axis of the
display unit 5 becomes a contactless operation instructing scrolling of the body text of the displayed email. - For this reason, when the
controller 15 detects a contactless operation that moves a finger over thedisplay unit 5 along the long axis of thedisplay unit 5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager 23, thecontroller 15 scrolls the body text of the email being displayed in accordance with the contactless operation. - Thus, when an email is received, the body text of the received email can be displayed on the
display unit 5 simply by conducting a contactless operation so as to horizontally swipe a finger across thedisplay unit 5. Also, when the body text of an email is being displayed on thedisplay unit 5, by simply conducting a contactless operation that moves a finger over thedisplay unit 5 along the long axis of thedisplay unit 5, the body text of the displayed email can be scrolled in accordance with the contactless operation along the long axis of thedisplay unit 5. - Next, if a given
hardware key 6 is operated, thecontroller 15 switches to a locked mode that cancels most key operations on the basis of a lock/unlock application program stored in thememory 14. - During such a locked mode, a contactless operation that horizontally swipes a finger across the
display unit 5 becomes a contactless operation instructing the mobile phone to accept a key operation and switch from the locked mode to an unlocked mode which conducts information processing corresponding to the key operation. - Also, during the unlocked mode, a contactless operation that horizontally swipes a finger across the
display unit 5 becomes a contactless operation instructing the mobile phone to switch from the unlocked mode to a locked mode which cancels most key operations. - Also, during the locked mode, a direct touch operation that slides an unlock slider on the
display unit 5 in an unlock direction becomes a direct touch operation instructing the mobile phone to switch from the locked mode to the unlocked mode. - Also, during the unlocked mode, a direct touch operation that slides an unlock slider on the
display unit 5 in a lock direction becomes a direct touch operation instructing the mobile phone to switch from the unlocked mode to the locked mode. - During the locked mode, if the
controller 15 detects a contactless operation that horizontally swipes a finger across thedisplay unit 5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager 23, thecontroller 15 switches to the unlocked mode in accordance with the contactless operation, accepts a key operation by the user, and controls execution of information processing corresponding to the key operation. - Also, during the unlocked mode, if the
controller 15 detects a contactless operation that horizontally swipes a finger across thedisplay unit 5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager 23, thecontroller 15 switches to the locked mode and cancels most key operations. - Thus, during a locked mode, the mobile phone can be switched to an unlocked mode simply by conducting a contactless operation that horizontally swipes a finger across the
display unit 5, while during an unlocked mode, the mobile phone can be switched to a locked mode simply by conducting a contactless operation that horizontally swipes a finger across thedisplay unit 5. - Also, during the locked mode, if the
controller 15 detects a direct touch operation that slides an unlock slider on thedisplay unit 5 in an unlock direction on the basis of the created two-dimensional coordinate information while functioning as thewindow manager 23, thecontroller 15 switches to an unlocked mode in accordance with the direct touch operation, accepts a key operation by the user, and controls execution of information processing corresponding to the key operation. - Also, during the unlocked mode, if the
controller 15 detects a direct touch operation that slides an unlock slider on thedisplay unit 5 in a lock direction on the basis of the created two-dimensional coordinate information while functioning as thewindow manager 23, thecontroller 15 switches to a locked mode and cancels most key operations. - Thus, during a locked mode, the mobile phone can be switched to an unlocked mode simply by conducting a direct touch operation that slides an unlock slider on the
display unit 5 in an unlock direction, while during an unlocked mode, the mobile phone can be switched to a locked mode simply by conducting a direct touch operation that slides an unlock slider on thedisplay unit 5 in a lock direction. - Next, in the case where a given web page is being displayed on the
display unit 5 on the basis of a web browsing program stored in thememory 14, a contactless operation that causes a finger to revolve in a clockwise direction over thedisplay unit 5 becomes a contactless operation instructing the mobile phone to enlarge the web page being displayed. - Also, in the above case where a web page is being displayed on the
display unit 5, a contactless operation that causes a finger to revolve in a counter-clockwise direction over thedisplay unit 5 becomes a contactless operation instructing the mobile phone to reduce the web page being displayed. - Also, in the above case where a web page is being displayed on the
display unit 5, a contactless operation that moves a finger over thedisplay unit 5 upwards along the long axis of thedisplay unit 5 becomes a contactless operation instructing the mobile phone to scroll up the web page being displayed. - Also, in the above case where a web page is being displayed on the
display unit 5, a contactless operation that moves a finger over thedisplay unit 5 downwards along the long axis of thedisplay unit 5 becomes a contactless operation instructing the mobile phone to scroll down the web page being displayed. - For this reason, if the
controller 15 detects a contactless operation that causes a finger over thedisplay unit 5 to revolve in a clockwise direction on the basis of the created two-dimensional coordinate information while functioning as thewindow manager 23, thecontroller 15 enlarges the web page being displayed at gradually larger display magnifications in accordance with the contactless operation. - Also, if the
controller 15 detects a contactless operation that causes a finger over thedisplay unit 5 to revolve in a counter-clockwise direction on the basis of the created two-dimensional coordinate information while functioning as thewindow manager 23, thecontroller 15 reduces the web page being displayed at gradually smaller display magnifications in accordance with the contactless operation. - Also, if the
controller 15 detects a contactless operation that moves a finger over thedisplay unit 5 upwards along the long axis of thedisplay unit 5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager 23, thecontroller 15 scrolls up the web page being displayed in accordance with the contactless operation. - Also, if the
controller 15 detects a contactless operation that moves a finger over thedisplay unit 5 downwards along the long axis of thedisplay unit 5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager 23, thecontroller 15 scrolls down the web page being displayed in accordance with the contactless operation. - Thus, when a web page is being displayed, the web page can be enlarged or reduced by conducting a contactless operation that causes a finger over the
display unit 5 to revolve in a clockwise or counter-clockwise direction. - Also, when a web page is being displayed, the web page can be scrolled up or scrolled down by conducting a contactless operation that moves a finger over the
display unit 5 upwards or downwards along the long axis of thedisplay unit 5. - In the foregoing embodiment, the mobile phone is configured to detect both contactless operations and direct touch operations, but the mobile phone may also detect only contactless operations and control execution of information processing in application program on the basis of detected contactless operations. Even in such cases, the same advantages as discussed above can be obtained.
- Although the foregoing embodiment was an example wherein the present disclosure was applied to a mobile phone, the present disclosure may also be applied to other electronic devices besides a mobile phone, such as a PHS phone (PHS: Personal Handyphone System), a PDA (PDA: Personal Digital Assistant), a digital camera, a digital video camera, a portable game console, or a notebook computer, for example. Moreover, the same advantages as those of the foregoing embodiment can still be obtained in any case.
- Lastly, the foregoing embodiment is one example of the present disclosure. Accordingly, the present disclosure is not limited to the foregoing embodiment, and various modifications, combinations, and other embodiment may occur depending on design or other factors while remaining within the scope of the claims of the present disclosure or their equivalents. This is naturally understood by those skilled in the art.
Claims (19)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/588,300 US20130093719A1 (en) | 2011-10-17 | 2012-08-17 | Information processing apparatus |
US16/058,368 US10877609B2 (en) | 2011-10-17 | 2018-08-08 | Information processing apparatus configured to control an application based on an input mode supported by the application |
US17/124,527 US11416097B2 (en) | 2011-10-17 | 2020-12-17 | Information processing apparatus configured to control an application based on an input mode supported by the application |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201161547783P | 2011-10-17 | 2011-10-17 | |
US13/588,300 US20130093719A1 (en) | 2011-10-17 | 2012-08-17 | Information processing apparatus |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/058,368 Continuation US10877609B2 (en) | 2011-10-17 | 2018-08-08 | Information processing apparatus configured to control an application based on an input mode supported by the application |
Publications (1)
Publication Number | Publication Date |
---|---|
US20130093719A1 true US20130093719A1 (en) | 2013-04-18 |
Family
ID=47008288
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/588,300 Abandoned US20130093719A1 (en) | 2011-10-17 | 2012-08-17 | Information processing apparatus |
US16/058,368 Active US10877609B2 (en) | 2011-10-17 | 2018-08-08 | Information processing apparatus configured to control an application based on an input mode supported by the application |
US17/124,527 Active US11416097B2 (en) | 2011-10-17 | 2020-12-17 | Information processing apparatus configured to control an application based on an input mode supported by the application |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/058,368 Active US10877609B2 (en) | 2011-10-17 | 2018-08-08 | Information processing apparatus configured to control an application based on an input mode supported by the application |
US17/124,527 Active US11416097B2 (en) | 2011-10-17 | 2020-12-17 | Information processing apparatus configured to control an application based on an input mode supported by the application |
Country Status (3)
Country | Link |
---|---|
US (3) | US20130093719A1 (en) |
EP (2) | EP3193245A1 (en) |
CN (1) | CN103076965B (en) |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015045909A (en) * | 2013-08-27 | 2015-03-12 | 株式会社東海理化電機製作所 | Operation input device |
US20170060335A1 (en) * | 2015-08-28 | 2017-03-02 | Mstar Semiconductor, Inc. | Method and associated controller for adaptively adjusting touch control threshold |
US9609124B2 (en) * | 2015-05-01 | 2017-03-28 | Vyng, Inc. | Methods and systems for management of video and ring tones among mobile devices |
WO2017071456A1 (en) * | 2015-10-28 | 2017-05-04 | 努比亚技术有限公司 | Terminal processing method, terminal, and storage medium |
US10104228B2 (en) | 2015-05-01 | 2018-10-16 | Vyng, Inc. | Methods and systems for management of media content associated with message context on mobile computing devices |
US10877609B2 (en) * | 2011-10-17 | 2020-12-29 | Sony Corporation | Information processing apparatus configured to control an application based on an input mode supported by the application |
US10931819B2 (en) | 2015-05-01 | 2021-02-23 | Vyng, Inc. | Management of media content associated with a user of a mobile computing device |
US10938984B2 (en) | 2015-05-01 | 2021-03-02 | Vyng, Inc. | Management of media content associated with ending a call on mobile computing devices |
US10944863B2 (en) | 2015-05-01 | 2021-03-09 | Vyng, Inc. | Management of media content derived from natural language processing on mobile computing devices |
US10951755B2 (en) | 2015-05-01 | 2021-03-16 | Vyng, Inc. | Management of media content for caller IDs on mobile computing devices |
US10965809B2 (en) | 2015-05-01 | 2021-03-30 | Vyng, Inc. | Management of media content associated with a call participant on mobile computing devices |
US10979558B2 (en) | 2015-05-01 | 2021-04-13 | Vyng, Inc. | Management of media content associated with time-sensitive offers on mobile computing devices |
US10979559B2 (en) | 2015-05-01 | 2021-04-13 | Vyng, Inc. | Management of calls on mobile computing devices based on call participants |
US11005990B2 (en) | 2015-05-01 | 2021-05-11 | Vyng, Inc. | Methods and systems for contact firewalls on mobile computing devices |
US11016573B2 (en) * | 2017-02-10 | 2021-05-25 | Panasonic Intellectual Property Management Co., Ltd. | Vehicular input apparatus |
US11368575B2 (en) | 2015-05-01 | 2022-06-21 | Vyng, Inc. | Management of calls and media content associated with a caller on mobile computing devices |
US11381679B2 (en) | 2015-05-01 | 2022-07-05 | Vyng, Inc. | Management of media content associated with call context on mobile computing devices |
US11394824B2 (en) | 2015-05-01 | 2022-07-19 | Vyng Inc. | Adjusting presentation on smart phone lockscreen of visual content associated with metadata of incoming call |
US11394823B2 (en) | 2015-05-01 | 2022-07-19 | Vyng Inc. | Configuring business application for utilization of sender controlled media service |
US11394821B2 (en) | 2015-05-01 | 2022-07-19 | Vyng Inc. | Curated search of entities from dial pad selections |
US11394822B2 (en) | 2015-05-01 | 2022-07-19 | Vyng Inc. | Incentivising answering call in smartphone lockscreen |
US11451659B2 (en) | 2015-05-01 | 2022-09-20 | Vyng Inc. | Dynamic rewardable activity value determination and allocation |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014135067A1 (en) * | 2013-10-22 | 2014-09-12 | 中兴通讯股份有限公司 | Mode switching method and device |
CN106940608B (en) * | 2017-03-07 | 2020-06-16 | Oppo广东移动通信有限公司 | Display screen control method, display screen and electronic equipment |
CN111506191A (en) | 2020-04-13 | 2020-08-07 | 维沃移动通信有限公司 | Control method and electronic device |
Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050097570A1 (en) * | 2003-11-03 | 2005-05-05 | Bomers Florian U. | Universal computer input event translator |
US20050229117A1 (en) * | 2002-02-08 | 2005-10-13 | Microsoft Corporation | Ink gestures |
US20060267958A1 (en) * | 2005-04-22 | 2006-11-30 | Microsoft Corporation | Touch Input Programmatical Interfaces |
US20080046425A1 (en) * | 2006-08-15 | 2008-02-21 | N-Trig Ltd. | Gesture detection for a digitizer |
US20080055269A1 (en) * | 2006-09-06 | 2008-03-06 | Lemay Stephen O | Portable Electronic Device for Instant Messaging |
US20090309851A1 (en) * | 2008-06-17 | 2009-12-17 | Jeffrey Traer Bernstein | Capacitive Sensor Panel Having Dynamically Reconfigurable Sensor Size and Shape |
US20100107067A1 (en) * | 2008-10-27 | 2010-04-29 | Nokia Corporation | Input on touch based user interfaces |
US20100295781A1 (en) * | 2009-05-22 | 2010-11-25 | Rachid Alameh | Electronic Device with Sensing Assembly and Method for Interpreting Consecutive Gestures |
US20100321334A1 (en) * | 2009-06-18 | 2010-12-23 | Wacom Co., Ltd. | Pointer detection apparatus and pointer detection method |
US20110083110A1 (en) * | 2009-10-07 | 2011-04-07 | Research In Motion Limited | Touch-sensitive display and method of control |
US20110093778A1 (en) * | 2009-10-20 | 2011-04-21 | Lg Electronics Inc. | Mobile terminal and controlling method thereof |
Family Cites Families (37)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7760187B2 (en) * | 2004-07-30 | 2010-07-20 | Apple Inc. | Visual expander |
US20050162402A1 (en) * | 2004-01-27 | 2005-07-28 | Watanachote Susornpol J. | Methods of interacting with a computer using a finger(s) touch sensing input device with visual feedback |
US7519223B2 (en) * | 2004-06-28 | 2009-04-14 | Microsoft Corporation | Recognizing gestures and using gestures for interacting with software applications |
US7653883B2 (en) | 2004-07-30 | 2010-01-26 | Apple Inc. | Proximity detector in handheld device |
US7567240B2 (en) * | 2005-05-31 | 2009-07-28 | 3M Innovative Properties Company | Detection of and compensation for stray capacitance in capacitive touch sensors |
JP2008134764A (en) * | 2006-11-28 | 2008-06-12 | Smk Corp | Noncontact input device |
JP2008165575A (en) * | 2006-12-28 | 2008-07-17 | Mitsubishi Electric Corp | Touch panel device |
US7777732B2 (en) * | 2007-01-03 | 2010-08-17 | Apple Inc. | Multi-event input system |
US8780046B2 (en) * | 2007-09-10 | 2014-07-15 | Blackberry Limited | Device and method for application navigation enhancement on a handheld electronic device |
JP5104150B2 (en) * | 2007-09-14 | 2012-12-19 | オムロン株式会社 | Detection apparatus and method, and program |
EP2104024B1 (en) * | 2008-03-20 | 2018-05-02 | LG Electronics Inc. | Portable terminal capable of sensing proximity touch and method for controlling screen using the same |
US20100229090A1 (en) * | 2009-03-05 | 2010-09-09 | Next Holdings Limited | Systems and Methods for Interacting With Touch Displays Using Single-Touch and Multi-Touch Gestures |
US9317140B2 (en) * | 2009-03-30 | 2016-04-19 | Microsoft Technology Licensing, Llc | Method of making a multi-touch input device for detecting touch on a curved surface |
US8982051B2 (en) * | 2009-03-30 | 2015-03-17 | Microsoft Technology Licensing, Llc | Detecting touch on a surface |
JP2010244302A (en) * | 2009-04-06 | 2010-10-28 | Sony Corp | Input device and input processing method |
US9024886B2 (en) * | 2009-04-14 | 2015-05-05 | Japan Display Inc. | Touch-panel device |
EP2425322A4 (en) * | 2009-04-30 | 2013-11-13 | Synaptics Inc | Control circuitry and method |
JP5295008B2 (en) * | 2009-06-18 | 2013-09-18 | 株式会社ワコム | Indicator detection device |
US9323398B2 (en) * | 2009-07-10 | 2016-04-26 | Apple Inc. | Touch and hover sensing |
JP2011053971A (en) * | 2009-09-02 | 2011-03-17 | Sony Corp | Apparatus, method and program for processing information |
JP4824805B2 (en) | 2009-09-28 | 2011-11-30 | 京セラ株式会社 | Mobile terminal device |
JP5493979B2 (en) | 2010-02-19 | 2014-05-14 | 富士通モバイルコミュニケーションズ株式会社 | Mobile device |
JP4727753B1 (en) * | 2010-03-04 | 2011-07-20 | Smk株式会社 | Capacitive touch panel |
KR101755394B1 (en) * | 2010-04-12 | 2017-07-11 | 삼성전자주식회사 | Method and apparatus for touch input in portable communication system |
US20140253486A1 (en) * | 2010-04-23 | 2014-09-11 | Handscape Inc. | Method Using a Finger Above a Touchpad During a Time Window for Controlling a Computerized System |
US9310990B2 (en) * | 2010-04-26 | 2016-04-12 | Blackberry Limited | Portable electronic device and method of controlling same |
US8954099B2 (en) * | 2010-06-16 | 2015-02-10 | Qualcomm Incorporated | Layout design of proximity sensors to enable shortcuts |
US20120054670A1 (en) * | 2010-08-27 | 2012-03-01 | Nokia Corporation | Apparatus and method for scrolling displayed information |
US9766718B2 (en) * | 2011-02-28 | 2017-09-19 | Blackberry Limited | Electronic device and method of displaying information in response to input |
KR101691478B1 (en) * | 2011-02-09 | 2016-12-30 | 삼성전자주식회사 | Operation Method based on multiple input And Portable device supporting the same |
CN102184067A (en) * | 2011-05-19 | 2011-09-14 | 苏州大学 | Non-contact type small size keyboard |
TWI469025B (en) * | 2011-08-25 | 2015-01-11 | Touch panel and its dynamic drive control method | |
US20130093719A1 (en) * | 2011-10-17 | 2013-04-18 | Sony Mobile Communications Japan, Inc. | Information processing apparatus |
US9582178B2 (en) * | 2011-11-07 | 2017-02-28 | Immersion Corporation | Systems and methods for multi-pressure interaction on touch-sensitive surfaces |
US8854335B2 (en) * | 2012-03-22 | 2014-10-07 | Eminent Electronic Technology Corp. Ltd. | Electronic apparatus having touch sensing ability and non-touch sensing ability and control method thereof |
KR102186393B1 (en) * | 2014-01-02 | 2020-12-03 | 삼성전자주식회사 | Method for processing input and an electronic device thereof |
US9898148B2 (en) * | 2015-05-27 | 2018-02-20 | Synaptics Incorporated | Capacitive stereoscopic image sensing |
-
2012
- 2012-08-17 US US13/588,300 patent/US20130093719A1/en not_active Abandoned
- 2012-08-30 EP EP17155041.1A patent/EP3193245A1/en not_active Ceased
- 2012-08-30 EP EP12182413.0A patent/EP2584439B1/en active Active
- 2012-09-27 CN CN201210365614.0A patent/CN103076965B/en active Active
-
2018
- 2018-08-08 US US16/058,368 patent/US10877609B2/en active Active
-
2020
- 2020-12-17 US US17/124,527 patent/US11416097B2/en active Active
Patent Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050229117A1 (en) * | 2002-02-08 | 2005-10-13 | Microsoft Corporation | Ink gestures |
US20050097570A1 (en) * | 2003-11-03 | 2005-05-05 | Bomers Florian U. | Universal computer input event translator |
US20060267958A1 (en) * | 2005-04-22 | 2006-11-30 | Microsoft Corporation | Touch Input Programmatical Interfaces |
US20080046425A1 (en) * | 2006-08-15 | 2008-02-21 | N-Trig Ltd. | Gesture detection for a digitizer |
US20080055269A1 (en) * | 2006-09-06 | 2008-03-06 | Lemay Stephen O | Portable Electronic Device for Instant Messaging |
US20090309851A1 (en) * | 2008-06-17 | 2009-12-17 | Jeffrey Traer Bernstein | Capacitive Sensor Panel Having Dynamically Reconfigurable Sensor Size and Shape |
US20100107067A1 (en) * | 2008-10-27 | 2010-04-29 | Nokia Corporation | Input on touch based user interfaces |
US20100295781A1 (en) * | 2009-05-22 | 2010-11-25 | Rachid Alameh | Electronic Device with Sensing Assembly and Method for Interpreting Consecutive Gestures |
US20100321334A1 (en) * | 2009-06-18 | 2010-12-23 | Wacom Co., Ltd. | Pointer detection apparatus and pointer detection method |
US20110083110A1 (en) * | 2009-10-07 | 2011-04-07 | Research In Motion Limited | Touch-sensitive display and method of control |
US20110093778A1 (en) * | 2009-10-20 | 2011-04-21 | Lg Electronics Inc. | Mobile terminal and controlling method thereof |
Cited By (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10877609B2 (en) * | 2011-10-17 | 2020-12-29 | Sony Corporation | Information processing apparatus configured to control an application based on an input mode supported by the application |
US11416097B2 (en) * | 2011-10-17 | 2022-08-16 | Sony Corporation | Information processing apparatus configured to control an application based on an input mode supported by the application |
JP2015045909A (en) * | 2013-08-27 | 2015-03-12 | 株式会社東海理化電機製作所 | Operation input device |
US10951755B2 (en) | 2015-05-01 | 2021-03-16 | Vyng, Inc. | Management of media content for caller IDs on mobile computing devices |
US10979558B2 (en) | 2015-05-01 | 2021-04-13 | Vyng, Inc. | Management of media content associated with time-sensitive offers on mobile computing devices |
US11936807B2 (en) | 2015-05-01 | 2024-03-19 | Digital Reef, Inc. | Dynamic rewardable activity value determination and allocation |
US10104228B2 (en) | 2015-05-01 | 2018-10-16 | Vyng, Inc. | Methods and systems for management of media content associated with message context on mobile computing devices |
US10511711B2 (en) | 2015-05-01 | 2019-12-17 | Vyng, Inc. | Methods and systems for management of media content associated with message context on mobile computing devices |
US11451659B2 (en) | 2015-05-01 | 2022-09-20 | Vyng Inc. | Dynamic rewardable activity value determination and allocation |
US10931819B2 (en) | 2015-05-01 | 2021-02-23 | Vyng, Inc. | Management of media content associated with a user of a mobile computing device |
US10938984B2 (en) | 2015-05-01 | 2021-03-02 | Vyng, Inc. | Management of media content associated with ending a call on mobile computing devices |
US10944863B2 (en) | 2015-05-01 | 2021-03-09 | Vyng, Inc. | Management of media content derived from natural language processing on mobile computing devices |
US9609124B2 (en) * | 2015-05-01 | 2017-03-28 | Vyng, Inc. | Methods and systems for management of video and ring tones among mobile devices |
US10965809B2 (en) | 2015-05-01 | 2021-03-30 | Vyng, Inc. | Management of media content associated with a call participant on mobile computing devices |
US9871902B2 (en) | 2015-05-01 | 2018-01-16 | Vyng, Inc. | Methods and systems for management of video and ring tones among mobile devices |
US10979559B2 (en) | 2015-05-01 | 2021-04-13 | Vyng, Inc. | Management of calls on mobile computing devices based on call participants |
US11005990B2 (en) | 2015-05-01 | 2021-05-11 | Vyng, Inc. | Methods and systems for contact firewalls on mobile computing devices |
US11394822B2 (en) | 2015-05-01 | 2022-07-19 | Vyng Inc. | Incentivising answering call in smartphone lockscreen |
US11368575B2 (en) | 2015-05-01 | 2022-06-21 | Vyng, Inc. | Management of calls and media content associated with a caller on mobile computing devices |
US11381679B2 (en) | 2015-05-01 | 2022-07-05 | Vyng, Inc. | Management of media content associated with call context on mobile computing devices |
US11394824B2 (en) | 2015-05-01 | 2022-07-19 | Vyng Inc. | Adjusting presentation on smart phone lockscreen of visual content associated with metadata of incoming call |
US11394823B2 (en) | 2015-05-01 | 2022-07-19 | Vyng Inc. | Configuring business application for utilization of sender controlled media service |
US11394821B2 (en) | 2015-05-01 | 2022-07-19 | Vyng Inc. | Curated search of entities from dial pad selections |
US20170060335A1 (en) * | 2015-08-28 | 2017-03-02 | Mstar Semiconductor, Inc. | Method and associated controller for adaptively adjusting touch control threshold |
US9939957B2 (en) * | 2015-08-28 | 2018-04-10 | Mstar Semiconductor, Inc. | Method and associated controller for adaptively adjusting touch control threshold |
WO2017071456A1 (en) * | 2015-10-28 | 2017-05-04 | 努比亚技术有限公司 | Terminal processing method, terminal, and storage medium |
US11016573B2 (en) * | 2017-02-10 | 2021-05-25 | Panasonic Intellectual Property Management Co., Ltd. | Vehicular input apparatus |
Also Published As
Publication number | Publication date |
---|---|
US20210117036A1 (en) | 2021-04-22 |
US10877609B2 (en) | 2020-12-29 |
CN103076965A (en) | 2013-05-01 |
EP2584439A2 (en) | 2013-04-24 |
US20190025958A1 (en) | 2019-01-24 |
EP2584439B1 (en) | 2020-01-08 |
EP3193245A1 (en) | 2017-07-19 |
US11416097B2 (en) | 2022-08-16 |
EP2584439A3 (en) | 2017-07-05 |
CN103076965B (en) | 2016-12-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11416097B2 (en) | Information processing apparatus configured to control an application based on an input mode supported by the application | |
US9753560B2 (en) | Input processing apparatus | |
EP2605117B1 (en) | Display processing device | |
RU2582854C2 (en) | Method and device for fast access to device functions | |
US8762893B2 (en) | Automatic derivation of analogous touch gestures from a user-defined gesture | |
JP5658765B2 (en) | Apparatus and method having multiple application display modes, including a mode with display resolution of another apparatus | |
RU2604990C2 (en) | Method of operating terminal based on multiple inputs and portable terminal supporting same | |
US20090146968A1 (en) | Input device, display device, input method, display method, and program | |
US20140270414A1 (en) | Auxiliary functionality control and fingerprint authentication based on a same user input | |
EP2081107A1 (en) | Electronic device capable of transferring object between two display units and controlling method thereof | |
US20100321323A1 (en) | Method and apparatus for reducing multi-touch input error in portable communication system | |
EP2575013A2 (en) | Pen system and method for performing input operations to mobile device via the same | |
CN102346592A (en) | Touch input transitions | |
US10248269B2 (en) | Information processing apparatus | |
JP2015007949A (en) | Display device, display controlling method, and computer program | |
US9658714B2 (en) | Electronic device, non-transitory storage medium, and control method for electronic device | |
EP3614239B1 (en) | Electronic device control in response to finger rotation upon fingerprint sensor and corresponding methods | |
US9298364B2 (en) | Mobile electronic device, screen control method, and storage medium strong screen control program | |
US9014762B2 (en) | Character input device, character input method, and character input program | |
US20130222334A1 (en) | Information processing apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SONY MOBILE COMMUNICATIONS JAPAN, INC., JAPAN Free format text: CHANGE OF NAME;ASSIGNOR:SONY ERICSSON MOBILE COMMUNICATIONS JAPAN, INC.;REEL/FRAME:034924/0228 Effective date: 20120308 Owner name: SONY MOBILE COMMUNICATIONS INC., JAPAN Free format text: CHANGE OF NAME;ASSIGNOR:SONY MOBILE COMMUNICATIONS JAPAN, INC.;REEL/FRAME:034924/0285 Effective date: 20130107 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION |
|
AS | Assignment |
Owner name: SONY CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SONY MOBILE COMMUNICATIONS, INC.;REEL/FRAME:048691/0134 Effective date: 20190325 |