JP2010204917A - Electronic apparatus, information processing system, method and program for controlling electronic apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an electronic apparatus having two display screens and high operability. <P>SOLUTION: The electronic apparatus 100 switches a mouse mode and a tablet mode of itself in response to pressing of a center key 242. In the mouse operation mode, the electronic apparatus 100 performs a mouse operation to a program for displaying an operation screen on a liquid crystal panel 140 in response to touching on a liquid crystal panel 240. In the tablet mode, the electronic apparatus 100 performs a program for displaying an operation screen on the liquid crystal panel 240 in response to touching on the liquid crystal panel 240. The electronic apparatus 100 stores an operation element to decide program behavior carried out in the tablet mode. The electronic apparatus 100 executes the program based on the stored operation element after resuming the tablet mode. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an electronic device, an information processing system, an electronic device control method, and an electronic device control program. In particular, the present invention relates to an electronic device or information processing system having two display screens, and a control method and control program for an electronic device having two display screens.

  At present, electronic devices such as personal computers and portable information terminals are widely used. Recently, electronic devices that display two screens are becoming popular.

  For example, a portable information terminal disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2000-172395) displays content on one of two screens. The portable information terminal also displays a content menu bar and slide bar on the other screen.

  Japanese Patent Application Laid-Open No. 2001-306291 discloses an information processing apparatus including a main display device and an auxiliary display device. This information processing apparatus normally displays both content and content additional information on the main display device. The information processing apparatus displays additional information on the sub-screen when displaying the content on the full screen.

  An electronic device disclosed in Patent Document 3 (Japanese Patent Laid-Open No. 2003-202948) has a main display unit and an auxiliary display unit, and displays a connection status of communication of the electronic device on the auxiliary display unit.

  The electronic device disclosed in Patent Document 4 (Japanese Patent Application Laid-Open No. 2003-216297) also includes a main display unit and an auxiliary display unit, similar to the electronic device described in Patent Document 3. The electronic device displays status information of the electronic device on the auxiliary display unit by a combination of a message by characters, a symbol, and a display color and / or blinking display of the auxiliary display unit.

  An information processing apparatus disclosed in Patent Document 5 (Japanese Patent Application Laid-Open No. 2004-5105) includes a main display, a main display controller, a display-integrated pointing device, and a controller dedicated to the pointing device. This information processing apparatus can perform system settings such as BIOS (Basic Input / Output System) by a pointing device before starting an OS (Operating System).

  Similarly to the information processing apparatus described in Patent Document 6 (Japanese Patent Application Laid-Open No. 2004-5212), a main display, a main display controller, a display-integrated pointing device, and a controller dedicated to the pointing device An information processing apparatus including the above is disclosed. The information processing apparatus disclosed in Patent Document 6 displays the display screen of the pointing device on the main display when the display screen of the pointing device is switched to prevent erroneous operation.

JP 2000-172395 A JP 2001-306291 A JP 2003-202948 A JP 2003-216297 A JP 2004-5105 A JP 2004-5212 A

  A sub-display capable of accepting external input as described in Patent Documents 5 and 6 can be used for various purposes.

  First, a user of an electronic device can use the sub-display in the same manner as a touch pad provided in many conventional notebook computers. That is, the user can use the sub display as a pointing device for moving the position of the cursor displayed on the main display, or giving an instruction to the application by clicking or dragging.

  Furthermore, when the operation screen of the application is displayed on the sub display as in the system setting screen of Patent Document 5, the user of the electronic device can operate the application by performing input on the sub display.

  In order to cause the sub-display to perform a plurality of types of operations, the electronic device switches the processing method of the input to the sub-display in accordance with key input or time passage. In a certain situation, the electronic device regards the input to the sub display as an instruction for moving the cursor on the main display or an operation instruction for an application whose execution screen is displayed on the main display. In another situation, the electronic device regards the input to the sub display as a direct instruction to the operation screen displayed on the sub display.

  By the way, Patent Documents 5 and 6 do not consider in detail how to control an electronic device when switching an input processing method. An electronic device that simply uses a conventional technique may cause a decrease in operability when the processing method is switched. For example, there may be a situation in which the operation of an application that operates in response to an input to the sub display is inconsistent before and after the processing method is switched.

  The present invention has been made to solve the above problems, and an object thereof is to provide an electronic device or an information processing system having two display screens and high operability. Furthermore, an object of the present invention is to provide an electronic device control method and a control program for improving the operability of an electronic device having two display screens.

  According to one aspect of the present invention, an electronic device is a display that displays a first screen, a display-integrated tablet that displays a second screen and can accept input from the outside, A storage unit, and a control unit that controls the display, the tablet, and the storage unit, the control unit switching a first operation mode and a second operation mode of the control unit, and executing the program And the execution means executes the program in response to an input to the tablet in the first operation mode, displays an image created by the executed program on the display, and in the second operation mode, the tablet The program is executed in response to the input to the image, the image created by the executed program is displayed on the tablet, and executed in the second operation mode. Storing the operating element that determines the operation of the program in a storage unit, after the resumed second operation mode, executing a program based on the stored operating element.

  Preferably, the operation element includes an input history representing a history of input to the tablet in the second operation mode.

  Preferably, the execution means counts an elapsed time from a predetermined event that has occurred in the second operation mode, the operation element includes the counted elapsed time, and the execution means includes the first operation mode during the first operation mode. Stop counting the elapsed time.

  More preferably, the execution means restarts the stopped counting operation in response to a start or end event of input to the tablet in the second operation mode.

  More preferably, the execution means recounts the elapsed time from the initial value when restarting the stopped count operation.

  More preferably, the execution means resets the elapsed time counted according to the transition from the second operation mode to the first operation mode.

  More preferably, the execution means executes a predetermined process by a program when the counted elapsed time exceeds a threshold value.

  More preferably, the execution means executes a predetermined process when the counted elapsed time from the input to the tablet in the second operation mode exceeds a threshold value.

  More preferably, the execution means recognizes the history of input to the tablet in the second operation mode when the counted elapsed time from the input to the tablet in the second operation mode exceeds a threshold.

  Preferably, the program includes a first program and a second program, and the execution means executes the first program in response to an input to the tablet in the first operation mode, and the second operation mode. The second program is executed in response to an input to the tablet.

  More preferably, the execution means includes a first execution means and a second execution means, and the first execution means executes the first program in response to an input to the tablet in the first operation mode. The second execution means executes the second program in response to an input to the tablet in the second operation mode.

  According to another aspect of the present invention, an information processing system including a first information processing unit and a second information processing unit, wherein the first information processing unit includes a display that displays a first screen; A first interface unit for exchanging data with the second information processing unit; and a first control unit for controlling the display and the first interface unit. The first control unit stores the first program A second execution unit for displaying an image generated by the executed first program and displaying an image generated by the executed first program on the display; A display-integrated tablet that can be received, a second interface unit that exchanges data with the first information processing unit, a tablet, and a second interface Second control means for controlling the second control means, wherein the second control means executes a second program, mode switching means for switching between the first operation mode and the second operation mode of the second control means. Second execution means for generating a command of the first program in response to an input to the tablet and controlling the second interface unit in the first operation mode. The command is transmitted to the first information processing unit, the second program is executed in response to the input to the tablet in the second operation mode, and the image created by the executed second program is displayed on the tablet. The operation element for determining the operation of the second program is stored in the storage device in the information processing system, and the second program is executed based on the stored operation element after the second operation mode is resumed. That.

  According to still another aspect of the present invention, a display that displays a first screen, a display-integrated tablet that displays a second screen and can accept external input, and an execution that executes a program And a step of switching between the first operation mode and the second operation mode of the electronic device, and in the first operation mode, by the execution means according to an input to the tablet Executing a program and displaying an image created by the executed program on a display; storing an operation element for determining an operation of the program executed in the second operation mode in a storage unit; In the operation mode, the program is executed by the execution means according to the input to the tablet, and the images created by the executed program are tabbed. And a step of displaying the Tsu bets, executing a program in the second mode of operation, after the resumed second mode of operation, comprising the step of executing a program based on the stored operating element.

  According to still another aspect of the present invention, control of an electronic device having a display that displays a first screen and a display-integrated tablet that displays a second screen and can accept external input. A step of switching between the first operation mode and the second operation mode of the electronic device and executing the program according to an input to the tablet in the first operation mode, and creating the executed program A step of displaying an image on the display, a step of storing an operation element for determining an operation of the program to be executed in the second operation mode in the storage unit, and a program in accordance with an input to the tablet in the second operation mode And displaying on the tablet an image created by the executed program, and in the second operation mode. Executing the program after the restart to a second operation mode, and executing the program based on the stored operating element.

  The electronic device (or information processing system) according to the present invention has a first operation mode and a second operation mode that can be switched to each other. The electronic device stores an operation element for determining the operation of the program executed in the second operation mode in the storage unit, and executes the program based on the stored operation element after the second operation mode is resumed. As a result, according to the present invention, it is possible to provide an electronic device or an information processing system that improves the operability of an electronic device having two display screens. Or according to this invention, the control method and control program of an electronic device which improve the operativity of the electronic device which has two display screens can be provided.

It is the schematic which shows the external appearance of an electronic device. It is a block diagram showing the hardware constitutions of an electronic device. It is the figure which showed the structure of the liquid crystal panel, and the peripheral circuit of the said liquid crystal panel. It is sectional drawing of a liquid crystal panel and a backlight. It is the figure which showed the timing chart at the time of operating an optical sensor circuit. It is sectional drawing which showed the structure which a photodiode receives the light from a backlight in the case of a scan. It is the figure which showed schematic structure of the command. It is a figure for demonstrating the command of classification "000". FIG. 10 is a diagram for explaining a command of type “001”. It is a figure for demonstrating the command of classification "010". FIG. 10 is a diagram for explaining a command of type “011”. It is a figure for demonstrating the command of classification "100". FIG. 10 is a diagram for explaining a command of type “101”. It is the figure which showed schematic structure of the response data. It is the figure which showed the image (scanned image) obtained by scanning a finger | toe. FIG. 4 is a circuit diagram of an optical sensor built-in liquid crystal panel different from that shown in FIG. 3. It is sectional drawing which showed the structure which a photodiode receives external light in the case of a scan. It is a block diagram showing the hardware constitutions of the modification of an electronic device. It is a figure which shows the functional structure of an electronic device in a block diagram format. It is a figure for demonstrating the screen which an electronic device displays in each of mouse mode and tablet mode. It is a figure which shows the specific example of a mouse | mouth screen. It is a figure which shows the specific example of a home menu screen. It is a transition diagram of the screen displayed on a liquid crystal panel in tablet mode. It is a figure for demonstrating operation | movement of the electronic device at the time of switching to mouse mode and tablet mode. It is the figure which showed typically the operation mode at the time of normal starting. It is the figure which showed typically the operation mode at the time of a return. It is a figure for demonstrating operation | movement of the electronic device at the time of tablet mode. It is a figure which shows an example of a character input screen. It is a 1st figure for demonstrating operation | movement of the electronic device at the time of utilization of a handwritten character input pad. It is a 2nd figure for demonstrating operation | movement of the electronic device at the time of use of a handwritten character input pad. It is a figure for demonstrating operation | movement of the electronic device when a text box is full. It is a figure which shows an example of an illustration input screen. It is a figure which shows an example of a calculator screen. It is a figure which shows an example of an internet screen. It is a figure which shows an example of the internet screen containing a scroll bar. It is a figure which shows an example of a dictionary selection screen. It is a figure which shows an example of the dictionary selection screen containing a scroll bar. It is a figure for demonstrating the inconvenience which may arise at the time of operation | movement of a handwritten character input application. It is a figure for demonstrating the count operation | movement of the electronic device shown in FIG. It is a figure for demonstrating the handwriting input operation | movement which concerns on this Embodiment. It is a figure for demonstrating a 1st time count. It is a figure for demonstrating a 2nd time count. It is a figure for demonstrating a 3rd time count. It is a figure for demonstrating a 4th time count. It is a figure which shows the flow of the process which an electronic device performs in a flowchart format. It is a figure which shows the flow of a process of mouse | mouth mode operation | movement in a flowchart format. It is a figure which shows the flow of a process of tablet mode operation | movement in the flowchart format. It is a figure which shows the flow of a process of mode switching (mouse mode to tablet mode) operation | movement in a flowchart format. It is a figure which shows the flow of the 1st process of mode switching (tablet mode to mouse mode) operation | movement in a flowchart format. It is a figure which shows the flow of the 2nd process of mode switching (tablet mode to mouse mode) operation | movement in a flowchart format. It is a figure which shows the flow of the 1st sub application operation | movement at the time of tablet mode in a flowchart format. It is a figure which shows the flow of the 2nd sub application operation | movement at the time of tablet mode in a flowchart format. It is a figure which shows the flow of the 3rd sub application operation | movement at the time of tablet mode in a flowchart format. It is a figure which shows the flow of the 4th sub application operation | movement at the time of tablet mode in a flowchart format.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<Appearance of electronic equipment>
FIG. 1 is a diagram illustrating an appearance of electronic device 100 according to the present embodiment. Referring to FIG. 1, electronic device 100 includes a first housing 100A and a second housing 100B.

  The first casing 100A and the second casing 100B are foldably connected by a hinge 100C. The first housing 100A includes a liquid crystal panel 140 with a built-in optical sensor. The second housing 100B includes an optical sensor built-in liquid crystal panel 240. As described above, the electronic device 100 includes two optical sensor built-in liquid crystal panels. The second casing 100B further includes a left click key 241, a center key 242, and a right click key 243.

  In the present embodiment, electronic device 100 is configured as a laptop personal computer. The electronic device 100 may be configured as a device having a display function, such as a PDA (Personal Digital Assistant), a mobile phone, or an electronic dictionary.

<About hardware configuration>
Next, an aspect of a specific configuration of the electronic device 100 will be described with reference to FIG. FIG. 2 is a block diagram illustrating a hardware configuration of electronic device 100.

  Electronic device 100 includes a first unit 1001 and a second unit 1002. The second unit 1002 is detachably connected to the first unit 1001 from the electronic device 100. The first unit 1001 includes a main body device 101 and a display device 102. The second unit 1002 includes a display device 103 and a main device 104.

  The first housing 100A includes a display device 102. Second housing 100B includes main device 101. The second housing 100B includes a second unit 1002.

(About the first unit)
The main unit 101 includes a CPU (Central Processing Unit) 110, an HDD (Hard Disc Drive) 170, a RAM (Random Access Memory) 171, a ROM (Read-Only Memory) 172, a memory card reader / writer 173, an external device Communication unit 174, microphone 175, speaker 176, operation key 177, power switch 191, power circuit 192, power detection unit 193, USB (Universal Serial Bus) connector 194, antenna 195, LAN ( Local Area Network) connector 196. Each component (110, 171 to 177, 193) is mutually connected by a data bus DB1. A memory card 1731 is attached to the memory card reader / writer 173.

  CPU 110 executes a program. The operation key 177 receives an instruction input from the user of the electronic device 100. The HDD 170 is a storage device that can write and read data. However, the HDD 170 is an example of such a storage device. The electronic device 100 may use a storage device such as a flash memory instead of the HDD 170. The RAM 171 stores data generated by the execution of the program by the CPU 110 or data input via the operation keys 177 in a volatile manner. The ROM 172 stores data in a nonvolatile manner. The ROM 172 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory.

  The external communication unit 174 communicates with other electronic devices. Specifically, the external communication unit 174 communicates with, for example, the second unit 1002 via the USB connector 194. The external communication unit 174 performs wireless communication with the second unit 1002 via the antenna 195, for example. Further, the external communication unit 174 performs wired communication with other electronic devices via the LAN connector 196.

  Note that the main device 101 may communicate with other electronic devices by wireless communication other than Bluetooth (registered trademark). For example, the external communication unit 174 may perform wireless communication with other electronic devices connected to the LAN via a wireless LAN antenna (not shown). Alternatively, wireless communication may be performed with another electronic device via an infrared port (not shown).

The power switch 191 is a switch for starting up the electronic device 100.
When the power switch 191 is turned on, the power supply circuit 192 supplies power to each component connected to the data bus DB1 and the display device 102 via the power detection unit 193. When the power switch 191 is turned on, the power circuit 192 supplies power to the external communication unit 174 without going through the power detection unit 193.

  The power supply detection unit 193 detects the output from the power supply circuit 192. In addition, the power supply detection unit 193 sends information (for example, a voltage value and a current value) regarding the detected output to the CPU 110.

  The USB connector 194 is used to connect the first unit 1001 to the second unit 1002. Note that the main device 101 may include other USB connectors in addition to the USB connector 194.

  The first unit 1001 transmits data to the second unit 1002 via the USB connector 194. The first unit 1001 receives data from the second unit 1002 via the USB connector 194. Further, the first unit 1001 supplies power to the second unit 1002 via the USB connector 194. The first unit 1001 may supply the power from the power supply circuit 192 directly to the second unit 1002 without using the USB connector 194.

  The antenna 195 is used for communication according to the Bluetooth (registered trademark) standard between the first unit 1001 and another communication apparatus (for example, the second unit 1002). The LAN connector 196 is used to connect the electronic device 100 to the LAN.

  The display device 102 includes a driver 130, an optical sensor built-in liquid crystal panel 140 (hereinafter referred to as a liquid crystal panel 140), an internal IF 178, a backlight 179, and an image processing engine 180.

  The driver 130 is a drive circuit for driving the liquid crystal panel 140 and the backlight 179. Various drive circuits included in the driver 130 will be described later.

  The liquid crystal panel 140 is a device having a liquid crystal display function and a photosensor function. That is, the liquid crystal panel 140 can perform image display using liquid crystal and sensing using an optical sensor. Details of the liquid crystal panel 140 will be described later.

  An internal IF (Interface) 178 mediates exchange of data between the main device 101 and the display device 102.

  The backlight 179 is a light source disposed on the back surface of the liquid crystal panel 140. The backlight 179 irradiates the back surface with uniform light.

  The image processing engine 180 controls the operation of the liquid crystal panel 140 via the driver 130. Here, the control is performed based on various data sent from the main apparatus 101 via the internal IF 178. Note that the various data includes commands to be described later. Further, the image processing engine 180 processes data output from the liquid crystal panel 140 and sends the processed data to the main apparatus 101 via the internal IF 178. Further, the image processing engine 180 includes a driver control unit 181, a timer 182, and a signal processing unit 183.

  The driver control unit 181 controls the operation of the driver 130 by sending a control signal to the driver 130. In addition, the driver control unit 181 analyzes a command transmitted from the main device 101. Then, the driver control unit 181 sends a control signal based on the analysis result to the driver 130. Details of the operation of the driver 130 will be described later.

The timer 182 generates time information and sends the time information to the signal processing unit 183.
The signal processing unit 183 receives data output from the optical sensor. Here, since the data output from the optical sensor is analog data, the signal processing unit 183 first converts the analog data into digital data. Further, the signal processing unit 183 performs data processing on the digital data in accordance with the content of the command sent from the main device 101. Then, the signal processing unit 183 sends data (hereinafter referred to as response data) including data after the above data processing and time information acquired from the timer 182 to the main unit 101. The signal processing unit 183 includes a RAM (not shown) that can store a plurality of scan data, which will be described later, continuously.

  The command includes a sensing command for instructing sensing by the optical sensor. Details of the sensing command and the response data will be described later (FIGS. 7, 8, and 14).

  Note that the timer 182 is not necessarily provided in the image processing engine 180. For example, the timer 182 may be provided outside the image processing engine 180 in the display device 102. Alternatively, the timer 182 may be provided in the main body device 101. Further, the microphone 175 and the speaker 176 are not always provided in the electronic device 100, and may be configured so as not to include either or both of the microphone 175 and the speaker 176 depending on the embodiment of the electronic device 100.

  Here, the display device 102 includes a system liquid crystal. The system liquid crystal is a device obtained by integrally forming peripheral devices of the liquid crystal panel 140 on the glass substrate of the liquid crystal panel 140. In the present embodiment, the driver 130 (excluding the circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are integrally formed on the glass substrate of the liquid crystal panel 140. Note that the display device 102 is not necessarily configured using a system liquid crystal, and the driver 130 (excluding a circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are included in the glass substrate. Other substrates may be configured.

(About the second unit)
The second unit 1002 receives power supply from the first unit 1001. Specifically, the second unit 1002 is supplied with power from the power supply circuit 192 of the first unit 1001 by connecting a USB connector 294 described later and the USB connector 194 of the first unit 1001. However, the method of supplying power to the second unit 1002 is not limited to this. For example, the second unit 1001 may receive power directly from the power supply circuit 192.

  The main body device 104 includes a CPU 210, a RAM 271, a ROM 272, a timer 273, a left click key 241, a center key 242, a right click key 243, an external communication unit 274, a power supply detection unit 293, and a USB connector 294. And an antenna 295 and a signal intensity detector 297. Each component (210, 241-243, 271, 272, 273, 274, 293) is mutually connected by a data bus DB2.

  The CPU 210 executes a program. The RAM 271 stores data generated by the execution of the program by the CPU 210 in a volatile manner. The ROM 272 stores data in a nonvolatile manner. The ROM 272 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory.

  The external communication unit 274 communicates with other electronic devices. Specifically, the external communication unit 274 communicates with, for example, the first unit 1001 via the USB connector 294. The external communication unit 274 communicates with the first unit 1001 through the antenna 295, for example.

  Note that main device 104 may communicate with another electronic device (for example, first unit 1001) by wireless communication other than Bluetooth (registered trademark). For example, the external communication unit 274 may perform wireless communication with other electronic devices via an infrared port (not shown).

  The signal strength detection unit 297 detects the strength of the signal received via the antenna 295. Then, the signal strength detection unit 297 sends the detected strength to the external communication unit 274.

  The USB connector 294 is used to connect the second unit 1002 to the first unit 1001.

  The second unit 1002 transmits data to the first unit 1001 via the USB connector 294. The second unit 1002 receives data from the first unit 1001 via the USB connector 294. Furthermore, the second unit 1002 receives power supply from the first unit 1001 via the USB connector 294 as described above. The second unit 1002 stores the electric power supplied from the first unit 1001 in a battery (not shown).

  The antenna 295 is used for communication between the second unit 1002 and the first unit 1001, for example, in accordance with the Bluetooth (registered trademark) standard.

  The power detection unit 293 detects the power supplied via the USB connector 294. In addition, the power supply detection unit 293 sends information about the detected power to the CPU 210.

Further, the main device 104 may have a function of performing infrared communication.
The display device 103 includes a driver 230, an optical sensor built-in liquid crystal panel 240 (hereinafter referred to as “liquid crystal panel 240”), an internal IF 278, a backlight 279, and an image processing engine 280. The image processing engine 280 includes a driver control unit 281, a timer 282, and a signal processing unit 283.

  The display device 103 has the same configuration as the display device 102. That is, the driver 230, the liquid crystal panel 240, the internal IF 278, the backlight 279, and the image processing engine 280 have the same configuration as the driver 130, the liquid crystal panel 140, the internal IF 178, the backlight 179, and the image processing engine 180 in the display device 102. Have each. The driver control unit 281, the timer 282, and the signal processing unit 283 have the same configurations as the driver control unit 181, the timer 182, and the signal processing unit 183 in the display device 102. Therefore, description of each functional block included in display device 103 will not be repeated.

  By the way, the processing in the electronic device 100 is realized by each hardware and software executed by the CPU 110. Such software may be stored in advance in the ROM 172 or the HDD 170. The software may be stored in a memory card 1731 or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by the memory card reader / writer 173 or other reading device, or downloaded via the communication unit 174 or communication IF (not shown), and then temporarily stored in the ROM 172 or the HDD 170. Stored. The software is read from the ROM 172 or the HDD 170 by the CPU 110 and stored in the RAM 171 in the form of an executable program. CPU 110 executes the program.

  Each component constituting the main device 101 of the electronic device 100 shown in FIG. 2 is a general one. Therefore, it can be said that the essential part of the present invention is the software stored in the RAM 171, the ROM 172 or the HDD 170, the memory card 1731 and other storage media, or the software downloadable via the network. Since the hardware operation of main device 101 of electronic device 100 is well known, detailed description will not be repeated.

  The storage medium is not limited to a memory card, but is a CD-ROM, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile). Disc)), IC (Integrated Circuit) cards (excluding memory cards), optical cards, mask ROM, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory), flash ROM, and other semiconductor memories, etc. It may be a medium to be used.

  The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

<Configuration and drive of liquid crystal panel with built-in optical sensor>
Next, the configuration of the liquid crystal panel 140 and the configuration of peripheral circuits of the liquid crystal panel 140 will be described. FIG. 3 is a diagram showing a configuration of the liquid crystal panel 140 and peripheral circuits of the liquid crystal panel 140.

  Referring to FIG. 3, the liquid crystal panel 140 includes a pixel circuit 141, an optical sensor circuit 144, a scanning signal line Gi, a data signal line SRj, a data signal line SGj, a data signal line SBj, and a sensor signal line. SSj, sensor signal line SDj, read signal line RWi, and reset signal line RSi are included. Note that i is a natural number satisfying 1 ≦ i ≦ m, and j is a natural number satisfying 1 ≦ j ≦ n.

  2 includes a scanning signal line driving circuit 131, a data signal line driving circuit 132, an optical sensor driving circuit 133, a switch 134, as peripheral circuits of the liquid crystal panel 140. And an amplifier 135.

  The scanning signal line drive circuit 131 receives the control signal TC1 from the driver control unit 181 shown in FIG. The scanning signal line drive circuit 131 applies a predetermined voltage in order from the scanning signal line G1 to each scanning signal line (G1 to Gm) based on the control signal TC1. More specifically, the scanning signal line driving circuit 131 sequentially selects one scanning signal line from the scanning signal lines (G1 to Gm) per unit time, and a TFT (to be described later) with respect to the selected scanning signal line. A voltage capable of turning on the gate of the thin film transistor 142 is applied (hereinafter referred to as a high level voltage). Note that a low level voltage is applied to a scanning signal line that is not selected without applying a high level voltage.

  The data signal line driving circuit 132 receives image data (DR, DG, DB) from the driver control unit 181 shown in FIG. The data signal line driving circuit 132 applies a voltage corresponding to one row of image data to the 3n data signal lines (SR1 to SRn, SG1 to SGn, SB1 to SBn) for each unit time. Apply sequentially.

  Note that although a driving method called a so-called line-sequential method has been described here, the driving method is not limited to this.

  The pixel circuit 141 is a circuit for setting the luminance (transmittance) of one pixel. Further, m × n pixel circuits 141 are arranged in a matrix. More specifically, m pixel circuits 141 are arranged in the vertical direction in FIG. 3 and n pixel circuits in the horizontal direction.

  The pixel circuit 141 includes an R subpixel circuit 141r, a G subpixel circuit 141g, and a B subpixel circuit 141b. Each of these three circuits (141r, 141g, 141b) includes a TFT 142, a pair of electrode pairs 143 including a pixel electrode and a counter electrode, and a capacitor (not shown).

  In addition, CMOS (Complementary Metal Oxide Semiconductor) capable of forming n-type transistors and p-type transistors can be realized, and the movement speed of carriers (electrons or holes) is several hundreds compared to amorphous silicon thin film transistors (a-Si TFTs). For example, a polycrystalline silicon thin film transistor (p-Si TFT) is used as the TFT 142 in the display device 102 because it is twice as fast. Note that the TFT 142 will be described as an n-channel field effect transistor. However, the TFT 142 may be a p-type channel field effect transistor.

  The source of the TFT 142 in the R subpixel circuit 141r is connected to the data signal line SRj. The gate of the TFT 142 is connected to the scanning signal line Gi. Further, the drain of the TFT 142 is connected to the pixel electrode of the electrode pair 143. A liquid crystal is disposed between the pixel electrode and the counter electrode. The G sub-pixel circuit 141g and the B sub-pixel circuit 141b have the same configuration as the R sub-pixel circuit 141r except that the data signal line to which the source of each TFT 142 is connected is different. Therefore, description of these two circuits (141g, 141b) will not be repeated.

  Here, setting of luminance in the pixel circuit 141 will be described. First, the high level voltage is applied to the scanning signal line Gi. By the application of the high level voltage, the gate of the TFT 142 is turned on. In this manner, with the gate of the TFT 142 turned on, a specified voltage (voltage corresponding to image data for one pixel) is applied to each data signal line (SRj, SGj, SBj). Thereby, a voltage based on the designated voltage is applied to the pixel electrode. As a result, a potential difference is generated between the pixel electrode and the counter electrode. Based on this potential difference, the liquid crystal responds and the luminance of the pixel is set to a predetermined luminance. Note that the potential difference is held by the capacitor (auxiliary capacitor) (not shown) until the scanning signal line Gi is selected in the next frame period.

  The optical sensor drive circuit 133 receives the control signal TC2 from the driver control unit 181 shown in FIG.

  Then, the optical sensor drive circuit 133 sequentially selects one signal line from the reset signal lines (RS1 to RSm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, the voltage VDDR that is higher than usual is applied. Note that a voltage VSSR lower than the voltage applied to the selected reset signal line is kept applied to the unselected reset signal line. For example, the voltage VDDR may be set to 0V and the voltage VSSR may be set to −5V.

  Further, the optical sensor driving circuit 133 sequentially selects one signal line from the readout signal lines (RW1 to RWm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, a voltage VDD higher than usual is applied. Note that the voltage VSSR is applied to the read signal line that is not selected. For example, the value of VDD may be set to 8V.

  The timing for applying the voltage VDDR and the timing for applying the voltage VDD will be described later.

  The optical sensor circuit 144 includes a photodiode 145, a capacitor 146, and a TFT 147. In the following description, it is assumed that the TFT 147 is an n-type channel field effect transistor. However, the TFT 147 may be a p-type channel field effect transistor.

  The anode of the photodiode 145 is connected to the reset signal line RSi. On the other hand, the cathode of the photodiode 145 is connected to one electrode of the capacitor 146. The other electrode of the capacitor 146 is connected to the read signal line RWi. Hereinafter, a connection point between the photodiode 145 and the capacitor 146 is referred to as a node N.

  The gate of the TFT 147 is connected to the node N. The drain of the TFT 147 is connected to the sensor signal line SDj. Further, the source of the TFT 147 is connected to the sensor signal line SSj. Details of sensing using the optical sensor circuit 144 will be described later.

  The switch 134 is a switch provided to switch whether or not to apply a predetermined voltage to the sensor signal lines (SD1 to SDn). The switching operation of the switch 134 is performed by the optical sensor driving circuit 133. Note that the voltage applied to the sensor signal lines (SD1 to SDn) when the switch 134 is turned on will be described later.

  The amplifier 135 amplifies the voltage output from each sensor signal line (SS1 to SSn). The amplified voltage is sent to the signal processing unit 183 shown in FIG.

  The image processing engine 180 controls the timing for displaying an image on the liquid crystal panel 140 using the pixel circuit 141 and the timing for sensing using the optical sensor circuit 144.

  FIG. 4 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179. Referring to FIG. 4, liquid crystal panel 140 includes an active matrix substrate 151A, a counter substrate 151B, and a liquid crystal layer 152. The counter substrate 151B is disposed to face the active matrix substrate 151A. The liquid crystal layer 152 is sandwiched between the active matrix substrate 151A and the counter substrate 151B. The backlight 179 is disposed on the opposite side of the liquid crystal layer 152 with respect to the active matrix substrate 151A.

  The active matrix substrate 151 </ b> A includes a polarizing filter 161, a glass substrate 162, a pixel electrode 143 a that constitutes the electrode pair 143, a photodiode 145, a data signal line 157, and an alignment film 164. Further, although not shown in FIG. 4, the active matrix substrate 151A includes the capacitor 146, the TFT 147, the TFT 142, and the scanning signal line Gi shown in FIG.

  In the active matrix substrate 151A, the polarizing filter 161, the glass substrate 162, the pixel electrode 143a, and the alignment film 164 are arranged in this order from the backlight 179 side. The photodiode 145 and the data signal line 157 are formed on the liquid crystal layer 152 side of the glass substrate 162.

  The counter substrate 151B includes a polarizing filter 161, a glass substrate 162, a light shielding film 163, color filters (153r, 153g, 153b), a counter electrode 143b constituting the electrode pair 143, and an alignment film 164.

  In the counter substrate 151B, the alignment film 164, the counter electrode 143b, the color filters (153r, 153g, 153b), the glass substrate 162, and the polarizing filter 161 are arranged in this order from the liquid crystal layer 152 side. The light shielding film 163 is formed in the same layer as the color filters (153r, 153g, 153b).

  The color filter 153r is a filter that transmits light having a red wavelength. The color filter 153g is a filter that transmits light having a green wavelength. The color filter 153b is a filter that transmits light having a blue wavelength. Here, the photodiode 145 is arranged at a position facing the color filter 153b.

  The liquid crystal panel 140 displays an image by blocking external light or light emitted from a light source such as a backlight 179 or transmitting the light. Specifically, the liquid crystal panel 140 changes the direction of liquid crystal molecules in the liquid crystal layer 152 by applying a voltage between the pixel electrode 143a and the counter electrode 143b, thereby blocking or transmitting the light. However, since the light cannot be completely blocked by the liquid crystal alone, a polarizing filter 161 that transmits only light having a specific polarization direction is provided.

  Note that the position of the photodiode 145 is not limited to the above position, and may be provided at a position facing the color filter 153r or a position facing the color filter 153g.

  Here, the operation of the optical sensor circuit 144 will be described. FIG. 5 is a diagram showing a timing chart when the optical sensor circuit 144 is operated. In FIG. 5, a voltage VINT indicates a potential at the node N in the photosensor circuit 144. A voltage VPIX is an output voltage from the sensor signal line SSj shown in FIG. 3 and is a voltage before being amplified by the amplifier 135.

  The following description will be divided into a reset period for resetting the optical sensor circuit 144, a sensing period for sensing light using the optical sensor circuit 144, and a readout period for reading the sensed result.

  First, the reset period will be described. In the reset period, the voltage applied to the reset signal line RSi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDDR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR). As described above, by applying the high-level voltage to the reset signal line RSi, a current starts to flow in the forward direction (from the anode side to the cathode side) of the photodiode 145. As a result, the voltage VINT which is the potential of the node N has a value represented by the following expression (1). In Equation (1), the forward voltage drop amount in the photodiode 145 is Vf.

VINT = VSSR + | VDDR−VSSR | −Vf (1)
Therefore, the potential of the node N is a value smaller by Vf than the voltage VDDR as shown in FIG.

  Here, since the voltage VINT is not more than the threshold value for turning on the gate of the TFT 147, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change. Further, a difference corresponding to the voltage VINT occurs between the electrodes of the capacitor 146. For this reason, the capacitor 146 accumulates charges corresponding to the difference.

  Next, the sensing period will be described. In the sensing period following the reset period, the voltage applied to the reset signal line RSi instantaneously switches from the high level (voltage VDDR) to the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR).

  Thus, by changing the voltage applied to the reset signal line RSi to the low level, the potential of the node N becomes higher than the voltage of the reset signal line RSi and the voltage of the read signal line RWi. For this reason, in the photodiode 145, the voltage on the cathode side becomes higher than the voltage on the anode side. That is, the photodiode 145 is in a reverse bias state. In such a reverse bias state, when the photodiode 145 receives light from the light source, current starts to flow from the cathode side to the anode side of the photodiode 145. As a result, as shown in FIG. 5, the potential of the node N (that is, the voltage VINT) becomes lower with the passage of time.

  Since the voltage VINT continues to decrease in this way, the gate of the TFT 147 does not turn on. Therefore, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change.

  Next, the reading period will be described. In the readout period following the sensing period, the voltage applied to the reset signal line RSi is kept at the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDD). Here, the voltage VDD is higher than the voltage VDDR.

  Thus, by applying a high level voltage instantaneously to the read signal line RWi, the potential of the node N is raised through the capacitor 146 as shown in FIG. Note that the increase width of the potential of the node N is a value corresponding to the voltage applied to the read signal line RWi. Here, since the potential of the node N (that is, the voltage VINT) is raised to a threshold value that turns on the gate of the TFT 147, the gate of the TFT 147 is turned on.

  At this time, if a constant voltage is applied in advance to the sensor signal line SDj (see FIG. 3) connected to the drain side of the TFT 147, the sensor signal line SSj connected to the source side of the TFT 147 will cause the VPIX in FIG. As shown in the graph, a voltage corresponding to the potential of the node N is output.

  Here, when the amount of light received by the photodiode 145 (hereinafter referred to as the amount of received light) is small, the slope of the straight line shown in the VINT graph of FIG. 5 becomes gentle. As a result, the voltage VPIX is higher than when the amount of received light is large. As described above, the optical sensor circuit 144 changes the value of the voltage output to the sensor signal line SSj in accordance with the amount of light received by the photodiode 145.

  By the way, in the above, the operation | movement was demonstrated paying attention to one optical sensor circuit 144 among the m * n optical sensor circuits which exist. Below, operation | movement of each photosensor circuit in the liquid crystal panel 140 is demonstrated.

  First, the optical sensor drive circuit 133 applies a predetermined voltage to all n sensor signal lines (SD1 to SDn). Next, the photosensor drive circuit 133 applies a voltage VDDR that is higher than normal to the reset signal line RS1. The other reset signal lines (RS2 to RSm) and read signal lines (RW1 to RWm) are kept in a state where a low level voltage is applied. As a result, the n photosensor circuits in the first row in FIG. 3 enter the reset period described above. Thereafter, the n photosensor circuits in the first row enter a sensing period. Further, thereafter, the n photosensor circuits in the first row enter a reading period.

  Note that the timing at which a predetermined voltage is applied to all n sensor signal lines (SD1 to SDn) is not limited to the above timing, and may be any timing that is applied at least before the readout period. .

  When the readout period of the n photosensor circuits in the first row is completed, the photosensor drive circuit 133 applies a voltage VDDR that is higher than usual to the reset signal line RS2. That is, the reset period of the n photosensor circuits in the second row starts. When the reset period ends, the n photosensor circuits in the second row enter a sensing period, and thereafter enter a reading period.

  Thereafter, the processing described above is sequentially performed on the n photosensor circuits in the third row, the n photosensor circuits in the fourth row,..., The n photosensor circuits in the m row. As a result, the sensing result of the first row, the sensing result of the second row,..., The sensing result of the m-th row are output in this order from the sensor signal lines (SS1 to SSn).

  In the display device 102, sensing is performed for each row as described above, and a sensing result is output from the liquid crystal panel 140 for each row. For this reason, hereinafter, the data after the signal processing unit 183 performs the above-described data processing on the data regarding the voltage for m rows from the first row to the m-th row output from the liquid crystal panel 140, This is called “scan data”. That is, scan data refers to image data obtained by scanning a scan target (for example, a user's finger). An image displayed based on the scan data is referred to as a “scanned image”. Further, in the following, sensing is referred to as “scan”.

  Moreover, in the above, although the structure which scans using all the m * n photosensor circuits was mentioned as an example, it is not limited to this. Scanning may be performed on a partial region of the surface of the liquid crystal panel 140 using a photosensor circuit selected in advance.

  In the following, it is assumed that the electronic device 100 can take either of the two configurations. Further, switching between the components is assumed to be performed by a command sent from the main device 101 based on an input via the operation key 177 or the like. Note that when scanning is performed on a partial area on the surface of the liquid crystal panel 140, the image processing engine 180 sets a scan target area. The setting of the area may be configured to be specified by the user via the operation key 177.

  As described above, when scanning is performed on a partial region of the surface of the liquid crystal panel 140, there are the following modes of use for displaying an image. The first is a mode in which an image is displayed in a surface area other than the partial area (hereinafter referred to as a scan area). The second is a mode in which no image is displayed in the surface area other than the scan area. Which mode is used is based on a command sent from the main apparatus 101 to the image processing engine 180.

  FIG. 6 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179, showing a configuration in which the photodiode 145 receives light from the backlight 179 during scanning.

  Referring to FIG. 6, when the user's finger 900 is in contact with the surface of the liquid crystal panel 140, a part of the light emitted from the backlight 179 is part of the user's finger 900 (abbreviated in the contacted area). Reflected on the plane). The photodiode 145 receives the reflected light.

  Even in a region where the finger 900 is not in contact, part of the light emitted from the backlight 179 is reflected by the user's finger 900. Even in this case, the photodiode 145 receives the reflected light. However, since the finger 900 is not in contact with the surface of the liquid crystal panel 140 in the region, the amount of light received by the photodiode 145 is smaller than the region in which the finger 900 is in contact. Note that the photodiode 145 cannot receive most of the light emitted from the backlight 179 that does not reach the user's finger 900.

  Here, by turning on the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the amount of light reflected by the user's finger 900 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter, also referred to as a reflected image) obtained by reflecting light with the finger 900.

Note that examples of the scan object other than the finger 900 include a stylus.
By the way, in the present embodiment, a liquid crystal panel is described as an example of the display device of electronic device 100, but another panel such as an organic EL (Electro-Luminescence) panel is used instead of the liquid crystal panel. Also good.

<About data>
Next, commands exchanged between the first unit 1001 and the second unit 1002 and commands exchanged between the main device 101 and the display device 102 in the first unit 1001 will be described.

  FIG. 7 is a diagram showing a schematic configuration of a command. Referring to FIG. 7, the command includes header DA01, first field DA02, second field DA03, third field DA04, fourth field DA05, fifth field DA06, and spare data area DA07. including.

  FIG. 8 is a diagram for explaining a command of type “000” (that is, a sensing command). The CPU 110 sends a command of type “000” (hereinafter referred to as “first command”) from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 110 sends the first command from the main body device 101 to the display device 102. In the following, the case where the CPU 110 sends the first command from the main unit 101 of the first unit 1001 to the second unit 1002 will be described as an example.

  The CPU 110 writes the command type (“000”), the command transmission destination, and the like in the header DA01. The CPU 110 writes the value of the timing whose number is “1” in the first field DA02. The CPU 110 writes the value of the data type with the number “2” in the second field DA03. The CPU 110 writes the value of the reading method whose number is “3” in the third field DA04. The CPU 110 writes the value of the image gradation number “4” in the fourth field DA05. The CPU 110 writes the value of the resolution with the number “5” in the fifth field DA06.

  The first command in which “00” is set in the first field DA02 requests the image processing engine 280 to transmit scan data at that time. That is, the sensing first command requests transmission of scan data obtained by scanning using the optical sensor circuit of the liquid crystal panel 240 after the image processing engine 280 receives the first command. Further, the first command in which “01” is set in the first field DA02 requests transmission of scan data when the scan result is changed. Further, the first command in which “10” is set in the first field DA02 requests transmission of scan data at regular intervals.

  The first command in which “001” is set in the second field DA03 requests transmission of the coordinate value of the center coordinate in the partial image. In addition, the first command in which “010” is set in the second field DA03 requests transmission of only the partial image whose scan result has changed. Note that the change in the scan result indicates that the previous scan result is different from the current scan result. Furthermore, the first command in which “100” is set in the second field DA03 requests transmission of the entire image.

  Here, the “whole image” is an image generated by the image processing engine 280 based on the voltage output from each optical sensor circuit when scanning using m × n optical sensor circuits. . The “partial image” is a part of the entire image. The reason why the partial image is requested to transmit only the partial image whose scan result has changed will be described later.

  The coordinate value and the partial image or the entire image may be requested at the same time. Further, in the case of a configuration in which scanning is performed on a partial area on the surface of the liquid crystal panel 240, the entire image is an image corresponding to the area to be scanned.

  The first sensing command in which “00” is set in the third field DA04 requests to scan with the backlight 279 turned on. In addition, the first command in which “01” is set in the third field DA04 requests that the backlight 279 be turned off to perform scanning. Note that the configuration for scanning with the backlight 279 off is described later (FIG. 17). Furthermore, the first command in which “10” is set in the third field DA04 requests scanning using both reflection and transmission. Note that the combined use of reflection and transmission refers to scanning a scanning object by switching between a method of scanning with the backlight 279 turned on and a method of scanning with the backlight turned off.

  The first command in which “00” is set in the fourth field DA05 requests monochrome binary image data. Further, the first command in which “01” is set in the fourth field DA05 requests multi-gradation image data. Further, the first command in which “10” is set in the fourth field DA05 requests RGB color image data.

  The first command in which “0” is set in the fifth field DA06 requests image data with high resolution. The first command in which “1” is set in the fifth field DA06 requests image data with a low resolution.

  In addition to the data shown in FIG. 8, the first command includes designation of a region to be scanned (pixel region that drives the optical sensor circuit 144), timing for scanning, lighting timing of the backlight 179, and the like. Is described.

  Note that the image processing engine 280 analyzes the content of the first command, and sends back data (that is, response data) according to the analysis result to the main device 101.

  FIG. 9 is a diagram for explaining a command of type “001” (hereinafter referred to as “second command”). The CPU 110 sends a second command from the main unit 101 of the first unit 1001 to the second unit 1002.

  The CPU 110 writes the command type (“001”), the command transmission destination, and the like in the header DA01. The CPU 110 writes the value of the display request with the number “1” in the first field DA02. CPU 110 writes information on the number / type of number “2” in second field DA03. The CPU 110 writes the value of the display range whose number is “3” in the third field DA04. The CPU 110 writes information related to the image data with the number “4” in the fourth field DA05.

  The second command in which “001” is set in the first field DA02 requests the image processing engine 280 to display an image on the liquid crystal panel 240 (sub-screen). The second command in which “010” is set in the first field DA02 requests the image processing engine 280 to display an icon on the liquid crystal panel 240. Further, the second command in which “011” is set in the first field DA02 requests the image processing engine 280 to display the handwriting area on the liquid crystal panel 240.

  The second field DA03 stores the number of images to be displayed on the liquid crystal panel 240 and a number for designating the type of handwriting language. The image processing engine 280 performs processing according to the number of the images or the language type.

  The second command in which “01” is set in the third field DA04 requests the image processing engine 280 to designate the display range on the liquid crystal panel 240 by coordinates. Further, the second command in which “10” is set in the third field DA04 requests the image processing engine 280 to set the display range on the liquid crystal panel 240 to the entire display area.

  The fourth field DA05 stores image data to be displayed on the liquid crystal panel 240 and position information for displaying the image data. The image processing engine 280 performs processing for displaying the image data at a position specified by the position information.

  FIG. 10 is a diagram for explaining a command of type “010” (hereinafter referred to as “third command”). The CPU 110 sends the third command from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 210 sends a third command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPUs 110 and 210 write the command type (“010”), the command transmission destination, and the like in the header DA01. The CPUs 110 and 210 write the value of the OS (Operating System) processing request with the number “1” in the first field DA02. The CPUs 110 and 210 write the value of the OS information with the number “2” in the second field DA03.

  The third command in which “01” or “10” is set in the first field DA02 is transmitted from the second unit 1002 to the first unit 1001.

  The third command in which “01” is set in the first field DA02 requests the first unit 1001 to transmit information indicating the type of OS of the first unit 1001 (main device). The third command in which “10” is set in the first field DA02 requests the first unit 1001 to start the OS specified by the OS information.

  The third command in which “000”, “001”, or “010” is set in the second field DA03 is transmitted from the second unit 1002 to the first unit 1001.

  The third command in which “000” is set in the second field DA03 does not request the activation of the OS in the first unit 1001. The third command in which “001” is set in the second field DA03 indicates that the second unit 1002 has selected to start the first OS. Furthermore, the third command in which “010” is set in the second field DA03 indicates that the second unit 1002 has selected to start the second OS.

  FIG. 11 is a diagram for explaining a command of type “011” (hereinafter referred to as “fourth command”). The CPU 210 sends a fourth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPU 210 writes the command type (“011”), the command transmission destination, and the like in the header DA01. The CPU 210 writes information related to the activated application with the number “1” in the first field DA02. The CPU 210 writes the startup information with the number “2” in the second field DA03.

  In the first field DA02, information specifying an application to be activated in the first unit 1001 is stored. The second field DA03 stores information used at the time of activation setting and information used after activation.

  FIG. 12 is a diagram for explaining a command of type “100” (hereinafter referred to as “fifth command”). The CPU 210 sends the fifth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPU 210 writes the command type (“100”), the command transmission destination, and the like in the header DA01. The CPU 210 writes information related to the reception request with the number “1” in the first field DA02. The CPU 210 writes information relating to the number “2” in the second field DA03. The CPU 210 writes information related to the file with the number “3” in the third field DA04.

  The fifth command in which “01” is set in the first field DA02 requests the first unit 1001 to receive a file. In the second field DA03, the number of files transmitted from the second unit 1002 to the first unit 1001 is stored. Further, a file transmitted from the second unit 1002 to the first unit 1001 is stored in the third field DA04.

  FIG. 13 is a diagram for explaining a command of type “101” (hereinafter referred to as “sixth command”). The CPU 110 sends the sixth command from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 210 sends the sixth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPUs 110 and 210 write the command type (“101”), the command transmission destination, and the like in the header DA01. The CPUs 110 and 210 write the value of the communication type with the number “1” in the first field DA02. The CPUs 110 and 210 write the value of the connection destination with the number “2” in the second field DA03. The CPUs 110 and 210 write the value of the transfer destination with the number “3” in the third field DA04. The CPUs 110 and 210 write the value of the signal strength acquisition timing number “4” in the fourth field DA05.

  The sixth command in which “001” is set in the first field DA02 requests the counterpart device to perform infrared communication. The sixth command in which “010” is set in the first field DA02 requests the counterpart device to perform wireless communication by Bluetooth (registered trademark). Furthermore, the sixth command in which “011” is set in the first field DA02 requests the counterpart device to perform LAN communication.

  The sixth command in which “000” is set in the second field DA03 indicates that there is no information for designating the connection destination of communication.

  Further, the sixth command in which “001” is set in the second field DA03 is transmitted by the first unit 1001 to the device to which the first unit 1001 is connected. Such a sixth command requests transmission of information related to a device to which the first unit 1001 is connected.

  Further, the sixth command in which “010” is set in the second field DA03 is transmitted by the second unit 1002 to the first unit 1001 to which the second unit 1002 is connected. Such a sixth command requests transmission of information regarding the first unit 1001 to which the second unit 1002 is connected.

  Further, the sixth command in which “011” is set in the second field DA03 is transmitted by the second unit 1002 to the first unit 1001 to which the second unit 1002 is connected. Such a sixth command requests that information regarding the second unit 1002 be set as connection destination device information.

  Further, the sixth command in which “100” is set in the second field DA03 is transmitted by the first unit 1001 to a device to which the first unit 1001 is connected (for example, the second unit 1002). Such a sixth command requests that information regarding the first unit 1001 be set as connection destination device information.

  The sixth command in which “000” is set in the third field DA04 indicates that it does not have information specifying the transfer destination of data (for example, a file).

  The sixth command in which “001” is set in the third field DA04 is transmitted by the first unit 1001 to the data transfer destination device. Such a sixth command requests transmission of information relating to the data transfer destination device.

  Further, the sixth command in which “010” is set in the third field DA04 is transmitted by the second unit 1002 to the first unit 1001 that is the data transfer destination. Such a sixth command requests transmission of information related to the first unit 1001 of the data transfer destination.

  The sixth command in which “011” is set in the third field DA04 is transmitted by the second unit 1002 to the first unit 1001 that is the data transfer destination. Such a sixth command requests that information regarding the second unit 1002 be set as device information of the data transfer source.

  Furthermore, the sixth command in which “100” is set in the third field DA04 is transmitted by the first unit 1001 to the data transfer destination device (for example, the second unit 1002). Such a sixth command requests that information regarding the first unit 1001 be set as device information of the data transfer source.

  The sixth command in which “00”, “01”, “10”, or “11” is set in the fourth field DA05 is transmitted by the first unit 1001 to the second unit 1002.

  The sixth command in which “00” is set in the fourth field DA05 does not request the second unit 1002 to transmit data indicating the signal strength. In addition, the sixth command in which “01” is set in the fourth field DA05 requests the signal strength detection unit 297 to transmit data indicating the signal strength at that time. Furthermore, the sixth command in which “10” is set in the fourth field DA05 requests transmission of data indicating the signal strength when the signal strength is changed. Further, the sixth command in which “11” is set in the fourth field DA05 requests the transmission of data indicating the signal strength at regular intervals.

  FIG. 14 is a diagram showing a schematic configuration of response data. The response data is data corresponding to the content of the first command (sensing command).

  When the first command is transmitted from the main device 101 to the second unit 1002, the CPU 210 transmits response data from the display device 103 to the first unit 1001. When the first command is transmitted from the main device 101 to the display device 102 of the first unit 1001, the image processing engine 180 transmits response data from the image processing engine 180 to the main device 101. Hereinafter, a case where the first command is transmitted from the main device 101 to the second unit 1002 will be described as an example.

  Referring to FIG. 14, the response data includes a header data area DA11, a coordinate data area DA12, a time data area DA13, and an image data area DA14. Here, the value of the center coordinates of the partial image is written in the data area DA12 indicating the coordinates. The time information acquired from the timer 282 of the image processing engine 280 is written in the data area indicating the time. Further, image data (that is, scan data) after being processed by the image processing engine 280 is written in the data area indicating the image.

  FIG. 15 is a diagram illustrating an image obtained by scanning the finger 900 (that is, a scanned image). Referring to FIG. 15, an image of a region W1 surrounded by a thick solid line is an entire image, and an image of a region P1 surrounded by a broken line is a partial image. The center point C1 of the cross indicated by a thick line is the center coordinate.

  In the present embodiment, a pixel (that is, a predetermined gradation or a pixel having a photosensor circuit which is a rectangular region and whose output voltage from the sensor signal line SSj is equal to or higher than a predetermined value). An area including all of the pixels having a luminance equal to or higher than a predetermined luminance is set as a partial image area.

  The center coordinates are coordinates determined in consideration of the gradation of each pixel in the partial image area. Specifically, for each pixel in the partial image, the center coordinate is determined by performing a weighting process based on the gradation of the pixel and the distance between the pixel and the rectangular center point (that is, the centroid). Is done. That is, the center coordinates do not necessarily match the centroid of the partial image.

  However, the position of the center coordinates is not necessarily limited to the above, and the center coordinates may be the coordinates of the centroid or the coordinates near the centroid.

  When “001” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the center coordinate in the data area DA12 indicating the coordinates. In this case, the image processing engine 280 does not write image data in the data area DA14 indicating an image. After writing the value of the center coordinate, the image processing engine 280 sends response data including the value of the center coordinate to the main body device 104. The main device 104 sends response data including the value of the center coordinate to the main device 101 of the first unit 1001. As described above, when “001” is set in the data area indicating the data type, the first command does not request output of the image data but requests output of the value of the center coordinate.

  When “010” is set in the data area indicating the data type of the first command, the image processing engine 280 stores the image data of the partial image whose scan result has changed in the data area DA14 indicating the image. Write. In this case, the image processing engine 280 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 280 writes the image data of the partial image whose scan result has changed, and then sends response data including the image data of the partial image to the main body device 104. The main device 104 sends response data including image data of the partial image to the main device 101 of the first unit 1001. Thus, when “010” is set in the data area indicating the data type, the first command does not request the output of the value of the center coordinate, and the image data of the partial image whose scan result has changed. Request output.

  As described above, the reason for requesting transmission of only the partial image whose scan result has changed is that the scan data of the partial image area of the scan data is more important than the scan data of the other area. This is because the data is high, and the scan data in the region corresponding to the region of the partial image in the scan data is likely to change due to the contact state with the scan object such as the finger 900.

  When “011” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the center coordinate in the data area DA12 indicating the coordinate and also indicates the data indicating the image. The image data of the partial image whose scan result has changed is written in the area DA14. Thereafter, the image processing engine 280 sends response data including the value of the center coordinate and the image data of the partial image to the main body device 104. The main device 104 sends response data including the value of the center coordinate and the image data of the partial image to the main device 101 of the first unit 1001. Thus, when “011” is set in the data area indicating the data type, the first command outputs the value of the center coordinate and the output of the image data of the partial image whose scan result has changed. Request.

  Further, when “100” is set in the data area indicating the data type of the first command, the image processing engine 280 displays the entire image in the data area DA14 indicating the image of the response data shown in FIG. Write image data. In this case, the image processing engine 280 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 280 writes the image data of the entire image, and then sends response data including the image data of the entire image to the main body device 104. The main device 104 sends response data including image data of the entire image to the main device 101 of the first unit 1001. As described above, when “100” is set in the data area indicating the data type, the first command requests the output of the image data of the entire image without requesting the output of the center coordinate value. .

  If “101” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the central coordinate in the data area DA12 indicating the coordinate and also indicates the data indicating the image. The image data of the entire image is written in the area DA14. Thereafter, the image processing engine 280 sends response data including the value of the center coordinate and the image data of the entire image to the main body device 104. The main device 104 sends response data including the value of the center coordinate and the image data of the entire image to the main device 101 of the first unit 1001. As described above, when “101” is set in the data area indicating the data type, the first command requests the output of the value of the center coordinate and the output of the image data of the entire image.

<Regarding First Modification of Configuration>
By the way, the structure of the liquid crystal panel 140 is not limited to the structure shown in FIG. Hereinafter, a liquid crystal panel having a mode different from that in FIG. 3 will be described.

  FIG. 16 is a circuit diagram of a photosensor built-in liquid crystal panel 140A according to the different embodiment. Referring to FIG. 16, photosensor built-in liquid crystal panel 140A (hereinafter referred to as liquid crystal panel 140A) includes three photosensor circuits (144r, 144g, 144b) in one pixel. Thus, the liquid crystal panel 140A is different from the liquid crystal panel 140 including one photosensor circuit in one pixel in that the liquid crystal panel 140A includes three photosensor circuits (144r, 144g, and 144b) in one pixel. The configuration of the optical sensor circuit 144 is the same as that of each of the three optical sensor circuits (144r, 144g, 144b).

  Further, the three photodiodes (145r, 145g, 145b) in one pixel are arranged at positions facing the color filter 153r, the color filter 153g, and the color filter 153b, respectively. Therefore, the photodiode 145r receives red light, the photodiode 145g receives green light, and the photodiode 145b receives blue light.

  In addition, since the liquid crystal panel 140 includes only one photosensor circuit 144 in one pixel, two data signal lines for the TFT 147 disposed in one pixel are a sensor signal line SSj and a sensor signal line SDj. Met. However, since the liquid crystal panel 140A includes three photosensor circuits (144r, 144g, 144b) in one pixel, there are six data signal lines for TFTs (147r, 147g, 147b) arranged in one pixel. It becomes.

  Specifically, the sensor signal line SSRj and the sensor signal line SDRj are arranged corresponding to the TFT 147r connected to the cathode of the photodiode 145r arranged at a position facing the color filter 153r. A sensor signal line SSGj and a sensor signal line SDGj are arranged corresponding to the TFT 147g connected to the cathode of the photodiode 145g arranged at a position facing the color filter 153g. Further, a sensor signal line SSBj and a sensor signal line SDBj are disposed corresponding to the TFT 147b connected to the cathode of the photodiode 145b disposed at a position facing the color filter 153b.

  In such a liquid crystal panel 140A, the white light emitted from the backlight 179 passes through the three color filters (153r, 153g, 153b), and red, green, and blue are displayed on the surface of the liquid crystal panel 140A. It mixes and becomes white light. Here, when white light is reflected by the scan object, a part of the white light is absorbed by the pigment on the surface of the scan object, and a part is reflected. And the reflected light permeate | transmits three color filters (153r, 153g, 153b) again.

  At this time, the color filter 153r transmits light having a red wavelength, and the photodiode 145r receives light having the red wavelength. The color filter 153g transmits light having a green wavelength, and the photodiode 145g receives light having the green wavelength. The color filter 153b transmits light having a blue wavelength, and the photodiode 145b receives light having the blue wavelength. That is, the light reflected by the scan object is separated into three primary colors (R, G, B) by three color filters (153r, 153g, 153b), and each photodiode (145r, 145g, 145b) The light of the color corresponding to is received.

  When a part of white light is absorbed by the pigment on the surface of the scan target, the amount of light received by each photodiode (145r, 145g, 145b) is different for each photodiode (145r, 145g, 145b). For this reason, the output voltages of the sensor signal line SSRj, the sensor signal line SSGj, and the sensor signal line SSBj are different from each other.

  Therefore, the image processing engine 180 determines the R gradation, the G gradation, and the B gradation according to each output voltage, so that the image processing engine 180 sends the RGB color image to the main body device 101. Can send.

  As described above, when the electronic apparatus 100 includes the liquid crystal panel 140A, the scan target can be scanned in color.

  Next, a scanning method different from the above-described scanning method (that is, the method of scanning the reflected image in FIG. 6) will be described with reference to FIG.

  FIG. 17 is a cross-sectional view showing a configuration in which a photodiode receives external light during scanning. As shown in the figure, part of the external light is blocked by the finger 900. Therefore, the photodiode disposed under the surface region of the liquid crystal panel 140 that is in contact with the finger 900 can hardly receive external light. In addition, although the photodiodes disposed under the surface area where the shadow of the finger 900 is formed can receive a certain amount of external light, the amount of external light received is small compared to the surface area where no shadow is formed.

  Here, by turning off the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the position of the finger 900 with respect to the surface of the liquid crystal panel 140 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), and the liquid crystal panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter also referred to as a shadow image) obtained by blocking external light with the finger 900.

  Further, the display device 102 may be configured to perform scanning by turning on the backlight 179 and then performing scanning again by turning off the backlight 179. Alternatively, the display device 102 may be configured to perform scanning by turning off the backlight 179 and then performing scanning again by turning on the backlight 179.

  In this case, since two scanning methods are used together, two scan data can be obtained. Therefore, it is possible to obtain a highly accurate result as compared with the case of scanning using only one scanning method.

<About display devices>
The operation of the display device 103 is controlled in accordance with a command (for example, a first command) from the main body device 101 as in the operation of the display device 102. The display device 103 has the same configuration as the display device 102. Therefore, when the display device 103 receives the same command as the display device 102 from the main body device 101, the display device 103 performs the same operation as the display device 102. For this reason, description of the configuration and operation of the display device 103 will not be repeated.

  Note that the main device 101 can send commands having different commands to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform different operations. Further, the main device 101 may send a command to either the display device 102 or the display device 103. In this case, only one display device performs an operation according to the command. Further, the main device 101 may send a command having the same command to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform the same operation.

  Note that the size of the liquid crystal panel 140 of the display device 102 and the size of the liquid crystal panel 240 of the display device 103 may be the same or different. Further, the resolution of the liquid crystal panel 140 and the resolution of the liquid crystal panel 240 may be the same or different.

<About Second Modification of Configuration>
In the present embodiment, a configuration in which electronic device 100 includes a liquid crystal panel with a built-in optical sensor such as liquid crystal panel 140 and liquid crystal panel 240 will be described. However, only one liquid crystal panel has a built-in photosensor. It may be.

  FIG. 18 is a block diagram illustrating a hardware configuration of electronic device 1300. Similar to the electronic device 100, the electronic device 1300 includes a first housing 100A and a second housing 100B. Referring to FIG. 18, electronic device 1300 includes a first unit 1001A and a second unit 1002. The first unit 1001A includes a main body device 101 and a display device 102A. The second unit 1002 includes a main device 104 and a display device 103.

  The display device 102A includes a liquid crystal panel that does not include a photosensor (that is, a liquid crystal panel having only a display function). The electronic device 1300 is different from the electronic device 100 in which the first unit 1001A includes a liquid crystal panel that does not include a photosensor, and the first unit 1001 includes a liquid crystal panel 240 that includes a photosensor. Such an electronic device 1300 performs the above-described sensing using the display device 103 of the second unit 1002.

  Further, the first unit 1001 may include, for example, a resistive film type or capacitive type touch panel, instead of the liquid crystal panel 140 incorporating the photosensor.

  In this embodiment, the display device 102 includes the timer 182 and the display device 103 includes the timer 282. However, the display device 102 and the display device 103 may share one timer. .

  In the present embodiment, electronic device 100 is described as a foldable device, but electronic device 100 is not necessarily limited to a foldable device. For example, the electronic device 100 may be a sliding device configured such that the first housing 100A slides with respect to the second housing 100B.

  Since electronic device 100 according to the present embodiment is configured as described above, second unit 1002 is detachable from first unit 1001 via USB connectors 194 and 294.

  And the electronic device 100 which concerns on this Embodiment can exhibit the following functions, for example at the time of power activation. First, when the user depresses the power switch 191 of the first unit 1001, the first unit 1001 activates BIOS (Basic Input / Output System) by using the power from the power supply circuit 192.

  The second unit 1002 acquires power from the first unit 1001 via the USB connectors 194 and 294. The second unit 1002 can transmit and receive data to and from the first unit 1001 by using the power. At this time, the CPU 210 of the second unit 1002 can display the OS (Operation System) type on the liquid crystal panel 240 in a selectable manner by using the power from the USB connectors 194 and 294.

  The user selects an OS to be started up via the liquid crystal panel 240. The CPU 210 transmits a command (for example, a “first OS” command shown in FIG. 10) specifying the OS to be activated to the first unit 1001 via the USB connectors 194 and 294 according to the user's selection. . The first unit 1001 starts the OS in response to the command.

  Further, for example, the second unit 1002 transmits / receives data to / from an external mobile phone or the like via the antenna 295. The CPU 210 of the second unit 1002 acquires photographic image data and corresponding thumbnail data from an external mobile phone via the antenna 295, and stores the photographic image data and corresponding thumbnail data in the RAM 271 or the like. The CPU 210 reads the thumbnail data from the RAM 271 and causes the liquid crystal panel 240 to display a thumbnail image of the photo in a selectable manner.

  In response to a selection command from the outside, the CPU 210 causes the liquid crystal panel 240 to display a photographic image. Alternatively, the CPU 210 displays a photographic image on the liquid crystal panel 140 or the display device 102A via the USB connector 294.

<Functional configuration>
FIG. 19 is a block diagram showing a functional configuration of electronic apparatus 100 according to the present embodiment. As described above, electronic device 100 includes first unit 1001 and second unit 1002. Hereinafter, the functional configuration of the electronic apparatus 100 will be described with reference to FIG.

  The first unit 1001 includes a display unit 310, an input unit 320, a storage unit 330, an interface unit 340, and a control unit 350. The first unit 1001 performs the main operation of the electronic device 100.

  The display unit 310 displays information inside the first unit 1001 to the outside. The input unit 320 receives an instruction from the outside. In the present embodiment, the liquid crystal panel 140 has the functions of the display unit 310 and the input unit 320. However, as the display unit 310, another display device, for example, a display such as an LCD (Liquid Crystal Display) may be used. The operation key 177 also functions as the input unit 320.

  The storage unit 330 stores information such as display data 333, a program 334, and an operation parameter 335 that are a basis of a screen displayed on the display unit 310 (liquid crystal panel 140) of the first unit 1001. The storage unit 330 generally stores a plurality of programs 334. Examples of the program 334 include general-purpose application software such as a word processor and a Web browser.

  The operation parameter 335 is information for giving an operation condition of the program 334. The operation parameter 335 includes, for example, data indicating an active window that operates in response to pressing of the operation key 177 in the multi-window program 334.

  The interface unit 340 exchanges information with the interface unit 440 on the second unit 1002 side. In the present embodiment, when the first unit 1001 and the second unit 1002 are directly connected, the USB connector 194 functions as the interface unit 340. When the first unit 1001 and the second unit 1002 are not directly connected, the antenna 195 functions as the interface unit 340. However, the method of exchanging information by the interface unit 340 is not limited to this.

  The control unit 350 controls operations of the display unit 310, the storage unit 330, and the interface unit 340 based on an instruction from the input unit 320 and the like. Control unit 350 includes an input processing unit 352, a display control unit 356, and a program execution unit 358. In the present embodiment, the CPU 110 and the image processing engine 180 correspond to the control unit 350. However, each function of the CPU 110 may be realized by hardware such as a dedicated circuit. Each function of the image processing engine 180 may be realized by the CPU 110 that executes software. That is, each function of the control unit 350 may be realized by hardware or may be realized by software.

  The input processing unit 352 transmits the signal received from the input unit 320 to the program execution unit 358. The display control unit 356 controls the operation of the display unit 310 based on the display data 333 stored in the storage unit 330. The program execution unit 358 executes the program 334 based on an instruction received from the input unit 320. Specifically, the CPU 110 that executes the program 334 using the RAM 171 as a working memory corresponds to the program execution unit 358.

  The second unit 1002 includes a display unit 410, an input unit 420, a storage unit 430, an interface unit 440, a control unit 450, and a timer 273.

  The display unit 410 displays information inside the second unit 1002 to the outside. The input unit 420 receives an instruction from the outside. In the present embodiment, the optical sensor built-in liquid crystal panel 240, the left click key 241, the center key 242, and the right click key 243 correspond to the input unit 420. In the present embodiment, the optical sensor built-in liquid crystal panel 240 also has the functions of the display unit 410 and the input unit 420 (panel input unit 422). However, as the display unit 410, another display device, for example, a display such as an LCD (Liquid Crystal Display) may be used. The input unit 420 is not limited to the optical sensor built-in liquid crystal panel 240, and a device (tablet) having a function of recognizing an input position can be used. For example, a capacitive touch panel may be used as the input unit 420. A component that realizes the functions of the display unit 410 and the input unit 420 in this way is referred to as a “display-integrated touchpad”.

  The storage unit 430 stores information such as input data 431, display data 433, a program 434, operation parameters 435, time data 436, and mode data 437.

  The input data 431 is data created based on the input received by the input unit 420. In particular, in the present embodiment, the input data 431 includes an input history 432 corresponding to the input history. The input history 432 includes handwritten character data 432a and illustration data 432b. Details of the handwritten character data 432a and the illustration data 432b will be described later.

  The display data 433 is a basis of a screen displayed on the display unit 410 (liquid crystal panel 240) of the second unit 1002. The display data 433 includes image data (such as wallpaper) stored in the storage unit 430 and image data created in accordance with the execution of the program 434.

  In the present embodiment, storage unit 430 stores a plurality of programs 434. The program 434 includes application software (for example, handwritten character input software, hand-drawn illustration input software, or calculator software) that displays an operation screen on the liquid crystal panel 240. Details of the program 434 will be described later.

  The operation parameter 435 is information for giving an operation condition of the program 334 like the operation parameter 335 in the first unit 1001. In particular, in the present embodiment, the operation parameter 435 includes a count value of elapsed time from a predetermined event created by the program 434.

  Time data 436 represents the time measured by the timer 273. The time data 436 is used when executing a program 434 that performs a predetermined operation with the passage of time.

  The mode data 437 is information representing the operation mode of the input processing unit 452. There are a plurality of operation modes of the input processing unit 452, and the mode data 437 represents the current operation mode. Specifically, for example, a flag stored in a predetermined storage area can be regarded as mode data 437. Details of the operation mode of the input processing unit 452 will be described later.

  The interface unit 440 exchanges information with the interface unit 340 on the first unit 1001 side. In the present embodiment, when the first unit 1001 and the second unit 1002 are directly connected, the USB connector 294 functions as the interface unit 440. When the first unit 1001 and the second unit 1002 are not directly connected, the antenna 295 functions as the interface unit 440. However, the method of exchanging information by the interface unit 440 is not limited to this.

  The control unit 450 controls operations of the display unit 410, the storage unit 430, and the interface unit 440 based on an instruction received by the input unit 420. Control unit 450 includes an input processing unit 452, a display control unit 456, and a program execution unit 458.

  The input processing unit 452 transmits the signal from the input unit 420 to the program execution unit 458 or the interface unit 440. The input processing unit 452 includes a panel input processing unit 453 and a mode setting unit 454.

  The panel input processing unit 453 processes a signal from the panel input unit 422. For example, the panel input processing unit 453 creates an input history 432 (handwritten character data 432a, illustration data 432b, etc.) based on the signal history. Details of the operation of the panel input processing unit 453 will be described later.

  The mode setting unit 454 sets the operation mode of the panel input processing unit 453 based on a predetermined signal from the input unit 420 (such as a signal generated by pressing the center key 242). Details of the operation of the mode setting unit 454 will be described later.

  The display control unit 456 controls the operation of the display unit 410 based on the display data 433. For example, the display control unit 456 displays a screen (operation screen) created as a result of the execution of the program 434 on the display unit 410.

  The program execution unit 458 executes the program 434 based on an instruction received from the input unit 420 or the like. Specifically, the CPU 210 that executes the program 434 using the RAM 271 as a working memory corresponds to the program execution unit 458.

<Outline of operation>
(Mouse mode and tablet mode)
The electronic device 100 issues an instruction for the operation of the application in response to an instruction to the liquid crystal panel 240, that is, an object (such as a finger 900 or a stylus 950) touching the liquid crystal panel 240. The electronic device 100 (more specifically, the panel input processing unit 453) has two operation modes of “mouse mode” and “tablet mode”. Electronic device 100 operates in accordance with the operation mode.

  In the mouse mode, the electronic device 100 executes the program 334 in response to an input to the liquid crystal panel 240, and an image created by the executed program 334 (hereinafter referred to as “program operation screen”) is displayed on the liquid crystal panel 140. To display.

  Specifically, in the mouse mode, electronic device 100 moves the cursor in the operation screen of liquid crystal panel 140 in real time according to the change in the input position to liquid crystal panel 240. Here, the “cursor” refers to a display that indicates an input position of a character, a figure, a display object, or the like.

  Further, in the mouse mode, electronic device 100 creates a command for instructing a predetermined operation to program 334 of first unit 1001 in response to a predetermined input to liquid crystal panel 240. For example, when the electronic device 100 determines that the liquid crystal panel 240 has received an input corresponding to a click, a double click, a drag, or the like, the electronic device 100 executes the program 334 according to the input. The operation performed by electronic device 100 at this time is determined by program 334.

  That is, when electronic device 100 is in the mouse mode, the user can use liquid crystal panel 240 as a touch pad. Hereinafter, for the sake of simplicity, the operation of the program 334 according to the movement of the cursor position and the predetermined input to the liquid crystal panel 240 will be collectively referred to as “mouse operation”.

  In the tablet mode, electronic device 100 executes program 434 (or program 334) in response to an input to liquid crystal panel 240, and displays an operation screen of the executed program on liquid crystal panel 240. In addition, electronic device 100 creates a command for a program for displaying an operation screen on liquid crystal panel 240. For example, when an operation button is displayed on the liquid crystal panel 240, the electronic device 100 executes a program and performs an operation according to the touched operation button. That is, when the electronic device 100 is in the tablet mode, the user can use the liquid crystal panel 240 as a touch screen.

  Electronic device 100 switches the operation mode based on a predetermined instruction. In the present embodiment, electronic device 100 switches the operation mode in response to pressing of center key 242.

  However, the operation mode switching instruction is not limited to pressing the center key 242. Electronic device 100 may switch the operation mode in response to pressing of operation key 177 other than center key 242. In addition, electronic device 100 may switch the operation mode in accordance with selection of an operation button displayed on liquid crystal panel 140 or liquid crystal panel 240. The electronic device 100 also switches the operation mode depending on the operation state of the electronic device (during startup processing, return from sleep or hibernation). This point will be described in detail later.

  Display on the liquid crystal panel 140 and the liquid crystal panel 240 in the mouse mode and the tablet mode will be described with reference to FIG. FIG. 20 is a diagram for explaining a screen displayed by the electronic device in each of the mouse mode and the tablet mode.

  In the mouse mode, the electronic device 100 displays an operation screen 500 of software such as word processing software or a Web browser on the liquid crystal panel 140. The screen 500 is similar to a display screen by a personal computer that is widely used at present (that is, having one display). However, in FIG. 20, the contents of the screen 500 are not drawn for simplicity. The screen 500 is not limited to this specific example.

  Screen 500 includes a cursor 510. Here, it is assumed that the cursor 510 is a pointer (mouse pointer) that can freely move on the screen 500. However, the cursor 510 is not limited to a pointer. The cursor 510 may be a display indicating the input position of a character or a display object. The display form of the cursor 510 is not limited to that shown in FIG. 20 (arrow). Also, electronic device 100 may change the display form of cursor 510 depending on the position pointed to.

  In the mouse mode, electronic device 100 displays screen 600 on liquid crystal panel 240. Hereinafter, the screen displayed on the liquid crystal panel 240 in the mouse mode is also referred to as a “mouse screen”. Referring to FIG. 20, screen 600 includes a guidance display 610.

  Guidance display 610 is a display for describing the operation of electronic device 100 when left click key 241, center key 242, and right click key 243 are pressed. Guidance display 610 includes a left guidance display 612, a center guidance display 614, and a right guidance display 616.

  The left guidance display 612, the center guidance display 614, and the right guidance display 616 are characters that describe the operation of the electronic device 100 when the left click key 241, the center key 242, and the right click key 243 are pressed, respectively. And / or symbols (though such characters and symbols are not shown in FIG. 20).

  The mouse screen may not always display the left guidance display 612, the center guidance display 614, and the right guidance display 616. In the present embodiment, electronic device 100 does not display left guidance display 612, center guidance display 614, and right guidance display 616 on liquid crystal panel 240 when the corresponding key is invalid.

  One specific example of the mouse screen is shown in FIG. Referring to FIG. 21, screen 600 includes a left guidance display 612, a center guidance display 614, and a right guidance display 616.

  In FIG. 21, the left guidance display 612 includes the characters “left click”. When this display is made, electronic device 100 performs a left click operation (decision process or the like) in response to pressing of left click key 241. The left click operation is determined by the running program.

  In FIG. 21, the center guidance display 614 includes the characters “touch screen operation”. This display is made when the electronic device 100 is in the mouse mode. When this display is made, the electronic device 100 shifts to the tablet mode in response to pressing of the center key 242.

  Electronic device 100 displays the character “mouse operation” on center guidance display 614 when in tablet mode. When this display is made, electronic device 100 shifts to the mouse mode in response to pressing of center key 242.

  The right guidance display 616 includes the characters “right click”. This indicates that electronic device 100 performs a right-click operation (menu display, etc.) in response to pressing of right-click key 243. The right click operation is determined by the running program.

  In this embodiment, the user can set the mouse screen. The user may be able to set an image such as a photograph stored in the electronic device 100 as the wallpaper of the mouse screen. The wallpaper may be an operation screen of an accessory such as a clock or calendar that does not require user operation. In addition, the wallpaper can be automatically changed depending on the state of the electronic device 100, such as during startup or recovery from the power saving state. Even when the user cannot set the mouse screen, the mouse screen is not limited to that shown in FIG.

  Returning to FIG. 20, the screen in the tablet mode will be described. In the tablet mode, the liquid crystal panel 140 displays an operation screen 700 of software such as word processing software or a Web browser. The content of the operation screen 700 is the same as that of the operation screen 500.

  In the tablet mode, the electronic device 100 displays the screen 800 on the liquid crystal panel 240. Hereinafter, the screen displayed on the liquid crystal panel 240 in the tablet mode is also referred to as a “tablet screen”. Referring to FIG. 20, screen 800 includes a guidance display 810 and an operation button display 820.

  The guidance display 810 includes a left guidance display 812, a center guidance display 814, and a right guidance display 816 similarly to the guidance display 610 in the mouse mode. These roles and operations are similar to left guidance display 612, center guidance display 614, and right guidance display 616, respectively, and detailed description thereof will not be repeated.

  The operation button display 820 is used for selecting an application. When electronic device 100 detects the contact of an external object (such as finger 900 or stylus 950) in the region corresponding to operation button display 820, electronic device 100 starts a predetermined operation corresponding to the region.

(About the program)
Here, a program to be executed in the electronic device 100 will be described.

  The programs executed by electronic device 100 according to the present embodiment include a program for displaying an operation screen on liquid crystal panel 140 (hereinafter referred to as “main application”) and a program for displaying an operation screen on liquid crystal panel 240. (Hereinafter referred to as “sub-application”).

  As the main application, an application that operates on a current electronic device such as a browser, a dictionary, a book viewer, and a photo viewer can be assumed. Examples of the sub-application include an input pad (handwritten character input pad, hand-drawn illustration input pad, calculator / number input pad, etc.) that uses input to the liquid crystal panel 240, and an application for assisting operation of the main application.

  In the description of the present embodiment, the sub application is assumed to be independent of the main application. The main application is stored in the storage unit 330 of the first unit 1001. On the other hand, the sub application is stored in the storage unit 430 of the second unit 1002.

  Thus, the main application is separated from the sub application. Therefore, a general-purpose application that operates on other electronic devices can be used as the main application. In this case, the specification of data exchange with the main application of the sub application is matched with the specification of the main application. For example, the instruction of the mouse operation by the sub application is matched with the operation instruction from the conventional touch pad or mouse.

  Furthermore, in the present embodiment, the program execution unit 358 that executes the main application and the program execution unit 458 that executes the sub-application are independent. In this way, it is possible to reduce the load on the processor (CPU 110 in this embodiment) that executes the main application. In particular, when the performance of the CPU of the electronic device 100 is low, dividing the program execution unit in this way is effective.

  By providing a control unit and a storage unit in each of the first unit 1001 and the second unit 1002 as in the present embodiment, data exchange between the first unit 1001 and the second unit 1002 can be reduced, and processing can be performed. Can be speeded up.

  However, the sub application is not necessarily independent of the main application. That is, the same program may function as both the main application and the sub application. Specifically, a part of the program may create a screen to be displayed on the liquid crystal panel 140, and another part of the program may create a screen to be displayed on the liquid crystal panel 240.

  Further, the main application and the sub application may be executed by the same processor. In this case, the processor that executes the application controls the operations of both the liquid crystal panel 140 and the liquid crystal panel 240.

(Sub application)
In the present embodiment, electronic device 100 executes one of a plurality of sub-applications in the tablet mode. One of the sub-applications is a “home application” for determining an application to be executed.

  The home application is a launcher for selecting an application. The home application displays on the liquid crystal panel 240 a screen for selecting one sub-application from the plurality of sub-applications (hereinafter “home menu screen”).

  FIG. 22 shows a specific example of the home menu screen. Referring to FIG. 22, the home menu screen includes a guidance display 810, operation button displays 820 a to 820 i, and a mouse disabled display 830. When the operation button displays 820a to 820i are touched, the home application calls a sub application corresponding to the operation button displays 820a to 820i. The mouse disabled display 830 is a display for indicating that the electronic device 100 does not perform a mouse operation by an input to the liquid crystal panel 240. This display helps the user to accurately understand the operation of the electronic device 100 in the tablet mode. It also helps the user to distinguish between mouse mode and tablet mode.

  In the present embodiment, it is assumed that electronic device 100 can customize the home menu screen based on a user instruction. Even when the home menu screen cannot be customized, the configuration of the home menu screen is not limited to that shown in FIG. For example, the number and arrangement of the operation button displays 820 are not limited to those shown in FIG. The mouse disabled display 830 is not limited to that shown in FIG. Alternatively, the mouse disabled display 830 may not be included in the home menu screen.

  It is assumed that the default sub application in the tablet mode is a home application. That is, when the electronic device 100 operates in the tablet mode for the first time after activation (after power is turned on), the home application is executed.

  The transition of the screen 800 displayed on the liquid crystal panel 240 in the tablet mode will be described with reference to FIG. FIG. 23 is a transition diagram of a screen 800 displayed on the liquid crystal panel 240 in the tablet mode.

  Referring to FIG. 23, when the user selects operation button display 820 (hand-drawn illustration) on home menu screen 800a, electronic device 100 executes the hand-drawn illustration application and displays screen 800b on liquid crystal panel 240. The screen 800b is an operation screen for a hand-drawn illustration application. In the screen 800b shown in FIG. 23, a picture of a dog is drawn in the hand-drawn input frame.

  When the user selects operation button display 820 (home) on screen 800b, electronic device 100 displays window 800d on liquid crystal panel 240. Here, the electronic device 100 may display the window 800d instead of the screen 800b, or may display the window 800d superimposed on the screen 800b. The window 800d includes text for inquiring whether to save the created hand-drawn illustration, and operation button displays 820 of “Yes”, “No”, and “Cancel”.

  When “Yes” in the window 800d is selected, the electronic device 100 displays a hand-drawn illustration in the input frame before the window 800d is displayed in the electronic device 100 or an external storage device (such as a hard disk or a flash memory). To store. In addition, the electronic device 100 displays the home menu screen 800a on the liquid crystal panel 240.

  When “No” in the window 800d is selected, the electronic device 100 displays the home menu screen 800a on the liquid crystal panel 240. In this case, electronic device 100 does not store the hand-drawn illustration made in the input frame before display of window 800d in the storage device.

  When “cancel” of the window 800d is selected, the electronic device 100 displays the screen 800b before the window 800d is displayed on the liquid crystal panel 240. That is, if the user selects cancel, the user can continue to create a hand-drawn illustration.

  When the user selects the operation button display 820 (Internet) on the home menu screen 800a, the electronic device 100 executes a Web browser and displays an operation screen 800c of the Web browser on the liquid crystal panel 140.

  When the user selects operation button display 820 (Home) on screen 800c, electronic device 100 displays liquid crystal panel 240 on home menu screen 800a. In this case, electronic device 100 does not perform the storage process of the input data. Therefore, electronic device 100 does not display an inquiry screen such as window 800d.

<Switching mode>
Hereafter, the operation of the electronic device 100 when switching between the mouse mode and the tablet mode will be described in detail with reference to FIG. FIG. 24 is a diagram for explaining the operation of the electronic device 100 when switching between the mouse mode and the tablet mode.

  In the present embodiment, electronic device 100 basically switches the operation mode in response to pressing of center key 242 as described above. The electronic device 100 can transition from the tablet mode to the mouse mode, or can transition from the mouse mode to the tablet mode.

  In the present embodiment, electronic device 100 changes the processing method of input to liquid crystal panel 240 when shifting from the tablet mode to the mouse mode. That is, electronic device 100 treats an input to liquid crystal panel 240 as a mouse operation instruction rather than an operation instruction to a sub application. In addition, the electronic device 100 displays a mouse screen on the liquid crystal panel 240 when shifting to the mouse mode.

  However, the electronic device 100 keeps running the application that has been executed in the tablet mode even after shifting to the mouse mode. Thus, the electronic device 100 can smoothly display the screen on the liquid crystal panel 240 when shifting from the mouse mode to the tablet mode. This is because the startup time of the sub application is unnecessary.

  Further, by continuing to operate the application that has been executed in the tablet mode, the operability when the user performs a mouse operation temporarily with the liquid crystal panel 140 during the tablet mode is improved. For example, consider that the electronic device 100 transitions from the tablet mode to the mouse mode and then transitions back to the tablet mode. Since the electronic device 100 operates as described above, the electronic device 100 displays the operation screen of the same sub application on the liquid crystal panel 240 before and after the transition to the mouse mode. Therefore, the user can continue to use the sub-application before the mouse operation after performing the mouse operation.

  The electronic device 100 displays the operation screen of the main application on the liquid crystal panel 140 not only in the mouse mode but also in the tablet mode. Therefore, electronic device 100 can allow the user to operate the sub application without impairing the visibility of the operation screen of the main application.

(Mode at the start of operation)
In particular, switching of operation modes related to the start of operation of electronic device 100 will be described. In the present embodiment, the operation of the electronic device 100 is roughly classified into (i) activation from a power-off state (hereinafter referred to as normal activation) and (ii) activation from a power saving state (hereinafter referred to as “power activation state”) , Called resumption).

  Here, the “power-off state” is a state in which the operation of each part of the electronic device 100 (excluding a part necessary for starting up the electronic device 100) is stopped. The “power saving state” is a state in which a part of the operation of the electronic device 100 is stopped.

  The power saving state includes a “standby state”, a “sleep state”, and a “sleep state” that is a combination of the standby state and the sleep state.

  When receiving the instruction to shift to the standby state, the electronic device 100 stores the data being worked in the RAM 171. In addition, the electronic device 100 stops supplying power to portions other than those necessary for restarting operations (such as the power supply circuit 192, the power supply detection unit 193, and the RAM 171).

  When receiving the instruction to shift to the hibernation state, the electronic device 100 stores the data being worked on the hard disk 170. In addition, the electronic device 100 stops supplying power to parts other than those necessary for restarting the operation (such as the power circuit 192 and the power detection unit 193).

  At the time of instructing the sleep process, the electronic device 100 first stores the data being worked on in the memory. Then, when a predetermined time elapses from the instruction in the sleep state, electronic device 100 moves the data stored in the memory from the memory to the hard disk.

  However, the types of power saving states are not limited to those described above. Moreover, the electronic device 100 does not necessarily need to prepare all these power saving processes.

  First, (i) the operation mode in normal startup will be described with reference to FIG. FIG. 25 is a diagram schematically showing an operation mode at the normal startup.

  At the normal startup, the electronic device 100 first performs a boot process for starting an OS (Operating System). During the boot process, the electronic device 100 displays a boot screen 2501 on the liquid crystal panel 140. During the boot process, the operation mode of the electronic device 100 is the mouse mode. The electronic device 100 displays a predetermined mouse screen (hereinafter referred to as a fixed screen) 2502 on the liquid crystal panel 240. The fixed screen 2502 does not include a guidance display. This is because mode switching cannot be executed during the boot process.

  When the boot process is completed, the electronic device 100 displays a login screen 2503 on the liquid crystal panel 140. Also at this time, the electronic device 100 displays the fixed screen 2504.

  When the login is completed, the electronic device 100 displays a desktop screen 2505 on the liquid crystal panel 140. In addition, the electronic device 100 displays a mouse screen 2506 on the liquid crystal panel 240. At this stage, the electronic device 100 can switch from the mouse mode to the tablet mode. Accordingly, the mouse screen 2506 includes a guidance display, similar to the mouse screen 600 shown in FIG.

  It is assumed that the default sub application in the tablet mode is a home application. That is, when the electronic device 100 operates in the tablet mode for the first time after activation (after power is turned on), the home application is executed.

  Next, (ii) the operation mode in return will be described with reference to FIG. FIG. 26 is a diagram schematically showing the operation mode at the time of return.

  At the time of resumption, the electronic device 100 first reads data relating to the work state stored in a storage device such as a memory or a hard disk. During this time, the electronic device 100 displays a resuming screen 2601 on the liquid crystal panel 140. During this time, the operation mode of the electronic device 100 is the mouse mode. The electronic device 100 displays a fixed screen 2602 on the liquid crystal panel 240.

  When the reading of the work state is completed, the electronic device 100 displays a login screen 2603 on the liquid crystal panel 140. Also at this time, the electronic device 100 displays the fixed screen 2604.

  When the login is completed, the electronic device 100 displays again the display screen 2605 displayed on the liquid crystal panel 140 immediately before shifting to the power saving state on the liquid crystal panel 140 based on the read work state. In addition, the electronic device 100 displays a display screen 2606 based on the sub application that was operating immediately before the transition to the power saving state on the liquid crystal panel 240.

  Electronic device 100 may always display a mouse screen on liquid crystal panel 140 after the login is completed. When it is expected that the first operation performed by the user after the restart is a mouse operation, the operability can be improved by this processing.

<Operation in tablet mode>
From here, the operation of the electronic device 100 in the tablet mode will be described in more detail with reference to FIG. FIG. 27 is a diagram for explaining the operation of the electronic device 100 in the tablet mode.

  In the tablet mode, the electronic device 100 roughly divides the home application, executes the input pad application, or executes the sub-screen utilization software.

  The electronic device 100 displays the home menu screen 800a on the liquid crystal panel 240 during execution of the home application. When the electronic device 100 receives a touch on a predetermined position (indicated by the operation button display) on the liquid crystal panel 240 during execution of the home application, the electronic device 100 calls the input pad. In addition, when the electronic device 100 receives a touch on a predetermined position on the liquid crystal panel 240 during execution of the home application, the electronic device 100 starts execution of the sub-screen utilization software.

  When the electronic device 100 calls the input pad, the electronic device 100 displays an input screen on the liquid crystal panel 240. Referring to FIG. 27, in the present embodiment, there are three types of input pads, that is, a handwritten character input pad, a hand-drawn illustration input pad, and a calculator / number input pad. Screens 2702 to 2704 displayed on liquid crystal panel 240 are operation screens for a handwritten character input pad, a handwritten illustration input pad, and a calculator / numeric input pad, respectively. Details of each input pad will be described later.

  In addition, when electronic device 100 receives a predetermined instruction during execution of the input pad, electronic device 100 activates the home application. In the present embodiment, the operation screen of the input pad includes a home button, and electronic device 100 ends the input pad and starts a home application in response to a touch on the home button.

  Referring to FIG. 27, the sub-screen utilization software in this embodiment includes a two-screen utilization guide, the Internet, a dictionary, a book, a photograph, and a game.

  The “2-screen usage guide” is an online manual. When the electronic device 100 executes the two-screen utilization guide, the electronic device 100 acquires the manual from the external server and displays the manual on the liquid crystal panel 240 (or the liquid crystal panel 140 or both the liquid crystal panel 140 and the liquid crystal panel 240).

  “Internet” is software for activating a Web browser and calling a home page, and is hereinafter referred to as Web page calling software. Details of the operation of this software will be described later.

  “Dictionary” is software for calling an electronic dictionary, and is hereinafter referred to as dictionary calling software. In the present embodiment, this software calls one electronic dictionary from a plurality of electronic dictionaries. Details of the operation of this software will be described later.

  “Book” is software for selecting an electronic book (for example, a book in XMDF format) to be viewed with a book viewer. “Photo” is software for displaying a photo slideshow.

  “Game” is game software for displaying a game screen on the liquid crystal panel 240. In the present embodiment, it is assumed that the “game” is a game utilizing the touch operation of the liquid crystal panel 240.

  When the electronic device 100 receives a predetermined instruction during execution of the sub-screen utilization software, the electronic device 100 ends the sub-screen utilization software and starts a home application. In the present embodiment, the operation screen of the sub screen utilization software includes a home button, and electronic device 100 activates a home application in response to a touch on the home button.

(Handwriting input pad)
The operation of electronic device 100 that executes the handwritten character input pad (specifically, CPU 210 that executes the application) will be described with reference to FIG. FIG. 28 is a diagram illustrating an example of a screen (character input screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the handwritten character input pad.

  Referring to FIG. 28, the character input screen includes a center guidance display 814, a mouse disabled display 830, a home button 840, a text box 2801, a paste button 2802, a candidate area 2803, a backward button 2804, handwriting. An area 2805, a recognition mode switching button 2806, a recognition button 2807, and an erase button 2808 are included.

  A text box 2801 displays characters that have been confirmed as a result of handwritten character recognition. The text box 2801 can display up to 10 confirmed characters. However, the maximum number of characters displayed in the text box 2801 is not limited to this.

  When the paste button 2802 is pressed when there is a character string (one or more characters) in the text box 2801, the electronic device 100 displays the character string on the active application displayed on the liquid crystal panel 140. Send.

  The paste button 2802 functions as an Enter key when there is no character string in the text box 2801. For example, the user can press the paste button 2802 to transmit a character string to the search box, and then press the paste button 2802 again to cause the application to execute a search. If there is no character string in text box 2801 (or paste button 2802 functions as an Enter key), electronic device 100 changes the character displayed on paste button 2802 to “Enter”.

  The candidate area 2803 displays input recognition candidates. In the present embodiment, the candidate area 2803 displays a maximum of five recognition candidates in order from the first candidate. However, the maximum number of candidates displayed in the candidate area 2803 is not limited to this.

  In the present embodiment, electronic device 100 automatically adds the first candidate (the first character in candidate area 2803) to text box 2801. In addition, electronic device 100 can change the added character in accordance with selection of a candidate in candidate area 2803. Since there is a high possibility that the first candidate is a character to be input by the user, the number of operations by the user can be reduced by automatically adding the first candidate to the text box 2801.

  When the backward button 2804 is pressed, the electronic device 100 erases the last character of the character string in the text box 2801. If there is no character string in text box 2801, electronic device 100 does not perform an operation associated with pressing back button 2804.

  The handwriting area 2805 accepts input from the outside. Electronic device 100 creates handwritten character data 432 a corresponding to the history of input to handwriting area 2805 and stores it in storage unit 430. For example, the electronic device 100 creates, as handwritten character data 432a, all coordinates that have been input within a predetermined time, or coordinates at the start and end of input that are temporally continuous within a predetermined time. In addition, electronic device 100 displays a graphic corresponding to coordinates (or handwritten character data 432a) for which input has been received in handwritten area 2805.

  In the present embodiment, handwriting area 2805 includes two regions (area 2805a and area 2805b). Electronic device 100 creates handwritten character data 432a for each of area 2805a and area 2805b. Note that the handwriting area 2805 is not limited to two areas.

  The recognition mode switching button 2806 switches the recognition mode for handwriting input. In the present embodiment, there are two recognition modes, “automatic mode” and “manual mode”. It is assumed that the recognition mode when the handwriting input pad is first activated is the automatic mode.

  In the “automatic mode”, electronic device 100 automatically starts character recognition for input to handwriting area 2805 after a predetermined time has elapsed since pen-up (end of input to handwriting area 2805). The automatic mode has an advantage that the number of user operations can be reduced. In the automatic mode, electronic device 100 may automatically start character recognition after a predetermined time from the start of input to another event, for example, handwriting area 2805, instead of pen-up. In addition, the start of input to a handwriting area 2805b (or 2805a) different from the handwriting area 2805a (or 2805b) where handwriting input has been performed is also an automatic start event for character recognition.

  In the “manual mode”, the electronic device 100 does not start character recognition until the recognition button 2807 is pressed. The manual mode has an advantage that the user can calmly input characters.

  When the recognition button 2807 is pressed, the electronic device 100 starts character recognition of the handwritten character data 432a based on the input to the input area 2805. Even in the automatic mode, if the recognition button 2807 is pressed before the recognition start time, the electronic device 100 starts character recognition of the handwritten character data 432a.

  When delete button 2808 is pressed, electronic device 100 deletes the graphic and handwritten character data 432a displayed in input area 2805. The erase button 2805 is used when a user rewrites a character handwritten before character recognition.

  The usage method of a handwritten character input pad is demonstrated with reference to FIG. 29 and FIG. FIGS. 29 and 30 are diagrams for explaining the operation of electronic device 100 when the handwritten character input pad is used. Here, a description will be given of an operation example in the case where the user searches for “Fugu” using the liquid crystal panel 240 while using an application (such as a Web browser) that displays a screen including a search box.

  FIG. 29A shows a screen displayed on the liquid crystal panel 140 and the liquid crystal panel 240 in the mouse mode. At this time, the liquid crystal panel 240 displays a mouse screen 2920. The liquid crystal panel 140 displays a main application operation screen 2910. Operation screen 2910 includes a search box 2912 and a search start button 2914. It is assumed that the search box 2912 is activated by the mouse operation accompanying the movement of the finger 900 on the liquid crystal panel 240.

FIG. 29B shows a display screen 2930 of the liquid crystal panel 240 after a mode switching instruction (specifically, pressing of the center key 242) in the state shown in FIG. 29A.
Here, display screen 2930 is a home menu screen.

  FIG. 29C shows the display on the liquid crystal panel 240 after the button for calling the handwritten character input pad (encircled in FIG. 29B) is pressed in the state shown in FIG. A screen 2940 is shown. Display screen 2940 is a character input screen.

  FIG. 29D shows a display screen 2950 of the liquid crystal panel 240 when a handwriting input is made on the display screen 2940 by the stylus 950. In the input area of the display screen 2950, a graphic 2952 corresponding to handwriting input is displayed.

  FIG. 29 (e) shows a display screen 2960 of the liquid crystal panel 240 when the stylus 950 shown in FIG. 29 (d) is separated from the liquid crystal panel 240. Candidate characters corresponding to handwriting input are displayed in the candidate area of the display screen 2950. In the text box, the first candidate character 2962 (here, the character “kawa”) is displayed.

  FIG. 29F shows the display screen 2970 of the liquid crystal panel 240 when the first candidate character 2962 is confirmed. Character 2972 is a confirmed character.

  FIG. 30G is a diagram showing the display screen 3010 of the liquid crystal panel 240 in a state where the user performs handwriting input with the stylus 950 after the time shown in FIG. Display screen 3010 includes a graphic 3012 corresponding to handwriting input.

  FIG. 30H shows a display screen 3020 of the liquid crystal panel 240 when the stylus 950 shown in FIG. Candidate characters corresponding to handwriting input are displayed in the candidate area of the display screen 2950. In the text box, the first candidate character 3022 (here, the character “knee”) is added to the right side of the already determined character “river”.

  FIG. 30I shows the display screen 3030 of the liquid crystal panel 240 when the character 3032 (“pig”) in the candidate area is pressed by the stylus 950 in the state shown in FIG. As the character 3032 is pressed, the character “knee” displayed in the text box changes to “pig” (character 3034).

  FIG. 30J shows the display screen 3040 of the liquid crystal panel 140 and the display screen 3050 of the liquid crystal panel 240 after the character 3034 (“pig”) is confirmed. Display screen 3040 includes a search box 3042 and a search start button 3044. After the character 3034 is confirmed, when the paste button in the display screen 3050 is pressed by the stylus 950, the character string “Kawaguchi” in the text box is input to the active search box 3042. Accordingly, the search string 3042 displays the character string “Kawaguchi”.

  FIG. 30 (k) shows the display screen 3060 of the liquid crystal panel 140 and the display screen 3070 of the liquid crystal panel 240 after the Enter button (paste button) is pressed after the state shown in FIG. 30 (j). Pressing the Enter button in the display screen 3070 has the same effect as pressing the search start key 3044. In other words, the main application executes a search for the character string “Kawaguchi” in the search box 3042 in response to pressing of the Enter button, and displays a display screen 3060 indicating the search result on the liquid crystal panel 140.

  In the present embodiment, when the text box is full (the maximum number of characters that can be input is input in the text box), electronic device 100 does not accept any more handwriting. The operation of electronic device 100 when the text box is full will be described with reference to FIG. FIG. 31 is a diagram for explaining the operation of electronic device 100 when the text box is full.

  FIG. 31A shows a state in which the user is about to input to the input area with the stylus 950 when the text box is already full (ten characters are in the text box). At this time, a character string 3112 (a 10-character string “Aiueokiki Kekko”) is displayed in the text box of the display screen 3110 of the liquid crystal screen 240.

  When the stylus 950 contacts the liquid crystal panel 240, the electronic device 100 displays a screen 3120 including a warning display 3122 on the liquid crystal panel 240 (see FIG. 31B). The warning display 3122 includes a character string that prompts the user to confirm the character, that is, press the paste button. Here, the character string included in the warning display 3122 is “Please touch the paste button”, but the character string is not limited to this.

  After the warning display 3122 is displayed, the liquid crystal panel 240 displays a display screen 3130 similar to the display screen 3110 (see FIG. 31C). While the display screen 3130 is displayed, the user can continue character input by pressing the paste button to confirm the character string or by pressing the backward button to delete the already input character. Note that the electronic device 100 automatically changes from the state of FIG. 31B to FIG. 31C, for example, automatically after a predetermined time has elapsed from the display of the warning display 3122 or in response to some instruction to the liquid crystal panel 240. ) State.

  In the present embodiment, handwritten character data 432a is temporarily stored in RAM 271 or the like, and electronic device 100 discards handwritten character data 432a when the handwritten character input pad ends. Therefore, when the handwritten character input pad is called again, the user can input a new character.

  However, the electronic device 100 may hold the handwritten character data 432a so that the user can perform handwriting input from the last time when the handwriting input pad is used again. In this case, electronic device 100 displays a graphic corresponding to handwritten character data 432a on liquid crystal panel 240 based on handwritten character data 432a when the handwritten character input pad is resumed.

  Note that the user may be able to select which of the above two operations is performed on the handwritten character input pad. In this case, the user can appropriately determine which of the above two operations is performed by the handwritten character input pad according to the usage mode of the handwritten character input.

(Hand-drawn illustration input pad)
The operation of electronic device 100 that executes the hand-drawn illustration input pad (specifically, CPU 210 that executes the application) will be described with reference to FIG. FIG. 32 is a diagram illustrating an example of a screen (illustration input screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the hand-drawn illustration input pad.

  Referring to FIG. 32, the illustration input screen includes a center guidance display 814, a mouse disabled display 830, a home button 840, a drawing area 3201, an undo button 3202, a pen / ruler / eraser button 3203, Pen thickness button 3204, pen color button 3205, stamp button 3206, frame button 3207, delete all button 3208, screen capture button 3209, mail attachment button 3210, file save button 3211, and paste button 3212 Including.

  The drawing area 3201 accepts input from the outside. The electronic device 100 creates the illustration data 432b based on the input to the drawing area 3201 and the drawing settings (input tool, pen thickness, pen color, etc.) and stores the illustration data 432b in the storage unit 430. Similar to the handwritten character data 432a, the illustration data 432b includes all the coordinates that have been input within a predetermined time, or the coordinates at the start and end of input that are temporally continuous within the predetermined time. The illustration data 432b further includes data related to drawing settings (input tool, pen thickness, pen color, etc.).

  In the present embodiment, the ratio of the horizontal length to the vertical length of the drawing area 3201 is 4: 3. This is because the photograph is also processed using the handwritten illustration input pad. However, the aspect ratio of the drawing area 3201 is not limited to this.

  An undo button 3202 is a button for canceling the previous input operation to the drawing area 3201. The electronic device 100 manages the inputs recorded in the input history 432 in order of time, and when the undo button 3202 is pressed, the previous input is deleted from the input history 432. At the same time, the electronic device 100 erases the graphic display based on the drawing data corresponding to the immediately preceding input from the drawing area 3201.

  A pen / ruler / eraser button 3203 is a button for selecting a drawing tool for the drawing area 3201. In response to pressing of the pen / ruler / eraser button 3203, the electronic device 100 switches the drawing tool in the order of pen, ruler, eraser, pen,.

  The pen thickness button 3204 is a button for setting the pen thickness. In response to pressing of the pen thickness button 3204, the electronic device 100 changes the setting of the thickness of a line drawn according to the input to the drawing area 3201 when the pen tool is selected. Alternatively, electronic device 100 may display on LCD panel 240 a screen for allowing the user to set the thickness of the line when pen thickness button 3204 is pressed.

  A pen color button 3205 is a button for setting a pen color. In response to pressing of the pen color button 3205, the setting of the color of the line to be drawn is changed according to the input to the drawing area 3201 when the pen tool is selected. Alternatively, electronic device 100 may display on LCD panel 240 a screen for allowing the user to set a line color when pen color button 3205 is pressed.

  The stamp button 3206 is a button for causing a stamp to be drawn in the drawing area 3201 in response to an input to the drawing area 3201.

  The frame button 3207 is a button for adding a frame such as a decorative frame to the illustration drawn in the drawing area 3201.

  The delete all button 3208 is a button for deleting all the illustration data 432b. By pressing this button, the user can set the drawing area 3201 to the state (plain color) when the hand-drawn illustration input pad is activated.

  A screen capture button 3209 is a button for saving the entire screen displayed on the liquid crystal panel 240. The mail attachment button 3210 is a button for attaching the illustration data 432b to an electronic mail. The file save button 3211 is a button for saving the illustration data 432b in a designated storage area. The area for storing the illustration data 432b may be fixed or may be designated by the user.

  The paste button 3212 is a button for sending the illustration data 432b to the active main application. When the paste button 3212 is pressed, the electronic device 100 provides the active main application with the illustration data 432b created by the hand-drawn illustration input pad. For example, the user can insert an illustration into a document being created by the main application using the hand-drawn illustration input pad.

  In this embodiment, as described with reference to FIG. 23, when the hand-drawn illustration input pad receives an instruction to end the hand-drawn illustration input pad, the hand-drawn illustration input pad 432b determines whether the created hand-drawn illustration data 432b needs to be saved. Inquire. However, this inquiry is not essential. The hand-drawn illustration input pad may automatically discard the hand-drawn illustration data 432b created so far at the end.

(Calculator / Number input pad)
The operation of electronic device 100 that executes the calculator / number input pad (specifically, CPU 210 that executes the application) will be described with reference to FIG. FIG. 33 is a diagram illustrating an example of a screen (calculator screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the calculator / number input pad.

  Referring to FIG. 33, the calculator screen includes a center guidance display 814, a mouse disabled display 830, a home button 840, a number box 3301, a pasting button 3302, a number button 3303, and a function button 3304.

  The number box 3301 displays the input number or the number of the calculation result. It is assumed that the maximum number of numbers that can be displayed in the number box 3301 is eight. However, the maximum number is not limited to eight.

  When paste button 3302 is pressed, electronic device 100 transmits the number displayed in number box 3301 to the active application displayed on liquid crystal panel 140.

  A number button 3303 is a button for inputting a number in the number box 3301. The function button 3304 is a button for instructing a predetermined calculation such as four arithmetic operations. Since the operation of electronic device 100 when numeric button 3303 and function button 3304 are pressed is the same as the operation of a normal calculator or a calculator application, detailed description thereof will not be repeated here.

(the Internet)
The operation of electronic device 100 (specifically, CPU 210 that executes an application) that executes Web page calling software that is one of the sub-screen utilization software will be described with reference to FIG. FIG. 34 is a diagram illustrating an example of a screen (Internet screen) displayed on the liquid crystal panel 240 by the electronic device 100 while the Web page calling software is activated.

  Referring to FIG. 34, the Internet screen includes a guidance display 810 (left guidance display 812, center guidance display 814 and right guidance display 816), a plurality of operation button displays 820, a mouse disabled display 830, and a home button 840. including.

  Each of the operation button displays 820 corresponds to a web page to be called. Each operation button display 820 includes characters representing the name of the corresponding Web page (such as “Internet 2” in the figure). When electronic device 100 detects a touch on an area corresponding to operation button display 820, electronic device 100 activates a web browser and displays the selected web page on liquid crystal panel 140 (or liquid crystal panel 240).

  Note that the web page calling software may display a scroll bar on the liquid crystal panel 240. An example of an internet screen including a scroll bar is shown in FIG. Referring to FIG. 35, the Internet screen includes a scroll bar 3500. When the user drags the slider 3502 in the scroll bar 3500, the Web page calling software scrolls the Internet screen.

(dictionary)
The operation of electronic device 100 (specifically, CPU 210 that executes an application) that executes dictionary calling software, which is one of the sub-screen utilization software, will be described with reference to FIG. FIG. 36 is a diagram illustrating an example of a screen (dictionary selection screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the dictionary calling software.

  Referring to FIG. 36, the dictionary selection screen includes a guidance display 810 (a left guidance display 812, a center guidance display 814, and a right guidance display 816), a plurality of operation button displays 820, a mouse disabled display 830, and a home button 840. Including.

  Each of the operation button displays 820 corresponds to the electronic dictionary to be called. Each operation button display 820 includes characters representing the name of the corresponding electronic dictionary (such as “English-Japanese dictionary” in the figure). When electronic device 100 detects a touch on an area corresponding to operation button display 820, electronic device 100 activates the electronic dictionary and displays the screen of the activated electronic dictionary on liquid crystal panel 140 (or liquid crystal panel 240).

  The dictionary calling software may display a scroll bar on the liquid crystal panel 240. An example of a dictionary selection screen including a scroll bar is shown in FIG. Referring to FIG. 37, the dictionary selection screen includes a scroll bar 3700. When the user drags slider 3702 in scroll bar 3700, the dictionary calling software scrolls the dictionary selection screen.

<Time count control>
As already described, in this embodiment, even after the transition from the tablet mode to the mouse mode, the sub-application that was operating in the tablet mode continues to operate. When the sub-application is operated in this way, there is a possibility that the sub-application may inconvenience the user's unexpected operation.

  As a specific example, inconveniences that may occur during the operation of the handwritten character input application will be described with reference to FIG. FIG. 38 is a diagram for explaining inconveniences that may occur during the operation of the handwritten character input application.

  FIG. 38A shows an operation screen 3810 of the handwritten character input application in the tablet mode. The operation screen 3810 is displayed when the user performs handwriting input to the input area with the stylus 950 and character recognition is not yet performed.

  In the state shown in FIG. 38A, when a mode switching instruction is issued by pressing the center key 242 or the like, the liquid crystal panel 240 displays a mouse screen 3820 on the liquid crystal panel 240.

  However, the handwritten character input application continues to operate while the mouse screen 3820 is displayed. The operation during this period will be described with reference to FIGS. 38 (c) and 38 (d). FIGS. 38C and 38D are diagrams showing virtual screens displayed on the liquid crystal panel 240 by the handwritten character input application when there is no switching to the mouse mode.

  In this example, it is assumed that the recognition mode of the handwritten character input application is “automatic mode”. Accordingly, the handwritten character input application starts character recognition after the first predetermined time (t1) has elapsed since the pen-up. A screen 3830 in FIG. 38C is a virtual screen after character recognition. It is assumed that the first candidate for recognition at this time is the character “洶”.

  Furthermore, the handwritten character input application determines the character after a second predetermined time (t2) has elapsed since the character recognition. A screen 3840 in FIG. 38D is a virtual screen after character determination.

  After a time exceeding t1 + t2 has elapsed since switching to the mouse mode, when there is an instruction to switch to the tablet mode again, the liquid crystal panel 240 displays the operation screen 3850 of the handwritten character input application.

  The counting operations of the electronic device 100 shown in FIG. 38 will be summarized with reference to FIG. FIG. 39 is a diagram for explaining the counting operation of electronic device 100 shown in FIG.

  While the handwritten character input application is being executed, the CPU 210 that executes the handwritten character input application counts the elapsed time from a predetermined event based on the time data 436 acquired from the timer 273. Specifically, CPU 210 counts the elapsed time from the end of touch on the handwriting area of liquid crystal panel 240 or the end of character recognition.

  More specifically, the CPU 210 increases the counter value of the elapsed time with the passage of time while not receiving a signal indicating a touch on the handwriting area. When the CPU 210 receives a signal indicating a touch on the handwriting area or ends character recognition, the CPU 210 resets the counter value (sets to 0).

  Even after the touch in the tablet mode is finished and the operation mode is changed from the tablet mode to the mouse mode, the CPU 210 continues counting elapsed time. Therefore, the counter value reaches the waiting time t1 for character recognition during the mouse mode. At this time, the CPU 210 performs character recognition.

  Furthermore, after character recognition, the CPU 210 resets the counter once and then continues to count elapsed time. During the mouse mode, the counter value reaches a waiting time t2 for character confirmation. At this point, the CPU 210 determines the character. If the counter value for determining the character is set to t1 + t2, the CPU 210 does not need to reset the counter when the character recognizer ends.

  As described with reference to FIGS. 38 and 39, when the sub application is simply operated during the mouse mode, the handwritten input performed by the user may be recognized and confirmed. There is a possibility that the operation of the sub application in the mouse mode is contrary to the user's intention.

  For example, the user may not be able to perform further handwriting input following the handwriting input performed before the transition to the mouse mode. Also, the confirmed user's character may be different from the intended character. In fact, in the case shown in FIG. 38, the user has input by hand “kawa”, but the confirmed character is “洶”. In such a case, the user has to perform extra operations such as cancellation of the confirmed character and rewriting by handwriting.

  Therefore, electronic device 100 according to the present embodiment stops counting the timer when the input is interrupted to shift to the mouse mode. Therefore, the user can perform handwriting input after resuming the tablet mode, following the handwriting input performed before shifting to the mouse mode. The handwriting input operation according to the present embodiment will be described with reference to FIG. FIG. 40 is a diagram for explaining the handwriting input operation according to the present embodiment.

  Screens 4010 to 4050 shown in FIGS. 40 (a) to 40 (e) correspond to screens 3810 to 3850 shown in FIGS. 38 (a) to 38 (e), respectively. It is assumed that the user performs the same operation as that shown in FIG.

  The difference from FIG. 38 is that the timer is not counted in the mouse mode. Therefore, the virtual screens 4030 and 4040 shown in FIGS. 40C and 40D are the same as the screen 4010 before switching to the mouse mode. The screen 4050 when the tablet mode is resumed is the same as the screen 4010. In other words, when the tablet mode is resumed, the user can touch the liquid crystal panel 240 displaying the screen 4050 having the same contents as the screen 4010 to perform input again from the continuation of the input performed before the transition to the mouse mode. it can.

  Hereinafter, the time count performed by the electronic device 100 will be specifically described. Electronic device 100 can perform the following four types of time counts. For example, the electronic device 100 performs one time count according to the setting by the user. Alternatively, electronic device 100 may determine a time count to be automatically executed according to the type of sub application. However, the electronic device 100 need not be able to perform all of the following four types of time counts. The electronic device 100 may perform at least one type of time count.

(First time count: Count stop)
When performing the first time count, the CPU 210 stops the time count during the mouse mode. That is, the CPU 210 does not increase the counter value even if time elapses during the mouse mode, and keeps the counter value at the value immediately before the transition to the mouse mode. Further, the CPU 210 resumes time counting in response to an instruction to switch from the mouse mode to the tablet mode.

  Referring to FIG. 41, the operation of CPU 210 when the first time count is performed will be described. FIG. 41 is a diagram for explaining the first time count. FIG. 41 shows an operation example of the CPU 210 when the user switches between the mouse mode and the tablet mode in the middle of handwriting input, as in FIGS. 38 and 39. As can be seen from FIG. 41, the counter operation stops when the tablet mode is switched to the mouse mode, and the counter value is constant during the mouse mode. Further, the count operation is restarted at the timing when the mouse mode is returned to the tablet mode, and the counter is increased until the handwriting input operation (touch) is started. When the touch is resumed, the counter is reset, and the count operation is suppressed until the next touch is finished (that is, the count operation is not performed during the touch), and when the counter reaches a predetermined value (t1) after the touch is finished. Start recognition operation. Note that the operation after the start of touch after restarting the tablet mode is the same as the normal handwriting recognition operation.

  According to the first time count, electronic device 100 does not recognize or confirm characters during the mouse mode. That is, electronic device 100 does not perform an operation not intended by the user during the mouse mode. Therefore, user operability is improved. Specifically, the user can temporarily perform the operation in the mouse mode during the operation in the tablet mode without being disturbed in the continuity of the operation in the tablet mode.

(Second time count: count stop + reset)
When performing the second time count, the CPU 210 stops the time count during the mouse mode and performs the time count in response to an instruction to switch from the mouse mode to the tablet mode as in the case of performing the first time count. Resume.

  Further, when the CPU 210 performs the second time count, the CPU 210 resets the counter value in response to an instruction to switch from the tablet mode to the mouse mode.

  Referring to FIG. 42, the operation of CPU 210 when the second time count is performed will be described. FIG. 42 is a diagram for explaining the second time count. FIG. 42 shows an operation example of the CPU 210 when the user switches between the mouse mode and the tablet mode in the middle of handwriting input, as in FIGS. 38 to 41. In FIG. 42, the change of the counter value when performing the second time count is indicated by a solid line. Further, in FIG. 42, for reference, a change in the counter value when the first time count is performed is indicated by a dotted line.

  According to the second time count, as in the first time count, electronic device 100 does not recognize or confirm characters during the mouse mode. That is, electronic device 100 does not perform an operation not intended by the user during the mouse mode. Therefore, user operability is improved.

  Further, according to the second time count, when restarting the tablet mode, the CPU 210 counts the time from the initial value (“0”). Therefore, compared to the first time count, the time from the restart of the tablet mode to the execution of operations such as character recognition and determination is longer. In the case of the first time count, if the user does nothing after restarting the tablet mode, character recognition is confirmed at the time surrounded by a circle in FIG. However, when the second time count is performed, character recognition is not yet determined at this point.

  Therefore, the user can operate the sub-application with a sufficient margin after restarting the tablet mode. In particular, this method is effective when the counter value at the time of returning from the tablet mode to the mouse mode is a value slightly lower than the threshold value (t1 or t2).

  In the above description, the CPU 210 resets the counter value in response to a switching instruction from the tablet mode to the mouse mode. However, the timing for resetting the counter value is not limited to this. The CPU 210 may reset the counter value so that counting can be performed from the initial value when the tablet mode is resumed. For example, the CPU 210 may reset the counter value when restarting the tablet mode.

(Third time count: count stop + restart at input start or end)
When performing the third time count, the CPU 210 stops the time count during the mouse mode as in the case of performing the first time count.

  Further, when the CPU 210 performs the third time count, the CPU 210 stops the time count operation until the start or end time of the touch on the liquid crystal panel 240 after restarting the tablet mode. That is, the CPU 210 resumes the time counting operation in response to the start or end event of the first touch on the liquid crystal panel 240. However, when restarting the time counting operation according to the touch start event, the counter does not increase until the touch operation actually has a function to suppress the count until the end of the touch. Apparently, it cannot be distinguished from the case where the counting operation is resumed from the end of the touch.

  Referring to FIG. 43, the operation of CPU 210 when the third time count is performed will be described. FIG. 43 is a diagram for explaining the third time count. FIG. 43 shows an operation example of the CPU 210 when the user switches between the mouse mode and the tablet mode in the middle of handwriting input, as in the case of FIGS. 38 to 42. In FIG. 43, the change in the counter value when the third time count is performed (when the count is restarted when the touch is started) is indicated by a solid line. At this time, the CPU 210 resets the counter value at the start of the touch, the count operation is suppressed until the touch is ended, and the count value increases with the end of the touch.

  Further, a change in the counter value when the count is restarted at the end of the touch is indicated by a dotted line I. In this case, the counter value is not reset until the end of touching, and is the same value as in the mouse mode.

  In the above description, the handwritten character input application has been described as starting the counting operation from the initial value at the start or end of the touch. However, in the present embodiment, the application does not have to reset the counter value after the start or end of the touch. In FIG. 43, a dotted line II shows a case where the counter value is not reset at the start or end of the touch.

  For reference, FIG. 43 shows a change in the counter value when performing the first time count with a dotted line (a dotted line extending to the upper right from the restart point of the tablet mode).

  According to the third time count, as in the first time count, electronic device 100 does not recognize or confirm characters during the mouse mode. That is, electronic device 100 does not perform an operation not intended by the user during the mouse mode. Therefore, user operability is improved.

  Furthermore, according to the third time count, when the tablet mode is resumed, the CPU 210 does not resume the time count operation until there is a touch start or end event on the handwriting input area of the liquid crystal panel 240. Therefore, compared to the first time count, the time from the restart of the tablet mode to the execution of operations such as character recognition and determination is longer. In the case of the first time count, if the user does nothing after restarting the tablet mode, character recognition is confirmed at the time surrounded by a circle in FIG. However, when the third time count is performed, character recognition is not yet determined at this point.

  Therefore, the user can operate the sub-application with a sufficient margin after restarting the tablet mode. Since the trigger for restarting the count operation is a user operation, the CPU 210 does not perform an operation unintended by the user without a user instruction after the tablet mode is restarted.

(4th time count: Count stop + Reset + Restart by input)
In the above description of the third time count, the CPU 210 resumes the count operation when the first touch starts or ends after the tablet mode is resumed. However, in addition to these operations, it is not essential for the CPU 210 to reset the counter value at the start or end of the first touch after restarting the tablet mode.

  Therefore, when performing the fourth time count, the CPU 210 stops the time count during the mouse mode as in the case of performing the first to third time counts. Further, as in the case of the second time count, the CPU 210 resets the counter value when shifting from the tablet mode to the mouse mode. Furthermore, as in the case of the third time count, the CPU 210 stops the counting operation until the touch start or end of the liquid crystal panel 240 after the tablet mode is resumed.

  Referring to FIG. 44, the operation of CPU 210 when the fourth time count is performed will be described. FIG. 44 is a diagram for describing the fourth time count. FIG. 44 shows an operation example of the CPU 210 when the user switches between the mouse mode and the tablet mode in the middle of handwriting input, as in the case of FIGS. 38 to 43. In FIG. 44, the change of the counter value when performing the fourth time count is indicated by a solid line. In FIG. 44, for reference, the change in the counter value when the first time count or the third time count is performed is indicated by a dotted line. As shown in FIG. 44, the CPU 210 resets the counter in addition to the transition from the tablet mode to the mouse mode as shown in FIG. 44, [I] when returning from the mouse mode to the tablet mode, and the liquid crystal panel 240 after the tablet mode is resumed. [II] at the start of touching the touch, [III] at the end of touch, etc.

  Compared to the third time count that triggers the restart of the counter when the touch operation start or end event is triggered without resetting the counter, the fourth time count provides the user with more operating margin. Can do.

<Process flow>
(Basic flow)
With reference to FIG. 45, the flow of processing performed by electronic device 100 according to the present embodiment will be described. FIG. 45 is a flowchart showing the flow of processing performed by electronic device 100. In FIG. 45, the processing performed by the control unit 350 in the first unit 1001 and the processing performed by the control unit 450 in the second unit 1002 are collectively shown.

  In step S <b> 101, the control unit 350 and the control unit 450 perform a normal activation process or a resumption process when receiving a normal activation instruction or a resumption instruction of the electronic device 100.

  Normal startup is startup from a power-off state, as already described. Examples of the normal startup process performed by the control unit 350 include a boot process and a boot screen display on the liquid crystal panel 140. The normal startup process performed by the control unit 450 includes displaying a boot screen on the liquid crystal panel 240.

  Control unit 350 and control unit 450 handle pressing of a predetermined button (such as power switch 191) in the power-off state as a normal activation instruction. One control unit (control unit 450 or 350) accepts a normal activation instruction, performs its own normal activation process, and instructs the other control unit (control unit 350 or 450) to perform a normal activation process. The structure to give may be sufficient.

  As already described, the restart is a start from the power saving state. The restart process performed by the control unit 350 includes reading of a work state stored in the RAM 171 or the HDD 170 or displaying a restarting screen on the liquid crystal panel 140. The restart process performed by the control unit 450 includes reading of a work state stored in the RAM 271 or the HDD 170 or displaying a restarting screen on the liquid crystal panel 240.

  The control unit 350 and the control unit 450 handle pressing of a predetermined button (power switch 191 or the like) in the power saving state, touching the liquid crystal panel 240, or the like as an instruction for restart processing. One control unit (450 or 350) may perform the restart process in response to an instruction from the other control unit (350 or 450) that has received the restart instruction.

  In step S103, the mode setting unit 454 included in the control unit 450 determines the operation mode to be either the mouse mode or the tablet mode.

  In the present embodiment, the operation mode for normal activation is determined to be the mouse mode. In the case of normal activation, mode setting unit 454 sets mode data 437 in storage unit 430 to data representing a mouse mode.

  In addition, the mode setting unit 454 determines the mode data 437 based on the operation mode of the mode data 437 before the power saving state when restarting. In this case, the control unit 450 stores the mode data 437 before the power saving state in the RAM 271 or the like when shifting to the power saving state or at the time of restarting. Alternatively, the mode setting unit 454 may always set the mode data 437 to data representing the mouse mode when resuming. In this case, the controller 450 does not have to perform the storage process of the mode data 437 before the power saving state.

  In step S105, the input processing unit 452 determines whether the operation mode is the mouse mode. That is, the input processing unit 452 determines whether the operation mode is the mouse mode based on the mode data 437.

  If the operation mode is the mouse mode (YES in step S105), control unit 350 and control unit 450 proceed to the mouse mode operation in step S107. If the operation mode is not the mouse mode (NO in step S105), control unit 350 and control unit 450 proceed to the tablet mode operation in step S113.

  In step S107, control unit 350 and control unit 450 perform a mouse mode operation. That is, the control unit 350 and the control unit 450 control each unit of the electronic device 100 so that the input to the liquid crystal panel 240 causes the mouse operation of the main application. Details of the mouse mode operation will be described later.

  In step S109, the mode setting unit 454 determines whether a mode switching instruction has been accepted. Specifically, mode setting unit 454 determines whether a signal in response to pressing of center key 242 has been received. However, the mode switching instruction is not limited to pressing the center key 242.

  If there is a mode switching instruction during the mouse mode (YES in step S109), control unit 450 executes a switching process from the mouse mode to the tablet mode in step S111. When there is no mode switching instruction (NO in step S109), control unit 350 and control unit 450 repeat the processing from step S107 (mouse mode operation).

  In step S111, the control unit 450 executes a switching process from the mouse mode to the tablet mode. For example, in step S111, the control unit 450 displays the operation screen of the sub application on the liquid crystal panel 240 and switches the operation of the panel input processing unit 453. Details of the switching process from the mouse mode to the tablet mode will be described later. After step S111 ends, control unit 350 and control unit 450 proceed to the tablet mode operation of step S113.

  In step S113, control unit 350 and control unit 450 perform a tablet mode operation. That is, the control unit 350 and the control unit 450 control each unit of the electronic device 100 so that the sub application operates in response to an input to the liquid crystal panel 240. Details of the tablet mode operation will be described later.

  In step S115, mode setting unit 454 determines whether a mode switching instruction has been accepted. Specifically, mode setting unit 454 determines whether a signal in response to pressing of center key 242 has been received. However, the mode switching instruction is not limited to pressing the center key 242.

  If there is a mode switching instruction during the tablet mode (YES in step S115), control unit 450 executes a switching process from the tablet mode to the mouse mode in step S117. When there is no mode switching instruction (NO in step S115), control unit 350 and control unit 450 repeat the processing from step S113 (tablet mode operation).

  In step S117, the control unit 450 executes switching processing from the tablet mode to the mouse mode. For example, the control unit 450 displays the mouse screen on the liquid crystal panel 240 and switches the operation of the panel input processing unit 453 in step S117. Details of the switching process from the tablet mode to the mouse mode will be described later. After completion of step S117, control unit 350 and control unit 450 proceed to the mouse mode operation of step S107.

  Note that the control unit 350 and the control unit 450 perform a power-off process or a power-saving state transition process when receiving a power-off instruction or a power-saving state transition instruction. These processes are interrupt processes, and are performed after any of the steps in FIG. However, these processes are not shown in FIG.

(Mouse mode operation)
The mouse mode operation in step S107 in FIG. 45 will be described in detail with reference to FIG. FIG. 46 is a flowchart illustrating the processing flow of the mouse mode operation.

  First, the operation of the control unit 350 on the first unit 1001 side will be described. The operation flow of the control unit 350 is shown on the left side in FIG.

  In step S <b> 201, the control unit 350 acquires coordinate data from the interface unit 340. The coordinate data is transmitted from the control unit 450 on the second unit 1002 side to the interface unit 340 via the interface unit 440.

  In step S203, control unit 350 determines the cursor position based on the coordinate data. More specifically, the program execution unit 358 included in the control unit 350 performs the process of step S203. The program execution unit 358 executes the program 334 to determine the cursor position.

  In step S <b> 205, the control unit 350 acquires a command from the interface unit 340. This command is transmitted from the control unit 450 on the second unit 1002 side to the interface unit 340 via the interface unit 440.

  In step S207, control unit 350 performs an application operation according to the command. Specifically, program execution unit 358 executes program 334 to perform application operations. The program execution unit 358 determines an application operation based on the application type, the cursor position, and the command type.

  The application operation is similar to the operation associated with a mouse click of a currently popular application. The application operation includes, for example, selection or activation of a file or folder at the cursor position, or execution of processing according to a button (minimum (maximum) button, close button, etc.) at the cursor position.

  Next, the operation of the control unit 450 on the second unit 1002 side will be described. The operation flow of the controller 450 is shown on the right side in FIG.

  In step S301, the panel input processing unit 453 included in the input processing unit 452 of the control unit 450 acquires a scan image from the liquid crystal panel 240 (panel input unit 422).

  In step S303, the panel input processing unit 453 calculates coordinate data specifying the input position to the liquid crystal panel 240 based on the scanned image acquired in step S301.

  In step S305, the panel input processing unit 453 controls the interface unit 440 and transmits coordinate data to the interface unit 340 on the first unit 1001 side.

  In step S307, the panel input processing unit 453 determines whether a predetermined scan cycle time has elapsed since the execution of step S301. If the scan cycle time has elapsed (YES in step S307), panel input processing unit 453 repeats the processing from step S301 (scanned image acquisition). If the scan cycle time has not elapsed (NO in step S307), control unit 450 proceeds to the process in step S309.

  In step S309, the input processing unit 452 determines whether there has been a click operation. Specifically, the input processing unit 452 determines whether the left click key 241 or the right click key 243 has been pressed. Note that the input processing unit 452 may determine that the tap operation on the liquid crystal panel 240 is a click operation. Specifically, the input processing unit 452 determines that a tap operation has been performed when the detection of an external object is started and ended in a specific area in the liquid crystal panel 240 within a predetermined short time.

  If there is no click operation (NO in step S309), control unit 450 repeats the processing from step S307. If there is a click operation (YES in step S309), control unit 450 proceeds to the process of step S311.

  In step S311, the input processing unit 452 controls the interface unit 440, and transmits a command to the interface unit 340 on the first unit 1001 side according to the click operation in step S309.

  Here, it is assumed that the input processing unit 452 determines the type of command according to the type of click operation. For example, the input processing unit 452 transmits different commands when the left click key 241 is pressed and when the right click key 243 is pressed.

  In the above, the command transmission by the click operation has been described. However, the operation that causes the command transmission is not limited to the click operation. For example, the input processing unit 452 may transmit a command corresponding to a double click or drag.

(Tablet mode operation)
The tablet mode operation in step S113 in FIG. 45 will be described in detail with reference to FIG. FIG. 47 is a flowchart showing the flow of processing of the tablet mode operation.

  First, the operation of the control unit 350 on the first unit 1001 side will be described. The operation flow of the control unit 350 is shown on the left side in FIG.

  In step S <b> 401, the control unit 350 acquires data from the interface unit 340. The “data” here is created by the control unit 450 on the second unit 1002 side executing the sub-application.

  Specifically, the “data” is, for example, character (or number) data or illustration data. The data may be a command. For example, when “Enter” on the operation screen of the handwritten character input pad is touched, the control unit 350 creates a command.

  In step S403, control unit 350 performs an application operation corresponding to the data. Specifically, program execution unit 358 executes program 334 to perform application operations. The program execution unit 358 processes the data acquired in step S401 by the main application being executed. When step S403 ends, the control unit 350 returns to the process of step S401.

  Next, the operation of the control unit 450 on the second unit 1002 side will be described. The flow of the operation of the control unit 450 is shown on the right side in FIG.

  In step S501, the panel input processing unit 453 included in the input processing unit 452 of the control unit 450 acquires a scan image from the liquid crystal panel 240 (panel input unit 422).

  In step S503, the panel input processing unit 453 calculates coordinate data specifying the input position to the liquid crystal panel 240 based on the scanned image acquired in step S501.

  In step S505, the panel input processing unit 453 determines whether a predetermined scan cycle time has elapsed since the execution of step S501. When the scan cycle time has elapsed (YES in step S505), panel input processing unit 453 repeats the processing from step S501 (scanned image acquisition). If the scan cycle time has not elapsed (NO in step S505), control unit 450 proceeds to the process in step S507.

  In step S507, control unit 450 performs a sub-application operation. The sub application operation includes operation execution by time count. In step S507, the control unit 450 may control the interface unit 440 to transmit data to the first unit 1001 side. Details of the sub application operation (sub application operation in the tablet mode) in step S507 will be described later.

  In step S <b> 509, the control unit 450 stores data (referred to as “operation element”) for determining the operation of the application being executed in the storage unit 430. In the present embodiment, the operation parameter 435 including the elapsed time from the event and the input history 432 correspond to the operation elements. Note that the control unit 450 performs the processing in step S509 at a predetermined time interval, when changing an operating element, when saving an operating element (for example, when saving an illustration), and the like. After executing step S509, control unit 450 repeats the processing from step S505.

(Mode switching: mouse mode to tablet mode)
The mode switching (from the mouse mode to the tablet mode) in step S111 in FIG. 45 will be described in detail with reference to FIG. FIG. 48 is a flowchart showing the flow of processing for mode switching (from mouse mode to tablet mode).

  In step S601, the mode setting unit 454 included in the control unit 450 loads the operation definition of the immediately preceding application stored in the storage unit 430.

  Here, the “immediate application” is a sub application that was last operated in the tablet mode before the mode setting unit 454 received a mode switching instruction to the tablet mode. In the present embodiment, since the sub application continues to operate even in the mouse mode, the immediately preceding application is the same as the sub application operating at the time of mode switching instruction.

  In step S603, the program execution unit 458 included in the control unit 450 executes the immediately preceding application based on the operation definition loaded in step S601. Then, the program execution unit 458 controls the display control unit 456 to display the operation screen of the sub application on the display unit 410.

  However, the program execution unit 458 may perform the process of step S601 at any time during the process of step S603. That is, during the execution of the immediately preceding application, the operation definition stored in the storage unit 430 may be read as necessary, and the sub application may be executed based on the read data.

  In step S605, the panel input processing unit 453 included in the control unit 450 switches the processing method of the signal from the panel input unit 422. That is, the panel input processing unit 453 converts the signal from the panel input unit 422 into an operation instruction of the sub application.

(Mode switching: tablet mode to mouse mode)
Hereafter, the mode switching (from the tablet mode to the mouse mode) in step S117 in FIG. 45 will be described in detail.

  First, a diagram illustrating an operation at the time of mode switching (tablet mode to mouse mode) of the control unit 450 when performing time count control (first time count and third time count described above) without resetting the time counter. This will be described with reference to FIG. FIG. 49 is a flowchart illustrating a first processing flow of mode switching (tablet mode to mouse mode) operation.

  In step S701, the program execution unit 458 included in the control unit 450 stops the operation of the time counter 459. When program execution unit 458 does not count the time, control unit 450 does not perform the process of step S701. In the case of not counting time, the operation definition is that an application that does not have an action related to time count from the beginning is executed, and that the application has an action that is related to time count but does not perform time count. There are cases where it is set (for example, the mode of the handwriting input pad is “manual mode”).

  In step S <b> 703, when the display control unit 456 included in the control unit 450 receives a signal indicating that a mode switching instruction has been received from the mode setting unit 454, the display unit 456 displays the mouse mode screen on the liquid crystal panel 240 based on the display data 433. Is displayed.

  In step S705, the panel input processing unit 453 included in the control unit 450 switches the processing method of the signal from the panel input unit 422. That is, the panel input processing unit 453 converts the signal from the panel input unit 422 into a mouse operation signal on the liquid crystal panel 140.

  Next, a diagram illustrating an operation at the time of mode switching (tablet mode to mouse mode) of the control unit 450 in the case of performing time count control (second time count and fourth time count described above) with resetting of the time counter. Reference is made to FIG. FIG. 50 is a flowchart illustrating the second processing flow of the mode switching (tablet mode to mouse mode) operation.

  In step S801, the program execution unit 458 included in the control unit 450 resets the time counter value by the time counter 459. That is, the program execution unit 458 sets the counter value to a predetermined initial value. The initial value is generally “0”, but is not limited thereto.

  The processing from step S803 to step S807 is the same as the processing from step S701 to step S705 shown in FIG. 49, respectively, and description thereof will not be repeated.

(Sub application operation in tablet mode)
From here, the application operation in step S507 of FIG. 47, that is, the sub-application operation in the tablet mode will be described in detail.

  First, the sub application operation in the tablet mode related to the first time count operation (count stop) will be described with reference to FIG. FIG. 51 is a diagram showing a flow of the first sub-application operation in the tablet mode in the form of a flowchart.

  In step S901, the program execution unit 458 included in the control unit 450 starts time counting by the sub application. However, when the time is not counted, program execution unit 458 does not perform the process of step S901.

  In step S903, the program execution unit 458 determines whether or not the time counter value has reached a predetermined value (timeout value). When the counter value reaches the timeout value (YES in step S903), program execution unit 458 proceeds to the process of step S905. If the counter value has not reached the timeout value (NO in step S903), program execution unit 458 proceeds to the process of step S907.

  In step S905, the program execution unit 458 performs a predetermined timeout operation. As described above, the timeout operation includes the start of character recognition and the confirmation of characters. After step S905, the program execution unit 458 proceeds to the process of step S907.

  In step S907, the program execution unit 458 determines whether a data transmission instruction has been received from the input processing unit 452. Here, the “data transmission instruction” is an instruction for instructing the program execution unit 458 to transmit data to the program execution unit 358 of the first unit 1001. Data that is instructed to be transmitted by the data transmission instruction includes data created by the sub-application (for example, characters and hand-drawn illustrations as a result of handwriting recognition), and operation instructions of the main application (for example, instructions when the ENTER button is pressed) including.

  When the data transmission instruction is accepted (YES in step S907), the program execution unit 458 controls the interface unit 440 and transmits data to the program execution unit 358 of the first unit 1001. Thereafter, the program execution unit 358 repeats the processing from step S903 (timeout determination).

  If a data transmission instruction has not been received (NO in step S907), program execution unit 358 repeats the processing from step S903 (timeout determination) without transmitting data.

  Next, the sub application operation in the tablet mode according to the second time count operation (count operation for resetting the time count when the tablet mode is resumed) will be described with reference to FIG. FIG. 52 is a diagram showing a flow of the second sub-application operation in the tablet mode in a flowchart format.

  In step S1001, the program execution unit 458 resets the time counter value. That is, similar to step S801 in FIG. 50, the program execution unit 458 sets the counter value to a predetermined initial value. The initial value is generally “0”, but is not limited thereto.

  The processing from step S1003 to step S1011 is the same as the processing from step S901 to step S909, respectively. Therefore, these descriptions will not be repeated.

  Next, with reference to FIG. 53, the sub-application operation in the tablet mode related to the third time count operation (the count operation for restarting the time count according to the start or end of input after restarting the tablet mode) will be described. To do. FIG. 53 is a flowchart illustrating the third sub-application operation flow in the tablet mode.

  In step S1101, the program execution unit 458 included in the control unit 450 determines whether a touch on the liquid crystal panel 240 has started (or ended). That is, the program execution unit 358 determines whether a signal corresponding to a predetermined input has been received from the input processing unit 452 (or the signal has ended). For example, the program execution unit 358 has started (or ended) a signal from the input processing unit 452 indicating a touch in the liquid crystal panel 240 or a predetermined area (such as a handwriting area) in the liquid crystal panel 240. Judge whether.

  If program execution unit 458 determines that touch on liquid crystal panel 240 has not started (or ended) (NO in step S1101), program execution unit 458 performs the process in step S1101 again. If program execution unit 458 determines that the touch on liquid crystal panel 240 has started (or ended) (YES in step S1101), the program execution unit 458 proceeds to the process of step S1103.

  The processing from step S1103 to step S1111 is the same as the processing from step S901 to step S909, respectively. Therefore, these descriptions will not be repeated.

  Finally, the sub application operation in the tablet mode related to the fourth time count operation (count stop + reset + input start or restart) will be described with reference to FIG. FIG. 54 is a flowchart illustrating the fourth sub-application operation flow in the tablet mode.

  In step S1201, the program execution unit 458 resets the time counter value. That is, the program execution unit 458 sets the counter value to a predetermined initial value.

  In step S1203, the program execution unit 458 included in the control unit 450 determines whether the touch on the liquid crystal panel 240 has started (or ended). This process is the same as step S1101. Therefore, detailed description thereof will not be repeated.

  When program execution unit 458 determines that touch on liquid crystal panel 240 has not started (or ended) (NO in step S1203), program execution unit 458 performs the process of step S1203 again. When program execution unit 458 determines that the touch on liquid crystal panel 240 has started (or ended) (YES in step S1203), the program execution unit 458 proceeds to the process of step S1205.

  The processing from step S1205 to step S1213 is the same as the processing from step S1005 to step S1011, respectively. Therefore, these descriptions will not be repeated.

[Others]
In the embodiment of the present invention, the liquid crystal panel 140 may be a display (single function display) that does not have a function as an input unit and that only displays a screen. This configuration is useful when the liquid crystal panel 140 is large and difficult to form a touch panel.

  The electronic device 100 according to the above-described embodiment continues to execute the sub application even during the mouse mode after the transition from the tablet mode to the mouse mode. However, the electronic device 100 may end the sub application when shifting to the mouse mode.

  In this case, electronic device 100 stores application settings in storage unit 430 during the tablet mode. This is to allow the user to continue to execute the sub-application before shifting to the mouse mode when the tablet mode is resumed. The electronic device 100 stores application settings in the storage unit 430, for example, at predetermined time intervals. Alternatively, electronic device 100 may store application settings in storage unit 430 when shifting to the mouse mode. The electronic device 100 may store the application settings in the storage unit 430 at both of the above timings.

  Furthermore, in the above, a handwritten character input pad has been described as a sub-application with time count processing. However, the sub application that is useful to be executed by electronic device 100 according to the present embodiment is not limited to this. The electronic device 100 can improve user convenience when executing a more general time-varying sub-application.

  For example, consider running video playback software using the electronic device 100. In this case, the electronic device 100 stops counting the moving image progression in the mouse mode, so that the user can continuously view the moving image before and after the mouse mode.

  The first unit 1001 and the second unit 1002 operate independently except for data exchange. Therefore, the second unit 1002 may be removable. Furthermore, the second unit 1002 may be replaceable with another unit (for example, a portable information terminal) having a function equivalent to that of the second unit 1002. Therefore, a system including an electronic device including the first unit 1001 and a unit connected to or connectable to the electronic device can be considered as one embodiment of the present invention.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  DESCRIPTION OF SYMBOLS 100 Electronic device, 100A housing | casing, 100B housing | casing, 100C Hinge, 101 Main body apparatus, 102 Display apparatus, 102A Display apparatus, 103 Display apparatus, 104 Main body apparatus, 130 Driver, 131 Scan signal line drive circuit, 132 Data signal line drive Circuit, 133 photosensor driving circuit, 134 switch, 135 amplifier, 140 photosensor built-in liquid crystal panel, 140A photosensor built-in liquid crystal panel, 141 pixel circuit, 141b subpixel circuit, 141g subpixel circuit, 141r subpixel circuit, 143 electrode pair , 143a Pixel electrode, 143b Counter electrode, 144 Photo sensor circuit, 145 photodiode, 145b photodiode, 145g photodiode, 145r photodiode, 146 capacitor, 151A AC Live matrix substrate, 151B Counter substrate, 152 Liquid crystal layer, 153b Color filter, 153g Color filter, 153r Color filter, 157 Data signal line, 161 Polarization filter, 162 Glass substrate, 163 Shading film, 164 Alignment film, 170 Hard disk, 173 Memory card Reader / writer, 174 External communication unit, 175 microphone, 176 speaker, 177 operation key, 179 backlight, 180 image processing engine, 181 driver control unit, 182 timer, 183 signal processing unit, 191 power switch, 192 power supply circuit, 193 power supply Detection unit, 194 connector, 195 antenna, 196 connector, 230 driver, 240 LCD panel with built-in optical sensor, 241 Left click key, 242 Center key, 243 Click key, 273 timer, 274 external communication unit, 279 backlight, 280 image processing engine, 281 driver control unit, 282 timer, 283 signal processing unit, 293 power supply detection unit, 294 connector, 295 antenna, 297 signal strength detection unit, 310 display unit, 320 input unit, 330 storage unit, 333 display data, 334 program, 335 operation parameter, 340 interface unit, 350 control unit, 352 input processing unit, 356 display control unit, 358 program execution unit, 410 display unit, 420 input unit, 422 panel input unit, 430 storage unit, 431 input data, 432 input history, 432a character data, 432b illustration data, 433 display data, 434 program, 435 operation parameter, 436 Time data, 437 mode data, 440 interface unit, 450 control unit, 452 input processing unit, 453 panel input processing unit, 454 mode setting unit, 456 display control unit, 458 program execution unit, 458 time counter, 500 operation screen, 510 Cursor, 600 mouse screen, 610 guidance display, 612 left guidance display, 614 center guidance display, 616 right guidance display, 700 operation screen, 800 screen, 800a home menu screen, 800b screen, 800c screen, 800d window, 810 guidance display, 812 Left guidance display, 814 Center guidance display, 816 Right guidance display, 820 Operation button display, 830 Mouse disabled display, 840 Home button, 900 finger, 9 0 stylus, 1001 first unit, 1001A unit, 1002 second unit, 1300 an electronic device, 1731 a memory card.

Claims (14)

A display for displaying the first screen;
A display-integrated tablet that displays a second screen and accepts external input;
A storage unit;
Control means for controlling the display, the tablet and the storage unit,
The control means includes
Mode switching means for switching between the first operation mode and the second operation mode of the control means;
Execution means for executing the program,
The execution means includes
In the first operation mode, the program is executed in response to the input to the tablet, and an image created by the executed program is displayed on the display.
In the second operation mode, the program is executed in response to the input to the tablet, and an image created by the executed program is displayed on the tablet.
An operation element that determines an operation of the program executed in the second operation mode is stored in the storage unit, and the program is executed based on the stored operation element after the second operation mode is resumed. Electronic equipment.   The electronic device according to claim 1, wherein the operation element includes an input history representing a history of the input to the tablet in the second operation mode. The execution means counts an elapsed time from a predetermined event that occurs in the second operation mode,
The operating element includes the elapsed time counted;
The electronic device according to claim 1, wherein the execution unit stops the counting operation of the elapsed time during the first operation mode.   The electronic device according to claim 3, wherein the execution unit restarts the stopped counting operation in response to a start or end event of the input to the tablet in the second operation mode.   5. The electronic device according to claim 3, wherein the execution unit recounts the elapsed time from an initial value when restarting the stopped count operation. 6.   5. The electronic device according to claim 3, wherein the execution unit resets the elapsed time counted according to the transition from the second operation mode to the first operation mode. 6.   The electronic device according to claim 3, wherein the execution unit executes a predetermined process by the program when the counted elapsed time exceeds a threshold value.   The electronic device according to claim 7, wherein the execution unit executes the predetermined process when the elapsed time counted from the input to the tablet in the second operation mode exceeds the threshold. .   The execution means, when the elapsed time counted from the input to the tablet in the second operation mode exceeds the threshold, the history of the input to the tablet in the second operation mode The electronic device according to claim 8, wherein character recognition is performed. The program includes a first program and a second program,
The execution means executes the first program in response to an input to the tablet in the first operation mode, and executes the second program in response to an input to the tablet in the second operation mode. The electronic device according to claim 1, wherein the electronic device executes a program. The execution means includes
First execution means;
Second execution means,
The first execution means executes the first program in response to an input to the tablet in the first operation mode,
The electronic device according to claim 10, wherein the second execution unit executes the second program in response to an input to the tablet in the second operation mode. An information processing system comprising a first information processing unit and a second information processing unit,
The first information processing unit includes:
A display for displaying the first screen;
A first interface unit for exchanging data with the second information processing unit;
First control means for controlling the display and the first interface unit,
The first control means includes
First execution means for executing a first program and displaying an image created by the executed first program on the display;
The second information processing unit is
A display-integrated tablet that displays a second screen and accepts external input;
A second interface unit for exchanging data with the first information processing unit;
A second control means for controlling the tablet and the second interface unit;
The second control means includes
Mode switching means for switching between the first operation mode and the second operation mode of the second control means;
Second execution means for executing a second program,
The second execution means includes
In the first operation mode, a command for the first program is created in response to the input to the tablet, and the command is transmitted to the first information processing unit by controlling the second interface unit. And
In the second operation mode, the second program is executed in response to the input to the tablet, and an image created by the executed second program is displayed on the tablet.
An operation element for determining the operation of the second program is stored in a storage device in the information processing system, and the second program is executed based on the stored operation element after the second operation mode is resumed. An information processing system to be executed. An electronic apparatus control method comprising: a display that displays a first screen; a display-integrated tablet that displays a second screen and that can accept external input; and an execution unit that executes a program. There,
Switching between a first operation mode and a second operation mode of the electronic device;
In the first operation mode, executing the program by the execution means in response to the input to the tablet, and displaying an image created by the executed program on the display;
Storing an operation element for determining an operation of the program executed in the second operation mode in the storage unit;
In the second operation mode, executing the program by the execution means in response to the input to the tablet, and displaying an image created by the executed program on the tablet,
The step of executing the program in the second operation mode includes a step of executing the program based on the stored operation element after resuming the second operation mode. A control program for an electronic device having a display that displays a first screen and a display-integrated tablet that displays a second screen and can accept an input from the outside,
Switching between a first operation mode and a second operation mode of the electronic device;
Executing a program in response to the input to the tablet in the first operation mode, and displaying an image created by the executed program on the display;
Storing an operation element that determines an operation of the program to be executed in the second operation mode in the storage unit;
Executing the program in response to the input to the tablet in the second operation mode, and displaying an image created by the executed program on the tablet,
The step of executing the program in the second operation mode includes the step of executing the program based on the stored operation element after the second operation mode is resumed.

Images