JP5526400B2 - Electronic device, information processing system, electronic device control method, and electronic device control program - Google Patents

Description

  The present invention relates to an electronic device, an information processing system, an electronic device control method, and an electronic device control program. In particular, the present invention relates to an electronic device or information processing system having two display screens, and a control method and control program for an electronic device having two display screens.

  In recent years, electronic devices such as personal computers and portable information terminals have become widespread. In recent years, electronic devices having two display screens are becoming popular.

  For example, Patent Document 1 (Japanese Patent Laid-Open No. 2000-172395) discloses a portable information terminal having two screens. This portable information terminal displays the content on one screen and displays the menu bar and slide bar of the content on the other screen.

  Japanese Patent Application Laid-Open No. 2001-306291 discloses an information processing apparatus including a main display device and an auxiliary display device. This information processing apparatus normally displays both content and content additional information on the main display device. The information processing apparatus displays additional information on the sub-screen when displaying the content on the full screen.

  An electronic device disclosed in Patent Document 3 (Japanese Patent Laid-Open No. 2003-202948) has a main display unit and an auxiliary display unit, and displays a connection status of communication of the electronic device on the auxiliary display unit.

  The electronic device disclosed in Patent Document 4 (Japanese Patent Application Laid-Open No. 2003-216297) is based on a combination of a text message, a symbol, a display color and / or blinking display of an auxiliary display unit, and status information of the electronic device Is displayed on the auxiliary display.

  An information processing apparatus disclosed in Patent Document 5 (Japanese Patent Application Laid-Open No. 2004-5105) includes a main display and a display-integrated pointing device. The information processing apparatus further includes a main display controller and a controller dedicated to the pointing device. This information processing apparatus can perform system settings such as BIOS (Basic Input / Output System) by a pointing device before starting an OS (Operating System).

  Similarly to the information processing apparatus described in Patent Document 6 (Japanese Patent Application Laid-Open No. 2004-5212), a main display, a main display controller, a display-integrated pointing device, and a controller dedicated to the pointing device An information processing apparatus including the above is disclosed. The information processing apparatus disclosed in Patent Document 6 displays the display screen of the pointing device on the main display when the display screen of the pointing device is switched to prevent erroneous operation.

JP 2000-172395 A JP 2001-306291 A JP 2003-202948 A JP 2003-216297 A JP 2004-5105 A JP 2004-5212 A

  A sub-display capable of accepting external input as described in Patent Documents 5 and 6 can be used for various purposes.

  First, a user of an electronic device can use the sub-display in the same manner as a touch pad provided in many conventional notebook computers. That is, the user can use the sub display as a pointing device for moving the position of the cursor displayed on the main display, or giving an instruction to the application by clicking or dragging.

  Furthermore, when the operation screen of the application is displayed on the sub display as in the system setting screen of Patent Document 5, the user of the electronic device can operate the application by performing input on the sub display.

  In order to cause the sub-display to perform a plurality of types of operations, the electronic device switches the processing method of input to the sub-display in accordance with key input, time passage, or the like. In a certain situation, the electronic device regards the input to the sub display as an instruction for moving the cursor on the main display or an operation instruction for an application whose execution screen is displayed on the main display. In another situation, the electronic device regards the input to the sub display as a direct instruction to the operation screen displayed on the sub display.

  By the way, Patent Documents 5 and 6 do not consider in detail how to control an electronic device when switching an input processing method. An electronic device that simply uses a conventional technique may cause a decrease in visibility and operability when the processing method is switched. For example, unless the display method of the cursor on the main display is appropriately controlled, the cursor display hinders the visibility of the screen, resulting in a decrease in operability.

  The present invention has been made to solve the above problems, and an object thereof is to provide an electronic device or an information processing system having two display screens and high operability. Furthermore, an object of the present invention is to provide an electronic device control method and a control program for improving the operability of an electronic device having two display screens.

  According to one aspect of the present invention, an electronic device is a display that displays a first screen, a display-integrated tablet that displays a second screen and can accept input from the outside, Control means for controlling operations of the display and the tablet, and the control means includes mode switching means for switching between the first operation mode and the second operation mode of the control means, and execution means for executing the program, In the first operation mode, the execution means executes the program according to the input to the tablet, displays an image created by the executed program on the display, and displays the image according to the change in the position of the input to the tablet. The displayed image is changed, and the program is executed according to the input to the tablet in the second operation mode. Image is displayed on the tablet was to control the display of the display independently of the change in the position of the input to the tablet.

In the first operation mode, the program is executed in accordance with the input to the tablet, and the image created by the executed program is displayed on the display. The cursor is positioned at the position in the display corresponding to the position of the input to the tablet. Display. In the second operation mode, executing a program in response to an input to the tablet and displaying the image created by the executed program on the tablet controls the position of the cursor independently of the position of the input to the tablet Including doing.

The execution means changes the display of the cursor in response to switching from the first operation mode to the second operation mode.

  More preferably, the execution means displays the cursor in a display form different from that in the first operation mode in the second operation mode.

  More preferably, the execution means displays the cursor lighter than the cursor in the first operation mode in the second operation mode.

  More preferably, the execution means stops displaying the cursor on the display in the second operation mode.

  More preferably, the execution means changes the display of the cursor when it is determined in the second operation mode that the cursor overlaps an active window in the display.

  More preferably, the execution means moves the cursor to a predetermined display position in response to switching from the first operation mode to the second operation mode.

  More preferably, the execution means displays the cursor on the end of the display in the second operation mode.

  More preferably, when the execution unit determines that the cursor overlaps an active window in the display in the second operation mode, the execution unit moves the cursor to a predetermined display position.

  More preferably, when the execution unit determines that the cursor is over the active window in the display in the second operation mode, the execution unit moves the cursor to an area outside the window.

  More preferably, when the execution unit determines that the cursor overlaps an active window in the display in the second operation mode, the execution unit displays the cursor on the edge of the display.

  More preferably, the electronic device further includes a storage device, and the execution means stores the position of the cursor in the storage device in the first operation mode, and switches from the second operation mode to the first operation mode. In response, the cursor is displayed at the position of the cursor stored in the storage device.

  More preferably, it further includes an interface to which an external pointing device can be connected, and when the execution means determines that the pointing device is connected to the interface, before and after switching from the first operation mode to the second operation mode. Will display the same cursor.

  More preferably, the program includes a first program and a second program, and the execution means executes the first program in response to an input to the tablet in the first operation mode, and performs a second operation. In the mode, the second program is executed in response to an input to the tablet.

  More preferably, the execution means includes a first execution means and a second execution means, and the first execution means executes the first program in response to an input to the tablet in the first operation mode. The second execution means executes the second program in response to an input to the tablet in the second operation mode.

According to another aspect of the present invention, an information processing system including a first information processing unit and a second information processing unit, wherein the first information processing unit includes a display that displays a first screen; A first interface unit for exchanging data with the second information processing unit; and a first control unit for controlling the display and the first interface unit. The first control unit stores the first program A second execution unit for displaying an image generated by the executed first program and displaying an image generated by the executed first program on the display; A display-integrated tablet that can be received, a second interface unit that exchanges data with the first information processing unit, a tablet, and a second interface Second control means for controlling the second control means, wherein the second control means executes a second program, mode switching means for switching between the first operation mode and the second operation mode of the second control means. Second execution means for generating a first command for changing the display of the first program in response to a change in the position of the input to the tablet in the first operation mode. The second interface unit is controlled to transmit the first command to the first information processing unit, and in the second operation mode, the second program is executed according to the input to the tablet and executed. The image created by the second program is displayed on the tablet, the second command of the first program is created independently of the change in the position of the input to the tablet, and the second interface unit is controlled to 2 Sending a command to the first information processing unit. In the first operation mode, executing the first program according to the input to the tablet and displaying the image created by the executed first program on the display corresponds to the position of the input to the tablet Including displaying a cursor at a position in the display. In the second operation mode, executing the second program in response to an input to the tablet and displaying the image created by the executed second program on the tablet is independent of the position of the input to the tablet. Including controlling the cursor position. The second execution means changes the display of the cursor in response to switching from the first operation mode to the second operation mode.

According to still another aspect of the present invention, a method for controlling an electronic device includes a display that displays a first screen, a display-integrated tablet that displays a second screen, and can accept external input. And a method for controlling the electronic device, the step of switching between the first operation mode and the second operation mode of the electronic device, and in the first operation mode, run the program in response to the input, Ru and a step of displaying on a display images executed program has created. The step of displaying the image on the display in the first operation mode includes the step of changing the image displayed on the display in response to a change in the position of the input to the tablet, and in response to a change in the position of the input to the tablet. The step of changing the image displayed on the display includes displaying a cursor at a position in the display corresponding to the position of the input to the tablet. In the second operation mode, the program is changed according to the input to the tablet. Executing and displaying an image created by the executed program on the tablet, and controlling the display on the display independently of the change in the position of the input to the tablet . The step of switching between the two operation modes is to switch from the first operation mode to the second operation mode. And Flip, including changing the display of the cursor.

According to still another aspect of the present invention, a control program for an electronic device includes a display that displays a first screen, a display-integrated tablet that displays a second screen, and can accept external input. And a control program for an electronic device having an execution means for executing the program, the step of switching between the first operation mode and the second operation mode of the electronic device, and in the first operation mode, run the program in response to the input, Ru and a step of displaying on a display images executed program has created. The step of displaying the image on the display in the first operation mode includes the step of changing the image displayed on the display in response to a change in the position of the input to the tablet, and in response to a change in the position of the input to the tablet. The step of changing the image displayed on the display includes displaying a cursor at a position in the display corresponding to the position of the input to the tablet. In the second operation mode, the program is changed according to the input to the tablet. Executing and displaying an image created by the executed program on the tablet, and controlling the display on the display independently of the change in the position of the input to the tablet . The step of switching between the two operation modes is to switch from the first operation mode to the second operation mode. And Flip, including changing the display of the cursor.

  An electronic device (or information processing system) according to the present invention includes a display that displays a first screen, and a display-integrated tablet that displays a second screen and can accept input from the outside. . Further, the electronic device has a first operation mode and a second operation mode that can be switched to each other.

  In the first operation mode, the electronic device changes the image displayed on the display based on the change in the input position to the tablet. On the other hand, in the second operation mode, the electronic device controls the display of the image on the display independently of the change in the input position to the tablet.

  More specifically, the electronic device directly operates the display based on the input position to the tablet in the first operation mode. In the second operation mode, the electronic device operates the display through an operation based on the input position of the tablet on a UI (user interface) screen displayed on the second screen tablet. In response to switching from the first operation mode to the second operation mode, the electronic device performs control such as suppressing an unnecessary display from appearing on the display due to input to the tablet.

  As a result, according to the present invention, it is possible to provide an electronic device or an information processing system that has two display screens and two types of operation modes and has high operability. Or according to this invention, the control method and control program of an electronic device which improve the operativity of the electronic device which has two display screens and two types of operation modes can be provided.

It is the schematic which shows the external appearance of an electronic device. It is a block diagram showing the hardware constitutions of an electronic device. It is the figure which showed the structure of the liquid crystal panel, and the peripheral circuit of the said liquid crystal panel. It is sectional drawing of a liquid crystal panel and a backlight. It is the figure which showed the timing chart at the time of operating an optical sensor circuit. It is sectional drawing which showed the structure which a photodiode receives the light from a backlight in the case of a scan. It is the figure which showed schematic structure of the command. It is a figure for demonstrating the command of classification "000". FIG. 10 is a diagram for explaining a command of type “001”. It is a figure for demonstrating the command of classification "010". FIG. 10 is a diagram for explaining a command of type “011”. It is a figure for demonstrating the command of classification "100". FIG. 10 is a diagram for explaining a command of type “101”. It is the figure which showed schematic structure of the response data. It is the figure which showed the image (scanned image) obtained by scanning a finger | toe. FIG. 4 is a circuit diagram of an optical sensor built-in liquid crystal panel different from that shown in FIG. 3. It is sectional drawing which showed the structure which a photodiode receives external light in the case of a scan. It is a block diagram showing the hardware constitutions of the modification of an electronic device. It is a figure which shows the functional structure of an electronic device in a block diagram format. It is a figure for demonstrating the screen which an electronic device displays in each of mouse mode and tablet mode. It is a figure which shows the specific example of a mouse | mouth screen. It is a figure which shows the specific example of a home menu screen. It is a transition diagram of the screen displayed on a liquid crystal panel in tablet mode. It is a figure for demonstrating operation | movement of the electronic device at the time of switching to mouse mode and tablet mode. It is the figure which showed typically the operation mode at the time of normal starting. It is the figure which showed typically the operation mode at the time of a return. It is a figure for demonstrating operation | movement of the electronic device at the time of tablet mode. It is a figure which shows an example of a character input screen. It is a 1st figure for demonstrating operation | movement of the electronic device at the time of utilization of a handwritten character input pad. It is a 2nd figure for demonstrating operation | movement of the electronic device at the time of use of a handwritten character input pad. It is a figure for demonstrating operation | movement of the electronic device when a text box is full. It is a figure which shows an example of an illustration input screen. It is a figure which shows an example of a calculator screen. It is a figure which shows an example of an internet screen. It is a figure which shows an example of the internet screen containing a scroll bar. It is a figure which shows an example of a dictionary selection screen. It is a figure which shows an example of the dictionary selection screen containing a scroll bar. It is a figure for demonstrating the change of the display control of the cursor which concerns on 1st Embodiment. It is a figure for demonstrating the change of the display control of the cursor which concerns on the modification of 1st Embodiment. It is a figure which shows the flow of the process which an electronic device performs in a flowchart format. It is a figure which shows the flow of a process of mouse | mouth mode operation | movement in a flowchart format. It is a figure which shows the flow of a process of tablet mode operation | movement in the flowchart format. It is a figure which shows the flow of a process of mode switching (mouse mode to tablet mode) operation | movement in a flowchart format. It is a figure which shows the flow of a process of mode switching (tablet mode to mouse mode) operation | movement in a flowchart format. It is a figure which shows the flow of a change process of the display form of a cursor in a flowchart format. It is a figure which shows the flow of the recovery process of the display form of a cursor in a flowchart format. It is a figure which shows the flow of the process in the case of non-display of a cursor in a flowchart format. It is a figure which shows the flow of a cursor redisplay process in the flowchart format. It is a figure which shows the flow of a process in the case of the movement of the cursor in the case of transfer from mouse mode to tablet mode in a flowchart format. It is a figure which shows the flow of a process in the case of the movement of the cursor in the case of transfer from tablet mode to mouse mode in a flowchart format. It is a figure for demonstrating the change of the display control of the cursor which concerns on 2nd Embodiment. It is a figure which shows the flow of a process of the mode switching (mouse mode to tablet mode) operation | movement which concerns on 2nd Embodiment in flowchart format. It is a figure which shows the flow of a process of the mode switching (tablet mode to mouse mode) operation | movement which concerns on 2nd Embodiment in flowchart format. It is a figure which shows the external appearance of the electronic device which concerns on 3rd Embodiment. It is a block diagram showing the hardware constitutions of an electronic device. It is a figure for demonstrating the change of the display control of the cursor which concerns on 3rd Embodiment. It is a figure which shows the flow of a process of the mouse | mouth mode operation | movement which concerns on 3rd Embodiment in the flowchart format. It is a figure which shows the flow of a process of the tablet mode operation | movement which concerns on 3rd Embodiment in the flowchart format. It is a figure which shows the flow of a process of the mode switching (from mouse mode to tablet mode) operation | movement which concerns on 3rd Embodiment in flowchart format. It is a figure which shows the flow of a process of the mode switching (tablet mode to mouse mode) operation | movement which concerns on 3rd Embodiment in flowchart format.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[First Embodiment]
<Appearance of electronic equipment>
FIG. 1 is a diagram illustrating an appearance of electronic device 100 according to the present embodiment. Referring to FIG. 1, electronic device 100 includes a first housing 100A and a second housing 100B.

  The first casing 100A and the second casing 100B are foldably connected by a hinge 100C. The first housing 100A includes a liquid crystal panel 140 with a built-in optical sensor. The second housing 100B includes an optical sensor built-in liquid crystal panel 240. As described above, the electronic device 100 includes two optical sensor built-in liquid crystal panels. The second casing 100B further includes a left click key 241, a center key 242, and a right click key 243.

  In the present embodiment, electronic device 100 is configured as a laptop personal computer. Electronic device 100 may be configured as a device having a display function, such as a PDA (Personal Digital Assistant), a mobile phone, or an electronic dictionary.

<About hardware configuration>
Next, an aspect of a specific configuration of the electronic device 100 will be described with reference to FIG. FIG. 2 is a block diagram illustrating a hardware configuration of electronic device 100.

  Electronic device 100 includes a first unit 1001 and a second unit 1002. The second unit 1002 is detachably connected to the first unit 1001 from the electronic device 100. The first unit 1001 includes a main body device 101 and a display device 102. The second unit 1002 includes a display device 103 and a main device 104.

  The first housing 100A includes a display device 102. Second housing 100B includes main device 101. The second housing 100B includes a second unit 1002.

(About the first unit)
The main unit 101 includes a CPU (Central Processing Unit) 110, an HDD (Hard Disc Drive) 170, a RAM (Random Access Memory) 171, a ROM (Read-Only Memory) 172, a memory card reader / writer 173, an external device Communication unit 174, microphone 175, speaker 176, operation key 177, power switch 191, power circuit 192, power detection unit 193, USB (Universal Serial Bus) connector 194, antenna 195, LAN ( Local Area Network) connector 196. Each component (110, 171 to 177, 193) is mutually connected by a data bus DB1. A memory card 1731 is attached to the memory card reader / writer 173.

  CPU 110 executes a program. The operation key 177 receives an instruction input from the user of the electronic device 100. The HDD 170 is a storage device that can write and read data. However, the HDD 170 is an example of such a storage device. The electronic device 100 may use a storage device such as a flash memory instead of the HDD 170. The RAM 171 stores data generated by the execution of the program by the CPU 110 or data input via the operation keys 177 in a volatile manner. The ROM 172 stores data in a nonvolatile manner. The ROM 172 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory.

  The external communication unit 174 communicates with other electronic devices. Specifically, the external communication unit 174 communicates with, for example, the second unit 1002 via the USB connector 194. The external communication unit 174 performs wireless communication with the second unit 1002 via the antenna 195, for example. Further, the external communication unit 174 performs wired communication with other electronic devices via the LAN connector 196.

  Note that the main device 101 may communicate with other electronic devices by wireless communication other than Bluetooth (registered trademark). For example, the external communication unit 174 may perform wireless communication with other electronic devices connected to the LAN via a wireless LAN antenna (not shown). Alternatively, wireless communication may be performed with another electronic device via an infrared port (not shown).

The power switch 191 is a switch for starting up the electronic device 100.
When the power switch 191 is turned on, the power supply circuit 192 supplies power to each component connected to the data bus DB1 and the display device 102 via the power detection unit 193. When the power switch 191 is turned on, the power circuit 192 supplies power to the external communication unit 174 without going through the power detection unit 193.

  The power supply detection unit 193 detects the output from the power supply circuit 192. In addition, the power supply detection unit 193 sends information (for example, a voltage value and a current value) regarding the detected output to the CPU 110.

  The USB connector 194 is used to connect the first unit 1001 to the second unit 1002. Note that the main device 101 may include other USB connectors in addition to the USB connector 194.

  The first unit 1001 transmits data to the second unit 1002 via the USB connector 194. The first unit 1001 receives data from the second unit 1002 via the USB connector 194. Further, the first unit 1001 supplies power to the second unit 1002 via the USB connector 194. The first unit 1001 may supply the power from the power supply circuit 192 directly to the second unit 1002 without using the USB connector 194.

  The antenna 195 is used for communication according to the Bluetooth (registered trademark) standard between the first unit 1001 and another communication apparatus (for example, the second unit 1002). The LAN connector 196 is used to connect the electronic device 100 to the LAN.

  The display device 102 includes a driver 130, an optical sensor built-in liquid crystal panel 140 (hereinafter referred to as a liquid crystal panel 140), an internal IF 178, a backlight 179, and an image processing engine 180.

  The driver 130 is a drive circuit for driving the liquid crystal panel 140 and the backlight 179. Various drive circuits included in the driver 130 will be described later.

  The liquid crystal panel 140 is a device having a liquid crystal display function and a photosensor function. That is, the liquid crystal panel 140 can perform image display using liquid crystal and sensing using an optical sensor. Details of the liquid crystal panel 140 will be described later.

  An internal IF (Interface) 178 mediates exchange of data between the main device 101 and the display device 102.

  The backlight 179 is a light source disposed on the back surface of the liquid crystal panel 140. The backlight 179 irradiates the back surface with uniform light.

  The image processing engine 180 controls the operation of the liquid crystal panel 140 via the driver 130. Here, the control is performed based on various data sent from the main apparatus 101 via the internal IF 178. Note that the various data includes commands to be described later. Further, the image processing engine 180 processes data output from the liquid crystal panel 140 and sends the processed data to the main apparatus 101 via the internal IF 178. Further, the image processing engine 180 includes a driver control unit 181, a timer 182, and a signal processing unit 183.

  The driver control unit 181 controls the operation of the driver 130 by sending a control signal to the driver 130. In addition, the driver control unit 181 analyzes a command transmitted from the main device 101. Then, the driver control unit 181 sends a control signal based on the analysis result to the driver 130. Details of the operation of the driver 130 will be described later.

The timer 182 generates time information and sends the time information to the signal processing unit 183.
The signal processing unit 183 receives data output from the optical sensor. Here, since the data output from the optical sensor is analog data, the signal processing unit 183 first converts the analog data into digital data. Further, the signal processing unit 183 performs data processing on the digital data in accordance with the content of the command sent from the main device 101. Then, the signal processing unit 183 sends data (hereinafter referred to as response data) including data after the above data processing and time information acquired from the timer 182 to the main unit 101. The signal processing unit 183 includes a RAM (not shown) that can store a plurality of scan data, which will be described later, continuously.

  The command includes a sensing command for instructing sensing by the optical sensor. Details of the sensing command and the response data will be described later (FIGS. 7, 8, and 14).

  Note that the timer 182 is not necessarily provided in the image processing engine 180. For example, the timer 182 may be provided outside the image processing engine 180 in the display device 102. Alternatively, the timer 182 may be provided in the main body device 101. Further, the microphone 175 and the speaker 176 are not always provided in the electronic device 100, and may be configured so as not to include either or both of the microphone 175 and the speaker 176 depending on the embodiment of the electronic device 100.

  Here, the display device 102 includes a system liquid crystal. The system liquid crystal is a device obtained by integrally forming peripheral devices of the liquid crystal panel 140 on the glass substrate of the liquid crystal panel 140. In the present embodiment, the driver 130 (excluding the circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are integrally formed on the glass substrate of the liquid crystal panel 140. Note that the display device 102 is not necessarily configured using the system liquid crystal, and the driver 130 (excluding a circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are included in the glass substrate. Other substrates may be configured.

(About the second unit)
The second unit 1002 receives power supply from the first unit 1001. Specifically, the second unit 1002 is supplied with power from the power supply circuit 192 of the first unit 1001 by connecting a USB connector 294 described later and the USB connector 194 of the first unit 1001. However, the method of supplying power to the second unit 1002 is not limited to this. For example, the second unit 1001 may receive power directly from the power supply circuit 192.

  The main body device 104 includes a CPU 210, a RAM 271, a ROM 272, a timer 273, a left click key 241, a center key 242, a right click key 243, an external communication unit 274, a power supply detection unit 293, and a USB connector 294. And an antenna 295 and a signal intensity detector 297. Each component (210, 241-243, 271, 272, 273, 274, 293) is mutually connected by a data bus DB2.

  The CPU 210 executes a program. The RAM 271 stores data generated by the execution of the program by the CPU 210 in a volatile manner. The ROM 272 stores data in a nonvolatile manner. The ROM 272 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory.

  The external communication unit 274 communicates with other electronic devices. Specifically, the external communication unit 274 communicates with, for example, the first unit 1001 via the USB connector 294. The external communication unit 274 communicates with the first unit 1001 through the antenna 295, for example.

  Note that main device 104 may communicate with another electronic device (for example, first unit 1001) by wireless communication other than Bluetooth (registered trademark). For example, the external communication unit 274 may perform wireless communication with other electronic devices via an infrared port (not shown).

  The signal strength detection unit 297 detects the strength of the signal received via the antenna 295. Then, the signal strength detection unit 297 sends the detected strength to the external communication unit 274.

  The USB connector 294 is used to connect the second unit 1002 to the first unit 1001.

  The second unit 1002 transmits data to the first unit 1001 via the USB connector 294. The second unit 1002 receives data from the first unit 1001 via the USB connector 294. Furthermore, the second unit 1002 receives power supply from the first unit 1001 via the USB connector 294 as described above. The second unit 1002 stores the electric power supplied from the first unit 1001 in a battery (not shown).

  The antenna 295 is used for communication between the second unit 1002 and the first unit 1001, for example, in accordance with the Bluetooth (registered trademark) standard.

  The power detection unit 293 detects the power supplied via the USB connector 294. In addition, the power supply detection unit 293 sends information about the detected power to the CPU 210.

Further, the main device 104 may have a function of performing infrared communication.
The display device 103 includes a driver 230, an optical sensor built-in liquid crystal panel 240 (hereinafter referred to as “liquid crystal panel 240”), an internal IF 278, a backlight 279, and an image processing engine 280. The image processing engine 280 includes a driver control unit 281, a timer 282, and a signal processing unit 283.

  The display device 103 has the same configuration as the display device 102. That is, the driver 230, the liquid crystal panel 240, the internal IF 278, the backlight 279, and the image processing engine 280 have the same configuration as the driver 130, the liquid crystal panel 140, the internal IF 178, the backlight 179, and the image processing engine 180 in the display device 102. Have each. The driver control unit 281, the timer 282, and the signal processing unit 283 have the same configurations as the driver control unit 181, the timer 182, and the signal processing unit 183 in the display device 102, respectively. Therefore, description of each functional block included in display device 103 will not be repeated.

  By the way, the processing in the electronic device 100 is realized by each hardware and software executed by the CPU 110. Such software may be stored in advance in the ROM 172 or the HDD 170. The software may be stored in a memory card 1731 or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by the memory card reader / writer 173 or other reading device, or downloaded via the communication unit 174 or communication IF (not shown), and then temporarily stored in the ROM 172 or the HDD 170. Stored. The software is read from the ROM 172 or the HDD 170 by the CPU 110 and stored in the RAM 171 in the form of an executable program. CPU 110 executes the program.

  Each component constituting the main device 101 of the electronic device 100 shown in FIG. 2 is a general one. Therefore, it can be said that the essential part of the present invention is software stored in the RAM 171, ROM 172 or HDD 170, memory card 1731 or other storage medium, or software downloadable via a network. Since the hardware operation of main device 101 of electronic device 100 is well known, detailed description will not be repeated.

  The storage medium is not limited to a memory card, but is a CD-ROM, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile). Disc)), IC (Integrated Circuit) cards (excluding memory cards), optical cards, mask ROM, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory), flash ROM, and other semiconductor memories, etc. It may be a medium to be used.

  The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

<Configuration and drive of liquid crystal panel with built-in optical sensor>
Next, the configuration of the liquid crystal panel 140 and the configuration of peripheral circuits of the liquid crystal panel 140 will be described. FIG. 3 is a diagram showing a configuration of the liquid crystal panel 140 and peripheral circuits of the liquid crystal panel 140.

  Referring to FIG. 3, the liquid crystal panel 140 includes a pixel circuit 141, an optical sensor circuit 144, a scanning signal line Gi, a data signal line SRj, a data signal line SGj, a data signal line SBj, and a sensor signal line. SSj, sensor signal line SDj, read signal line RWi, and reset signal line RSi are included. Note that i is a natural number satisfying 1 ≦ i ≦ m, and j is a natural number satisfying 1 ≦ j ≦ n.

  2 includes a scanning signal line driving circuit 131, a data signal line driving circuit 132, an optical sensor driving circuit 133, a switch 134, as peripheral circuits of the liquid crystal panel 140. And an amplifier 135.

  The scanning signal line drive circuit 131 receives the control signal TC1 from the driver control unit 181 shown in FIG. The scanning signal line drive circuit 131 applies a predetermined voltage in order from the scanning signal line G1 to each scanning signal line (G1 to Gm) based on the control signal TC1. More specifically, the scanning signal line driving circuit 131 sequentially selects one scanning signal line from the scanning signal lines (G1 to Gm) per unit time, and a TFT (to be described later) with respect to the selected scanning signal line. A voltage capable of turning on the gate of the thin film transistor 142 is applied (hereinafter referred to as a high level voltage). Note that a low level voltage is applied to a scanning signal line that is not selected without applying a high level voltage.

  The data signal line driving circuit 132 receives image data (DR, DG, DB) from the driver control unit 181 shown in FIG. The data signal line driving circuit 132 applies a voltage corresponding to one row of image data to the 3n data signal lines (SR1 to SRn, SG1 to SGn, SB1 to SBn) for each unit time. Apply sequentially.

  Note that although a driving method called a so-called line-sequential method has been described here, the driving method is not limited to this.

  The pixel circuit 141 is a circuit for setting the luminance (transmittance) of one pixel. Further, m × n pixel circuits 141 are arranged in a matrix. More specifically, m pixel circuits 141 are arranged in the vertical direction in FIG. 3 and n pixel circuits in the horizontal direction.

  The pixel circuit 141 includes an R subpixel circuit 141r, a G subpixel circuit 141g, and a B subpixel circuit 141b. Each of these three circuits (141r, 141g, 141b) includes a TFT 142, a pair of electrode pairs 143 including a pixel electrode and a counter electrode, and a capacitor (not shown).

  In addition, CMOS (Complementary Metal Oxide Semiconductor) capable of forming n-type transistors and p-type transistors can be realized, and the movement speed of carriers (electrons or holes) is several hundreds compared to amorphous silicon thin film transistors (a-Si TFTs). For example, a polycrystalline silicon thin film transistor (p-Si TFT) is used as the TFT 142 in the display device 102 because it is twice as fast. Note that the TFT 142 will be described as an n-channel field effect transistor. However, the TFT 142 may be a p-type channel field effect transistor.

  The source of the TFT 142 in the R subpixel circuit 141r is connected to the data signal line SRj. The gate of the TFT 142 is connected to the scanning signal line Gi. Further, the drain of the TFT 142 is connected to the pixel electrode of the electrode pair 143. A liquid crystal is disposed between the pixel electrode and the counter electrode. The G sub-pixel circuit 141g and the B sub-pixel circuit 141b have the same configuration as the R sub-pixel circuit 141r except that the data signal line to which the source of each TFT 142 is connected is different. Therefore, description of these two circuits (141g, 141b) will not be repeated.

  Here, setting of luminance in the pixel circuit 141 will be described. First, the high level voltage is applied to the scanning signal line Gi. By the application of the high level voltage, the gate of the TFT 142 is turned on. In this manner, with the gate of the TFT 142 turned on, a specified voltage (voltage corresponding to image data for one pixel) is applied to each data signal line (SRj, SGj, SBj). Thereby, a voltage based on the designated voltage is applied to the pixel electrode. As a result, a potential difference is generated between the pixel electrode and the counter electrode. Based on this potential difference, the liquid crystal responds and the luminance of the pixel is set to a predetermined luminance. Note that the potential difference is held by the capacitor (auxiliary capacitor) (not shown) until the scanning signal line Gi is selected in the next frame period.

  The optical sensor drive circuit 133 receives the control signal TC2 from the driver control unit 181 shown in FIG.

  Then, the optical sensor drive circuit 133 sequentially selects one signal line from the reset signal lines (RS1 to RSm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, the voltage VDDR that is higher than usual is applied. Note that a voltage VSSR lower than the voltage applied to the selected reset signal line is kept applied to the unselected reset signal line. For example, the voltage VDDR may be set to 0V and the voltage VSSR may be set to −5V.

  Further, the optical sensor driving circuit 133 sequentially selects one signal line from the readout signal lines (RW1 to RWm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, a voltage VDD higher than usual is applied. Note that the voltage VSSR is applied to the read signal line that is not selected. For example, the value of VDD may be set to 8V.

  The timing for applying the voltage VDDR and the timing for applying the voltage VDD will be described later.

  The optical sensor circuit 144 includes a photodiode 145, a capacitor 146, and a TFT 147. In the following description, it is assumed that the TFT 147 is an n-type channel field effect transistor. However, the TFT 147 may be a p-type channel field effect transistor.

  The anode of the photodiode 145 is connected to the reset signal line RSi. On the other hand, the cathode of the photodiode 145 is connected to one electrode of the capacitor 146. The other electrode of the capacitor 146 is connected to the read signal line RWi. Hereinafter, a connection point between the photodiode 145 and the capacitor 146 is referred to as a node N.

  The gate of the TFT 147 is connected to the node N. The drain of the TFT 147 is connected to the sensor signal line SDj. Further, the source of the TFT 147 is connected to the sensor signal line SSj. Details of sensing using the optical sensor circuit 144 will be described later.

  The switch 134 is a switch provided to switch whether or not to apply a predetermined voltage to the sensor signal lines (SD1 to SDn). The switching operation of the switch 134 is performed by the optical sensor driving circuit 133. Note that the voltage applied to the sensor signal lines (SD1 to SDn) when the switch 134 is turned on will be described later.

  The amplifier 135 amplifies the voltage output from each sensor signal line (SS1 to SSn). The amplified voltage is sent to the signal processing unit 183 shown in FIG.

  The image processing engine 180 controls the timing for displaying an image on the liquid crystal panel 140 using the pixel circuit 141 and the timing for sensing using the optical sensor circuit 144.

  FIG. 4 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179. Referring to FIG. 4, liquid crystal panel 140 includes an active matrix substrate 151A, a counter substrate 151B, and a liquid crystal layer 152. The counter substrate 151B is disposed to face the active matrix substrate 151A. The liquid crystal layer 152 is sandwiched between the active matrix substrate 151A and the counter substrate 151B. The backlight 179 is disposed on the opposite side of the liquid crystal layer 152 with respect to the active matrix substrate 151A.

  The active matrix substrate 151 </ b> A includes a polarizing filter 161, a glass substrate 162, a pixel electrode 143 a that constitutes the electrode pair 143, a photodiode 145, a data signal line 157, and an alignment film 164. Further, although not shown in FIG. 4, the active matrix substrate 151A includes the capacitor 146, the TFT 147, the TFT 142, and the scanning signal line Gi shown in FIG.

  In the active matrix substrate 151A, the polarizing filter 161, the glass substrate 162, the pixel electrode 143a, and the alignment film 164 are arranged in this order from the backlight 179 side. The photodiode 145 and the data signal line 157 are formed on the liquid crystal layer 152 side of the glass substrate 162.

  The counter substrate 151B includes a polarizing filter 161, a glass substrate 162, a light shielding film 163, color filters (153r, 153g, 153b), a counter electrode 143b constituting the electrode pair 143, and an alignment film 164.

  In the counter substrate 151B, the alignment film 164, the counter electrode 143b, the color filters (153r, 153g, 153b), the glass substrate 162, and the polarizing filter 161 are arranged in this order from the liquid crystal layer 152 side. The light shielding film 163 is formed in the same layer as the color filters (153r, 153g, 153b).

  The color filter 153r is a filter that transmits light having a red wavelength. The color filter 153g is a filter that transmits light having a green wavelength. The color filter 153b is a filter that transmits light having a blue wavelength. Here, the photodiode 145 is arranged at a position facing the color filter 153b.

  The liquid crystal panel 140 displays an image by blocking external light or light emitted from a light source such as a backlight 179 or transmitting the light. Specifically, the liquid crystal panel 140 changes the direction of liquid crystal molecules in the liquid crystal layer 152 by applying a voltage between the pixel electrode 143a and the counter electrode 143b, thereby blocking or transmitting the light. However, since the light cannot be completely blocked by the liquid crystal alone, a polarizing filter 161 that transmits only light having a specific polarization direction is provided.

  Note that the position of the photodiode 145 is not limited to the above position, and may be provided at a position facing the color filter 153r or a position facing the color filter 153g.

  Here, the operation of the optical sensor circuit 144 will be described. FIG. 5 is a diagram showing a timing chart when the optical sensor circuit 144 is operated. In FIG. 5, a voltage VINT indicates a potential at the node N in the photosensor circuit 144. A voltage VPIX is an output voltage from the sensor signal line SSj shown in FIG. 3 and is a voltage before being amplified by the amplifier 135.

  The following description will be divided into a reset period for resetting the optical sensor circuit 144, a sensing period for sensing light using the optical sensor circuit 144, and a readout period for reading the sensed result.

  First, the reset period will be described. In the reset period, the voltage applied to the reset signal line RSi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDDR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR). As described above, by applying the high-level voltage to the reset signal line RSi, a current starts to flow in the forward direction (from the anode side to the cathode side) of the photodiode 145. As a result, the voltage VINT which is the potential of the node N has a value represented by the following expression (1). In Equation (1), the forward voltage drop amount in the photodiode 145 is Vf.

VINT = VSSR + | VDDR−VSSR | −Vf (1)
Therefore, the potential of the node N is a value smaller by Vf than the voltage VDDR as shown in FIG.

  Here, since the voltage VINT is not more than the threshold value for turning on the gate of the TFT 147, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change. Further, a difference corresponding to the voltage VINT occurs between the electrodes of the capacitor 146. For this reason, the capacitor 146 accumulates charges corresponding to the difference.

  Next, the sensing period will be described. In the sensing period following the reset period, the voltage applied to the reset signal line RSi instantaneously switches from the high level (voltage VDDR) to the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR).

  Thus, by changing the voltage applied to the reset signal line RSi to the low level, the potential of the node N becomes higher than the voltage of the reset signal line RSi and the voltage of the read signal line RWi. For this reason, in the photodiode 145, the voltage on the cathode side becomes higher than the voltage on the anode side. That is, the photodiode 145 is in a reverse bias state. In such a reverse bias state, when the photodiode 145 receives light from the light source, current starts to flow from the cathode side to the anode side of the photodiode 145. As a result, as shown in FIG. 5, the potential of the node N (that is, the voltage VINT) becomes lower with the passage of time.

  Since the voltage VINT continues to decrease in this way, the gate of the TFT 147 does not turn on. Therefore, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change.

  Next, the reading period will be described. In the readout period following the sensing period, the voltage applied to the reset signal line RSi is kept at the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDD). Here, the voltage VDD is higher than the voltage VDDR.

  Thus, by applying a high level voltage instantaneously to the read signal line RWi, the potential of the node N is raised through the capacitor 146 as shown in FIG. Note that the increase width of the potential of the node N is a value corresponding to the voltage applied to the read signal line RWi. Here, since the potential of the node N (that is, the voltage VINT) is raised to a threshold value that turns on the gate of the TFT 147, the gate of the TFT 147 is turned on.

  At this time, if a constant voltage is applied in advance to the sensor signal line SDj (see FIG. 3) connected to the drain side of the TFT 147, the sensor signal line SSj connected to the source side of the TFT 147 will cause the VPIX in FIG. As shown in the graph, a voltage corresponding to the potential of the node N is output.

  Here, when the amount of light received by the photodiode 145 (hereinafter referred to as the amount of received light) is small, the slope of the straight line shown in the VINT graph of FIG. 5 becomes gentle. As a result, the voltage VPIX is higher than when the amount of received light is large. As described above, the optical sensor circuit 144 changes the value of the voltage output to the sensor signal line SSj in accordance with the amount of light received by the photodiode 145.

  By the way, in the above, the operation | movement was demonstrated paying attention to one optical sensor circuit 144 among the m * n optical sensor circuits which exist. Below, operation | movement of each photosensor circuit in the liquid crystal panel 140 is demonstrated.

  First, the optical sensor drive circuit 133 applies a predetermined voltage to all n sensor signal lines (SD1 to SDn). Next, the photosensor drive circuit 133 applies a voltage VDDR that is higher than normal to the reset signal line RS1. The other reset signal lines (RS2 to RSm) and read signal lines (RW1 to RWm) are kept in a state where a low level voltage is applied. As a result, the n photosensor circuits in the first row in FIG. 3 enter the reset period described above. Thereafter, the n photosensor circuits in the first row enter a sensing period. Further, thereafter, the n photosensor circuits in the first row enter a reading period.

  Note that the timing at which a predetermined voltage is applied to all n sensor signal lines (SD1 to SDn) is not limited to the above timing, and may be any timing that is applied at least before the readout period. .

  When the readout period of the n photosensor circuits in the first row is completed, the photosensor drive circuit 133 applies a voltage VDDR that is higher than usual to the reset signal line RS2. That is, the reset period of the n photosensor circuits in the second row starts. When the reset period ends, the n photosensor circuits in the second row enter a sensing period, and thereafter enter a reading period.

  Thereafter, the processing described above is sequentially performed on the n photosensor circuits in the third row, the n photosensor circuits in the fourth row,..., The n photosensor circuits in the m row. As a result, the sensing result of the first row, the sensing result of the second row,..., The sensing result of the m-th row are output in this order from the sensor signal lines (SS1 to SSn).

  In the display device 102, sensing is performed for each row as described above, and a sensing result is output from the liquid crystal panel 140 for each row. For this reason, hereinafter, the data after the signal processing unit 183 performs the above-described data processing on the data regarding the voltage for m rows from the first row to the m-th row output from the liquid crystal panel 140, This is called “scan data”. That is, scan data refers to image data obtained by scanning a scan target (for example, a user's finger). An image displayed based on the scan data is referred to as a “scanned image”. Further, in the following, sensing is referred to as “scan”.

  Moreover, in the above, although the structure which scans using all the m * n photosensor circuits was mentioned as an example, it is not limited to this. Scanning may be performed on a partial region of the surface of the liquid crystal panel 140 using a photosensor circuit selected in advance.

  In the following, it is assumed that the electronic device 100 can take either of the two configurations. Further, switching between the components is assumed to be performed by a command sent from the main device 101 based on an input via the operation key 177 or the like. Note that when scanning is performed on a partial area on the surface of the liquid crystal panel 140, the image processing engine 180 sets a scan target area. The setting of the area may be configured to be specified by the user via the operation key 177.

  As described above, when scanning is performed on a partial region of the surface of the liquid crystal panel 140, there are the following modes of use for displaying an image. The first is a mode in which an image is displayed in a surface area other than the partial area (hereinafter referred to as a scan area). The second is a mode in which no image is displayed in the surface area other than the scan area. Which mode is used is based on a command sent from the main apparatus 101 to the image processing engine 180.

  FIG. 6 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179, showing a configuration in which the photodiode 145 receives light from the backlight 179 during scanning.

  Referring to FIG. 6, when the user's finger 900 is in contact with the surface of the liquid crystal panel 140, a part of the light emitted from the backlight 179 is part of the user's finger 900 (abbreviated in the contacted area). Reflected on the plane). The photodiode 145 receives the reflected light.

  Even in a region where the finger 900 is not in contact, part of the light emitted from the backlight 179 is reflected by the user's finger 900. Even in this case, the photodiode 145 receives the reflected light. However, since the finger 900 is not in contact with the surface of the liquid crystal panel 140 in the region, the amount of light received by the photodiode 145 is smaller than the region in which the finger 900 is in contact. Note that the photodiode 145 cannot receive most of the light emitted from the backlight 179 that does not reach the user's finger 900.

  Here, by turning on the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the amount of light reflected by the user's finger 900 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), and the liquid crystal panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter, also referred to as a reflected image) obtained by reflecting light with the finger 900.

Note that examples of the scan object other than the finger 900 include a stylus.
By the way, in the present embodiment, a liquid crystal panel is described as an example of the display device of electronic device 100, but another panel such as an organic EL (Electro-Luminescence) panel is used instead of the liquid crystal panel. Also good.

<About data>
Next, commands exchanged between the first unit 1001 and the second unit 1002 and commands exchanged between the main device 101 and the display device 102 in the first unit 1001 will be described.

  FIG. 7 is a diagram showing a schematic configuration of a command. Referring to FIG. 7, the command includes a header DA01, a first field DA02, a second field DA03, a third field DA04, a fourth field DA05, a fifth field DA06, and a spare data area DA07. including.

  FIG. 8 is a diagram for explaining a command of type “000” (that is, a sensing command). The CPU 110 sends a command of type “000” (hereinafter referred to as “first command”) from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 110 sends the first command from the main body device 101 to the display device 102. In the following, the case where the CPU 110 sends the first command from the main unit 101 of the first unit 1001 to the second unit 1002 will be described as an example.

  The CPU 110 writes the command type (“000”), the command transmission destination, and the like in the header DA01. The CPU 110 writes the value of the timing whose number is “1” in the first field DA02. The CPU 110 writes the value of the data type with the number “2” in the second field DA03. The CPU 110 writes the value of the reading method whose number is “3” in the third field DA04. The CPU 110 writes the value of the image gradation number “4” in the fourth field DA05. The CPU 110 writes the value of the resolution with the number “5” in the fifth field DA06.

  The first command in which “00” is set in the first field DA02 requests the image processing engine 280 to transmit scan data at that time. That is, the sensing first command requests transmission of scan data obtained by scanning using the optical sensor circuit of the liquid crystal panel 240 after the image processing engine 280 receives the first command. Further, the first command in which “01” is set in the first field DA02 requests transmission of scan data when the scan result is changed. Further, the first command in which “10” is set in the first field DA02 requests transmission of scan data at regular intervals.

  The first command in which “001” is set in the second field DA03 requests transmission of the coordinate value of the center coordinate in the partial image. In addition, the first command in which “010” is set in the second field DA03 requests transmission of only the partial image whose scan result has changed. Note that the change in the scan result indicates that the previous scan result is different from the current scan result. Furthermore, the first command in which “100” is set in the second field DA03 requests transmission of the entire image.

  Here, the “whole image” is an image generated by the image processing engine 280 based on the voltage output from each optical sensor circuit when scanning using m × n optical sensor circuits. . The “partial image” is a part of the entire image. The reason why the partial image is requested to transmit only the partial image whose scan result has changed will be described later.

  The coordinate value and the partial image or the entire image may be requested at the same time. Further, in the case of a configuration in which scanning is performed on a partial area on the surface of the liquid crystal panel 240, the entire image is an image corresponding to the area to be scanned.

  The first sensing command in which “00” is set in the third field DA04 requests to scan with the backlight 279 turned on. In addition, the first command in which “01” is set in the third field DA04 requests that the backlight 279 be turned off to perform scanning. Note that the configuration for scanning with the backlight 279 off is described later (FIG. 17). Furthermore, the first command in which “10” is set in the third field DA04 requests scanning using both reflection and transmission. Note that the combined use of reflection and transmission refers to scanning a scanning object by switching between a method of scanning with the backlight 279 turned on and a method of scanning with the backlight turned off.

  The first command in which “00” is set in the fourth field DA05 requests monochrome binary image data. Further, the first command in which “01” is set in the fourth field DA05 requests multi-gradation image data. Further, the first command in which “10” is set in the fourth field DA05 requests RGB color image data.

  The first command in which “0” is set in the fifth field DA06 requests image data with high resolution. The first command in which “1” is set in the fifth field DA06 requests image data with a low resolution.

  In addition to the data shown in FIG. 8, the first command includes designation of a region to be scanned (pixel region that drives the optical sensor circuit 144), timing for scanning, lighting timing of the backlight 179, and the like. Is described.

  Note that the image processing engine 280 analyzes the content of the first command, and sends back data (that is, response data) according to the analysis result to the main device 101.

  FIG. 9 is a diagram for explaining a command of type “001” (hereinafter referred to as “second command”). The CPU 110 sends a second command from the main unit 101 of the first unit 1001 to the second unit 1002.

  The CPU 110 writes the command type (“001”), the command transmission destination, and the like in the header DA01. The CPU 110 writes the value of the display request with the number “1” in the first field DA02. CPU 110 writes information on the number / type of number “2” in second field DA03. The CPU 110 writes the value of the display range whose number is “3” in the third field DA04. The CPU 110 writes information related to the image data with the number “4” in the fourth field DA05.

  The second command in which “001” is set in the first field DA02 requests the image processing engine 280 to display an image on the liquid crystal panel 240 (sub-screen). The second command in which “010” is set in the first field DA02 requests the image processing engine 280 to display an icon on the liquid crystal panel 240. Further, the second command in which “011” is set in the first field DA02 requests the image processing engine 280 to display the handwriting area on the liquid crystal panel 240.

  The second field DA03 stores the number of images to be displayed on the liquid crystal panel 240 and a number for designating the type of handwriting language. The image processing engine 280 performs processing according to the number of the images or the language type.

  The second command in which “01” is set in the third field DA04 requests the image processing engine 280 to designate the display range on the liquid crystal panel 240 by coordinates. Further, the second command in which “10” is set in the third field DA04 requests the image processing engine 280 to set the display range on the liquid crystal panel 240 to the entire display area.

  The fourth field DA05 stores image data to be displayed on the liquid crystal panel 240 and position information for displaying the image data. The image processing engine 280 performs processing for displaying the image data at a position specified by the position information.

  FIG. 10 is a diagram for explaining a command of type “010” (hereinafter referred to as “third command”). The CPU 110 sends the third command from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 210 sends a third command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPUs 110 and 210 write the command type (“010”), the command transmission destination, and the like in the header DA01. The CPUs 110 and 210 write the value of the OS (Operating System) processing request with the number “1” in the first field DA02. The CPUs 110 and 210 write the value of the OS information with the number “2” in the second field DA03.

  The third command in which “01” or “10” is set in the first field DA02 is transmitted from the second unit 1002 to the first unit 1001.

  The third command in which “01” is set in the first field DA02 requests the first unit 1001 to transmit information indicating the type of OS of the first unit 1001 (main device). The third command in which “10” is set in the first field DA02 requests the first unit 1001 to start the OS specified by the OS information.

  The third command in which “000”, “001”, or “010” is set in the second field DA03 is transmitted from the second unit 1002 to the first unit 1001.

  The third command in which “000” is set in the second field DA03 does not request the activation of the OS in the first unit 1001. The third command in which “001” is set in the second field DA03 indicates that the second unit 1002 has selected to start the first OS. Furthermore, the third command in which “010” is set in the second field DA03 indicates that the second unit 1002 has selected to start the second OS.

  FIG. 11 is a diagram for explaining a command of type “011” (hereinafter referred to as “fourth command”). The CPU 210 sends a fourth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPU 210 writes the command type (“011”), the command transmission destination, and the like in the header DA01. The CPU 210 writes information related to the activated application with the number “1” in the first field DA02. The CPU 210 writes the startup information with the number “2” in the second field DA03.

  In the first field DA02, information specifying an application to be activated in the first unit 1001 is stored. The second field DA03 stores information used at the time of activation setting and information used after activation.

  FIG. 12 is a diagram for explaining a command of type “100” (hereinafter referred to as “fifth command”). The CPU 210 sends the fifth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPU 210 writes the command type (“100”), the command transmission destination, and the like in the header DA01. The CPU 210 writes information related to the reception request with the number “1” in the first field DA02. The CPU 210 writes information relating to the number “2” in the second field DA03. The CPU 210 writes information related to the file with the number “3” in the third field DA04.

  The fifth command in which “01” is set in the first field DA02 requests the first unit 1001 to receive a file. In the second field DA03, the number of files transmitted from the second unit 1002 to the first unit 1001 is stored. Further, a file transmitted from the second unit 1002 to the first unit 1001 is stored in the third field DA04.

  FIG. 13 is a diagram for explaining a command of type “101” (hereinafter referred to as “sixth command”). The CPU 110 sends the sixth command from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 210 sends the sixth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPUs 110 and 210 write the command type (“101”), the command transmission destination, and the like in the header DA01. The CPUs 110 and 210 write the value of the communication type with the number “1” in the first field DA02. The CPUs 110 and 210 write the value of the connection destination with the number “2” in the second field DA03. The CPUs 110 and 210 write the value of the transfer destination with the number “3” in the third field DA04. The CPUs 110 and 210 write the value of the signal strength acquisition timing number “4” in the fourth field DA05.

  The sixth command in which “001” is set in the first field DA02 requests the counterpart device to perform infrared communication. The sixth command in which “010” is set in the first field DA02 requests the counterpart device to perform wireless communication by Bluetooth (registered trademark). Furthermore, the sixth command in which “011” is set in the first field DA02 requests the counterpart device to perform LAN communication.

  The sixth command in which “000” is set in the second field DA03 indicates that there is no information for designating the connection destination of communication.

  Further, the sixth command in which “001” is set in the second field DA03 is transmitted by the first unit 1001 to the device to which the first unit 1001 is connected. Such a sixth command requests transmission of information related to a device to which the first unit 1001 is connected.

  Further, the sixth command in which “010” is set in the second field DA03 is transmitted by the second unit 1002 to the first unit 1001 to which the second unit 1002 is connected. Such a sixth command requests transmission of information regarding the first unit 1001 to which the second unit 1002 is connected.

  Further, the sixth command in which “011” is set in the second field DA03 is transmitted by the second unit 1002 to the first unit 1001 to which the second unit 1002 is connected. Such a sixth command requests that information regarding the second unit 1002 be set as connection destination device information.

  Further, the sixth command in which “100” is set in the second field DA03 is transmitted by the first unit 1001 to a device to which the first unit 1001 is connected (for example, the second unit 1002). Such a sixth command requests that information regarding the first unit 1001 be set as connection destination device information.

  The sixth command in which “000” is set in the third field DA04 indicates that it does not have information specifying the transfer destination of data (for example, a file).

  The sixth command in which “001” is set in the third field DA04 is transmitted by the first unit 1001 to the data transfer destination device. Such a sixth command requests transmission of information relating to the data transfer destination device.

  Further, the sixth command in which “010” is set in the third field DA04 is transmitted by the second unit 1002 to the first unit 1001 that is the data transfer destination. Such a sixth command requests transmission of information related to the first unit 1001 of the data transfer destination.

  The sixth command in which “011” is set in the third field DA04 is transmitted by the second unit 1002 to the first unit 1001 that is the data transfer destination. Such a sixth command requests that information regarding the second unit 1002 be set as device information of the data transfer source.

  Furthermore, the sixth command in which “100” is set in the third field DA04 is transmitted by the first unit 1001 to the data transfer destination device (for example, the second unit 1002). Such a sixth command requests that information regarding the first unit 1001 be set as device information of the data transfer source.

  The sixth command in which “00”, “01”, “10”, or “11” is set in the fourth field DA05 is transmitted by the first unit 1001 to the second unit 1002.

  The sixth command in which “00” is set in the fourth field DA05 does not request the second unit 1002 to transmit data indicating the signal strength. In addition, the sixth command in which “01” is set in the fourth field DA05 requests the signal strength detection unit 297 to transmit data indicating the signal strength at that time. Furthermore, the sixth command in which “10” is set in the fourth field DA05 requests transmission of data indicating the signal strength when the signal strength is changed. Further, the sixth command in which “11” is set in the fourth field DA05 requests the transmission of data indicating the signal strength at regular intervals.

  FIG. 14 is a diagram showing a schematic configuration of response data. The response data is data corresponding to the content of the first command (sensing command).

  When the first command is transmitted from the main device 101 to the second unit 1002, the CPU 210 transmits response data from the display device 103 to the first unit 1001. When the first command is transmitted from the main device 101 to the display device 102 of the first unit 1001, the image processing engine 180 transmits response data from the image processing engine 180 to the main device 101. Hereinafter, a case where the first command is transmitted from the main device 101 to the second unit 1002 will be described as an example.

  Referring to FIG. 14, the response data includes a header data area DA11, a coordinate data area DA12, a time data area DA13, and an image data area DA14. Here, the value of the center coordinates of the partial image is written in the data area DA12 indicating the coordinates. The time information acquired from the timer 282 of the image processing engine 280 is written in the data area indicating the time. Further, image data (that is, scan data) after being processed by the image processing engine 280 is written in the data area indicating the image.

  FIG. 15 is a diagram illustrating an image obtained by scanning the finger 900 (that is, a scanned image). Referring to FIG. 15, an image of a region W1 surrounded by a thick solid line is an entire image, and an image of a region P1 surrounded by a broken line is a partial image. The center point C1 of the cross indicated by a thick line is the center coordinate.

  In the present embodiment, a pixel (that is, a predetermined gradation or a pixel having a photosensor circuit which is a rectangular region and whose output voltage from the sensor signal line SSj is equal to or higher than a predetermined value). An area including all of the pixels having a luminance equal to or higher than a predetermined luminance is set as a partial image area.

  The center coordinates are coordinates determined in consideration of the gradation of each pixel in the partial image area. Specifically, for each pixel in the partial image, the center coordinate is determined by performing a weighting process based on the gradation of the pixel and the distance between the pixel and the rectangular center point (that is, the centroid). Is done. That is, the center coordinates do not necessarily match the centroid of the partial image.

  However, the position of the center coordinates is not necessarily limited to the above, and the center coordinates may be the coordinates of the centroid or the coordinates near the centroid.

  When “001” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the center coordinate in the data area DA12 indicating the coordinates. In this case, the image processing engine 280 does not write image data in the data area DA14 indicating an image. After writing the value of the center coordinate, the image processing engine 280 sends response data including the value of the center coordinate to the main body device 104. The main device 104 sends response data including the value of the center coordinate to the main device 101 of the first unit 1001. As described above, when “001” is set in the data area indicating the data type, the first command does not request output of the image data but requests output of the value of the center coordinate.

  When “010” is set in the data area indicating the data type of the first command, the image processing engine 280 stores the image data of the partial image whose scan result has changed in the data area DA14 indicating the image. Write. In this case, the image processing engine 280 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 280 writes the image data of the partial image whose scan result has changed, and then sends response data including the image data of the partial image to the main body device 104. The main device 104 sends response data including image data of the partial image to the main device 101 of the first unit 1001. Thus, when “010” is set in the data area indicating the data type, the first command does not request the output of the value of the center coordinate, and the image data of the partial image whose scan result has changed. Request output.

  As described above, the reason for requesting transmission of only the partial image whose scan result has changed is that the scan data of the partial image area of the scan data is more important than the scan data of the other area. This is because the data is high, and the scan data in the region corresponding to the region of the partial image in the scan data is likely to change due to the contact state with the scan object such as the finger 900.

  When “011” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the center coordinate in the data area DA12 indicating the coordinate and also indicates the data indicating the image. The image data of the partial image whose scan result has changed is written in the area DA14. Thereafter, the image processing engine 280 sends response data including the value of the center coordinate and the image data of the partial image to the main body device 104. The main device 104 sends response data including the value of the center coordinate and the image data of the partial image to the main device 101 of the first unit 1001. Thus, when “011” is set in the data area indicating the data type, the first command outputs the value of the center coordinate and the output of the image data of the partial image whose scan result has changed. Request.

  Further, when “100” is set in the data area indicating the data type of the first command, the image processing engine 280 displays the entire image in the data area DA14 indicating the image of the response data shown in FIG. Write image data. In this case, the image processing engine 280 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 280 writes the image data of the entire image, and then sends response data including the image data of the entire image to the main body device 104. The main device 104 sends response data including image data of the entire image to the main device 101 of the first unit 1001. As described above, when “100” is set in the data area indicating the data type, the first command requests the output of the image data of the entire image without requesting the output of the center coordinate value. .

  If “101” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the central coordinate in the data area DA12 indicating the coordinate and also indicates the data indicating the image. The image data of the entire image is written in the area DA14. Thereafter, the image processing engine 280 sends response data including the value of the center coordinate and the image data of the entire image to the main body device 104. The main device 104 sends response data including the value of the center coordinate and the image data of the entire image to the main device 101 of the first unit 1001. As described above, when “101” is set in the data area indicating the data type, the first command requests the output of the value of the center coordinate and the output of the image data of the entire image.

<Regarding First Modification of Configuration>
By the way, the structure of the liquid crystal panel 140 is not limited to the structure shown in FIG. Hereinafter, a liquid crystal panel having a mode different from that in FIG. 3 will be described.

  FIG. 16 is a circuit diagram of a photosensor built-in liquid crystal panel 140A according to the different embodiment. Referring to FIG. 16, photosensor built-in liquid crystal panel 140A (hereinafter referred to as liquid crystal panel 140A) includes three photosensor circuits (144r, 144g, 144b) in one pixel. Thus, the liquid crystal panel 140A is different from the liquid crystal panel 140 including one photosensor circuit in one pixel in that the liquid crystal panel 140A includes three photosensor circuits (144r, 144g, and 144b) in one pixel. The configuration of the optical sensor circuit 144 is the same as that of each of the three optical sensor circuits (144r, 144g, 144b).

  Further, the three photodiodes (145r, 145g, 145b) in one pixel are arranged at positions facing the color filter 153r, the color filter 153g, and the color filter 153b, respectively. Therefore, the photodiode 145r receives red light, the photodiode 145g receives green light, and the photodiode 145b receives blue light.

  In addition, since the liquid crystal panel 140 includes only one photosensor circuit 144 in one pixel, two data signal lines for the TFT 147 disposed in one pixel are a sensor signal line SSj and a sensor signal line SDj. Met. However, since the liquid crystal panel 140A includes three photosensor circuits (144r, 144g, 144b) in one pixel, there are six data signal lines for TFTs (147r, 147g, 147b) arranged in one pixel. It becomes.

  Specifically, the sensor signal line SSRj and the sensor signal line SDRj are arranged corresponding to the TFT 147r connected to the cathode of the photodiode 145r arranged at a position facing the color filter 153r. A sensor signal line SSGj and a sensor signal line SDGj are arranged corresponding to the TFT 147g connected to the cathode of the photodiode 145g arranged at a position facing the color filter 153g. Further, a sensor signal line SSBj and a sensor signal line SDBj are disposed corresponding to the TFT 147b connected to the cathode of the photodiode 145b disposed at a position facing the color filter 153b.

  In such a liquid crystal panel 140A, the white light emitted from the backlight 179 passes through the three color filters (153r, 153g, 153b), and red, green, and blue are displayed on the surface of the liquid crystal panel 140A. It mixes and becomes white light. Here, when white light is reflected by the scan object, a part of the white light is absorbed by the pigment on the surface of the scan object, and a part is reflected. And the reflected light permeate | transmits three color filters (153r, 153g, 153b) again.

  At this time, the color filter 153r transmits light having a red wavelength, and the photodiode 145r receives light having the red wavelength. The color filter 153g transmits light having a green wavelength, and the photodiode 145g receives light having the green wavelength. The color filter 153b transmits light having a blue wavelength, and the photodiode 145b receives light having the blue wavelength. That is, the light reflected by the scan object is separated into three primary colors (R, G, B) by the three color filters (153r, 153g, 153b), and each photodiode (145r, 145g, 145b) The light of the color corresponding to is received.

  When a part of white light is absorbed by the pigment on the surface of the scan target, the amount of light received by each photodiode (145r, 145g, 145b) is different for each photodiode (145r, 145g, 145b). For this reason, the output voltages of the sensor signal line SSRj, the sensor signal line SSGj, and the sensor signal line SSBj are different from each other.

  Therefore, the image processing engine 180 determines the R gradation, the G gradation, and the B gradation according to each output voltage, so that the image processing engine 180 sends the RGB color image to the main body device 101. Can send.

  As described above, when the electronic apparatus 100 includes the liquid crystal panel 140A, the scan target can be scanned in color.

  Next, a scanning method different from the above-described scanning method (that is, the method of scanning the reflected image in FIG. 6) will be described with reference to FIG.

  FIG. 17 is a cross-sectional view showing a configuration in which a photodiode receives external light during scanning. As shown in the figure, part of the external light is blocked by the finger 900. Therefore, the photodiode disposed under the surface region of the liquid crystal panel 140 that is in contact with the finger 900 can hardly receive external light. In addition, although the photodiodes disposed under the surface area where the shadow of the finger 900 is formed can receive a certain amount of external light, the amount of external light received is small compared to the surface area where no shadow is formed.

  Here, by turning off the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the position of the finger 900 with respect to the surface of the liquid crystal panel 140 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), and the liquid crystal panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter also referred to as a shadow image) obtained by blocking external light with the finger 900.

  Further, the display device 102 may be configured to perform scanning by turning on the backlight 179 and then performing scanning again by turning off the backlight 179. Alternatively, the display device 102 may be configured to perform scanning by turning off the backlight 179 and then performing scanning again by turning on the backlight 179.

  In this case, since two scanning methods are used together, two scan data can be obtained. Therefore, it is possible to obtain a highly accurate result as compared with the case of scanning using only one scanning method.

<About display devices>
The operation of the display device 103 is controlled in accordance with a command (for example, a first command) from the main body device 101 as in the operation of the display device 102. The display device 103 has the same configuration as the display device 102. Therefore, when the display device 103 receives the same command as the display device 102 from the main body device 101, the display device 103 performs the same operation as the display device 102. For this reason, description of the configuration and operation of the display device 103 will not be repeated.

  Note that the main device 101 can send commands having different commands to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform different operations. Further, the main device 101 may send a command to either the display device 102 or the display device 103. In this case, only one display device performs an operation according to the command. Further, the main device 101 may send a command having the same command to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform the same operation.

  Note that the size of the liquid crystal panel 140 of the display device 102 and the size of the liquid crystal panel 240 of the display device 103 may be the same or different. Further, the resolution of the liquid crystal panel 140 and the resolution of the liquid crystal panel 240 may be the same or different.

<About Second Modification of Configuration>
In the present embodiment, a configuration in which electronic device 100 includes a liquid crystal panel with a built-in optical sensor such as liquid crystal panel 140 and liquid crystal panel 240 will be described. However, only one liquid crystal panel has a built-in photosensor. It may be.

  FIG. 18 is a block diagram illustrating a hardware configuration of electronic device 1300. Similar to the electronic device 100, the electronic device 1300 includes a first housing 100A and a second housing 100B. Referring to FIG. 18, electronic device 1300 includes a first unit 1001A and a second unit 1002. The first unit 1001A includes a main body device 101 and a display device 102A. The second unit 1002 includes a main device 104 and a display device 103.

  The display device 102A includes a liquid crystal panel that does not include a photosensor (that is, a liquid crystal panel having only a display function). The electronic device 1300 is different from the electronic device 100 in which the first unit 1001A includes a liquid crystal panel that does not include a photosensor, and the first unit 1001 includes a liquid crystal panel 240 that includes a photosensor. Such an electronic device 1300 performs the above-described sensing using the display device 103 of the second unit 1002.

  Further, the first unit 1001 may include, for example, a resistive film type or capacitive type touch panel, instead of the liquid crystal panel 140 incorporating the photosensor.

  In this embodiment, the display device 102 includes the timer 182 and the display device 103 includes the timer 282. However, the display device 102 and the display device 103 may share one timer. .

  In the present embodiment, electronic device 100 is described as a foldable device, but electronic device 100 is not necessarily limited to a foldable device. For example, the electronic device 100 may be a sliding device configured such that the first housing 100A slides with respect to the second housing 100B.

  Since electronic device 100 according to the present embodiment is configured as described above, second unit 1002 is detachable from first unit 1001 via USB connectors 194 and 294.

  And the electronic device 100 which concerns on this Embodiment can exhibit the following functions, for example at the time of power activation. First, when the user depresses the power switch 191 of the first unit 1001, the first unit 1001 activates BIOS (Basic Input / Output System) by using the power from the power supply circuit 192.

  The second unit 1002 acquires power from the first unit 1001 via the USB connectors 194 and 294. The second unit 1002 can transmit and receive data to and from the first unit 1001 by using the power. At this time, the CPU 210 of the second unit 1002 can display the OS (Operation System) type on the liquid crystal panel 240 in a selectable manner by using the power from the USB connectors 194 and 294.

  The user selects an OS to be started up via the liquid crystal panel 240. The CPU 210 transmits a command (for example, a “first OS” command shown in FIG. 10) specifying the OS to be activated to the first unit 1001 via the USB connectors 194 and 294 according to the user's selection. . The first unit 1001 starts the OS in response to the command.

  Further, for example, the second unit 1002 transmits / receives data to / from an external mobile phone or the like via the antenna 295. The CPU 210 of the second unit 1002 acquires photographic image data and corresponding thumbnail data from an external mobile phone via the antenna 295, and stores the photographic image data and corresponding thumbnail data in the RAM 271 or the like. The CPU 210 reads the thumbnail data from the RAM 271 and causes the liquid crystal panel 240 to display a thumbnail image of the photo in a selectable manner.

  In response to a selection command from the outside, the CPU 210 causes the liquid crystal panel 240 to display a photographic image. Alternatively, the CPU 210 displays a photographic image on the liquid crystal panel 140 or the display device 102A via the USB connector 294.

<Functional configuration>
FIG. 19 is a block diagram showing a functional configuration of electronic apparatus 100 according to the present embodiment. As described above, electronic device 100 includes first unit 1001 and second unit 1002. Hereinafter, the functional configuration of the electronic apparatus 100 will be described with reference to FIG.

  The first unit 1001 includes a display unit 310, an input unit 320, a storage unit 330, an interface unit 340, and a control unit 350. The first unit 1001 performs the main operation of the electronic device 100.

  The display unit 310 displays information inside the first unit 1001 to the outside. The input unit 320 receives an instruction from the outside. In the present embodiment, the liquid crystal panel 140 has the functions of the display unit 310 and the input unit 320. However, as the display unit 310, another display device, for example, a display such as an LCD (Liquid Crystal Display) may be used. The operation key 177 also functions as the input unit 320.

  The storage unit 330 stores information such as display data 333, a program 334, and an operation parameter 335 that are a basis of a screen displayed on the display unit 310 (liquid crystal panel 140) of the first unit 1001. The storage unit 330 generally stores a plurality of programs 334. Examples of the program 334 include general-purpose application software such as a word processor and a Web browser.

  The operation parameter 335 is information for giving an operation condition of the program 334. The operation parameter 335 includes, for example, data indicating an active window that operates in response to pressing of the operation key 177 in the multi-window program 334.

  The interface unit 340 exchanges information with the interface unit 440 on the second unit 1002 side. In the present embodiment, when the first unit 1001 and the second unit 1002 are directly connected, the USB connector 194 functions as the interface unit 340. When the first unit 1001 and the second unit 1002 are not directly connected, the antenna 195 functions as the interface unit 340. However, the method of exchanging information by the interface unit 340 is not limited to this.

  The control unit 350 controls operations of the display unit 310, the storage unit 330, and the interface unit 340 based on an instruction from the input unit 320 and the like. Control unit 350 includes an input processing unit 352, a display control unit 356, and a program execution unit 358. In the present embodiment, the CPU 110 and the image processing engine 180 correspond to the control unit 350. However, each function of the CPU 110 may be realized by hardware such as a dedicated circuit. Each function of the image processing engine 180 may be realized by the CPU 110 that executes software. That is, each function of the control unit 350 may be realized by hardware or may be realized by software.

  The input processing unit 352 transmits the signal received from the input unit 320 to the program execution unit 358. The display control unit 356 controls the operation of the display unit 310 based on the display data 333 stored in the storage unit 330. The program execution unit 358 executes the program 334 based on an instruction received from the input unit 320. Specifically, the CPU 110 that executes the program 334 using the RAM 171 as a working memory corresponds to the program execution unit 358.

  The second unit 1002 includes a display unit 410, an input unit 420, a storage unit 430, an interface unit 440, a control unit 450, and a timer 273.

  The display unit 410 displays information inside the second unit 1002 to the outside. The input unit 420 receives an instruction from the outside. In the present embodiment, the optical sensor built-in liquid crystal panel 240, the left click key 241, the center key 242, and the right click key 243 correspond to the input unit 420. In the present embodiment, the optical sensor built-in liquid crystal panel 240 also has the functions of the display unit 410 and the input unit 420 (panel input unit 422). However, as the display unit 410, another display device, for example, a display such as an LCD (Liquid Crystal Display) may be used. The input unit 420 is not limited to the optical sensor built-in liquid crystal panel 240, and a device (tablet) having a function of recognizing an input position can be used. For example, a capacitive touch panel may be used as the input unit 420. A component that realizes the functions of the display unit 410 and the input unit 420 in this way is referred to as a “display-integrated touchpad”.

  The storage unit 430 stores information such as input data 431, display data 433, a program 434, operation parameters 435, time data 436, and mode data 437.

  The input data 431 is data created based on the input received by the input unit 420. In particular, in the present embodiment, the input data 431 includes an input history 432 corresponding to the input history. The input history 432 includes handwritten character data 432a and illustration data 432b. Details of the handwritten character data 432a and the illustration data 432b will be described later.

  The display data 433 is a basis of a screen displayed on the display unit 410 (liquid crystal panel 240) of the second unit 1002. The display data 433 includes image data (such as wallpaper) stored in the storage unit 430 and image data created in accordance with the execution of the program 434.

  In the present embodiment, storage unit 430 stores a plurality of programs 434. The program 434 includes application software (for example, handwritten character input software, hand-drawn illustration input software, or calculator software) that displays an operation screen on the liquid crystal panel 240. Details of the program 434 will be described later.

  The operation parameter 435 is information for giving an operation condition of the program 334 like the operation parameter 335 in the first unit 1001. In particular, in the present embodiment, the operation parameter 435 includes a count value of elapsed time from a predetermined event created by the program 434.

  Time data 436 represents the time measured by the timer 273. The time data 436 is used when executing a program 434 that performs a predetermined operation with the passage of time.

  The mode data 437 is information representing the operation mode of the input processing unit 452. There are a plurality of operation modes of the input processing unit 452, and the mode data 437 represents the current operation mode. Specifically, for example, a flag stored in a predetermined storage area can be regarded as mode data 437. Details of the operation mode of the input processing unit 452 will be described later.

  The interface unit 440 exchanges information with the interface unit 340 on the first unit 1001 side. In the present embodiment, when the first unit 1001 and the second unit 1002 are directly connected, the USB connector 294 functions as the interface unit 440. When the first unit 1001 and the second unit 1002 are not directly connected, the antenna 295 functions as the interface unit 440. However, the method of exchanging information by the interface unit 440 is not limited to this.

  The control unit 450 controls operations of the display unit 410, the storage unit 430, and the interface unit 440 based on an instruction received by the input unit 420. Control unit 450 includes an input processing unit 452, a display control unit 456, and a program execution unit 458.

  The input processing unit 452 transmits the signal from the input unit 420 to the program execution unit 458 or the interface unit 440. The input processing unit 452 includes a panel input processing unit 453 and a mode setting unit 454.

  The panel input processing unit 453 processes a signal from the panel input unit 422. For example, the panel input processing unit 453 creates an input history 432 (handwritten character data 432a, illustration data 432b, etc.) based on the signal history. Details of the operation of the panel input processing unit 453 will be described later.

  The mode setting unit 454 sets the operation mode of the panel input processing unit 453 based on a predetermined signal from the input unit 420 (such as a signal generated by pressing the center key 242). Details of the operation of the mode setting unit 454 will be described later.

  The display control unit 456 controls the operation of the display unit 410 based on the display data 433. For example, the display control unit 456 displays a screen (operation screen) created as a result of the execution of the program 434 on the display unit 410.

  The program execution unit 458 executes the program 434 based on an instruction received from the input unit 420 or the like. Specifically, the CPU 210 that executes the program 434 using the RAM 271 as a working memory corresponds to the program execution unit 458.

<Outline of operation>
(Mouse mode and tablet mode)
The electronic device 100 issues an instruction for the operation of the application in response to an instruction to the liquid crystal panel 240, that is, an object (such as a finger 900 or a stylus 950) touching the liquid crystal panel 240. The electronic device 100 (more specifically, the panel input processing unit 453) has two operation modes of “mouse mode” and “tablet mode”. Electronic device 100 operates in accordance with the operation mode.

  In the mouse mode, the electronic device 100 executes the program 334 in response to an input to the liquid crystal panel 240, and an image created by the executed program 334 (hereinafter referred to as “program operation screen”) is displayed on the liquid crystal panel 140. To display.

  Specifically, in the mouse mode, electronic device 100 moves the cursor in the operation screen of liquid crystal panel 140 in real time according to the change in the input position to liquid crystal panel 240. Here, the “cursor” refers to a display that indicates an input position of a character, a figure, a display object, or the like.

  Further, in the mouse mode, electronic device 100 creates a command for instructing a predetermined operation to program 334 of first unit 1001 in response to a predetermined input to liquid crystal panel 240. For example, when the electronic device 100 determines that the liquid crystal panel 240 has received an input corresponding to a click, a double click, a drag, or the like, the electronic device 100 executes the program 334 according to the input. The operation performed by electronic device 100 at this time is determined by program 334.

  That is, when electronic device 100 is in the mouse mode, the user can use liquid crystal panel 240 as a touch pad. Hereinafter, for the sake of simplicity, the operation of the program 334 according to the movement of the cursor position and the predetermined input to the liquid crystal panel 240 will be collectively referred to as “mouse operation”.

  In the tablet mode, electronic device 100 executes program 434 (or program 334) in response to an input to liquid crystal panel 240, and displays an operation screen of the executed program on liquid crystal panel 240. In addition, electronic device 100 creates a command for a program for displaying an operation screen on liquid crystal panel 240. For example, when an operation button is displayed on the liquid crystal panel 240, the electronic device 100 executes a program and performs an operation according to the touched operation button. That is, when the electronic device 100 is in the tablet mode, the user can use the liquid crystal panel 240 as a touch screen.

  Electronic device 100 switches the operation mode based on a predetermined instruction. In the present embodiment, electronic device 100 switches the operation mode in response to pressing of center key 242.

  However, the operation mode switching instruction is not limited to pressing the center key 242. Electronic device 100 may switch the operation mode in response to pressing of operation key 177 other than center key 242. In addition, electronic device 100 may switch the operation mode in accordance with selection of an operation button displayed on liquid crystal panel 140 or liquid crystal panel 240. The electronic device 100 also switches the operation mode depending on the operation state of the electronic device (during startup processing, return from sleep or hibernation). This point will be described in detail later.

  Display on the liquid crystal panel 140 and the liquid crystal panel 240 in the mouse mode and the tablet mode will be described with reference to FIG. FIG. 20 is a diagram for explaining a screen displayed by the electronic device in each of the mouse mode and the tablet mode.

  In the mouse mode, the electronic device 100 displays an operation screen 500 of software such as word processing software or a Web browser on the liquid crystal panel 140. The screen 500 is similar to a display screen by a personal computer that is widely used at present (that is, having one display). However, in FIG. 20, the contents of the screen 500 are not drawn for simplicity. The screen 500 is not limited to this specific example.

  Screen 500 includes a cursor 510. Here, it is assumed that the cursor 510 is a pointer (mouse pointer) that can freely move on the screen 500. However, the cursor 510 is not limited to a pointer. The cursor 510 may be a display indicating the input position of a character or a display object. The display form of the cursor 510 is not limited to that shown in FIG. 20 (arrow). Also, electronic device 100 may change the display form of cursor 510 depending on the position pointed to.

  In the mouse mode, electronic device 100 displays screen 600 on liquid crystal panel 240. Hereinafter, the screen displayed on the liquid crystal panel 240 in the mouse mode is also referred to as a “mouse screen”. Referring to FIG. 20, screen 600 includes a guidance display 610.

  Guidance display 610 is a display for describing the operation of electronic device 100 when left click key 241, center key 242, and right click key 243 are pressed. Guidance display 610 includes a left guidance display 612, a center guidance display 614, and a right guidance display 616.

  The left guidance display 612, the center guidance display 614, and the right guidance display 616 are characters that describe the operation of the electronic device 100 when the left click key 241, the center key 242, and the right click key 243 are pressed, respectively. And / or symbols (though such characters and symbols are not shown in FIG. 20).

  The mouse screen may not always display the left guidance display 612, the center guidance display 614, and the right guidance display 616. In the present embodiment, electronic device 100 does not display left guidance display 612, center guidance display 614, and right guidance display 616 on liquid crystal panel 240 when the corresponding key is invalid.

  One specific example of the mouse screen is shown in FIG. Referring to FIG. 21, screen 600 includes a left guidance display 612, a center guidance display 614, and a right guidance display 616.

  In FIG. 21, the left guidance display 612 includes the characters “left click”. When this display is made, electronic device 100 performs a left click operation (decision process or the like) in response to pressing of left click key 241. The left click operation is determined by the running program.

  In FIG. 21, the center guidance display 614 includes the characters “touch screen operation”. This display is made when the electronic device 100 is in the mouse mode. When this display is made, the electronic device 100 shifts to the tablet mode in response to pressing of the center key 242.

  Electronic device 100 displays the character “mouse operation” on center guidance display 614 when in tablet mode. When this display is made, electronic device 100 shifts to the mouse mode in response to pressing of center key 242.

  The right guidance display 616 includes the characters “right click”. This indicates that electronic device 100 performs a right-click operation (menu display, etc.) in response to pressing of right-click key 243. The right click operation is determined by the running program.

  In this embodiment, the user can set the mouse screen. The user may be able to set an image such as a photograph stored in the electronic device 100 as the wallpaper of the mouse screen. The wallpaper may be an operation screen of an accessory such as a clock or calendar that does not require user operation. In addition, the wallpaper can be automatically changed depending on the state of the electronic device 100, such as during startup or recovery from the power saving state. Even when the user cannot set the mouse screen, the mouse screen is not limited to that shown in FIG.

  Returning to FIG. 20, the screen in the tablet mode will be described. In the tablet mode, the liquid crystal panel 140 displays an operation screen 700 of software such as word processing software or a Web browser. The content of the operation screen 700 is the same as that of the operation screen 500.

  In the tablet mode, the electronic device 100 displays the screen 800 on the liquid crystal panel 240. Hereinafter, the screen displayed on the liquid crystal panel 240 in the tablet mode is also referred to as a “tablet screen”. Referring to FIG. 20, screen 800 includes a guidance display 810 and an operation button display 820.

  The guidance display 810 includes a left guidance display 812, a center guidance display 814, and a right guidance display 816 similarly to the guidance display 610 in the mouse mode. These roles and operations are similar to left guidance display 612, center guidance display 614, and right guidance display 616, respectively, and detailed description thereof will not be repeated.

  The operation button display 820 is used for selecting an application. When electronic device 100 detects the contact of an external object (such as finger 900 or stylus 950) in the region corresponding to operation button display 820, electronic device 100 starts a predetermined operation corresponding to the region.

(About the program)
Here, a program to be executed in the electronic device 100 will be described.

  The programs executed by electronic device 100 according to the present embodiment include a program for displaying an operation screen on liquid crystal panel 140 (hereinafter referred to as “main application”) and a program for displaying an operation screen on liquid crystal panel 240. (Hereinafter referred to as “sub-application”).

  As the main application, an application that operates on a current electronic device such as a browser, a dictionary, a book viewer, and a photo viewer can be assumed. Examples of the sub-application include an input pad (handwritten character input pad, hand-drawn illustration input pad, calculator / number input pad, etc.) that uses input to the liquid crystal panel 240, and an application for assisting operation of the main application.

  In the description of the present embodiment, the sub application is assumed to be independent of the main application. The main application is stored in the storage unit 330 of the first unit 1001. On the other hand, the sub application is stored in the storage unit 430 of the second unit 1002.

  Thus, the main application is separated from the sub application. Therefore, a general-purpose application that operates on other electronic devices can be used as the main application. In this case, the specification of data exchange with the main application of the sub application is matched with the specification of the main application. For example, the instruction of the mouse operation by the sub application is matched with the operation instruction from the conventional touch pad or mouse.

  Furthermore, in the present embodiment, the program execution unit 358 that executes the main application and the program execution unit 458 that executes the sub-application are independent. In this way, it is possible to reduce the load on the processor (CPU 110 in this embodiment) that executes the main application. In particular, when the performance of the CPU of the electronic device 100 is low, dividing the program execution unit in this way is effective.

  By providing a control unit and a storage unit in each of the first unit 1001 and the second unit 1002 as in the present embodiment, data exchange between the first unit 1001 and the second unit 1002 can be reduced, and processing can be performed. Can be speeded up.

  However, the sub application is not necessarily independent of the main application. That is, the same program may function as both the main application and the sub application. Specifically, a part of the program may create a screen to be displayed on the liquid crystal panel 140, and another part of the program may create a screen to be displayed on the liquid crystal panel 240.

  Further, the main application and the sub application may be executed by the same processor. In this case, the processor that executes the application controls the operations of both the liquid crystal panel 140 and the liquid crystal panel 240.

(Sub application)
In the present embodiment, electronic device 100 executes one of a plurality of sub-applications in the tablet mode. One of the sub-applications is a “home application” for determining an application to be executed.

  The home application is a launcher for selecting an application. The home application displays on the liquid crystal panel 240 a screen for selecting one sub-application from the plurality of sub-applications (hereinafter “home menu screen”).

  FIG. 22 shows a specific example of the home menu screen. Referring to FIG. 22, the home menu screen includes a guidance display 810, operation button displays 820 a to 820 i, and a mouse disabled display 830. When the operation button displays 820a to 820i are touched, the home application calls a sub application corresponding to the operation button displays 820a to 820i. The mouse disabled display 830 is a display for indicating that the electronic device 100 does not perform a mouse operation by an input to the liquid crystal panel 240. This display helps the user to accurately understand the operation of the electronic device 100 in the tablet mode. It also helps the user to distinguish between mouse mode and tablet mode.

  In the present embodiment, it is assumed that electronic device 100 can customize the home menu screen based on a user instruction. Even when the home menu screen cannot be customized, the configuration of the home menu screen is not limited to that shown in FIG. For example, the number and arrangement of the operation button displays 820 are not limited to those shown in FIG. The mouse disabled display 830 is not limited to that shown in FIG. Alternatively, the mouse disabled display 830 may not be included in the home menu screen.

  It is assumed that the default sub application in the tablet mode is a home application. That is, when the electronic device 100 operates in the tablet mode for the first time after activation (after power is turned on), the home application is executed.

  The transition of the screen 800 displayed on the liquid crystal panel 240 in the tablet mode will be described with reference to FIG. FIG. 23 is a transition diagram of a screen 800 displayed on the liquid crystal panel 240 in the tablet mode.

  Referring to FIG. 23, when the user selects operation button display 820 (hand-drawn illustration) on home menu screen 800a, electronic device 100 executes the hand-drawn illustration application and displays screen 800b on liquid crystal panel 240. The screen 800b is an operation screen for a hand-drawn illustration application. In the screen 800b shown in FIG. 23, a picture of a dog is drawn in the hand-drawn input frame.

  When the user selects operation button display 820 (home) on screen 800b, electronic device 100 displays window 800d on liquid crystal panel 240. Here, the electronic device 100 may display the window 800d instead of the screen 800b, or may display the window 800d superimposed on the screen 800b. The window 800d includes text for inquiring whether to save the created hand-drawn illustration, and operation button displays 820 of “Yes”, “No”, and “Cancel”.

  When “Yes” in the window 800d is selected, the electronic device 100 displays a hand-drawn illustration in the input frame before the window 800d is displayed in the electronic device 100 or an external storage device (such as a hard disk or a flash memory). To store. In addition, the electronic device 100 displays the home menu screen 800a on the liquid crystal panel 240.

  When “No” in the window 800d is selected, the electronic device 100 displays the home menu screen 800a on the liquid crystal panel 240. In this case, electronic device 100 does not store the hand-drawn illustration made in the input frame before display of window 800d in the storage device.

  When “cancel” of the window 800d is selected, the electronic device 100 displays the screen 800b before the window 800d is displayed on the liquid crystal panel 240. That is, if the user selects cancel, the user can continue to create a hand-drawn illustration.

  When the user selects the operation button display 820 (Internet) on the home menu screen 800a, the electronic device 100 executes a Web browser and displays an operation screen 800c of the Web browser on the liquid crystal panel 140.

  When the user selects operation button display 820 (Home) on screen 800c, electronic device 100 displays liquid crystal panel 240 on home menu screen 800a. In this case, electronic device 100 does not perform the storage process of the input data. Therefore, electronic device 100 does not display an inquiry screen such as window 800d.

<Switching mode>
Hereafter, the operation of the electronic device 100 when switching between the mouse mode and the tablet mode will be described in detail with reference to FIG. FIG. 24 is a diagram for explaining the operation of the electronic device 100 when switching between the mouse mode and the tablet mode.

  In the present embodiment, electronic device 100 basically switches the operation mode in response to pressing of center key 242 as described above. The electronic device 100 can transition from the tablet mode to the mouse mode, or can transition from the mouse mode to the tablet mode.

  In the present embodiment, electronic device 100 changes the processing method of input to liquid crystal panel 240 when shifting from the tablet mode to the mouse mode. That is, electronic device 100 treats an input to liquid crystal panel 240 as a mouse operation instruction rather than an operation instruction to a sub application. In addition, the electronic device 100 displays a mouse screen on the liquid crystal panel 240 when shifting to the mouse mode.

  However, the electronic device 100 keeps running the application that has been executed in the tablet mode even after shifting to the mouse mode. Thus, the electronic device 100 can smoothly display the screen on the liquid crystal panel 240 when shifting from the mouse mode to the tablet mode. This is because the startup time of the sub application is unnecessary.

  Further, by continuing to operate the application that has been executed in the tablet mode, the operability when the user performs a mouse operation temporarily with the liquid crystal panel 140 during the tablet mode is improved. For example, consider that the electronic device 100 transitions from the tablet mode to the mouse mode and then transitions back to the tablet mode. Since the electronic device 100 operates as described above, the electronic device 100 displays the operation screen of the same sub application on the liquid crystal panel 240 before and after the transition to the mouse mode. Therefore, the user can continue to use the sub-application before the mouse operation after performing the mouse operation.

  The electronic device 100 displays the operation screen of the main application on the liquid crystal panel 140 not only in the mouse mode but also in the tablet mode. Therefore, electronic device 100 can allow the user to operate the sub application without impairing the visibility of the operation screen of the main application.

(Mode at the start of operation)
In particular, switching of operation modes related to the start of operation of electronic device 100 will be described. In the present embodiment, the operation of the electronic device 100 is roughly classified into (i) activation from a power-off state (hereinafter referred to as normal activation) and (ii) activation from a power saving state (hereinafter referred to as “power activation state”). , Called resumption).

  Here, the “power-off state” is a state in which the operation of each part of the electronic device 100 (excluding a part necessary for starting up the electronic device 100) is stopped. The “power saving state” is a state in which a part of the operation of the electronic device 100 is stopped.

  The power saving state includes a “standby state”, a “sleep state”, and a “sleep state” that is a combination of the standby state and the sleep state.

  When receiving the instruction to shift to the standby state, the electronic device 100 stores the data being worked in the RAM 171. In addition, the electronic device 100 stops supplying power to portions other than those necessary for restarting operations (such as the power supply circuit 192, the power supply detection unit 193, and the RAM 171).

  When receiving the instruction to shift to the hibernation state, the electronic device 100 stores the data being worked on the hard disk 170. In addition, the electronic device 100 stops supplying power to portions other than those necessary for restarting the operation (such as the power supply circuit 192 and the power supply detection unit 193).

  At the time of instructing the sleep process, the electronic device 100 first stores the data being worked on in the memory. Then, when a predetermined time elapses from the instruction in the sleep state, electronic device 100 moves the data stored in the memory from the memory to the hard disk.

  However, the types of power saving states are not limited to those described above. Moreover, the electronic device 100 does not necessarily need to prepare all these power saving processes.

  First, (i) the operation mode in normal startup will be described with reference to FIG. FIG. 25 is a diagram schematically showing an operation mode at the normal startup.

  At the normal startup, the electronic device 100 first performs a boot process for starting an OS (Operating System). During the boot process, the electronic device 100 displays a boot screen 2501 on the liquid crystal panel 140. During the boot process, the operation mode of the electronic device 100 is the mouse mode. The electronic device 100 displays a predetermined mouse screen (hereinafter referred to as a fixed screen) 2502 on the liquid crystal panel 240. The fixed screen 2502 does not include a guidance display. This is because mode switching cannot be executed during the boot process.

  When the boot process is completed, the electronic device 100 displays a login screen 2503 on the liquid crystal panel 140. Also at this time, the electronic device 100 displays the fixed screen 2504.

  When the login is completed, the electronic device 100 displays a desktop screen 2505 on the liquid crystal panel 140. In addition, the electronic device 100 displays a mouse screen 2506 on the liquid crystal panel 240. At this stage, the electronic device 100 can switch from the mouse mode to the tablet mode. Accordingly, the mouse screen 2506 includes a guidance display, similar to the mouse screen 600 shown in FIG.

  It is assumed that the default sub application in the tablet mode is a home application. That is, when the electronic device 100 operates in the tablet mode for the first time after activation (after power is turned on), the home application is executed.

  Next, (ii) the operation mode in return will be described with reference to FIG. FIG. 26 is a diagram schematically showing the operation mode at the time of return.

  At the time of resumption, the electronic device 100 first reads data relating to the work state stored in a storage device such as a memory or a hard disk. During this time, the electronic device 100 displays a resuming screen 2601 on the liquid crystal panel 140. During this time, the operation mode of the electronic device 100 is the mouse mode. The electronic device 100 displays a fixed screen 2602 on the liquid crystal panel 240.

  When the reading of the work state is completed, the electronic device 100 displays a login screen 2603 on the liquid crystal panel 140. Also at this time, the electronic device 100 displays the fixed screen 2604.

  When the login is completed, the electronic device 100 displays again the display screen 2605 displayed on the liquid crystal panel 140 immediately before shifting to the power saving state on the liquid crystal panel 140 based on the read work state. In addition, the electronic device 100 displays a display screen 2606 based on the sub application that was operating immediately before the transition to the power saving state on the liquid crystal panel 240.

  Electronic device 100 may always display a mouse screen on liquid crystal panel 140 after the login is completed. When it is expected that the first operation performed by the user after the restart is a mouse operation, the operability can be improved by this processing.

<Operation in tablet mode>
From here, the operation of the electronic device 100 in the tablet mode will be described in more detail with reference to FIG. FIG. 27 is a diagram for explaining the operation of the electronic device 100 in the tablet mode.

  In the tablet mode, the electronic device 100 roughly divides the home application, executes the input pad application, or executes the sub-screen utilization software.

  The electronic device 100 displays the home menu screen 800a on the liquid crystal panel 240 during execution of the home application. When the electronic device 100 receives a touch on a predetermined position (indicated by the operation button display) on the liquid crystal panel 240 during execution of the home application, the electronic device 100 calls the input pad. In addition, when the electronic device 100 receives a touch on a predetermined position on the liquid crystal panel 240 during execution of the home application, the electronic device 100 starts execution of the sub-screen utilization software.

  When the electronic device 100 calls the input pad, the electronic device 100 displays an input screen on the liquid crystal panel 240. Referring to FIG. 27, in the present embodiment, there are three types of input pads, that is, a handwritten character input pad, a hand-drawn illustration input pad, and a calculator / number input pad. Screens 2702 to 2704 displayed on liquid crystal panel 240 are operation screens for a handwritten character input pad, a handwritten illustration input pad, and a calculator / numeric input pad, respectively. Details of each input pad will be described later.

  In addition, when electronic device 100 receives a predetermined instruction during execution of the input pad, electronic device 100 activates the home application. In the present embodiment, the operation screen of the input pad includes a home button, and electronic device 100 ends the input pad and starts a home application in response to a touch on the home button.

  Referring to FIG. 27, the sub-screen utilization software in this embodiment includes a two-screen utilization guide, the Internet, a dictionary, a book, a photograph, and a game.

  The “2-screen usage guide” is an online manual. When the electronic device 100 executes the two-screen utilization guide, the electronic device 100 acquires the manual from the external server and displays the manual on the liquid crystal panel 240 (or the liquid crystal panel 140 or both the liquid crystal panel 140 and the liquid crystal panel 240).

  “Internet” is software for activating a Web browser and calling a home page, and is hereinafter referred to as Web page calling software. Details of the operation of this software will be described later.

  “Dictionary” is software for calling an electronic dictionary, and is hereinafter referred to as dictionary calling software. In the present embodiment, this software calls one electronic dictionary from a plurality of electronic dictionaries. Details of the operation of this software will be described later.

  “Book” is software for selecting an electronic book (for example, a book in XMDF format) to be viewed with a book viewer. “Photo” is software for displaying a photo slideshow.

  “Game” is game software for displaying a game screen on the liquid crystal panel 240. In the present embodiment, it is assumed that the “game” is a game utilizing the touch operation of the liquid crystal panel 240.

  When the electronic device 100 receives a predetermined instruction during execution of the sub-screen utilization software, the electronic device 100 ends the sub-screen utilization software and starts a home application. In the present embodiment, the operation screen of the sub screen utilization software includes a home button, and electronic device 100 activates a home application in response to a touch on the home button.

(Handwriting input pad)
The operation of electronic device 100 that executes the handwritten character input pad (specifically, CPU 210 that executes the application) will be described with reference to FIG. FIG. 28 is a diagram illustrating an example of a screen (character input screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the handwritten character input pad.

  Referring to FIG. 28, the character input screen includes a center guidance display 814, a mouse disabled display 830, a home button 840, a text box 2801, a paste button 2802, a candidate area 2803, a backward button 2804, handwriting. An area 2805, a recognition mode switching button 2806, a recognition button 2807, and an erase button 2808 are included.

  A text box 2801 displays characters that have been confirmed as a result of handwritten character recognition. The text box 2801 can display up to 10 confirmed characters. However, the maximum number of characters displayed in the text box 2801 is not limited to this.

  When the paste button 2802 is pressed when there is a character string (one or more characters) in the text box 2801, the electronic device 100 displays the character string on the active application displayed on the liquid crystal panel 140. Send.

  The paste button 2802 functions as an Enter key when there is no character string in the text box 2801. For example, the user can press the paste button 2802 to transmit a character string to the search box, and then press the paste button 2802 again to cause the application to execute a search. If there is no character string in text box 2801 (or paste button 2802 functions as an Enter key), electronic device 100 changes the character displayed on paste button 2802 to “Enter”.

  The candidate area 2803 displays input recognition candidates. In the present embodiment, the candidate area 2803 displays a maximum of five recognition candidates in order from the first candidate. However, the maximum number of candidates displayed in the candidate area 2803 is not limited to this.

  In the present embodiment, electronic device 100 automatically adds the first candidate (the first character in candidate area 2803) to text box 2801. In addition, electronic device 100 can change the added character in accordance with selection of a candidate in candidate area 2803. Since there is a high possibility that the first candidate is a character to be input by the user, the number of operations by the user can be reduced by automatically adding the first candidate to the text box 2801.

  When the backward button 2804 is pressed, the electronic device 100 erases the last character of the character string in the text box 2801. If there is no character string in text box 2801, electronic device 100 does not perform an operation associated with pressing back button 2804.

  The handwriting area 2805 accepts input from the outside. Electronic device 100 creates handwritten character data 432 a corresponding to the history of input to handwriting area 2805 and stores it in storage unit 430. For example, the electronic device 100 creates, as handwritten character data 432a, all coordinates that have been input within a predetermined time, or coordinates at the start and end of input that are temporally continuous within a predetermined time. In addition, electronic device 100 displays a graphic corresponding to coordinates (or handwritten character data 432a) for which input has been received in handwritten area 2805.

  In the present embodiment, handwriting area 2805 includes two regions (area 2805a and area 2805b). Electronic device 100 creates handwritten character data 432a for each of area 2805a and area 2805b. Note that the handwriting area 2805 is not limited to two areas.

  The recognition mode switching button 2806 switches the recognition mode for handwriting input. In the present embodiment, there are two recognition modes, “automatic mode” and “manual mode”. It is assumed that the recognition mode when the handwriting input pad is first activated is the automatic mode.

  In the “automatic mode”, electronic device 100 automatically starts character recognition for input to handwriting area 2805 after a predetermined time has elapsed since pen-up (end of input to handwriting area 2805). The automatic mode has an advantage that the number of user operations can be reduced. In the automatic mode, electronic device 100 may automatically start character recognition after a predetermined time from the start of input to another event, for example, handwriting area 2805, instead of pen-up. In addition, the start of input to a handwriting area 2805b (or 2805a) different from the handwriting area 2805a (or 2805b) where handwriting input has been performed is also an automatic start event for character recognition.

  In the “manual mode”, the electronic device 100 does not start character recognition until the recognition button 2807 is pressed. The manual mode has an advantage that the user can calmly input characters.

  When the recognition button 2807 is pressed, the electronic device 100 starts character recognition of the handwritten character data 432a based on the input to the input area 2805. Even in the automatic mode, if the recognition button 2807 is pressed before the recognition start time, the electronic device 100 starts character recognition of the handwritten character data 432a.

  When delete button 2808 is pressed, electronic device 100 deletes the graphic and handwritten character data 432a displayed in input area 2805. The erase button 2805 is used when a user rewrites a character handwritten before character recognition.

  The usage method of a handwritten character input pad is demonstrated with reference to FIG. 29 and FIG. FIGS. 29 and 30 are diagrams for explaining the operation of electronic device 100 when the handwritten character input pad is used. Here, a description will be given of an operation example in the case where the user searches for “Fugu” using the liquid crystal panel 240 while using an application (such as a Web browser) that displays a screen including a search box.

  FIG. 29A shows a screen displayed on the liquid crystal panel 140 and the liquid crystal panel 240 in the mouse mode. At this time, the liquid crystal panel 240 displays a mouse screen 2920. The liquid crystal panel 140 displays a main application operation screen 2910. Operation screen 2910 includes a search box 2912 and a search start button 2914. It is assumed that the search box 2912 is activated by the mouse operation accompanying the movement of the finger 900 on the liquid crystal panel 240.

FIG. 29B shows a display screen 2930 of the liquid crystal panel 240 after a mode switching instruction (specifically, pressing of the center key 242) in the state shown in FIG. 29A.
Here, display screen 2930 is a home menu screen.

  FIG. 29C shows the display on the liquid crystal panel 240 after the button for calling the handwritten character input pad (encircled in FIG. 29B) is pressed in the state shown in FIG. A screen 2940 is shown. Display screen 2940 is a character input screen.

  FIG. 29D shows a display screen 2950 of the liquid crystal panel 240 when a handwriting input is made on the display screen 2940 by the stylus 950. In the input area of the display screen 2950, a graphic 2952 corresponding to handwriting input is displayed.

  FIG. 29 (e) shows a display screen 2960 of the liquid crystal panel 240 when the stylus 950 shown in FIG. 29 (d) is separated from the liquid crystal panel 240. Candidate characters corresponding to handwriting input are displayed in the candidate area of the display screen 2950. In the text box, the first candidate character 2962 (here, the character “kawa”) is displayed.

  FIG. 29F shows the display screen 2970 of the liquid crystal panel 240 when the first candidate character 2962 is confirmed. Character 2972 is a confirmed character.

  FIG. 30G is a diagram showing the display screen 3010 of the liquid crystal panel 240 in a state where the user performs handwriting input with the stylus 950 after the time shown in FIG. Display screen 3010 includes a graphic 3012 corresponding to handwriting input.

  FIG. 30H shows a display screen 3020 of the liquid crystal panel 240 when the stylus 950 shown in FIG. Candidate characters corresponding to handwriting input are displayed in the candidate area of the display screen 2950. In the text box, the first candidate character 3022 (here, the character “knee”) is added to the right side of the already determined character “river”.

  FIG. 30I shows the display screen 3030 of the liquid crystal panel 240 when the character 3032 (“pig”) in the candidate area is pressed by the stylus 950 in the state shown in FIG. As the character 3032 is pressed, the character “knee” displayed in the text box changes to “pig” (character 3034).

  FIG. 30J shows the display screen 3040 of the liquid crystal panel 140 and the display screen 3050 of the liquid crystal panel 240 after the character 3034 (“pig”) is confirmed. Display screen 3040 includes a search box 3042 and a search start button 3044. After the character 3034 is confirmed, when the paste button in the display screen 3050 is pressed by the stylus 950, the character string “Kawaguchi” in the text box is input to the active search box 3042. Accordingly, the search string 3042 displays the character string “Kawaguchi”.

  FIG. 30 (k) shows the display screen 3060 of the liquid crystal panel 140 and the display screen 3070 of the liquid crystal panel 240 after the Enter button (paste button) is pressed after the state shown in FIG. 30 (j). Pressing the Enter button in the display screen 3070 has the same effect as pressing the search start key 3044. In other words, the main application executes a search for the character string “Kawaguchi” in the search box 3042 in response to pressing of the Enter button, and displays a display screen 3060 indicating the search result on the liquid crystal panel 140.

  In the present embodiment, when the text box is full (the maximum number of characters that can be input is input in the text box), electronic device 100 does not accept any more handwriting. The operation of electronic device 100 when the text box is full will be described with reference to FIG. FIG. 31 is a diagram for explaining the operation of electronic device 100 when the text box is full.

  FIG. 31A shows a state in which the user is about to input to the input area with the stylus 950 when the text box is already full (ten characters are in the text box). At this time, a character string 3112 (a 10-character string “Aiueokiki Kekko”) is displayed in the text box of the display screen 3110 of the liquid crystal screen 240.

  When the stylus 950 contacts the liquid crystal panel 240, the electronic device 100 displays a screen 3120 including a warning display 3122 on the liquid crystal panel 240 (see FIG. 31B). The warning display 3122 includes a character string that prompts the user to confirm the character, that is, press the paste button. Here, the character string included in the warning display 3122 is “Please touch the paste button”, but the character string is not limited to this.

  After the warning display 3122 is displayed, the liquid crystal panel 240 displays a display screen 3130 similar to the display screen 3110 (see FIG. 31C). While the display screen 3130 is displayed, the user can continue character input by pressing the paste button to confirm the character string or by pressing the backward button to delete the already input character. Note that the electronic device 100 automatically changes from the state of FIG. 31B to FIG. 31C, for example, automatically after a predetermined time has elapsed from the display of the warning display 3122 or in response to some instruction to the liquid crystal panel 240. ) State.

  In the present embodiment, handwritten character data 432a is temporarily stored in RAM 271 or the like, and electronic device 100 discards handwritten character data 432a when the handwritten character input pad ends. Therefore, when the handwritten character input pad is called again, the user can input a new character.

  However, the electronic device 100 may hold the handwritten character data 432a so that the user can perform handwriting input from the last time when the handwriting input pad is used again. In this case, electronic device 100 displays a graphic corresponding to handwritten character data 432a on liquid crystal panel 240 based on handwritten character data 432a when the handwritten character input pad is resumed.

  Note that the user may be able to select which of the above two operations is performed on the handwritten character input pad. In this case, the user can appropriately determine which of the above two operations is performed by the handwritten character input pad according to the usage mode of the handwritten character input.

(Hand-drawn illustration input pad)
The operation of electronic device 100 that executes the hand-drawn illustration input pad (specifically, CPU 210 that executes the application) will be described with reference to FIG. FIG. 32 is a diagram illustrating an example of a screen (illustration input screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the hand-drawn illustration input pad.

  Referring to FIG. 32, the illustration input screen includes a center guidance display 814, a mouse disabled display 830, a home button 840, a drawing area 3201, an undo button 3202, a pen / ruler / eraser button 3203, Pen thickness button 3204, pen color button 3205, stamp button 3206, frame button 3207, delete all button 3208, screen capture button 3209, mail attachment button 3210, file save button 3211, and paste button 3212 Including.

  The drawing area 3201 accepts input from the outside. The electronic device 100 creates the illustration data 432b based on the input to the drawing area 3201 and the drawing settings (input tool, pen thickness, pen color, etc.) and stores the illustration data 432b in the storage unit 430. Similar to the handwritten character data 432a, the illustration data 432b includes all the coordinates that have been input within a predetermined time, or the coordinates at the start and end of input that are temporally continuous within the predetermined time. The illustration data 432b further includes data related to drawing settings (input tool, pen thickness, pen color, etc.).

  In the present embodiment, the ratio of the horizontal length to the vertical length of the drawing area 3201 is 4: 3. This is because the photograph is also processed using the handwritten illustration input pad. However, the aspect ratio of the drawing area 3201 is not limited to this.

  An undo button 3202 is a button for canceling the previous input operation to the drawing area 3201. The electronic device 100 manages the inputs recorded in the input history 432 in order of time, and when the undo button 3202 is pressed, the previous input is deleted from the input history 432. At the same time, the electronic device 100 erases the graphic display based on the drawing data corresponding to the immediately preceding input from the drawing area 3201.

  A pen / ruler / eraser button 3203 is a button for selecting a drawing tool for the drawing area 3201. In response to pressing of the pen / ruler / eraser button 3203, the electronic device 100 switches the drawing tool in the order of pen, ruler, eraser, pen,.

  The pen thickness button 3204 is a button for setting the pen thickness. In response to pressing of the pen thickness button 3204, the electronic device 100 changes the setting of the thickness of a line drawn according to the input to the drawing area 3201 when the pen tool is selected. Alternatively, electronic device 100 may display on LCD panel 240 a screen for allowing the user to set the thickness of the line when pen thickness button 3204 is pressed.

  A pen color button 3205 is a button for setting a pen color. In response to pressing of the pen color button 3205, the setting of the color of the line to be drawn is changed according to the input to the drawing area 3201 when the pen tool is selected. Alternatively, electronic device 100 may display on LCD panel 240 a screen for allowing the user to set a line color when pen color button 3205 is pressed.

  The stamp button 3206 is a button for causing a stamp to be drawn in the drawing area 3201 in response to an input to the drawing area 3201.

  The frame button 3207 is a button for adding a frame such as a decorative frame to the illustration drawn in the drawing area 3201.

  The delete all button 3208 is a button for deleting all the illustration data 432b. By pressing this button, the user can set the drawing area 3201 to the state (plain color) when the hand-drawn illustration input pad is activated.

  A screen capture button 3209 is a button for saving the entire screen displayed on the liquid crystal panel 240. The mail attachment button 3210 is a button for attaching the illustration data 432b to an electronic mail. The file save button 3211 is a button for saving the illustration data 432b in a designated storage area. The area for storing the illustration data 432b may be fixed or may be designated by the user.

  The paste button 3212 is a button for sending the illustration data 432b to the active main application. When the paste button 3212 is pressed, the electronic device 100 provides the active main application with the illustration data 432b created by the hand-drawn illustration input pad. For example, the user can insert an illustration into a document being created by the main application using the hand-drawn illustration input pad.

  In this embodiment, as described with reference to FIG. 23, when the hand-drawn illustration input pad receives an instruction to end the hand-drawn illustration input pad, the hand-drawn illustration input pad 432b determines whether the created hand-drawn illustration data 432b needs to be saved. Inquire. However, this inquiry is not essential. The hand-drawn illustration input pad may automatically discard the hand-drawn illustration data 432b created so far at the end.

(Calculator / Number input pad)
The operation of electronic device 100 that executes the calculator / number input pad (specifically, CPU 210 that executes the application) will be described with reference to FIG. FIG. 33 is a diagram illustrating an example of a screen (calculator screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the calculator / number input pad.

  Referring to FIG. 33, the calculator screen includes a center guidance display 814, a mouse disabled display 830, a home button 840, a number box 3301, a pasting button 3302, a number button 3303, and a function button 3304.

  The number box 3301 displays the input number or the number of the calculation result. It is assumed that the maximum number of numbers that can be displayed in the number box 3301 is eight. However, the maximum number is not limited to eight.

  When paste button 3302 is pressed, electronic device 100 transmits the number displayed in number box 3301 to the active application displayed on liquid crystal panel 140.

  A number button 3303 is a button for inputting a number in the number box 3301. The function button 3304 is a button for instructing a predetermined calculation such as four arithmetic operations. Since the operation of electronic device 100 when numeric button 3303 and function button 3304 are pressed is the same as the operation of a normal calculator or a calculator application, detailed description thereof will not be repeated here.

(the Internet)
The operation of electronic device 100 (specifically, CPU 210 that executes an application) that executes Web page calling software that is one of the sub-screen utilization software will be described with reference to FIG. FIG. 34 is a diagram illustrating an example of a screen (Internet screen) displayed on the liquid crystal panel 240 by the electronic device 100 while the Web page calling software is activated.

  Referring to FIG. 34, the Internet screen includes a guidance display 810 (left guidance display 812, center guidance display 814 and right guidance display 816), a plurality of operation button displays 820, a mouse disabled display 830, and a home button 840. including.

  Each of the operation button displays 820 corresponds to a web page to be called. Each operation button display 820 includes characters representing the name of the corresponding Web page (such as “Internet 2” in the figure). When electronic device 100 detects a touch on an area corresponding to operation button display 820, electronic device 100 activates a web browser and displays the selected web page on liquid crystal panel 140 (or liquid crystal panel 240).

  Note that the web page calling software may display a scroll bar on the liquid crystal panel 240. An example of an internet screen including a scroll bar is shown in FIG. Referring to FIG. 35, the Internet screen includes a scroll bar 3500. When the user drags the slider 3502 in the scroll bar 3500, the Web page calling software scrolls the Internet screen.

(dictionary)
The operation of electronic device 100 (specifically, CPU 210 that executes an application) that executes dictionary calling software, which is one of the sub-screen utilization software, will be described with reference to FIG. FIG. 36 is a diagram illustrating an example of a screen (dictionary selection screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the dictionary calling software.

  Referring to FIG. 36, the dictionary selection screen includes a guidance display 810 (a left guidance display 812, a center guidance display 814, and a right guidance display 816), a plurality of operation button displays 820, a mouse disabled display 830, and a home button 840. Including.

  Each of the operation button displays 820 corresponds to the electronic dictionary to be called. Each operation button display 820 includes characters representing the name of the corresponding electronic dictionary (such as “English-Japanese dictionary” in the figure). When electronic device 100 detects a touch on an area corresponding to operation button display 820, electronic device 100 activates the electronic dictionary and displays the screen of the activated electronic dictionary on liquid crystal panel 140 (or liquid crystal panel 240).

  The dictionary calling software may display a scroll bar on the liquid crystal panel 240. An example of a dictionary selection screen including a scroll bar is shown in FIG. Referring to FIG. 37, the dictionary selection screen includes a scroll bar 3700. When the user drags slider 3702 in scroll bar 3700, the dictionary calling software scrolls the dictionary selection screen.

<Cursor display control>
In the mouse mode, electronic device 100 can move the cursor displayed on liquid crystal panel 140 by changing the input position on liquid crystal panel 240 (such as sliding an object on liquid crystal panel 240).

  On the other hand, in the tablet mode, electronic device 100 does not move the cursor by changing the input position to liquid crystal panel 240 (such as sliding an object on liquid crystal panel 240). That is, in the tablet mode, the cursor display is not directly related to the operation of the main application. Therefore, in order not to hinder the visibility of the image displayed on the liquid crystal panel 140, it is preferable to make the cursor display inconspicuous in the tablet mode.

  Therefore, in the present embodiment, electronic device 100 displays the cursor in a display mode that is less noticeable than the display mode in mouse mode during the tablet mode. Specifically, the electronic device 100 displays the cursor lighter in the tablet mode than in the mouse mode. More specifically, the electronic device 100 reduces the brightness of the cursor, for example. Alternatively, electronic device 100 increases the transparency of the display color of the cursor.

  With reference to FIG. 38, a change in cursor display control according to the present embodiment will be described. FIG. 38 is a diagram for explaining a change in cursor display control according to the first embodiment.

  FIG. 38A shows a display screen 3810 of the liquid crystal panel 140 and a display screen 3820 of the liquid crystal panel 240 in the mouse mode. Display screen 3810 includes a cursor 3812. An operation screen of the main application is displayed in a portion other than the cursor 3812 in the display screen 3810. The display screen 3820 displays a mouse screen such as wallpaper. However, in FIG. 38A, for the sake of simplicity, the operation screen and the mouse screen of the main application are not shown in detail. This also applies to the following drawings.

  In the mouse mode, electronic device 100 moves cursor 3812 in real time in accordance with a change in the input position to liquid crystal panel 240. This is indicated by a solid arrow from the liquid crystal panel 240 to the liquid crystal panel 140 in FIG.

  FIG. 38B shows a display screen 3830 of the liquid crystal panel 140 and a display screen 3840 of the liquid crystal panel 240 in the tablet mode. Here, the case where the electronic apparatus 100 starts the book viewer application from the tablet mode home application is shown as an example. However, the description of the operation of the electronic device 100 here also applies when the electronic device 100 starts a sub application other than the book viewer.

  Display screen 3830 includes a cursor 3832 and a window 3834. The window 3834 is created by the executed book viewer application. Window 3834 displays an operation screen of the book viewer application.

  The cursor 3832 is thinner than the cursor 3812 in the mouse mode. In FIG. 38B, the cursor 3832 is hatched to indicate that the cursor 3832 is thin. Actually, the cursor 3832 need not be displayed with diagonal lines.

  The display screen 3840 displays an operation assistance screen for the book viewer application. The user can turn pages by operating the operation assist screen. However, the user cannot move the cursor 3832 by changing the input position on the liquid crystal panel 240. That is, in the tablet mode, electronic device 100 does not reflect the change in the input position to liquid crystal panel 240 in the movement of cursor 3832. In FIG. 38 (b), this is indicated by a dashed arrow from the liquid crystal panel 240 toward the liquid crystal panel 140.

  If the electronic apparatus 100 changes the position of the cursor 3832 depending on the input position to the liquid crystal panel 240, the cursor 3832 moves each time the user performs an operation such as turning the page, so that the visibility of the liquid crystal panel 140 may be deteriorated. is there. As in the present embodiment, the electronic device 100 prevents the display screen of the liquid crystal panel 140 from being difficult to see by controlling the cursor position of the liquid crystal panel 140 independently of the input position to the tablet in the tablet mode. be able to.

  In addition, since the display mode of the cursor changes between the mouse mode and the tablet mode, the user can easily grasp whether or not the cursor can be moved by an input to the liquid crystal panel 240.

  FIG. 38C is a diagram showing the display screen 3850 of the liquid crystal panel 140 and the display screen 3860 of the liquid crystal panel 240 after shifting from the state shown in FIG. 38B (tablet mode) to the mouse mode.

  Display screen 3850 includes a cursor 3852 and a window 3854. Window 3854 is the same as window 3834 in tablet mode. In other words, even after the electronic device 100 shifts from the tablet mode to the mouse mode, the operation screen of the application displayed on the liquid crystal panel 140 in the tablet mode is continuously displayed on the liquid crystal panel 140.

  On the contrary, the electronic device 100 continues to display the operation screen of the application displayed on the liquid crystal panel 140 in the mouse mode on the liquid crystal panel 140 even after shifting from the mouse mode to the tablet mode.

  Thus, the electronic device 100 displays the same operation screen on the liquid crystal panel 140 before and after switching between the mouse mode and the tablet mode. Therefore, the operability of the electronic device 100 is improved. While viewing the operation screen of the application displayed on the liquid crystal panel 140, the user selectively selects the operation of the mouse or the operation of the application that assists the operation of the application displayed on the liquid crystal panel 140 as necessary. Can be done.

  The cursor 3852 has the same display form as the cursor 3812 shown in FIG. That is, electronic device 100 displays the same type of cursor on liquid crystal panel 140 in the mouse mode.

  As in the case shown in FIG. 38A, the display screen 3860 is a mouse screen. In addition, electronic device 100 moves cursor 3852 in accordance with a change in the input position to liquid crystal panel 240.

  Heretofore, an example in which the electronic device 100 displays the cursor lighter in the tablet mode than in the mouse mode has been described with reference to FIG. However, the method of making the cursor inconspicuous in the tablet mode is not limited to this.

  For example, the electronic device 100 may stop the cursor display in the tablet mode. That is, electronic device 100 may hide the cursor in the tablet mode. This can be regarded as one form of changing the display form. That is, electronic device 100 can hide the cursor by setting the cursor to be completely transparent.

  Alternatively, electronic device 100 may hide the cursor by creating display data that does not include the cursor in the tablet mode. In this case, electronic device 100 displays a cursor at a predetermined position (for example, a corner of the screen of liquid crystal panel 140) after shifting from the tablet mode to the mouse mode. Alternatively, the electronic device 100 stores the cursor position in a storage device such as the RAM 271 when shifting to the tablet mode, reads the cursor position from the storage device when shifting to the mouse mode, and moves the cursor to the read cursor position. May be displayed.

  Further, electronic device 100 may move the display position of the cursor in order to make the cursor inconspicuous in the tablet mode. An example of this operation will be described with reference to FIG. FIG. 39 is a diagram for describing a change in cursor display control according to a modification of the first embodiment.

  FIG. 39A shows a display screen 3910 of the liquid crystal panel 140 and a display screen 3920 of the liquid crystal panel 240 in the mouse mode. Display screen 3910 includes a cursor 3912. Display screen 3920 is a mouse screen. In the state shown in FIG. 39A, the electronic device 100 operates in the same manner as in the state shown in FIG. That is, electronic device 100 moves cursor 3912 according to the input position on liquid crystal panel 240.

  FIG. 39B shows a display screen 3930 of the liquid crystal panel 140 and a display screen 3940 of the liquid crystal panel 240 in the tablet mode. Here, as in FIG. 38B, the case where the electronic device 100 is executing a book viewer is shown as an example. However, the description of the operation of the electronic device 100 here also applies to the case where the electronic device 100 is executing an application other than the book viewer.

  Display screen 3930 includes a cursor 3932 and a window 3934. The window 3934 is created by an application in the same manner as the window 3834 in FIG.

  The cursor 3932 is in the lower right corner of the screen. That is, electronic device 100 moves the cursor in mouse mode to a predetermined position in tablet mode. By changing the display position of the cursor in this way, it is possible to prevent the window 3934 from being difficult to see with the cursor. In FIG. 39B, a virtual cursor 3932 # corresponding to the position of the cursor 3912 in the mouse mode is indicated by a broken line for easy understanding.

  In the present embodiment, electronic device 100 moves the cursor to the lower right corner of the display screen. Thus, the visibility of the screen can be improved more reliably by moving the cursor to the end of the display screen. Here, the “end portions” are not limited to the four corners of the screen, and may be each side of the screen or a region within a predetermined distance from each side. Further, all the cursors may not be displayed, and some or almost all of them may be hidden at the edge of the screen.

  In the present embodiment, the display form of the cursor 3932 is assumed to be the same as the display form in the mouse mode, as shown in FIG. However, the electronic device 100 may change the display form such as displaying the cursor after movement lightly compared to the mouse mode.

  FIG. 39C is a diagram showing the display screen 3950 of the liquid crystal panel 140 and the display screen 3960 of the liquid crystal panel 240 after the transition to the mouse mode from the state shown in FIG. 39B (tablet mode).

  Display screen 3950 includes a cursor 3952 and a window 3954. The relationship between the window 3934 and the window 3954 in FIG. 39B is the same as the relationship between the window 3834 in FIG. 38B and the window 3854 in FIG.

  The cursor 3952 is displayed at the same position as the cursor 3812 shown in FIG. That is, electronic device 100 displays a cursor at the same position in the screen of liquid crystal panel 140 before the transition from the mouse mode to the tablet mode and after the return from the tablet mode to the mouse mode. In FIG. 39C, a virtual cursor 3952 # corresponding to the position of the cursor 3932 in the tablet mode is indicated by a broken line for easy understanding.

  In order to move the cursor, the electronic device 100 stores the cursor position in a storage device such as the RAM 271 when the mouse mode is changed to the tablet mode. When the electronic device 100 returns from the tablet mode to the mouse mode, the electronic device 100 reads the stored cursor position and displays the cursor at the read cursor position.

  In this way, by displaying the cursor at the same position in the mouse mode before and after the tablet mode, even if the user once shifts from the mouse mode to the tablet mode, the user can move the mouse from the same position after returning to the mouse mode again. Can be done.

  However, when the electronic device 100 shifts from the tablet mode to the mouse mode, the cursor does not necessarily have to be moved to a position before the shift to the tablet mode. If the cursor is not moved at this time, the user may easily operate the mouse.

  As in the case shown in FIG. 39A, the display screen 3960 is a mouse screen. In addition, the electronic device 100 moves the cursor 3952 in accordance with a change in the input position to the liquid crystal panel 240.

<Process flow>
(Basic flow)
With reference to FIG. 40, the flow of processing performed by electronic device 100 according to the present embodiment will be described. FIG. 40 is a diagram illustrating a flow of processing performed by the electronic device 100 in a flowchart format. In FIG. 40, the processing performed by the control unit 350 in the first unit 1001 and the processing performed by the control unit 450 in the second unit 1002 are collectively shown.

  In step S <b> 101, the control unit 350 and the control unit 450 perform a normal activation process or a resumption process when receiving a normal activation instruction or a resumption instruction of the electronic device 100.

  Normal startup is startup from a power-off state, as already described. Examples of the normal startup process performed by the control unit 350 include a boot process and a boot screen display on the liquid crystal panel 140. The normal startup process performed by the control unit 450 includes displaying a boot screen on the liquid crystal panel 240.

  Control unit 350 and control unit 450 handle pressing of a predetermined button (such as power switch 191) in the power-off state as a normal activation instruction. One control unit (control unit 450 or 350) accepts a normal activation instruction, performs its own normal activation process, and instructs the other control unit (control unit 350 or 450) to perform a normal activation process. The structure to give may be sufficient.

  As already described, the restart is a start from the power saving state. The restart process performed by the control unit 350 includes reading of a work state stored in the RAM 171 or the HDD 170 or displaying a restarting screen on the liquid crystal panel 140. The restart process performed by the control unit 450 includes reading of a work state stored in the RAM 271 or the HDD 170 or displaying a restarting screen on the liquid crystal panel 240.

  The control unit 350 and the control unit 450 handle pressing of a predetermined button (power switch 191 or the like) in the power saving state, touching the liquid crystal panel 240, or the like as an instruction for restart processing. One control unit (450 or 350) may perform the restart process in response to an instruction from the other control unit (350 or 450) that has received the restart instruction.

  In step S103, the mode setting unit 454 included in the control unit 450 determines the operation mode to be either the mouse mode or the tablet mode.

  In the present embodiment, the operation mode for normal activation is determined to be the mouse mode. In the case of normal activation, mode setting unit 454 sets mode data 437 in storage unit 430 to data representing a mouse mode.

  In addition, the mode setting unit 454 determines the mode data 437 based on the operation mode of the mode data 437 before the power saving state when restarting. In this case, the control unit 450 stores the mode data 437 before the power saving state in the RAM 271 or the like when shifting to the power saving state or at the time of restarting. Alternatively, the mode setting unit 454 may always set the mode data 437 to data representing the mouse mode when resuming. In this case, the controller 450 does not have to perform the storage process of the mode data 437 before the power saving state.

  In step S105, the input processing unit 452 determines whether the operation mode is the mouse mode. That is, the input processing unit 452 determines whether the operation mode is the mouse mode based on the mode data 437.

  If the operation mode is the mouse mode (YES in step S105), control unit 350 and control unit 450 proceed to the mouse mode operation in step S107. If the operation mode is not the mouse mode (NO in step S105), control unit 350 and control unit 450 proceed to the tablet mode operation in step S113.

  In step S107, control unit 350 and control unit 450 perform a mouse mode operation. That is, the control unit 350 and the control unit 450 control each unit of the electronic device 100 so that the input to the liquid crystal panel 240 causes the mouse operation of the main application. Details of the mouse mode operation will be described later.

  In step S109, the mode setting unit 454 determines whether a mode switching instruction has been accepted. Specifically, mode setting unit 454 determines whether a signal in response to pressing of center key 242 has been received. However, the mode switching instruction is not limited to pressing the center key 242.

  If there is a mode switching instruction during the mouse mode (YES in step S109), control unit 450 executes a switching process from the mouse mode to the tablet mode in step S111. When there is no mode switching instruction (NO in step S109), control unit 350 and control unit 450 repeat the processing from step S107 (mouse mode operation).

  In step S111, the control unit 450 executes a switching process from the mouse mode to the tablet mode. For example, in step S111, the control unit 450 displays the operation screen of the sub application on the liquid crystal panel 240 and switches the operation of the panel input processing unit 453. Control unit 350 also changes the cursor display. Details of the switching process from the mouse mode to the tablet mode will be described later. After step S111 ends, control unit 350 and control unit 450 proceed to the tablet mode operation of step S113.

  In step S113, control unit 350 and control unit 450 perform a tablet mode operation. That is, the control unit 350 and the control unit 450 control each unit of the electronic device 100 so that the sub application operates in response to an input to the liquid crystal panel 240. Details of the tablet mode operation will be described later.

  In step S115, mode setting unit 454 determines whether a mode switching instruction has been accepted. Specifically, mode setting unit 454 determines whether a signal in response to pressing of center key 242 has been received. However, the mode switching instruction is not limited to pressing the center key 242.

  If there is a mode switching instruction during the tablet mode (YES in step S115), control unit 450 executes a switching process from the tablet mode to the mouse mode in step S117. When there is no mode switching instruction (NO in step S115), control unit 350 and control unit 450 repeat the processing from step S113 (tablet mode operation).

  In step S117, the control unit 350 and the control unit 450 execute switching processing from the tablet mode to the mouse mode. For example, the control unit 450 displays the mouse screen on the liquid crystal panel 240 and switches the operation of the panel input processing unit 453 in step S117. In addition, control unit 350 restores the cursor display. Details of the switching process from the tablet mode to the mouse mode will be described later. After completion of step S117, control unit 350 and control unit 450 proceed to the mouse mode operation of step S107.

  Note that the control unit 350 and the control unit 450 perform a power-off process or a power-saving state transition process when receiving a power-off instruction or a power-saving state transition instruction. These processes are interrupt processes, and are performed after any of the steps in FIG. However, these processes are not shown in FIG.

(Mouse mode operation)
The mouse mode operation in step S107 in FIG. 40 will be described in detail with reference to FIG. FIG. 41 is a flowchart illustrating the processing flow of the mouse mode operation.

  First, the operation of the control unit 350 on the first unit 1001 side will be described. The operation flow of the control unit 350 is shown on the left side in FIG.

  In step S <b> 201, the control unit 350 acquires coordinate data from the interface unit 340. The coordinate data is transmitted from the control unit 450 on the second unit 1002 side to the interface unit 340 via the interface unit 440.

  In step S203, control unit 350 determines the cursor position based on the coordinate data. More specifically, the program execution unit 358 included in the control unit 350 performs the process of step S203. The program execution unit 358 executes the program 334 to determine the cursor position.

  In step S <b> 205, the control unit 350 acquires a command from the interface unit 340. This command is a click event transmitted from the control unit 450 on the second unit 1002 side to the interface unit 340 via the interface unit 440.

  In step S207, control unit 350 performs an application operation according to the command. Specifically, program execution unit 358 executes program 334 to perform application operations. The program execution unit 358 determines an application operation based on the application type, the cursor position, and the command type.

  The application operation is similar to the operation associated with a mouse click of a currently popular application. The application operation includes, for example, selection or activation of a file or folder at the cursor position, or execution of processing according to a button (minimum (maximum) button, close button, etc.) at the cursor position.

  Next, the operation of the control unit 450 on the second unit 1002 side will be described. The flow of the operation of the control unit 450 is shown on the right side in FIG.

  In step S301, the panel input processing unit 453 included in the input processing unit 452 of the control unit 450 acquires a scan image from the liquid crystal panel 240 (panel input unit 422).

  In step S303, the panel input processing unit 453 calculates coordinate data specifying the input position to the liquid crystal panel 240 based on the scanned image acquired in step S301.

  In step S305, the panel input processing unit 453 controls the interface unit 440 and transmits coordinate data to the interface unit 340 on the first unit 1001 side.

  In step S307, the panel input processing unit 453 determines whether a predetermined scan cycle time has elapsed since the execution of step S301. If the scan cycle time has elapsed (YES in step S307), panel input processing unit 453 repeats the processing from step S301 (scanned image acquisition). If the scan cycle time has not elapsed (NO in step S307), control unit 450 proceeds to the process in step S309.

  In step S309, the input processing unit 452 determines whether there has been a click operation. Specifically, the input processing unit 452 determines whether the left click key 241 or the right click key 243 has been pressed. Note that the input processing unit 452 may determine that the tap operation on the liquid crystal panel 240 is a click operation. Specifically, the input processing unit 452 determines that a tap operation has been performed when the detection of an external object is started and ended in a specific area in the liquid crystal panel 240 within a predetermined short time.

  If there is no click operation (NO in step S309), control unit 450 repeats the processing from step S307. If there is a click operation (YES in step S309), control unit 450 proceeds to the process of step S311.

  In step S311, the input processing unit 452 controls the interface unit 440, and transmits a command to the interface unit 340 on the first unit 1001 side according to the click operation in step S309.

  Here, it is assumed that the input processing unit 452 determines the type of command according to the type of click operation. For example, the input processing unit 452 transmits different commands when the left click key 241 is pressed and when the right click key 243 is pressed.

  In the above, the command transmission by the click operation has been described. However, the operation that causes the command transmission is not limited to the click operation. For example, the input processing unit 452 may transmit a command corresponding to double click, drag, multi-tap, gesture operation, or the like.

(Tablet mode operation)
The tablet mode operation in step S113 in FIG. 40 will be described in detail with reference to FIG. FIG. 42 is a flowchart illustrating the processing flow of the tablet mode operation.

  First, the operation of the control unit 350 on the first unit 1001 side will be described. The operation flow of the control unit 350 is shown on the left side in FIG.

  In step S <b> 401, the control unit 350 acquires data from the interface unit 340. The “data” here is created by the control unit 450 on the second unit 1002 side executing the sub-application.

  Specifically, the “data” is, for example, character (or number) data or illustration data. The data may be a command. For example, when “Enter” on the operation screen of the handwritten character input pad is touched, the control unit 350 creates a command.

  In step S403, control unit 350 performs an application operation corresponding to the data. Specifically, program execution unit 358 executes program 334 to perform application operations. The program execution unit 358 processes the data acquired in step S401 by the main application being executed. When step S403 ends, the control unit 350 returns to the process of step S401.

  Next, the operation of the control unit 450 on the second unit 1002 side will be described. The operation flow of the controller 450 is shown on the right side in FIG.

  In step S501, the panel input processing unit 453 included in the input processing unit 452 of the control unit 450 acquires a scan image from the liquid crystal panel 240 (panel input unit 422).

  In step S503, the panel input processing unit 453 calculates coordinate data specifying the input position to the liquid crystal panel 240 based on the scanned image acquired in step S501.

  In step S505, the panel input processing unit 453 determines whether a predetermined scan cycle time has elapsed since the execution of step S501. When the scan cycle time has elapsed (YES in step S505), panel input processing unit 453 repeats the processing from step S501 (scanned image acquisition). If the scan cycle time has not elapsed (NO in step S505), control unit 450 proceeds to the process in step S507.

  In step S507, control unit 450 performs a sub-application operation. The sub application operation includes operation execution by time count. In step S507, the control unit 450 may control the interface unit 440 to transmit data to the first unit 1001 side.

  In step S <b> 509, the control unit 450 stores data (referred to as “operation element”) for determining the operation of the application being executed in the storage unit 430. In the present embodiment, the operation parameter 435 including the elapsed time from the event and the input history 432 correspond to the operation elements. Note that the control unit 450 performs the processing in step S509 at a predetermined time interval, when changing an operating element, when saving an operating element (for example, when saving an illustration), and the like. After executing step S509, control unit 450 repeats the processing from step S505.

(Mode switching: mouse mode to tablet mode)
Mode switching (from the mouse mode to the tablet mode) in step S111 in FIG. 40 will be described in detail with reference to FIG. FIG. 43 is a flowchart illustrating the process flow of the mode switching (from the mouse mode to the tablet mode) operation.

  In step S601, the mode setting unit 454 included in the control unit 450 loads the operation element of the immediately preceding application stored in the storage unit 430.

  Here, the “immediate application” is a sub application that was last operated in the tablet mode before the mode setting unit 454 received a mode switching instruction to the tablet mode. In the present embodiment, since the sub application continues to operate even in the mouse mode, the immediately preceding application is the same as the sub application operating at the time of mode switching instruction.

  In step S603, the program execution unit 458 included in the control unit 450 executes the immediately preceding application based on the operation element loaded in step S601. Then, the program execution unit 458 controls the display control unit 456 to display the operation screen of the sub application on the display unit 410.

  However, the program execution unit 458 may perform the process of step S601 at any time during the process of step S603. In other words, during the execution of the immediately preceding application, the operation element stored in the storage unit 430 may be read as necessary, and the sub application may be executed based on the read data.

  In step S605, the panel input processing unit 453 included in the control unit 450 switches the processing method of the signal from the panel input unit 422. That is, the panel input processing unit 453 converts the signal from the panel input unit 422 into an operation instruction of the sub application.

  In step S607, the program execution unit 358 and the display control unit 356 change the display of the cursor in the liquid crystal panel 140. Details of the processing for changing the cursor display will be described later.

(Mode switching: tablet mode to mouse mode)
Hereafter, the mode switching (from the tablet mode to the mouse mode) in step S117 in FIG. 40 will be described in detail with reference to FIG. FIG. 44 is a flowchart showing the processing flow of the mode switching (tablet mode to mouse mode) operation.

  In step S <b> 701, when the display control unit 456 included in the control unit 450 receives a signal indicating that there has been a mode switching instruction from the mode setting unit 454, the display unit 456 displays the mouse mode screen on the liquid crystal panel 240 based on the display data 433. Is displayed.

  In step S <b> 703, the panel input processing unit 453 included in the control unit 450 switches the processing method of the signal from the panel input unit 422. That is, the panel input processing unit 453 converts the signal from the panel input unit 422 into a mouse operation signal on the liquid crystal panel 140.

  In step S705, the program execution unit 358 and the display control unit 356 included in the control unit 350 perform processing for restoring the cursor display that has been changed when the tablet mode is changed from the mouse mode. Details of the cursor recovery process will be described later.

(Display change and restoration of cursor [Part 1]: Change display mode)
From here, the details of the process in step S607 of FIG. 43 (cursor display change) and the process in step S705 of FIG. 44 (cursor recovery) will be described.

  First, processing performed by the control unit 350 when the cursor display mode is changed between the mouse mode and the tablet mode, such as displaying the cursor lightly in the tablet mode, will be described with reference to FIG. FIG. 45 is a flowchart showing the flow of processing for changing the cursor display form.

  In step S <b> 801, the display control unit 356 included in the control unit 350 loads cursor display data for tablet mode from the storage unit 330. It is assumed that the cursor display data for the tablet mode is stored in the storage unit 330 in advance. The cursor display data for the tablet mode is different from the cursor display data for the mouse mode (which is also stored in the storage unit 330). Specifically, the cursor display data for the tablet mode is set to have lower brightness or higher transparency than the cursor display data for the mouse mode.

  In step S803, the display control unit 356 displays a touch mode cursor on the liquid crystal panel 140 based on the tablet mode cursor display data loaded in step S801.

  The cursor recovery processing corresponding to the change in the cursor display mode described with reference to FIG. 45 will be described with reference to FIG. FIG. 46 is a diagram showing a flow of restoration processing of the cursor display form in a flowchart form.

  In step S <b> 901, the display control unit 356 included in the control unit 350 loads cursor display data for mouse mode from the storage unit 330.

  In step S903, the display control unit 356 displays a mouse mode cursor on the liquid crystal panel 140 based on the mouse mode cursor display data loaded in step S901.

  Here, the display control unit 356 stores the cursor display data for the tablet mode and the mouse mode in advance. However, the display control unit 356 may create one cursor display data based on the other cursor display data. For example, the display control unit 356 may create the cursor display data by performing predetermined change processing relating to the brightness and the transparency on the cursor display data for the mouse mode stored in advance.

(Change and restore cursor display [Part 2]: Display / hide cursor)
Next, processing performed by the control unit 350 when the cursor is hidden in the tablet mode will be described with reference to FIG. FIG. 47 is a flowchart showing the flow of processing when the cursor is not displayed.

  In step S <b> 1001, the control unit 350 stores the current cursor position in the storage unit 330. Here, “present” refers to a point in time when the control unit 350 receives a mode switching instruction from the mouse mode to the tablet mode.

  In step S1003, control unit 350 hides the cursor of liquid crystal panel 140. Specifically, the program execution unit 358 included in the control unit 350 creates display data based on the result of executing the program excluding the portion related to cursor display. Display control unit 356 displays a screen based on the generated display data on liquid crystal panel 140.

  Note that the control unit 350 may hide the cursor by making the cursor display completely transparent. In this case, the control unit 350 hides the cursor by calling the cursor display data corresponding to colorlessness by a process similar to the process shown in FIG. Alternatively, the control unit 350 may hide the cursor by changing display data of the currently displayed cursor (that is, the cursor in the mouse mode).

  A cursor recovery process corresponding to the non-display of the cursor described with reference to FIG. 47 will be described with reference to FIG. FIG. 48 is a flowchart showing the flow of the cursor re-display process.

  In step S1101, the display control unit 356 included in the control unit 350 loads the “current cursor position” stored in the storage unit 330 in step S1001 of FIG. 47 from the storage unit 330.

  In step S1103, the display control unit 356 displays a cursor at the cursor position loaded in step S1101. At this time, the display control unit 350 displays a mouse mode cursor based on the mouse mode cursor display data.

(Change and restore cursor display [Part 3]: Move cursor)
Subsequently, processing performed by the control unit 350 when the cursor is moved during the transition from the mouse mode to the tablet mode will be described with reference to FIG. FIG. 49 is a flowchart showing the flow of processing for moving the cursor during the transition from the mouse mode to the tablet mode.

  In step S <b> 1201, the control unit 350 stores the current cursor position in the storage unit 330. Here, “present” refers to the time point when the control unit 350 receives a mode switching instruction from the mouse mode to the tablet mode, as in the previous description.

  In step S1203, control unit 350 moves the cursor of liquid crystal panel 140. Specifically, the program execution unit 358 included in the control unit 350 reads the designated position of the cursor in the tablet mode stored in advance in the storage unit 330. Then, the program execution unit 358 creates display data for displaying a cursor at the read designated position. Display control unit 356 displays an image based on the generated display data on liquid crystal panel 140.

  The cursor recovery processing corresponding to the cursor movement described with reference to FIG. 49 will be described with reference to FIG. FIG. 50 is a flowchart showing the flow of processing when moving the cursor during the transition from the tablet mode to the mouse mode.

  In step S1301, the display control unit 356 included in the control unit 350 loads the “current cursor position” stored in the storage unit 330 in step S1201 of FIG.

  In step S1303, the display control unit 356 displays a cursor at the cursor position loaded in step S3101. At this time, the display control unit 350 displays a mouse mode cursor based on the mouse mode cursor display data.

  If the cursor position is not returned to the cursor position in the previous mouse mode when returning to the mouse mode, the processing in steps S1201 and S1301 is not necessary. In this case, the storage unit 330 stores in advance the cursor display position when the cursor is redisplayed. The display control unit 356 loads a predetermined display position from the storage unit 330 when the cursor is displayed again. Then, the display control unit 356 displays a cursor at the loaded predetermined display position.

[Second Embodiment]
The electronic device 100 according to the first embodiment always changes the display of the cursor when shifting from the mouse mode to the tablet mode. On the other hand, electronic device 100 according to the second embodiment changes the display of the cursor only when the cursor overlaps the active window in the display screen of liquid crystal panel 140.

  A change in cursor display control according to the second embodiment will be described with reference to FIG. FIG. 51 is a diagram for explaining a change in cursor display control according to the second embodiment.

  FIG. 51A shows the display screen 5110 of the liquid crystal panel 140 and the display screen 5120 of the liquid crystal panel 240 in the mouse mode. Display screen 5110 includes a cursor 5112. In FIG. 51A, a cursor 5112 is shown at two different positions for explanation. However, actually, there is one cursor 5112 displayed on the liquid crystal panel 140, and two cursors 5112 are not displayed on the liquid crystal panel 140 at the same time. In FIG. 51A, this is indicated by putting parentheses on one cursor 5112. This display method is the same for FIGS. 51 (b) and 51 (c).

  FIG. 51B shows the display screen 5130 of the liquid crystal panel 140 and the display screen 5140 of the liquid crystal panel 240 in the tablet mode. Here, as in FIG. 38B and the like, a case where the electronic device 100 is executing a book viewer is shown as an example.

  Display screen 5130 includes a cursor 5132 and a window 5134. Window 5134 is an active window created by the executed book viewer application.

  Here, the operation when electronic device 100 moves the cursor will be described. In the present embodiment, electronic device 100 determines the display position of cursor 5132 based on the positional relationship between window 5134 and the display position in mouse mode before switching to tablet mode.

  When the cursor position in the mouse mode does not overlap with the display area of the window 5134, the electronic device 100 displays the cursor at the same position as the display position in the mouse mode in the tablet mode. That is, when the electronic device 100 displays the cursor 5112 without parentheses shown in FIG. 51A in the mouse mode, the electronic device 100 displays the cursor 5132 without parentheses in the tablet mode.

  On the other hand, when the cursor position in the mouse mode overlaps with the display area of the window 5134, the electronic device 100 moves the cursor during the transition from the mouse mode to the tablet mode, and displays the window 5134 in the tablet mode. Display the cursor outside the area. That is, in the mouse mode, when the cursor 5112 with parentheses shown in FIG. 51A is displayed in the mouse mode, the electronic device 100 displays the cursor 5132 with parentheses in the tablet mode. In FIG. 51B, a virtual cursor 5132 # corresponding to the position of the cursor 5112 in the mouse mode is indicated by a broken line for easy understanding.

  FIG. 51C is a diagram showing the display screen 5150 of the liquid crystal panel 140 and the display screen 5160 of the liquid crystal panel 240 after shifting from the state shown in FIG. 51B (tablet mode) to the mouse mode.

  Display screen 5150 includes a cursor 5152 and a window 5154. The relationship between the window 5134 and the window 5154 in FIG. 51B is the same as the relationship between the window 3834 in FIG. 38B and the window 3854 in FIG.

  When the display position of the cursor is not moved at the time of transition from the mouse mode to the tablet mode, the electronic device 100 displays the cursor at the same position as before the return to the mouse mode (in FIG. 51 (c), parentheses). A cursor 5152 without a mark.

  On the other hand, if the display position of the cursor has been moved at the time of transition from the mouse mode to the tablet mode, the electronic device 100 moves the cursor to the same position as the position in the mouse mode before the transition to the tablet mode. (Cursor 5152 in parentheses in FIG. 51 (c)). In FIG. 51C, for easy understanding, a virtual cursor 5152 # corresponding to the position of the parenthesis cursor 5132 in the tablet mode is indicated by a broken line.

  The movement of the cursor when the cursor overlaps the active window has been described above, but the electronic device 100 may change the display form such as displaying the cursor lightly when the cursor overlaps the window. Good.

  The hardware configuration and functional configuration of electronic device 100 according to the second embodiment are substantially the same as those of the first embodiment, and detailed description thereof will not be repeated. However, as will be described below, the processing performed by the control unit 350 and the control unit 450 is different from the first embodiment and the second embodiment.

  Further, the basic flow of processing of the electronic device 100 according to the second embodiment is the same as that of the first embodiment (FIG. 40) and will not be repeated. Also, the mouse mode operation and the tablet mode operation are the same as those in the first embodiment (FIGS. 41 and 42) and will not be repeated. However, the mode switching process is different from the first embodiment as described below.

  Mode switching (from the mouse mode to the tablet mode) according to the second embodiment will be described in detail with reference to FIG. FIG. 52 is a flowchart showing the process flow of the mode switching (mouse mode to tablet mode) operation according to the second embodiment.

  In step S1401, the mode setting unit 454 included in the control unit 450 loads the operation element of the immediately preceding application stored in the storage unit 430. Here, the “immediate application” is a sub-application that was last operated in the tablet mode before the mode setting unit 454 accepted the mode switching instruction to the tablet mode, as already described.

  In step S1403, the program execution unit 458 included in the control unit 450 executes the immediately preceding application based on the operation element loaded in step S1401. Then, the program execution unit 458 controls the display control unit 456 to display the operation screen of the sub application on the display unit 410.

  However, the program execution unit 458 may perform the process of step S1401 at any time during the process of step S1403. In other words, during the execution of the immediately preceding application, the operation element stored in the storage unit 430 may be read as necessary, and the sub application may be executed based on the read data.

  In step S1405, the panel input processing unit 453 included in the control unit 450 switches the processing method of the signal from the panel input unit 422. That is, the panel input processing unit 453 converts the signal from the panel input unit 422 into an operation instruction of the sub application.

  In step S1407, the program execution unit 358 determines whether or not the cursor overlaps the window display area. Specifically, the program execution unit 358 reads data indicating the display area of the active window and data specifying the display position of the cursor from the storage unit 330. The program execution unit 358 determines that the cursor overlaps the display area of the window when there is something common in the positions specified by the respective data.

  Note that the program execution unit 358 may determine the overlap between the cursor and the window using a window display area or cursor display position wider than the actual one. In this way, electronic device 100 is located in the vicinity of the window that is outside the window, and can move a cursor that may hinder visual recognition of the window.

  If control unit 350 determines that the cursor overlaps the window (YES in step S1407), control unit 350 proceeds to step S1409. In step S1409, control unit 350 performs a display change process for the cursor displayed on liquid crystal panel 140. Specifically, control unit 350 performs processing similar to the processing described in FIG. 45, FIG. 47, or FIG.

  In the above example, the electronic device 100 moves the cursor to the corner of the screen. However, the electronic device 100 may move the cursor to an area that does not hinder the viewing of the window. For example, electronic device 100 may move the cursor to the edge of the screen, such as on each side of the screen, as in the first embodiment. The electronic device 100 may move the cursor to at least an area that does not obstruct the active window. For example, the electronic device 100 may move to the edge of the window or just outside the window.

  On the other hand, when control unit 350 determines that the cursor does not overlap the window (NO in step S1407), control unit 350 switches from the mouse mode to the tablet mode without performing the cursor display change process (without executing step S1409). The mode switching process ends.

(Mode switching: tablet mode to mouse mode)
From now on, mode switching (from tablet mode to mouse mode) according to the second embodiment will be described in detail with reference to FIG. FIG. 53 is a flowchart showing the process flow of the mode switching (tablet mode to mouse mode) operation according to the second embodiment.

  In step S1501, when the display control unit 456 included in the control unit 450 receives a signal indicating that a mode switching instruction has been received from the mode setting unit 454, the display unit 456 displays a mouse mode screen on the liquid crystal panel 240 based on the display data 433. Is displayed.

  In step S1503, the panel input processing unit 453 included in the control unit 450 switches the processing method of the signal from the panel input unit 422. That is, the panel input processing unit 453 converts the signal from the panel input unit 422 into a mouse operation signal on the display panel 140.

  In step S1505, the program execution unit 358 included in the control unit 350 determines whether or not the cursor display has been changed at the time of transition from the mouse mode to the tablet mode.

  For example, the program execution unit 358 stores the display mode (or display position) of the cursor in the mouse mode before the transition in the storage unit 330 when shifting from the mouse mode to the tablet mode. In step S1505, the program execution unit 358 compares the stored display form (or display position) with the cursor display form (or display position) in the tablet mode, and whether the cursor display has been changed. Judge whether.

  Alternatively, the program execution unit 358 stores data representing the determination result in the storage unit 330 in the determination of step S1407 in FIG. In step S1505, it is determined whether the display of the cursor has been changed based on the stored data representing the determination result.

  If the cursor display has been changed (YES in step S1505), in step S1507, the program execution unit 356 and the display control unit 356 included in the control unit 350 display the cursor changed when the mouse mode is changed to the tablet mode. Perform display restoration processing. Specifically, control unit 350 performs processing similar to the processing described in FIG. 46, FIG. 48, or FIG. If the cursor display has not been changed (NO in step S1505), control unit 350 ends the mode switching process without performing the cursor display recovery process.

[Third Embodiment]
In the third embodiment, an electronic apparatus 100 that can be connected to an external pointing device will be described. The external appearance of the electronic device 100 according to the third embodiment is shown in FIG. FIG. 54 is a diagram illustrating an appearance of an electronic device 100 according to the third embodiment.

  Referring to FIG. 54, the configuration of electronic device 100 is the same as that of the first embodiment except that mouse 1100 is connected. Note that the mouse 1100 is an example of a pointing device. The following description also applies to the electronic device 100 to which other pointing devices can be connected.

  The position aspect of the hardware configuration of electronic device 100 according to the third embodiment will be described with reference to FIG. FIG. 55 is a block diagram illustrating a hardware configuration of electronic device 100.

  The hardware configuration of the electronic device 100 shown in FIG. 55 is obtained by adding a mouse connector 197 to the configuration shown in FIG. 2 in the first embodiment. A mouse 1100 can be attached to and detached from the mouse connector 197. The mouse connector 197 detects the connection of the mouse 1100 and the removal of the mouse 1100, and sends a signal indicating the connection state of the mouse 1100 to the CPU 110 or the like.

  As the mouse connector 197, a connector such as a USB connector that physically connects the terminals on the mouse side can be used. However, the mouse connector 197 is an example of an interface between the electronic device 100 and an external pointing device, and is not limited to the above. For example, the electronic device 100 may include an interface for connecting to a pointing device using wireless or the like.

  Also, the hardware configuration of the electronic device 100 is not limited to that shown in FIG. For example, as in the first embodiment, a mouse unit 197 added to the first unit 1001A may be used instead of the first unit 1001.

  A change in cursor display control according to the third embodiment will be described with reference to FIG. FIG. 56 is a diagram for explaining a change in cursor display control according to the third embodiment.

  FIG. 56A shows a display screen 5610 of the liquid crystal panel 140 and a display screen 5620 of the liquid crystal panel 240 in the mouse mode. Display screen 5610 includes a cursor 5612.

  Similar to the first embodiment and the second embodiment, electronic device 100 changes the position of cursor 5612 in accordance with the change of the input position to liquid crystal panel 240. Further, in the present embodiment, electronic device 100 moves cursor 5612 in accordance with the movement of mouse 1100. In FIG. 56A and the subsequent drawings, this is indicated by an arrow from the mouse 1100 to the liquid crystal panel 140.

  FIG. 56B is a diagram showing the display screen 5630 of the liquid crystal panel 140 and the display screen 5640 of the liquid crystal panel 240 in the tablet mode. Here, as in FIG. 38B and the like, a case where the electronic device 100 is executing a book viewer is shown as an example. Display screen 5630 includes a cursor 5632 and a window 5634. Window 5634 is created by the executed book viewer application.

  The electronic device 100 according to the third embodiment does not change the cursor display when the mouse 1100 is connected to the electronic device 100 when the mouse mode is changed to the tablet mode. That is, electronic device 100 displays the same cursor before and after mode switching. More specifically, electronic device 100 displays, in tablet mode, cursor 5632 having the same display form as cursor 5612 at the same position as cursor 5612 in mouse mode before the mode is switched.

  In the present embodiment, electronic device 100 can move cursor 5632 in accordance with the movement of mouse 1100 even during the tablet mode. FIG. 56C shows a state where the user has moved the cursor 5632 to the right. Thus, the user can move the cursor even in the tablet mode.

  Therefore, electronic device 100 according to the present embodiment does not move the cursor when shifting to the tablet mode. This is because if the cursor that the user can move becomes inconspicuous or moves without user operation, the operability of the cursor by the user is impaired. Even if the cursor overlaps the window, the user can move the cursor to a position where the window is easy to see by moving the mouse 1100, so that visibility is not hindered.

  The electronic device 100 does not change the display of the mouse even when shifting from the tablet mode to the mouse mode. FIG. 56C shows a display screen 5650 of the liquid crystal panel 140 and a display screen 5660 of the liquid crystal panel 240 when the state is changed from the state shown in FIG. 56B (tablet mode) to the mouse mode. The display screen 5650 includes a cursor 5562 and a window 5654 similarly to the display screen 5630 shown in FIG.

  The basic flow of processing of electronic device 100 according to the third embodiment is the same as that of the first embodiment (FIG. 40) and will not be repeated. However, since there is also an operation by the mouse 1100, the mouse mode operation and the tablet mode operation are different from the first embodiment and the second embodiment. Also, the mode switching process is different from the first embodiment and the second embodiment, as described below. Hereinafter, a flow of processing different from that of the other embodiments will be described.

(Mouse mode operation)
The mouse mode operation according to the third embodiment will be described in detail with reference to FIG. FIG. 57 is a flowchart illustrating the processing flow of the mouse mode operation according to the third embodiment.

  First, the operation of the control unit 350 on the first unit 1001 side will be described. The operation flow of the controller 350 is shown on the left side in FIG.

  In step S <b> 1601, the control unit 350 determines whether the mouse 1100 is connected to the mouse connector 197.

  When control unit 350 determines that mouse 1100 is not connected (NO in step S1601), control unit 350 proceeds from step S1603 to step S1609. Since the processing from step S1603 to step S1609 is the same as the processing from step S201 to step S207 in FIG. 41, respectively, description thereof will not be repeated.

  If controller 350 determines that mouse 1100 is connected (YES in step S1601), it calculates position coordinates (mouse coordinates) designated by mouse 1100 in step S1611. Specifically, control unit 350 calculates mouse coordinates based on the amount of movement of mouse 1100 determined from the movement of the mouse ball and the like, the definition of the amount of change in coordinates corresponding to the amount of movement, and the like.

  In step S1613, control unit 350 acquires panel coordinates. This is the same as the processing in step S1603, and the description thereof will not be repeated.

  In step S1615, control unit 350 determines the cursor position based on the mouse coordinates and panel coordinates.

  In step S1617, control unit 350 determines whether mouse 1100 has been clicked or not. Specifically, control unit 350 determines whether a signal corresponding to a mouse 1100 click has been received from mouse connector 197.

  If mouse 1100 has been clicked (YES in step S1617), control unit 350 executes an application operation based on the command corresponding to mouse 1100 click in step S1621. Note that the command is not limited to a click operation, and may correspond to a double click, a drag, or the like.

  On the other hand, when mouse 1100 has not been clicked (NO in step S1617), control unit 350 receives a command from second unit 1002 in step S1619. Thereafter, in step S1621, control unit 350 executes an application operation corresponding to the command. The operation in step S1621 at this time is similar to that in step S1609, and the description thereof will not be repeated.

  After step S1621, control unit 350 repeats the processing from step S1601.

  The operation of the control unit 450 on the second unit 1002 side is shown on the right side in FIG. The processing from step S1701 to step S1711 is the same as the processing from step S301 to step S311 in FIG. 41, and detailed description thereof will not be repeated.

(Tablet mode operation)
The tablet mode operation according to the third embodiment will be described with reference to FIG. FIG. 58 is a flowchart illustrating the processing flow of the tablet mode operation according to the third embodiment.

  First, the operation of the control unit 350 on the first unit 1001 side will be described. The flow of the operation of the control unit 350 is shown on the left side in FIG.

  In step S <b> 1701, the control unit 350 determines whether the mouse 1100 is connected to the mouse connector 197.

  When control unit 350 determines that mouse 1100 is not connected (NO in step S1701), control unit 350 proceeds to the processing in steps S1803 and S1805. Since the processes in steps S1803 and S1805 are the same as the processes in steps S401 and S403 in FIG. 42, respectively, the description thereof will not be repeated.

  When control unit 350 determines that mouse 1100 is connected (YES in step S1701), control unit 350 proceeds from step S1807 to step S1815. The processing from step S1807 to step S1815 is the same as the processing from step S1611 and step S1615 to step S1621, respectively, in FIG. 57, and thus description thereof will not be repeated.

  The operation of the control unit 450 on the second unit 1002 side is shown on the right side in FIG. The processing from step S1901 to step S1909 is the same as the processing from step S501 to step S509 in FIG. 42, and detailed description thereof will not be repeated.

  Mode switching (from the mouse mode to the tablet mode) according to the third embodiment will be described in detail with reference to FIG. FIG. 59 is a flowchart showing the process flow of the mode switching (mouse mode to tablet mode) operation according to the third embodiment.

  The processing from step S2001 to step S2005 is similar to the processing from step S1401 to step S1405 in FIG. 52, and description thereof will not be repeated.

  In step S2007, the control unit 350 determines whether the mouse 1100 is connected to the mouse connector 197.

  When control unit 350 determines that mouse 1100 is not connected (NO in step S2007), control unit 350 proceeds to the process of step S2009 (cursor display change). The process in step S2009 is similar to the process in step S1409 of FIG. 52, and the detailed description thereof will not be repeated.

  On the other hand, when control unit 350 determines that mouse 1100 is connected (YES in step S2007), it does not perform cursor display change processing (without executing step S2009) and switches from mouse mode to tablet mode. The mode switching process is terminated.

(Mode switching: tablet mode to mouse mode)
FIG. 60 shows mode switching (tablet mode to mouse mode) according to the third embodiment. FIG. 60 is a flowchart showing a process flow of the mode switching (tablet mode to mouse mode) operation according to the third embodiment. Each process (steps S2101 to S2115) in FIG. 60 is the same as steps S1501 to S1507 in FIG. 53 described in the second embodiment, and detailed description thereof will not be repeated.

[Others]
In the embodiment of the present invention, the liquid crystal panel 140 may be a display (single function display) that does not have a function as an input unit and that only displays a screen. This configuration is useful when the liquid crystal panel 140 is large and difficult to form a touch panel.

  Further, it is a matter of course that the present invention also includes a configuration in which the embodiments are appropriately combined. For example, an electronic device in which the second embodiment and the third embodiment are combined can be considered as one embodiment of the present invention. That is, an electronic device that changes the display of the cursor when an external pointing device is not connected and the cursor overlaps the window in the tablet mode is also an embodiment of the present invention.

  The first unit 1001 and the second unit 1002 operate independently except for data exchange. Therefore, the second unit 1002 may be removable. Furthermore, the second unit 1002 may be replaceable with another unit (for example, a portable information terminal) having a function equivalent to that of the second unit 1002. Therefore, a system including an electronic device including the first unit 1001 and a unit connected to or connectable to the electronic device can be considered as one embodiment of the present invention.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  DESCRIPTION OF SYMBOLS 100 Electronic device, 100A housing | casing, 100B housing | casing, 100C Hinge, 101 Main body apparatus, 102 Display apparatus, 102A Display apparatus, 103 Display apparatus, 104 Main body apparatus, 130 Driver, 131 Scan signal line drive circuit, 132 Data signal line drive Circuit, 133 photosensor driving circuit, 134 switch, 135 amplifier, 140 photosensor built-in liquid crystal panel, 140A photosensor built-in liquid crystal panel, 141 pixel circuit, 141b subpixel circuit, 141g subpixel circuit, 141r subpixel circuit, 143 electrode pair , 143a Pixel electrode, 143b Counter electrode, 144 Photo sensor circuit, 145 photodiode, 145b photodiode, 145g photodiode, 145r photodiode, 146 capacitor, 151A AC Live matrix substrate, 151B Counter substrate, 152 Liquid crystal layer, 153b Color filter, 153g Color filter, 153r Color filter, 157 Data signal line, 161 Polarization filter, 162 Glass substrate, 163 Shading film, 164 Alignment film, 170 Hard disk, 173 Memory card Reader / writer, 174 External communication unit, 175 microphone, 176 speaker, 177 operation key, 179 backlight, 180 image processing engine, 181 driver control unit, 182 timer, 183 signal processing unit, 191 power switch, 192 power supply circuit, 193 power supply Detection unit, 194 connector, 195 antenna, 196 connector, 197 mouse connector, 230 driver, 240 light sensor built-in liquid crystal panel, 241 left click key, 242 Center key, 243 Right click key, 273 Timer, 274 External communication unit, 279 Backlight, 280 Image processing engine, 281 Driver control unit, 282 Timer, 283 Signal processing unit, 293 Power supply detection unit, 294 connector, 295 antenna, 297 Signal intensity detection unit, 310 display unit, 320 input unit, 330 storage unit, 333 display data, 334 program, 335 operation parameter, 340 interface unit, 350 control unit, 352 input processing unit, 356 display control unit, 358 program execution unit , 410 display unit, 420 input unit, 422 panel input unit, 430 storage unit, 431 input data, 432 input history, 432a character data, 432b illustration data, 433 display data, 434 program, 435 Operation parameters, 436 time data, 437 mode data, 440 interface unit, 450 control unit, 452 input processing unit, 453 panel input processing unit, 454 mode setting unit, 456 display control unit, 458 program execution unit, 458 time counter, 500 Operation screen, 510 cursor, 600 mouse screen, 610 guidance display, 612 left guidance display, 614 center guidance display, 616 right guidance display, 700 operation screen, 800 screen, 800a home menu screen, 800b screen, 800c screen, 800d window, 810 Guidance display, 812 Left guidance display, 814 Center guidance display, 816 Right guidance display, 820 Operation button display, 830 Mouse disabled display, 840 Ho Button, 900 fingers, 950 stylus, 1001 first unit, 1001A unit, 1002 second unit, 1100 mouse, 1300 electronic device, 1731 a memory card.

Claims (17)

A display for displaying the first screen;
A display-integrated tablet that displays a second screen and accepts external input;
Control means for controlling the operation of the display and the tablet,
The control means includes
Mode switching means for switching between the first operation mode and the second operation mode of the control means;
Execution means for executing the program,
The execution means includes
In the first operation mode, the program is executed in response to the input to the tablet, an image created by the executed program is displayed on the display, and the position of the input to the tablet is changed. Change the image displayed on the display accordingly,
In the second operation mode, the program is executed in response to the input to the tablet, an image created by the executed program is displayed on the tablet, and the position of the input to the tablet is changed. Controls the display of the display independently ,
In the first operation mode, executing the program in response to the input to the tablet and displaying an image created by the executed program on the display is at a position of the input to the tablet. Displaying a cursor at a position in the display in response,
In the second operation mode, executing the program in response to the input to the tablet, and displaying an image created by the executed program on the tablet is a position of the input to the tablet. Includes independently controlling the position of the cursor;
The execution device changes the display of the cursor according to switching from the first operation mode to the second operation mode . The execution means, in said second operation mode, the cursor is displayed in a different display form than the first operation mode, the electronic device according to claim 1. The electronic device according to claim 2 , wherein the execution unit displays the cursor thinner than the cursor in the first operation mode in the second operation mode. The execution means, in said second operation mode, to stop the display of the cursor to the display, the electronic device according to claim 1. The execution means, in said second operation mode, when the cursor is determined to overlap the active window within the display changes the display of the cursor, any one of claims 1 to 4 1 The electronic device as described in the paragraph. The execution means, from said first mode of operation in response to switching to the second mode of operation, moving the cursor to a predetermined display position, the electronic device according to claim 1. The electronic device according to claim 6 , wherein the execution unit displays the cursor on an end portion of the display in the second operation mode. The electronic device according to claim 6 , wherein in the second operation mode, the execution unit moves the cursor to the predetermined display position when determining that the cursor overlaps an active window in the display. machine. The electronic device according to claim 8 , wherein the execution unit moves the cursor to an area outside the window when determining that the cursor overlaps an active window in the display in the second operation mode. machine. The execution means, in said second operation mode, when the cursor is determined to overlap the active window in the display, and displays the cursor at the end of the display, according to claim 8 or claim 9 The electronic device as described in. A storage device;
The execution means stores the position of the cursor in the storage device in the first operation mode, and stores it in the storage device in response to switching from the second operation mode to the first operation mode. The electronic device according to any one of claims 6 to 10 , wherein the cursor is displayed at a position of the cursor. It further includes an interface that can connect an external pointing device,
The execution means displays the same cursor before and after switching from the first operation mode to the second operation mode when it is determined that the pointing device is connected to the interface. the electronic device according to any one of 1 to claim 11. The program includes a first program and a second program,
The execution means executes the first program in response to an input to the tablet in the first operation mode, and executes the second program in response to an input to the tablet in the second operation mode. The electronic device according to claim 1, wherein the electronic device executes a program. The execution means includes
First execution means;
Second execution means,
The first execution means executes the first program in response to an input to the tablet in the first operation mode,
The electronic device according to claim 13 , wherein the second execution unit executes the second program in response to an input to the tablet in the second operation mode. An information processing system comprising a first information processing unit and a second information processing unit,
The first information processing unit includes:
A display for displaying the first screen;
A first interface unit for exchanging data with the second information processing unit;
First control means for controlling the display and the first interface unit,
The first control means includes
First execution means for executing a first program and displaying an image created by the executed first program on the display;
The second information processing unit is
A display-integrated tablet that displays a second screen and accepts external input;
A second interface unit for exchanging data with the first information processing unit;
A second control means for controlling the tablet and the second interface unit;
The second control means includes
Mode switching means for switching between the first operation mode and the second operation mode of the second control means;
Second execution means for executing a second program,
The second execution means includes
In the first operation mode, a first command for changing the display of the first program according to a change in the position of the input to the tablet is created, and the second interface unit is controlled to control the second interface unit. Sending a first command to the first information processing unit;
In the second operation mode, the second program is executed in response to the input to the tablet, an image created by the executed second program is displayed on the tablet, and the tablet Creating a second command of the first program independently of a change in input position, controlling the second interface unit to send the second command to the first information processing unit ;
In the first operation mode, executing the first program in response to the input to the tablet and displaying an image created by the executed first program on the display Displaying a cursor at a position in the display according to the position of the input,
In the second operation mode, executing the second program in response to the input to the tablet and displaying an image created by the executed second program on the tablet Controlling the position of the cursor independently of the position of the input,
The information processing system , wherein the second execution unit changes the display of the cursor in response to switching from the first operation mode to the second operation mode . An electronic apparatus control method comprising: a display that displays a first screen; a display-integrated tablet that displays a second screen and that can accept external input; and an execution unit that executes a program. There,
Switching between a first operation mode and a second operation mode of the electronic device;
Executing the program in response to the input to the tablet in the first operation mode, and displaying an image created by the executed program on the display.
Displaying the image on the display in the first operation mode includes changing the image displayed on the display in response to a change in a position of the input to the tablet;
The step of changing the image displayed on the display in response to the change in the position of the input to the tablet includes displaying a cursor at a position in the display in accordance with the position of the input to the tablet. ,
In the second operation mode, the program is executed in response to the input to the tablet, an image created by the executed program is displayed on the tablet, and the position of the input to the tablet is changed. Further comprising independently controlling the display of the display ;
The step of switching between the first operation mode and the second operation mode of the electronic device includes changing the display of the cursor in response to switching from the first operation mode to the second operation mode. A method for controlling an electronic device. An electronic device control program comprising: a display that displays a first screen; a display-integrated tablet that displays a second screen and that can accept external input; and an execution unit that executes the program. There,
Switching between a first operation mode and a second operation mode of the electronic device;
Executing the program in response to the input to the tablet in the first operation mode, and displaying an image created by the executed program on the display.
Displaying the image on the display in the first operation mode includes changing the image displayed on the display in response to a change in a position of the input to the tablet;
The step of changing the image displayed on the display in response to the change in the position of the input to the tablet includes displaying a cursor at a position in the display in accordance with the position of the input to the tablet. ,
In the second operation mode, the program is executed in response to the input to the tablet, an image created by the executed program is displayed on the tablet, and the position of the input to the tablet is changed. Further comprising independently controlling the display of the display ;
The step of switching between the first operation mode and the second operation mode of the electronic device includes changing the display of the cursor in response to switching from the first operation mode to the second operation mode. Including electronic device control programs.

Images