JP2010211393A - Electronic equipment, method and program for controlling the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve operability of electronic equipment equipped with two display devices. <P>SOLUTION: The electronic equipment can operate in two operation modes of a "mouse mode" and "tablet mode". In the mouse mode, an operation screen of a program created by executing a program is displayed on a main screen according to input to a liquid crystal panel 240 which includes a display integrated tablet. In the tablet mode, the operation screen of the program generated by executing the program is displayed on the liquid crystal panel 240 according to the input to the liquid crystal panel 240. In a sub application so called a "book", history information of an electronic book selected so as to be browsed by a book viewer by then is stored, and a softkey of each electronic book is displayed in the liquid crystal panel 240 in the order according to the history information concerned. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an electronic device, a control method for the electronic device, and a control program for the electronic device. In particular, the present invention relates to an electronic apparatus having two display devices, a control method thereof, and a control program.

  At present, electronic devices such as personal computers and portable information terminals are widely used. Recently, electronic devices having two display devices are becoming widespread.

  For example, a portable information terminal disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2000-172395) has two screens, and displays content on one of the two screens. The portable information terminal also displays a content menu bar and slide bar on the other screen.

  Japanese Patent Application Laid-Open No. 2001-306291 discloses an information processing apparatus including a main display device and an auxiliary display device. This information processing apparatus normally displays both content and content additional information on the main display device. The information processing apparatus displays additional information on the sub-screen when displaying the content on the full screen.

  An electronic device disclosed in Patent Document 3 (Japanese Patent Laid-Open No. 2003-202948) has a main display unit and an auxiliary display unit, and displays a connection status of communication of the electronic device on the auxiliary display unit.

  The electronic device disclosed in Patent Document 4 (Japanese Patent Application Laid-Open No. 2003-216297) also includes a main display unit and an auxiliary display unit, similar to the electronic device described in Patent Document 3. The electronic device displays status information of the electronic device on the auxiliary display unit by a combination of a message by characters, a symbol, and a display color and / or blinking display of the auxiliary display unit.

  An information processing apparatus disclosed in Patent Document 5 (Japanese Patent Application Laid-Open No. 2004-5105) includes a main display, a main display controller, a display-integrated pointing device, and a controller dedicated to the pointing device. This information processing apparatus can perform system settings such as BIOS (Basic Input / Output System) by a pointing device before starting an OS (Operating System).

  Similarly to the information processing apparatus described in Patent Document 6 (Japanese Patent Application Laid-Open No. 2004-5212), a main display, a main display controller, a display-integrated pointing device, and a controller dedicated to the pointing device An information processing apparatus including the above is disclosed. The information processing apparatus disclosed in Patent Document 6 displays the display screen of the pointing device on the main display when the display screen of the pointing device is switched to prevent erroneous operation.

JP 2000-172395 A JP 2001-306291 A JP 2003-202948 A JP 2003-216297 A JP 2004-5105 A JP 2004-5212 A

  In the electronic device as described above, the user can use the sub-display in the same manner as a touch pad provided in many conventional notebook personal computers. That is, in such an electronic device, the input to the sub display is processed as information for moving the position of the cursor displayed on the main display or giving an instruction to the application by clicking or dragging.

  By the way, in recent years, in portable information terminals, it has become possible to operate an application by displaying an operation screen of the application on a sub display and performing input on the sub display. That is, the input to the sub display is processed as the input to the operation screen of the application displayed on the sub display.

  In the conventional portable information terminal, when the processing method of the input to the sub display is switched between the two processing methods, detailed consideration is not given as to how the electronic device should be controlled. .

  Therefore, in an electronic device that simply uses a conventional technique, there is a possibility that the operability may be reduced when the processing method is switched.

  The present invention has been conceived in view of such circumstances, and an object thereof is to improve the operability of an electronic apparatus including two display devices.

  An electronic apparatus according to the present invention includes a first display unit, a second display unit, a storage unit, and a control unit that controls display modes of the first and second display devices. The display means is a display-integrated tablet capable of accepting input from the outside, and the control means is a first mode for causing the first display means to display a screen created by processing executed in response to input to the tablet. And a second mode in which a screen created by processing executed in response to an input to the tablet is displayed on the second display means, and a mode in which the operation can be performed from the second mode to the first mode. In the case of switching the mode, the operation information that is the information for specifying the content of the operation in the second mode is stored in the storage means, and in the case of switching the mode operating from the first mode to the second mode The information corresponding to the stored operation information in the storage means is displayed on the second display means.

  In the electronic device of the present invention, when the control unit switches the operation mode from the first mode to the second mode, the control unit arranges the softkey array as information corresponding to the operation information stored in the storage unit. The screen including it is displayed on the second display means.

  In the electronic device of the present invention, the content can be selected in the second mode, and the storage unit stores, as operation information, information related to the history of selection of the content in the second mode, and a soft key Is a soft key for selecting content arranged in correspondence with information on the history of selection of the content stored in the storage means.

  In the electronic device of the present invention, the storage unit stores information specifying content that can be selected in the second mode in an order according to the name, and the control unit selects a soft key and the content is selected. The input of information for selecting whether to arrange in accordance with the information related to the history or in order according to the name is accepted.

  An electronic device control method according to the present invention includes: a first display unit; a second display unit including a display-integrated tablet capable of accepting an external input; an operation unit; and a first and a second unit. A control method for an electronic device comprising a control means for controlling the display mode of the display device in a first mode in which a screen created by processing executed in response to an input to the tablet is displayed on the first display means. A step that operates, a step that operates in a second mode in which a screen created by processing executed in response to an input to the tablet is displayed on the second display means, and a mode that operates from the second mode to the first mode. Storing the operation information, which is information for specifying the content of the operation in the second mode, in the storage means when switching between the first mode and the second mode. When switching the mode to create, and a step of displaying information corresponding to the stored operation information in the storage means to the second display means.

  An electronic device control program according to the present invention includes: a first display unit; a second display unit including a display-integrated tablet capable of accepting an external input; an operation unit; A computer-readable control program that is executed in an electronic device that includes a control unit that controls a display mode of the display device of the display device, wherein a screen created by processing executed in response to an input to the tablet is displayed on the electronic device. Operating in a first mode for displaying on one display means, operating in a second mode for displaying on a second display means a screen created by processing executed in response to an input to the tablet, When switching the mode that operates from the second mode to the first mode, the operation information is information that specifies the content of the operation in the second mode. Storing the information in the storage means, and displaying the information corresponding to the operation information stored in the storage means on the second display means when switching from the first mode to the second mode. Is executed.

  According to the present invention, in an electronic device that is a display-integrated tablet that includes first and second display means, and in which the second display means that is one of them can accept input from the outside, The electronic device has two operation modes, that is, a first mode in which a screen created by processing executed in response to input to the tablet is displayed on the first display means, and processing executed in response to input to the tablet It is possible to operate in the second mode in which the screen created in (2) is displayed on the second display means. And in the said electronic device, when switching the mode which operate | moves from 2nd mode to 1st mode, the operation | movement information which is the information which specifies the content of the operation | movement in 2nd mode is memorize | stored in a memory | storage means, When the operation mode is switched from the first mode to the second mode, information corresponding to the operation information stored in the storage unit is displayed on the second display unit.

  As a result, the user can use an electronic apparatus including two display devices (first and second display means) in both the first mode and the second mode, and in these modes. When the operation mode is changed from one mode to the other mode, the display based on the information specifying the content of the operation in the other mode can be seen on the second display means.

  Therefore, in the electronic device, when the operation mode is switched between a plurality of modes in which the usage modes of the two display devices are different, the mode after the switching has been executed in the mode after the switching so far. The content of the operation can be reflected on the second display means, and the operability of the electronic device including the two display devices can be improved.

It is the schematic which shows the external appearance of an electronic device. It is a block diagram showing the hardware constitutions of an electronic device. It is the figure which showed the structure of the liquid crystal panel, and the peripheral circuit of the said liquid crystal panel. It is sectional drawing of a liquid crystal panel and a backlight. It is the figure which showed the timing chart at the time of operating an optical sensor circuit. It is sectional drawing which showed the structure which a photodiode receives the light from a backlight in the case of a scan. It is the figure which showed schematic structure of the command. It is a figure for demonstrating the command of classification "000". FIG. 10 is a diagram for explaining a command of type “001”. It is a figure for demonstrating the command of classification "010". FIG. 10 is a diagram for explaining a command of type “011”. It is a figure for demonstrating the command of classification "100". FIG. 10 is a diagram for explaining a command of type “101”. It is the figure which showed schematic structure of the response data. It is the figure which showed the image (scanned image) obtained by scanning a finger | toe. FIG. 4 is a circuit diagram of an optical sensor built-in liquid crystal panel different from that shown in FIG. 3. It is sectional drawing which showed the structure which a photodiode receives external light in the case of a scan. It is a block diagram showing the hardware constitutions of the modification of an electronic device. It is the schematic which shows the external appearance of the 3rd modification of an electronic device. It is a block diagram showing the hardware constitutions of the electronic device of FIG. It is a figure which shows the functional structure of an electronic device in a block diagram format. It is a figure for demonstrating the screen which an electronic device displays in each of mouse mode and tablet mode. It is a figure which shows the specific example of a mouse | mouth screen. It is a figure which shows the specific example of a home menu screen. It is a transition diagram of the screen displayed on a liquid crystal panel in tablet mode. It is a figure for demonstrating operation | movement of the electronic device at the time of switching to mouse mode and tablet mode. It is the figure which showed typically the operation mode at the time of normal starting. It is the figure which showed typically the operation mode at the time of a return. It is a figure for demonstrating operation | movement of the electronic device at the time of tablet mode. It is a figure which shows an example of a character input screen. It is a 1st figure for demonstrating operation | movement of the electronic device at the time of utilization of a handwritten character input pad. It is a 2nd figure for demonstrating operation | movement of the electronic device at the time of use of a handwritten character input pad. It is a figure for demonstrating operation | movement of the electronic device when a text box is full. It is a figure which shows an example of an illustration input screen. It is a figure which shows an example of a calculator screen. It is a figure which shows an example of an internet screen. It is a figure which shows an example of the internet screen containing a scroll bar. It is a figure which shows an example of a dictionary selection screen. It is a figure which shows an example of the dictionary selection screen containing a scroll bar. It is a figure which shows the flow of the process which an electronic device performs in a flowchart format. It is a figure which shows the flow of a process of mouse | mouth mode operation | movement in a flowchart format. It is a figure which shows the flow of a process of tablet mode operation | movement in the flowchart format. It is a figure which shows the flow of a process of mode switching (mouse mode to tablet mode) operation | movement in a flowchart format. It is a figure which shows the flow of the 1st process of mode switching (tablet mode to mouse mode) operation | movement in a flowchart format. It is a figure for demonstrating the state in execution of sub application "book" of an electronic device. It is a figure which shows the modification of the block diagram of FIG. It is a figure for demonstrating the state in execution of sub application "book" of an electronic device.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<Appearance of electronic equipment>
FIG. 1 is a diagram illustrating an appearance of electronic device 100 according to the present embodiment. Referring to FIG. 1, electronic device 100 includes a first housing 100A and a second housing 100B.

  The first casing 100A and the second casing 100B are foldably connected by a hinge 100C. The first housing 100A includes a liquid crystal panel 140 with a built-in optical sensor. The second housing 100B includes an optical sensor built-in liquid crystal panel 240. As described above, the electronic device 100 includes two optical sensor built-in liquid crystal panels.

  The electronic device 100 is configured as a portable device having a display function such as a PDA (Personal Digital Assistant), a notebook personal computer, a portable telephone, and an electronic dictionary.

<About hardware configuration>
Next, an aspect of a specific configuration of the electronic device 100 will be described with reference to FIG. FIG. 2 is a block diagram illustrating a hardware configuration of electronic device 100.

  Electronic device 100 includes a first unit 1001 and a second unit 1002. The second unit 1002 is detachably connected to the first unit 1001 from the electronic device 100. The first unit 1001 includes a main body device 101 and a display device 102. The second unit 1002 includes a display device 103 and a main device 104.

  The first housing 100A includes a display device 102. Second housing 100B includes main device 101. The second housing 100B includes a second unit 1002.

(About the first unit)
The main unit 101 includes a CPU (Central Processing Unit) 110, a RAM (Random Access Memory) 171, a ROM (Read-Only Memory) 172, a memory card reader / writer 173, an external communication unit 174, a microphone 175, A speaker 176, an operation key 177, a power switch 191, a power circuit 192, a power detection unit 193, a USB (Universal Serial Bus) connector 194, an antenna 195, and a LAN (Local Area Network) connector 196 are included. . Each component (110, 171 to 177, 193) is mutually connected by a data bus DB1. A memory card 1731 is attached to the memory card reader / writer 173.

  CPU 110 executes a program. The operation key 177 receives an instruction input from the user of the electronic device 100. The RAM 171 stores data generated by the execution of the program by the CPU 110 or data input via the operation keys 177 in a volatile manner. The ROM 172 stores data in a nonvolatile manner. The ROM 172 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory.

  The external communication unit 174 communicates with other electronic devices. Specifically, the external communication unit 174 communicates with, for example, the second unit 1002 via the USB connector 194. The external communication unit 174 performs wireless communication with the second unit 1002 via the antenna 195, for example. Further, the external communication unit 174 performs wired communication with other electronic devices via the LAN connector 196.

  Note that the main device 101 may communicate with other electronic devices by wireless communication other than Bluetooth (registered trademark). For example, the external communication unit 174 may perform wireless communication with other electronic devices connected to the LAN via a wireless LAN antenna (not shown). Alternatively, wireless communication may be performed with another electronic device via an infrared port (not shown).

The power switch 191 is a switch for starting up the electronic device 100.
When the power switch 191 is turned on, the power supply circuit 192 supplies power to each component connected to the data bus DB1 and the display device 102 via the power detection unit 193. When the power switch 191 is turned on, the power circuit 192 supplies power to the external communication unit 174 without going through the power detection unit 193.

  The power supply detection unit 193 detects the output from the power supply circuit 192. In addition, the power supply detection unit 193 sends information (for example, a voltage value and a current value) regarding the detected output to the CPU 110.

  The USB connector 194 is used to connect the first unit 1001 to the second unit 1002. Note that the main device 101 may include other USB connectors in addition to the USB connector 194.

  The first unit 1001 transmits data to the second unit 1002 via the USB connector 194. The first unit 1001 receives data from the second unit 1002 via the USB connector 194. Further, the first unit 1001 supplies power to the second unit 1002 via the USB connector 194.

  The antenna 195 is used for communication according to the Bluetooth (registered trademark) standard between the first unit 1001 and another communication apparatus (for example, the second unit 1002). The LAN connector 196 is used to connect the electronic device 100 to the LAN.

  The display device 102 includes a driver 130, an optical sensor built-in liquid crystal panel 140 (hereinafter referred to as a liquid crystal panel 140), an internal IF 178, a backlight 179, and an image processing engine 180.

  The driver 130 is a drive circuit for driving the liquid crystal panel 140 and the backlight 179. Various drive circuits included in the driver 130 will be described later.

  The liquid crystal panel 140 is a device having a liquid crystal display function and a photosensor function. That is, the liquid crystal panel 140 can perform image display using liquid crystal and sensing using an optical sensor. Details of the liquid crystal panel 140 will be described later.

  An internal IF (Interface) 178 mediates exchange of data between the main device 101 and the display device 102.

  The backlight 179 is a light source disposed on the back surface of the liquid crystal panel 140. The backlight 179 irradiates the back surface with uniform light.

  The image processing engine 180 controls the operation of the liquid crystal panel 140 via the driver 130. Here, the control is performed based on various data sent from the main apparatus 101 via the internal IF 178. Note that the various data includes commands to be described later. Further, the image processing engine 180 processes data output from the liquid crystal panel 140 and sends the processed data to the main apparatus 101 via the internal IF 178. Further, the image processing engine 180 includes a driver control unit 181, a timer 182, and a signal processing unit 183.

  The driver control unit 181 controls the operation of the driver 130 by sending a control signal to the driver 130. In addition, the driver control unit 181 analyzes a command transmitted from the main device 101. Then, the driver control unit 181 sends a control signal based on the analysis result to the driver 130. Details of the operation of the driver 130 will be described later.

The timer 182 generates time information and sends the time information to the signal processing unit 183.
The signal processing unit 183 receives data output from the optical sensor. Here, since the data output from the optical sensor is analog data, the signal processing unit 183 first converts the analog data into digital data. Further, the signal processing unit 183 performs data processing on the digital data in accordance with the content of the command sent from the main device 101. Then, the signal processing unit 183 sends data (hereinafter referred to as response data) including data after the above data processing and time information acquired from the timer 182 to the main unit 101. The signal processing unit 183 includes a RAM (not shown) that can store a plurality of scan data, which will be described later, continuously.

  The command includes a sensing command for instructing sensing by the optical sensor. Details of the sensing command and the response data will be described later (FIGS. 7, 8, and 14).

  Note that the timer 182 is not necessarily provided in the image processing engine 180. For example, the timer 182 may be provided outside the image processing engine 180 in the display device 102. Alternatively, the timer 182 may be provided in the main body device 101. Further, the microphone 175 and the speaker 176 are not always provided in the electronic device 100, and may be configured so as not to include either or both of the microphone 175 and the speaker 176 depending on the embodiment of the electronic device 100.

  Here, the display device 102 includes a system liquid crystal. The system liquid crystal is a device obtained by integrally forming peripheral devices of the liquid crystal panel 140 on the glass substrate of the liquid crystal panel 140. In the present embodiment, the driver 130 (excluding the circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are integrally formed on the glass substrate of the liquid crystal panel 140. Note that the display device 102 is not necessarily configured using a system liquid crystal, and the driver 130 (excluding a circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are included in the glass substrate. Other substrates may be configured.

(About the second unit)
The second unit 1002 receives power supply from the first unit 1001. Specifically, the second unit 1002 is supplied with power from the power supply circuit 192 of the first unit 1001 by connecting a USB connector 294 described later and the USB connector 194 of the first unit 1001.

  The main body device 104 includes a CPU 210, a RAM 271, a ROM 272, an external communication unit 274, a power supply detection unit 293, a USB connector 294, an antenna 295, and a signal strength detection unit 297. Each component (210, 271, 272, 274, 293) is connected to each other by a data bus DB2.

  The CPU 210 executes a program. The RAM 271 stores data generated by the execution of the program by the CPU 210 in a volatile manner. The ROM 272 stores data in a nonvolatile manner. The ROM 272 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory.

  The external communication unit 274 communicates with other electronic devices. Specifically, the external communication unit 274 communicates with, for example, the first unit 1001 via the USB connector 294. The external communication unit 274 communicates with the first unit 1001 through the antenna 295, for example.

  Note that main device 104 may communicate with another electronic device (for example, first unit 1001) by wireless communication other than Bluetooth (registered trademark). For example, the external communication unit 274 may perform wireless communication with other electronic devices via an infrared port (not shown).

  The signal strength detection unit 297 detects the strength of the signal received via the antenna 295. Then, the signal strength detection unit 297 sends the detected strength to the external communication unit 274.

  The USB connector 294 is used to connect the second unit 1002 to the first unit 1001.

  The second unit 1002 transmits data to the first unit 1001 via the USB connector 294. The second unit 1002 receives data from the first unit 1001 via the USB connector 294. Furthermore, the second unit 1002 receives power supply from the first unit 1001 via the USB connector 294 as described above. The second unit 1002 stores the electric power supplied from the first unit 1001 in a battery (not shown).

  The antenna 295 is used for communication between the second unit 1002 and the first unit 1001, for example, in accordance with the Bluetooth (registered trademark) standard.

  The power detection unit 293 detects the power supplied via the USB connector 294. In addition, the power supply detection unit 293 sends information about the detected power to the CPU 210.

Further, the main device 104 may have a function of performing infrared communication.
The display device 103 includes a driver 230, an optical sensor built-in liquid crystal panel 240 (hereinafter referred to as “liquid crystal panel 240”), an internal IF 278, a backlight 279, and an image processing engine 280. The image processing engine 280 includes a driver control unit 281, a timer 282, and a signal processing unit 283.

  The display device 103 has the same configuration as the display device 102. That is, the driver 230, the liquid crystal panel 240, the internal IF 278, the backlight 279, and the image processing engine 280 have the same configuration as the driver 130, the liquid crystal panel 140, the internal IF 178, the backlight 179, and the image processing engine 180 in the display device 102. Have each. The driver control unit 281, the timer 282, and the signal processing unit 283 have the same configurations as the driver control unit 181, the timer 182, and the signal processing unit 183 in the display device 102. Therefore, description of each functional block included in display device 103 will not be repeated.

  By the way, the processing in the electronic device 100 is realized by each hardware and software executed by the CPU 110. Such software may be stored in the ROM 172 in advance. The software may be stored in a memory card 1731 or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by the memory card reader / writer 173 or other reading device, or downloaded via the communication unit 174 or communication IF (not shown), and then temporarily stored in the ROM 172. The The software is read from the ROM 172 by the CPU 110 and stored in the RAM 171 in the form of an executable program. CPU 110 executes the program.

  Each component constituting the main device 101 of the electronic device 100 shown in FIG. 2 is a general one. Therefore, it can be said that the essential part of the present invention is the software stored in the RAM 171, the ROM 172, the memory card 1731 and other storage media, or the software downloadable via the network. Since the hardware operation of main device 101 of electronic device 100 is well known, detailed description will not be repeated.

  The storage medium is not limited to a memory card, but is a CD-ROM, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile). Disc)), IC (Integrated Circuit) cards (excluding memory cards), optical cards, mask ROM, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory), flash ROM, and other semiconductor memories, etc. It may be a medium to be used.

  The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

<Configuration and drive of liquid crystal panel with built-in optical sensor>
Next, the configuration of the liquid crystal panel 140 and the configuration of peripheral circuits of the liquid crystal panel 140 will be described. FIG. 3 is a diagram showing a configuration of the liquid crystal panel 140 and peripheral circuits of the liquid crystal panel 140.

  Referring to FIG. 3, the liquid crystal panel 140 includes a pixel circuit 141, an optical sensor circuit 144, a scanning signal line Gi, a data signal line SRj, a data signal line SGj, a data signal line SBj, and a sensor signal line. SSj, sensor signal line SDj, read signal line RWi, and reset signal line RSi are included. Note that i is a natural number satisfying 1 ≦ i ≦ m, and j is a natural number satisfying 1 ≦ j ≦ n.

  2 includes a scanning signal line driving circuit 131, a data signal line driving circuit 132, an optical sensor driving circuit 133, a switch 134, as peripheral circuits of the liquid crystal panel 140. And an amplifier 135.

  The scanning signal line drive circuit 131 receives the control signal TC1 from the driver control unit 181 shown in FIG. The scanning signal line drive circuit 131 applies a predetermined voltage in order from the scanning signal line G1 to each scanning signal line (G1 to Gm) based on the control signal TC1. More specifically, the scanning signal line driving circuit 131 sequentially selects one scanning signal line from the scanning signal lines (G1 to Gm) per unit time, and a TFT (to be described later) with respect to the selected scanning signal line. A voltage capable of turning on the gate of the thin film transistor 142 is applied (hereinafter referred to as a high level voltage). Note that a low level voltage is applied to a scanning signal line that is not selected without applying a high level voltage.

  The data signal line driving circuit 132 receives image data (DR, DG, DB) from the driver control unit 181 shown in FIG. The data signal line driving circuit 132 applies a voltage corresponding to one row of image data to the 3n data signal lines (SR1 to SRn, SG1 to SGn, SB1 to SBn) for each unit time. Apply sequentially.

  Note that although a driving method called a so-called line-sequential method has been described here, the driving method is not limited to this.

  The pixel circuit 141 is a circuit for setting the luminance (transmittance) of one pixel. Further, m × n pixel circuits 141 are arranged in a matrix. More specifically, m pixel circuits 141 are arranged in the vertical direction in FIG. 3 and n pixel circuits in the horizontal direction.

  The pixel circuit 141 includes an R subpixel circuit 141r, a G subpixel circuit 141g, and a B subpixel circuit 141b. Each of these three circuits (141r, 141g, 141b) includes a TFT 142, a pair of electrode pairs 143 including a pixel electrode and a counter electrode, and a capacitor (not shown).

  In addition, CMOS (Complementary Metal Oxide Semiconductor) capable of forming n-type transistors and p-type transistors can be realized, and the movement speed of carriers (electrons or holes) is several hundreds compared to amorphous silicon thin film transistors (a-Si TFTs). For example, a polycrystalline silicon thin film transistor (p-Si TFT) is used as the TFT 142 in the display device 102 because it is twice as fast. Note that the TFT 142 will be described as an n-channel field effect transistor. However, the TFT 142 may be a p-type channel field effect transistor.

  The source of the TFT 142 in the R subpixel circuit 141r is connected to the data signal line SRj. The gate of the TFT 142 is connected to the scanning signal line Gi. Further, the drain of the TFT 142 is connected to the pixel electrode of the electrode pair 143. A liquid crystal is disposed between the pixel electrode and the counter electrode. The G sub-pixel circuit 141g and the B sub-pixel circuit 141b have the same configuration as the R sub-pixel circuit 141r except that the data signal line to which the source of each TFT 142 is connected is different. Therefore, description of these two circuits (141g, 141b) will not be repeated.

  Here, setting of luminance in the pixel circuit 141 will be described. First, the high level voltage is applied to the scanning signal line Gi. By the application of the high level voltage, the gate of the TFT 142 is turned on. In this manner, with the gate of the TFT 142 turned on, a specified voltage (voltage corresponding to image data for one pixel) is applied to each data signal line (SRj, SGj, SBj). Thereby, a voltage based on the designated voltage is applied to the pixel electrode. As a result, a potential difference is generated between the pixel electrode and the counter electrode. Based on this potential difference, the liquid crystal responds and the luminance of the pixel is set to a predetermined luminance. Note that the potential difference is held by the capacitor (auxiliary capacitor) (not shown) until the scanning signal line Gi is selected in the next frame period.

  The optical sensor drive circuit 133 receives the control signal TC2 from the driver control unit 181 shown in FIG.

  Then, the optical sensor drive circuit 133 sequentially selects one signal line from the reset signal lines (RS1 to RSm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, the voltage VDDR that is higher than usual is applied. Note that a voltage VSSR lower than the voltage applied to the selected reset signal line is kept applied to the unselected reset signal line. For example, the voltage VDDR may be set to 0V and the voltage VSSR may be set to −5V.

  Further, the optical sensor driving circuit 133 sequentially selects one signal line from the readout signal lines (RW1 to RWm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, a voltage VDD higher than usual is applied. Note that the voltage VSSR is applied to the read signal line that is not selected. For example, the value of VDD may be set to 8V.

  The timing for applying the voltage VDDR and the timing for applying the voltage VDD will be described later.

  The optical sensor circuit 144 includes a photodiode 145, a capacitor 146, and a TFT 147. In the following description, it is assumed that the TFT 147 is an n-type channel field effect transistor. However, the TFT 147 may be a p-type channel field effect transistor.

  The anode of the photodiode 145 is connected to the reset signal line RSi. On the other hand, the cathode of the photodiode 145 is connected to one electrode of the capacitor 146. The other electrode of the capacitor 146 is connected to the read signal line RWi. Hereinafter, a connection point between the photodiode 145 and the capacitor 146 is referred to as a node N.

  The gate of the TFT 147 is connected to the node N. The drain of the TFT 147 is connected to the sensor signal line SDj. Further, the source of the TFT 147 is connected to the sensor signal line SSj. Details of sensing using the optical sensor circuit 144 will be described later.

  The switch 134 is a switch provided to switch whether or not to apply a predetermined voltage to the sensor signal lines (SD1 to SDn). The switching operation of the switch 134 is performed by the optical sensor driving circuit 133. Note that the voltage applied to the sensor signal lines (SD1 to SDn) when the switch 134 is turned on will be described later.

  The amplifier 135 amplifies the voltage output from each sensor signal line (SS1 to SSn). The amplified voltage is sent to the signal processing unit 183 shown in FIG.

  The image processing engine 180 controls the timing for displaying an image on the liquid crystal panel 140 using the pixel circuit 141 and the timing for sensing using the optical sensor circuit 144.

  FIG. 4 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179. Referring to FIG. 4, liquid crystal panel 140 includes an active matrix substrate 151A, a counter substrate 151B, and a liquid crystal layer 152. The counter substrate 151B is disposed to face the active matrix substrate 151A. The liquid crystal layer 152 is sandwiched between the active matrix substrate 151A and the counter substrate 151B. The backlight 179 is disposed on the opposite side of the liquid crystal layer 152 with respect to the active matrix substrate 151A.

  The active matrix substrate 151 </ b> A includes a polarizing filter 161, a glass substrate 162, a pixel electrode 143 a that constitutes the electrode pair 143, a photodiode 145, a data signal line 157, and an alignment film 164. Further, although not shown in FIG. 4, the active matrix substrate 151A includes the capacitor 146, the TFT 147, the TFT 142, and the scanning signal line Gi shown in FIG.

  In the active matrix substrate 151A, the polarizing filter 161, the glass substrate 162, the pixel electrode 143a, and the alignment film 164 are arranged in this order from the backlight 179 side. The photodiode 145 and the data signal line 157 are formed on the liquid crystal layer 152 side of the glass substrate 162.

  The counter substrate 151B includes a polarizing filter 161, a glass substrate 162, a light shielding film 163, color filters (153r, 153g, 153b), a counter electrode 143b constituting the electrode pair 143, and an alignment film 164.

  In the counter substrate 151B, the alignment film 164, the counter electrode 143b, the color filters (153r, 153g, 153b), the glass substrate 162, and the polarizing filter 161 are arranged in this order from the liquid crystal layer 152 side. The light shielding film 163 is formed in the same layer as the color filters (153r, 153g, 153b).

  The color filter 153r is a filter that transmits light having a red wavelength. The color filter 153g is a filter that transmits light having a green wavelength. The color filter 153b is a filter that transmits light having a blue wavelength. Here, the photodiode 145 is arranged at a position facing the color filter 153b.

  The liquid crystal panel 140 displays an image by blocking external light or light emitted from a light source such as a backlight 179 or transmitting the light. Specifically, the liquid crystal panel 140 changes the direction of liquid crystal molecules in the liquid crystal layer 152 by applying a voltage between the pixel electrode 143a and the counter electrode 143b, thereby blocking or transmitting the light. However, since the light cannot be completely blocked by the liquid crystal alone, a polarizing filter 161 that transmits only light having a specific polarization direction is provided.

  Note that the position of the photodiode 145 is not limited to the above position, and may be provided at a position facing the color filter 153r or a position facing the color filter 153g.

  Here, the operation of the optical sensor circuit 144 will be described. FIG. 5 is a diagram showing a timing chart when the optical sensor circuit 144 is operated. In FIG. 5, a voltage VINT indicates a potential at the node N in the photosensor circuit 144. A voltage VPIX is an output voltage from the sensor signal line SSj shown in FIG. 3 and is a voltage before being amplified by the amplifier 135.

  The following description will be divided into a reset period for resetting the optical sensor circuit 144, a sensing period for sensing light using the optical sensor circuit 144, and a readout period for reading the sensed result.

  First, the reset period will be described. In the reset period, the voltage applied to the reset signal line RSi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDDR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR). As described above, by applying the high-level voltage to the reset signal line RSi, a current starts to flow in the forward direction (from the anode side to the cathode side) of the photodiode 145. As a result, the voltage VINT which is the potential of the node N has a value represented by the following expression (1). In Equation (1), the forward voltage drop amount in the photodiode 145 is Vf.

VINT = VSSR + | VDDR−VSSR | −Vf (1)
Therefore, the potential of the node N is a value smaller by Vf than the voltage VDDR as shown in FIG.

  Here, since the voltage VINT is not more than the threshold value for turning on the gate of the TFT 147, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change. Further, a difference corresponding to the voltage VINT occurs between the electrodes of the capacitor 146. For this reason, the capacitor 146 accumulates charges corresponding to the difference.

  Next, the sensing period will be described. In the sensing period following the reset period, the voltage applied to the reset signal line RSi instantaneously switches from the high level (voltage VDDR) to the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR).

  Thus, by changing the voltage applied to the reset signal line RSi to the low level, the potential of the node N becomes higher than the voltage of the reset signal line RSi and the voltage of the read signal line RWi. For this reason, in the photodiode 145, the voltage on the cathode side becomes higher than the voltage on the anode side. That is, the photodiode 145 is in a reverse bias state. In such a reverse bias state, when the photodiode 145 receives light from the light source, current starts to flow from the cathode side to the anode side of the photodiode 145. As a result, as shown in FIG. 5, the potential of the node N (that is, the voltage VINT) becomes lower with the passage of time.

  Since the voltage VINT continues to decrease in this way, the gate of the TFT 147 does not turn on. Therefore, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change.

  Next, the reading period will be described. In the readout period following the sensing period, the voltage applied to the reset signal line RSi is kept at the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDD). Here, the voltage VDD is higher than the voltage VDDR.

  Thus, by applying a high level voltage instantaneously to the read signal line RWi, the potential of the node N is raised through the capacitor 146 as shown in FIG. Note that the increase width of the potential of the node N is a value corresponding to the voltage applied to the read signal line RWi. Here, since the potential of the node N (that is, the voltage VINT) is raised to a threshold value that turns on the gate of the TFT 147, the gate of the TFT 147 is turned on.

  At this time, if a constant voltage is applied in advance to the sensor signal line SDj (see FIG. 3) connected to the drain side of the TFT 147, the sensor signal line SSj connected to the source side of the TFT 147 will cause the VPIX in FIG. As shown in the graph, a voltage corresponding to the potential of the node N is output.

  Here, when the amount of light received by the photodiode 145 (hereinafter referred to as the amount of received light) is small, the slope of the straight line shown in the VINT graph of FIG. 5 becomes gentle. As a result, the voltage VPIX is higher than when the amount of received light is large. As described above, the optical sensor circuit 144 changes the value of the voltage output to the sensor signal line SSj in accordance with the amount of light received by the photodiode 145.

  By the way, in the above, the operation | movement was demonstrated paying attention to one optical sensor circuit 144 among the m * n optical sensor circuits which exist. Below, operation | movement of each photosensor circuit in the liquid crystal panel 140 is demonstrated.

  First, the optical sensor drive circuit 133 applies a predetermined voltage to all n sensor signal lines (SD1 to SDn). Next, the photosensor drive circuit 133 applies a voltage VDDR that is higher than normal to the reset signal line RS1. The other reset signal lines (RS2 to RSm) and read signal lines (RW1 to RWm) are kept in a state where a low level voltage is applied. As a result, the n photosensor circuits in the first row in FIG. 3 enter the reset period described above. Thereafter, the n photosensor circuits in the first row enter a sensing period. Further, thereafter, the n photosensor circuits in the first row enter a reading period.

  Note that the timing at which a predetermined voltage is applied to all n sensor signal lines (SD1 to SDn) is not limited to the above timing, and may be any timing that is applied at least before the readout period. .

  When the readout period of the n photosensor circuits in the first row is completed, the photosensor drive circuit 133 applies a voltage VDDR that is higher than usual to the reset signal line RS2. That is, the reset period of the n photosensor circuits in the second row starts. When the reset period ends, the n photosensor circuits in the second row enter a sensing period, and thereafter enter a reading period.

  Thereafter, the processing described above is sequentially performed on the n photosensor circuits in the third row, the n photosensor circuits in the fourth row,..., The n photosensor circuits in the m row. As a result, the sensing result of the first row, the sensing result of the second row,..., The sensing result of the m-th row are output in this order from the sensor signal lines (SS1 to SSn).

  In the display device 102, sensing is performed for each row as described above, and a sensing result is output from the liquid crystal panel 140 for each row. For this reason, hereinafter, the data after the signal processing unit 183 performs the above-described data processing on the data regarding the voltage for m rows from the first row to the m-th row output from the liquid crystal panel 140, This is called “scan data”. That is, scan data refers to image data obtained by scanning a scan target (for example, a user's finger). An image displayed based on the scan data is referred to as a “scanned image”. Further, in the following, sensing is referred to as “scan”.

  Moreover, in the above, although the structure which scans using all the m * n photosensor circuits was mentioned as an example, it is not limited to this. Scanning may be performed on a partial region of the surface of the liquid crystal panel 140 using a photosensor circuit selected in advance.

  In the following, it is assumed that the electronic device 100 can take either of the two configurations. Further, switching between the components is assumed to be performed by a command sent from the main device 101 based on an input via the operation key 177 or the like. Note that when scanning is performed on a partial area on the surface of the liquid crystal panel 140, the image processing engine 180 sets a scan target area. The setting of the area may be configured to be specified by the user via the operation key 177.

  As described above, when scanning is performed on a partial region of the surface of the liquid crystal panel 140, there are the following modes of use for displaying an image. The first is a mode in which an image is displayed in a surface area other than the partial area (hereinafter referred to as a scan area). The second is a mode in which no image is displayed in the surface area other than the scan area. Which mode is used is based on a command sent from the main apparatus 101 to the image processing engine 180.

  FIG. 6 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179, showing a configuration in which the photodiode 145 receives light from the backlight 179 during scanning.

  Referring to FIG. 6, when the user's finger 900 is in contact with the surface of the liquid crystal panel 140, a part of the light emitted from the backlight 179 is part of the user's finger 900 (abbreviated in the contacted area). Reflected on the plane). The photodiode 145 receives the reflected light.

  Even in a region where the finger 900 is not in contact, part of the light emitted from the backlight 179 is reflected by the user's finger 900. Even in this case, the photodiode 145 receives the reflected light. However, since the finger 900 is not in contact with the surface of the liquid crystal panel 140 in the region, the amount of light received by the photodiode 145 is smaller than the region in which the finger 900 is in contact. Note that the photodiode 145 cannot receive most of the light emitted from the backlight 179 that does not reach the user's finger 900.

  Here, by turning on the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the amount of light reflected by the user's finger 900 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter, also referred to as a reflected image) obtained by reflecting light with the finger 900.

Note that examples of the scan object other than the finger 900 include a stylus.
By the way, in the present embodiment, a liquid crystal panel is described as an example of the display device of electronic device 100, but another panel such as an organic EL (Electro-Luminescence) panel is used instead of the liquid crystal panel. Also good.

<About data>
Next, commands exchanged between the first unit 1001 and the second unit 1002 and commands exchanged between the main device 101 and the display device 102 in the first unit 1001 will be described.

  FIG. 7 is a diagram showing a schematic configuration of a command. Referring to FIG. 7, the command includes header DA01, first field DA02, second field DA03, third field DA04, fourth field DA05, fifth field DA06, and spare data area DA07. including.

  FIG. 8 is a diagram for explaining a command of type “000” (that is, a sensing command). The CPU 110 sends a command of type “000” (hereinafter referred to as “first command”) from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 110 sends the first command from the main body device 101 to the display device 102. In the following, the case where the CPU 110 sends the first command from the main unit 101 of the first unit 1001 to the second unit 1002 will be described as an example.

  The CPU 110 writes the command type (“000”), the command transmission destination, and the like in the header DA01. The CPU 110 writes the value of the timing whose number is “1” in the first field DA02. The CPU 110 writes the value of the data type with the number “2” in the second field DA03. The CPU 110 writes the value of the reading method whose number is “3” in the third field DA04. The CPU 110 writes the value of the image gradation number “4” in the fourth field DA05. The CPU 110 writes the value of the resolution with the number “5” in the fifth field DA06.

  The first command in which “00” is set in the first field DA02 requests the image processing engine 280 to transmit scan data at that time. That is, the sensing first command requests transmission of scan data obtained by scanning using the optical sensor circuit of the liquid crystal panel 240 after the image processing engine 280 receives the first command. Further, the first command in which “01” is set in the first field DA02 requests transmission of scan data when the scan result is changed. Further, the first command in which “10” is set in the first field DA02 requests transmission of scan data at regular intervals.

  The first command in which “001” is set in the second field DA03 requests transmission of the coordinate value of the center coordinate in the partial image. In addition, the first command in which “010” is set in the second field DA03 requests transmission of only the partial image whose scan result has changed. Note that the change in the scan result indicates that the previous scan result is different from the current scan result. Furthermore, the first command in which “100” is set in the second field DA03 requests transmission of the entire image.

  Here, the “whole image” is an image generated by the image processing engine 280 based on the voltage output from each optical sensor circuit when scanning using m × n optical sensor circuits. . The “partial image” is a part of the entire image. The reason why the partial image is requested to transmit only the partial image whose scan result has changed will be described later.

  The coordinate value and the partial image or the entire image may be requested at the same time. Further, in the case of a configuration in which scanning is performed on a partial area on the surface of the liquid crystal panel 240, the entire image is an image corresponding to the area to be scanned.

  The first sensing command in which “00” is set in the third field DA04 requests to scan with the backlight 279 turned on. In addition, the first command in which “01” is set in the third field DA04 requests that the backlight 279 be turned off to perform scanning. Note that the configuration for scanning with the backlight 279 off is described later (FIG. 17). Furthermore, the first command in which “10” is set in the third field DA04 requests scanning using both reflection and transmission. Note that the combined use of reflection and transmission refers to scanning a scanning object by switching between a method of scanning with the backlight 279 turned on and a method of scanning with the backlight turned off.

  The first command in which “00” is set in the fourth field DA05 requests monochrome binary image data. Further, the first command in which “01” is set in the fourth field DA05 requests multi-gradation image data. Further, the first command in which “10” is set in the fourth field DA05 requests RGB color image data.

  The first command in which “0” is set in the fifth field DA06 requests image data with high resolution. The first command in which “1” is set in the fifth field DA06 requests image data with a low resolution.

  In addition to the data shown in FIG. 8, the first command includes designation of a region to be scanned (pixel region that drives the optical sensor circuit 144), timing for scanning, lighting timing of the backlight 179, and the like. Is described.

  Note that the image processing engine 280 analyzes the content of the first command, and sends back data (that is, response data) according to the analysis result to the main device 101.

  FIG. 9 is a diagram for explaining a command of type “001” (hereinafter referred to as “second command”). The CPU 110 sends a second command from the main unit 101 of the first unit 1001 to the second unit 1002.

  The CPU 110 writes the command type (“001”), the command transmission destination, and the like in the header DA01. The CPU 110 writes the value of the display request with the number “1” in the first field DA02. CPU 110 writes information on the number / type of number “2” in second field DA03. The CPU 110 writes the value of the display range whose number is “3” in the third field DA04. The CPU 110 writes information related to the image data with the number “4” in the fourth field DA05.

  The second command in which “001” is set in the first field DA02 requests the image processing engine 280 to display an image on the liquid crystal panel 240 (sub-screen). The second command in which “010” is set in the first field DA02 requests the image processing engine 280 to display an icon on the liquid crystal panel 240. Further, the second command in which “011” is set in the first field DA02 requests the image processing engine 280 to display the handwriting area on the liquid crystal panel 240.

  The second field DA03 stores the number of images to be displayed on the liquid crystal panel 240 and a number for designating the type of handwriting language. The image processing engine 280 performs processing according to the number of the images or the language type.

  The second command in which “01” is set in the third field DA04 requests the image processing engine 280 to designate the display range on the liquid crystal panel 240 by coordinates. Further, the second command in which “10” is set in the third field DA04 requests the image processing engine 280 to set the display range on the liquid crystal panel 240 to the entire display area.

  The fourth field DA05 stores image data to be displayed on the liquid crystal panel 240 and position information for displaying the image data. The image processing engine 280 performs processing for displaying the image data at a position specified by the position information.

  FIG. 10 is a diagram for explaining a command of type “010” (hereinafter referred to as “third command”). The CPU 110 sends the third command from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 210 sends a third command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPUs 110 and 210 write the command type (“010”), the command transmission destination, and the like in the header DA01. The CPUs 110 and 210 write the value of the OS (Operating System) processing request with the number “1” in the first field DA02. The CPUs 110 and 210 write the value of the OS information with the number “2” in the second field DA03.

  The third command in which “01” or “10” is set in the first field DA02 is transmitted from the second unit 1002 to the first unit 1001.

  The third command in which “01” is set in the first field DA02 requests the first unit 1001 to transmit information indicating the type of OS of the first unit 1001 (main device). The third command in which “10” is set in the first field DA02 requests the first unit 1001 to start the OS specified by the OS information.

  The third command in which “000”, “001”, or “010” is set in the second field DA03 is transmitted from the second unit 1002 to the first unit 1001.

  The third command in which “000” is set in the second field DA03 does not request the activation of the OS in the first unit 1001. The third command in which “001” is set in the second field DA03 indicates that the second unit 1002 has selected to start the first OS. Furthermore, the third command in which “010” is set in the second field DA03 indicates that the second unit 1002 has selected to start the second OS.

  FIG. 11 is a diagram for explaining a command of type “011” (hereinafter referred to as “fourth command”). The CPU 210 sends a fourth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPU 210 writes the command type (“011”), the command transmission destination, and the like in the header DA01. The CPU 210 writes information related to the activated application with the number “1” in the first field DA02. The CPU 210 writes the startup information with the number “2” in the second field DA03.

  In the first field DA02, information specifying an application to be activated in the first unit 1001 is stored. The second field DA03 stores information used at the time of activation setting and information used after activation.

  FIG. 12 is a diagram for explaining a command of type “100” (hereinafter referred to as “fifth command”). The CPU 210 sends the fifth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPU 210 writes the command type (“100”), the command transmission destination, and the like in the header DA01. The CPU 210 writes information related to the reception request with the number “1” in the first field DA02. The CPU 210 writes information relating to the number “2” in the second field DA03. The CPU 210 writes information related to the file with the number “3” in the third field DA04.

  The fifth command in which “01” is set in the first field DA02 requests the first unit 1001 to receive a file. In the second field DA03, the number of files transmitted from the second unit 1002 to the first unit 1001 is stored. Further, a file transmitted from the second unit 1002 to the first unit 1001 is stored in the third field DA04.

  FIG. 13 is a diagram for explaining a command of type “101” (hereinafter referred to as “sixth command”). The CPU 110 sends the sixth command from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 210 sends the sixth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPUs 110 and 210 write the command type (“101”), the command transmission destination, and the like in the header DA01. The CPUs 110 and 210 write the value of the communication type with the number “1” in the first field DA02. The CPUs 110 and 210 write the value of the connection destination with the number “2” in the second field DA03. The CPUs 110 and 210 write the value of the transfer destination with the number “3” in the third field DA04. The CPUs 110 and 210 write the value of the signal strength acquisition timing number “4” in the fourth field DA05.

  The sixth command in which “001” is set in the first field DA02 requests the counterpart device to perform infrared communication. The sixth command in which “010” is set in the first field DA02 requests the counterpart device to perform wireless communication by Bluetooth (registered trademark). Furthermore, the sixth command in which “011” is set in the first field DA02 requests the counterpart device to perform LAN communication.

  The sixth command in which “000” is set in the second field DA03 indicates that there is no information for designating the connection destination of communication.

  Further, the sixth command in which “001” is set in the second field DA03 is transmitted by the first unit 1001 to the device to which the first unit 1001 is connected. Such a sixth command requests transmission of information related to a device to which the first unit 1001 is connected.

  Further, the sixth command in which “010” is set in the second field DA03 is transmitted by the second unit 1002 to the first unit 1001 to which the second unit 1002 is connected. Such a sixth command requests transmission of information regarding the first unit 1001 to which the second unit 1002 is connected.

  Further, the sixth command in which “011” is set in the second field DA03 is transmitted by the second unit 1002 to the first unit 1001 to which the second unit 1002 is connected. Such a sixth command requests that information regarding the second unit 1002 be set as connection destination device information.

  Further, the sixth command in which “100” is set in the second field DA03 is transmitted by the first unit 1001 to a device to which the first unit 1001 is connected (for example, the second unit 1002). Such a sixth command requests that information regarding the first unit 1001 be set as connection destination device information.

  The sixth command in which “000” is set in the third field DA04 indicates that it does not have information specifying the transfer destination of data (for example, a file).

  The sixth command in which “001” is set in the third field DA04 is transmitted by the first unit 1001 to the data transfer destination device. Such a sixth command requests transmission of information relating to the data transfer destination device.

  Further, the sixth command in which “010” is set in the third field DA04 is transmitted by the second unit 1002 to the first unit 1001 that is the data transfer destination. Such a sixth command requests transmission of information related to the first unit 1001 of the data transfer destination.

  The sixth command in which “011” is set in the third field DA04 is transmitted by the second unit 1002 to the first unit 1001 that is the data transfer destination. Such a sixth command requests that information regarding the second unit 1002 be set as device information of the data transfer source.

  Furthermore, the sixth command in which “100” is set in the third field DA04 is transmitted by the first unit 1001 to the data transfer destination device (for example, the second unit 1002). Such a sixth command requests that information regarding the first unit 1001 be set as device information of the data transfer source.

  The sixth command in which “00”, “01”, “10”, or “11” is set in the fourth field DA05 is transmitted by the first unit 1001 to the second unit 1002.

  The sixth command in which “00” is set in the fourth field DA05 does not request the second unit 1002 to transmit data indicating the signal strength. In addition, the sixth command in which “01” is set in the fourth field DA05 requests the signal strength detection unit 297 to transmit data indicating the signal strength at that time. Furthermore, the sixth command in which “10” is set in the fourth field DA05 requests transmission of data indicating the signal strength when the signal strength is changed. Further, the sixth command in which “11” is set in the fourth field DA05 requests the transmission of data indicating the signal strength at regular intervals.

  FIG. 14 is a diagram showing a schematic configuration of response data. The response data is data corresponding to the content of the first command (sensing command).

  When the first command is transmitted from the main device 101 to the second unit 1002, the CPU 210 transmits response data from the display device 103 to the first unit 1001. When the first command is transmitted from the main device 101 to the display device 102 of the first unit 1001, the image processing engine 180 transmits response data from the image processing engine 180 to the main device 101. Hereinafter, a case where the first command is transmitted from the main device 101 to the second unit 1002 will be described as an example.

  Referring to FIG. 14, the response data includes a header data area DA11, a coordinate data area DA12, a time data area DA13, and an image data area DA14. Here, the value of the center coordinates of the partial image is written in the data area DA12 indicating the coordinates. The time information acquired from the timer 282 of the image processing engine 280 is written in the data area indicating the time. Further, image data (that is, scan data) after being processed by the image processing engine 280 is written in the data area indicating the image.

  FIG. 15 is a diagram illustrating an image obtained by scanning the finger 900 (that is, a scanned image). Referring to FIG. 15, an image of a region W1 surrounded by a thick solid line is an entire image, and an image of a region P1 surrounded by a broken line is a partial image. The center point C1 of the cross indicated by a thick line is the center coordinate.

  In the present embodiment, a pixel (that is, a predetermined gradation or a pixel having a photosensor circuit which is a rectangular region and whose output voltage from the sensor signal line SSj is equal to or higher than a predetermined value). An area including all of the pixels having a luminance equal to or higher than a predetermined luminance is set as a partial image area.

  The center coordinates are coordinates determined in consideration of the gradation of each pixel in the partial image area. Specifically, for each pixel in the partial image, the center coordinate is determined by performing a weighting process based on the gradation of the pixel and the distance between the pixel and the rectangular center point (that is, the centroid). Is done. That is, the center coordinates do not necessarily match the centroid of the partial image.

  However, the position of the center coordinates is not necessarily limited to the above, and the center coordinates may be the coordinates of the centroid or the coordinates near the centroid.

  When “001” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the center coordinate in the data area DA12 indicating the coordinates. In this case, the image processing engine 280 does not write image data in the data area DA14 indicating an image. After writing the value of the center coordinate, the image processing engine 280 sends response data including the value of the center coordinate to the main body device 104. The main device 104 sends response data including the value of the center coordinate to the main device 101 of the first unit 1001. As described above, when “001” is set in the data area indicating the data type, the first command does not request output of the image data but requests output of the value of the center coordinate.

  When “010” is set in the data area indicating the data type of the first command, the image processing engine 280 stores the image data of the partial image whose scan result has changed in the data area DA14 indicating the image. Write. In this case, the image processing engine 280 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 280 writes the image data of the partial image whose scan result has changed, and then sends response data including the image data of the partial image to the main body device 104. The main device 104 sends response data including image data of the partial image to the main device 101 of the first unit 1001. Thus, when “010” is set in the data area indicating the data type, the first command does not request the output of the value of the center coordinate, and the image data of the partial image whose scan result has changed. Request output.

  As described above, the reason for requesting transmission of only the partial image whose scan result has changed is that the scan data of the partial image area of the scan data is more important than the scan data of the other area. This is because the data is high, and the scan data in the region corresponding to the region of the partial image in the scan data is likely to change due to the contact state with the scan object such as the finger 900.

  When “011” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the center coordinate in the data area DA12 indicating the coordinate and also indicates the data indicating the image. The image data of the partial image whose scan result has changed is written in the area DA14. Thereafter, the image processing engine 280 sends response data including the value of the center coordinate and the image data of the partial image to the main body device 104. The main device 104 sends response data including the value of the center coordinate and the image data of the partial image to the main device 101 of the first unit 1001. Thus, when “011” is set in the data area indicating the data type, the first command outputs the value of the center coordinate and the output of the image data of the partial image whose scan result has changed. Request.

  Further, when “100” is set in the data area indicating the data type of the first command, the image processing engine 280 displays the entire image in the data area DA14 indicating the image of the response data shown in FIG. Write image data. In this case, the image processing engine 280 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 280 writes the image data of the entire image, and then sends response data including the image data of the entire image to the main body device 104. The main device 104 sends response data including image data of the entire image to the main device 101 of the first unit 1001. As described above, when “100” is set in the data area indicating the data type, the first command requests the output of the image data of the entire image without requesting the output of the center coordinate value. .

  If “101” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the central coordinate in the data area DA12 indicating the coordinate and also indicates the data indicating the image. The image data of the entire image is written in the area DA14. Thereafter, the image processing engine 280 sends response data including the value of the center coordinate and the image data of the entire image to the main body device 104. The main device 104 sends response data including the value of the center coordinate and the image data of the entire image to the main device 101 of the first unit 1001. As described above, when “101” is set in the data area indicating the data type, the first command requests the output of the value of the center coordinate and the output of the image data of the entire image.

<Regarding First Modification of Configuration>
By the way, the structure of the liquid crystal panel 140 is not limited to the structure shown in FIG. Hereinafter, a liquid crystal panel having a mode different from that in FIG. 3 will be described.

  FIG. 16 is a circuit diagram of a photosensor built-in liquid crystal panel 140A according to the different embodiment. Referring to FIG. 16, photosensor built-in liquid crystal panel 140A (hereinafter referred to as liquid crystal panel 140A) includes three photosensor circuits (144r, 144g, 144b) in one pixel. Thus, the liquid crystal panel 140A is different from the liquid crystal panel 140 including one photosensor circuit in one pixel in that the liquid crystal panel 140A includes three photosensor circuits (144r, 144g, and 144b) in one pixel. The configuration of the optical sensor circuit 144 is the same as that of each of the three optical sensor circuits (144r, 144g, 144b).

  Further, the three photodiodes (145r, 145g, 145b) in one pixel are arranged at positions facing the color filter 153r, the color filter 153g, and the color filter 153b, respectively. Therefore, the photodiode 145r receives red light, the photodiode 145g receives green light, and the photodiode 145b receives blue light.

  In addition, since the liquid crystal panel 140 includes only one photosensor circuit 144 in one pixel, two data signal lines for the TFT 147 disposed in one pixel are a sensor signal line SSj and a sensor signal line SDj. Met. However, since the liquid crystal panel 140A includes three photosensor circuits (144r, 144g, 144b) in one pixel, there are six data signal lines for TFTs (147r, 147g, 147b) arranged in one pixel. It becomes.

  Specifically, the sensor signal line SSRj and the sensor signal line SDRj are arranged corresponding to the TFT 147r connected to the cathode of the photodiode 145r arranged at a position facing the color filter 153r. A sensor signal line SSGj and a sensor signal line SDGj are arranged corresponding to the TFT 147g connected to the cathode of the photodiode 145g arranged at a position facing the color filter 153g. Further, a sensor signal line SSBj and a sensor signal line SDBj are disposed corresponding to the TFT 147b connected to the cathode of the photodiode 145b disposed at a position facing the color filter 153b.

  In such a liquid crystal panel 140A, the white light emitted from the backlight 179 passes through the three color filters (153r, 153g, 153b), and red, green, and blue are displayed on the surface of the liquid crystal panel 140A. It mixes and becomes white light. Here, when white light is reflected by the scan object, a part of the white light is absorbed by the pigment on the surface of the scan object, and a part is reflected. And the reflected light permeate | transmits three color filters (153r, 153g, 153b) again.

  At this time, the color filter 153r transmits light having a red wavelength, and the photodiode 145r receives light having the red wavelength. The color filter 153g transmits light having a green wavelength, and the photodiode 145g receives light having the green wavelength. The color filter 153b transmits light having a blue wavelength, and the photodiode 145b receives light having the blue wavelength. That is, the light reflected by the scan object is separated into three primary colors (R, G, B) by three color filters (153r, 153g, 153b), and each photodiode (145r, 145g, 145b) The light of the color corresponding to is received.

  When a part of white light is absorbed by the pigment on the surface of the scan target, the amount of light received by each photodiode (145r, 145g, 145b) is different for each photodiode (145r, 145g, 145b). For this reason, the output voltages of the sensor signal line SSRj, the sensor signal line SSGj, and the sensor signal line SSBj are different from each other.

  Therefore, the image processing engine 180 determines the R gradation, the G gradation, and the B gradation according to each output voltage, so that the image processing engine 180 sends the RGB color image to the main body device 101. Can send.

  As described above, when the electronic apparatus 100 includes the liquid crystal panel 140A, the scan target can be scanned in color.

  Next, a scanning method different from the above-described scanning method (that is, the method of scanning the reflected image in FIG. 6) will be described with reference to FIG.

  FIG. 17 is a cross-sectional view showing a configuration in which a photodiode receives external light during scanning. As shown in the figure, part of the external light is blocked by the finger 900. Therefore, the photodiode disposed under the surface region of the liquid crystal panel 140 that is in contact with the finger 900 can hardly receive external light. In addition, although the photodiodes disposed under the surface area where the shadow of the finger 900 is formed can receive a certain amount of external light, the amount of external light received is small compared to the surface area where no shadow is formed.

  Here, by turning off the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the position of the finger 900 with respect to the surface of the liquid crystal panel 140 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), and the liquid crystal panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter also referred to as a shadow image) obtained by blocking external light with the finger 900.

  Further, the display device 102 may be configured to perform scanning by turning on the backlight 179 and then performing scanning again by turning off the backlight 179. Alternatively, the display device 102 may be configured to perform scanning by turning off the backlight 179 and then performing scanning again by turning on the backlight 179.

  In this case, since two scanning methods are used together, two scan data can be obtained. Therefore, it is possible to obtain a highly accurate result as compared with the case of scanning using only one scanning method.

<About display devices>
The operation of the display device 103 is controlled in accordance with a command (for example, a first command) from the main body device 101 as in the operation of the display device 102. The display device 103 has the same configuration as the display device 102. Therefore, when the display device 103 receives the same command as the display device 102 from the main body device 101, the display device 103 performs the same operation as the display device 102. For this reason, description of the configuration and operation of the display device 103 will not be repeated.

  Note that the main device 101 can send commands having different commands to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform different operations. Further, the main device 101 may send a command to either the display device 102 or the display device 103. In this case, only one display device performs an operation according to the command. Further, the main device 101 may send a command having the same command to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform the same operation.

  Note that the size of the liquid crystal panel 140 of the display device 102 and the size of the liquid crystal panel 240 of the display device 103 may be the same or different. Further, the resolution of the liquid crystal panel 140 and the resolution of the liquid crystal panel 240 may be the same or different.

<About Second Modification of Configuration>
In the present embodiment, a configuration in which electronic device 100 includes a liquid crystal panel with a built-in optical sensor such as liquid crystal panel 140 and liquid crystal panel 240 will be described. However, only one liquid crystal panel has a built-in photosensor. It may be.

  FIG. 18 is a block diagram illustrating a hardware configuration of electronic device 100300. Similar to the electronic device 100, the electronic device 100300 includes a first housing 100A and a second housing 100B. Referring to FIG. 18, electronic device 100300 includes a first unit 1001A and a second unit 1002. The first unit 1001A includes a main body device 101 and a display device 102A. The second unit 1002 includes a main device 104 and a display device 103.

  The display device 102A includes a liquid crystal panel that does not include a photosensor (that is, a liquid crystal panel having only a display function). The electronic device 100300 is different from the electronic device 100 in which the first unit 1001A includes a liquid crystal panel that does not include a photosensor, and the first unit 1001 includes a liquid crystal panel 240 that includes a photosensor. Such an electronic device 100300 performs the above-described sensing using the display device 103 of the second unit 1002.

  Further, the first unit 1001 may include, for example, a resistive film type or capacitive type touch panel, instead of the liquid crystal panel 140 incorporating the photosensor.

  In this embodiment, the display device 102 includes the timer 182 and the display device 103 includes the timer 282. However, the display device 102 and the display device 103 may share one timer. .

  In the present embodiment, electronic device 100 is described as a foldable device, but electronic device 100 is not necessarily limited to a foldable device. For example, the electronic device 100 may be a sliding device configured such that the first housing 100A slides with respect to the second housing 100B.

  Since electronic device 100 according to the present embodiment is configured as described above, second unit 1002 is detachable from first unit 1001 via USB connectors 194 and 294.

  And the electronic device 100 which concerns on this Embodiment can exhibit the following functions, for example at the time of power activation. First, when the user depresses the power switch 191 of the first unit 1001, the first unit 1001 activates BIOS (Basic Input / Output System) by using the power from the power supply circuit 192.

  The second unit 1002 acquires power from the first unit 1001 via the USB connectors 194 and 294. The second unit 1002 can transmit and receive data to and from the first unit 1001 by using the power. At this time, the CPU 210 of the second unit 1002 can display the OS (Operation System) type on the liquid crystal panel 240 in a selectable manner by using the power from the USB connectors 194 and 294. The second unit 1002 may be directly supplied with power from the power source 192 without going through the USB connectors 194 and 294.

  The user selects an OS to be started up via the liquid crystal panel 240. The CPU 210 transmits a command (for example, a “first OS” command shown in FIG. 10) specifying the OS to be activated to the first unit 1001 via the USB connectors 194 and 294 according to the user's selection. . The first unit 1001 starts the OS in response to the command.

  Further, for example, the second unit 1002 transmits / receives data to / from an external mobile phone or the like via the antenna 295. The CPU 210 of the second unit 1002 acquires photographic image data and corresponding thumbnail data from an external mobile phone via the antenna 295, and stores the photographic image data and corresponding thumbnail data in the RAM 271 or the like. The CPU 210 reads the thumbnail data from the RAM 271 and causes the liquid crystal panel 240 to display a thumbnail image of the photo in a selectable manner.

  In response to a selection command from the outside, the CPU 210 causes the liquid crystal panel 240 to display a photographic image. Alternatively, the CPU 210 displays a photographic image on the liquid crystal panel 140 or the display device 102A via the USB connector 294.

<Third Modification of Configuration>
In the present embodiment, electronic device 100 may further include a key operation unit in second housing 100B as shown in FIG. In the present embodiment, electronic device 100 is configured as a laptop personal computer. Electronic device 100 may be configured as a device having a display function, such as a PDA (Personal Digital Assistant), a mobile phone, or an electronic dictionary. FIG. 20 is a block diagram showing a hardware configuration of electronic device 100 shown in FIG.

  A main device 101 shown in FIG. 20 further includes an HDD (Hard Disc Drive) 170 in addition to the main device 101 of the electronic apparatus 100 shown in FIG.

  In electronic device 100, CPU 110 executes a program. The operation key 177 receives an instruction input from the user of the electronic device 100. The HDD 170 is a storage device that can write and read data. However, the HDD 170 is an example of such a storage device. In the electronic device 100, a storage device such as a flash memory may be used instead of the HDD 170.

  20 is different from the main device 104 shown in FIG. 2 in that the timer 273 and the key operation unit (the left click key 241, the center key 242, and the right click key 243). And further including. Each component (210, 241 to 243, 271, 272, 274, 273, 293) is mutually connected by a data bus DB2.

FIG. 21 is a block diagram illustrating a functional configuration of the electronic device 100 of FIG.
As described above, electronic device 100 includes first unit 1001 and second unit 1002. Hereinafter, the functional configuration of the electronic apparatus 100 will be described with reference to FIG.

  The first unit 1001 includes a display unit 310, an input unit 320, a storage unit 330, an interface unit 340, and a control unit 350. The first unit 1001 performs the main operation of the electronic device 100.

  The display unit 310 displays information inside the first unit 1001 to the outside. The input unit 320 receives an instruction from the outside. In the present embodiment, the liquid crystal panel 140 has the functions of the display unit 310 and the input unit 320. However, as the display unit 310, another display device, for example, a display such as an LCD (Liquid Crystal Display) may be used. The operation key 177 also functions as the input unit 320.

  The storage unit 330 stores information such as display data 333, a program 334, and an operation parameter 335 that are a basis of a screen displayed on the display unit 310 (liquid crystal panel 140) of the first unit 1001. The storage unit 330 generally stores a plurality of programs 334. Examples of the program 334 include general-purpose application software such as a word processor and a Web browser.

  The operation parameter 335 is information for giving an operation condition of the program 334. The operation parameter 335 includes, for example, data indicating an active window that operates in response to pressing of the operation key 177 in the multi-window program 334.

  The interface unit 340 exchanges information with the interface unit 440 on the second unit 1002 side. In the present embodiment, when the first unit 1001 and the second unit 1002 are directly connected, the USB connector 194 functions as the interface unit 340. When the first unit 1001 and the second unit 1002 are not directly connected, the antenna 195 functions as the interface unit 340. However, the method of exchanging information by the interface unit 340 is not limited to this.

  The control unit 350 controls operations of the display unit 310, the storage unit 330, and the interface unit 340 based on an instruction from the input unit 320 and the like. Control unit 350 includes an input processing unit 352, a display control unit 356, and a program execution unit 358. In the present embodiment, the CPU 110 and the image processing engine 180 correspond to the control unit 350. However, each function of the CPU 110 may be realized by hardware such as a dedicated circuit. Each function of the image processing engine 180 may be realized by the CPU 110 that executes software. That is, each function of the control unit 350 may be realized by hardware or may be realized by software.

  The input processing unit 352 transmits the signal received from the input unit 320 to the program execution unit 358. The display control unit 356 controls the operation of the display unit 310 based on the display data 333 stored in the storage unit 330. The program execution unit 358 executes the program 334 based on an instruction received from the input unit 320. Specifically, the CPU 110 that executes the program 334 using the RAM 171 as a working memory corresponds to the program execution unit 358.

  The second unit 1002 includes a display unit 410, an input unit 420, a storage unit 430, an interface unit 440, a control unit 450, and a timer 273.

  The display unit 410 displays information inside the second unit 1002 to the outside. The input unit 420 receives an instruction from the outside. In the present embodiment, the optical sensor built-in liquid crystal panel 240, the left click key 241, the center key 242, and the right click key 243 correspond to the input unit 420. In the present embodiment, the optical sensor built-in liquid crystal panel 240 also has the functions of the display unit 410 and the input unit 420 (panel input unit 422). However, as the display unit 410, another display device, for example, a display such as an LCD may be used. The input unit 320 is not limited to the optical sensor built-in liquid crystal panel 240, and a device (tablet) having a function of recognizing an input position can be used. For example, a capacitive touch panel may be used as the input unit 420. A component that realizes the functions of the display unit 410 and the input unit 420 in this way is referred to as a “display-integrated touchpad”.

  The storage unit 430 stores information such as input data 431, display data 433, a program 434, operation parameters 435, time data 436, and mode data 437.

  The input data 431 is data created based on the input received by the input unit 420. In particular, in the present embodiment, the input data 431 includes an input history 432 corresponding to the input history. The input history 432 includes handwritten character data 432a and illustration data 432b. Details of the handwritten character data 432a and the illustration data 432b will be described later.

  The display data 433 is a basis of a screen displayed on the display unit 410 (liquid crystal panel 240) of the second unit 1002. The display data 433 includes image data (such as wallpaper) stored in the storage unit 430 and image data created in accordance with the execution of the program 434.

  In the present embodiment, storage unit 430 stores a plurality of programs 434. The program 434 includes application software (for example, handwritten character input software, hand-drawn illustration input software, or calculator software) that displays an operation screen on the liquid crystal panel 240. Details of the program 434 will be described later.

  The operation parameter 435 is information for giving an operation condition of the program 434 as with the operation parameter 335 in the first unit 1001. In particular, in the present embodiment, the operation parameter 435 includes a count value of elapsed time from a predetermined event created by the program 434.

  Time data 436 represents the time measured by the timer 273. The time data 436 is used when executing a program 434 that performs a predetermined operation with the passage of time.

  The mode data 437 is information representing the operation mode of the input processing unit 452. There are a plurality of operation modes of the input processing unit 452, and the mode data 437 represents the current operation mode. Specifically, for example, a flag stored in a predetermined storage area can be regarded as mode data 437. Details of the operation mode of the input processing unit 452 will be described later.

  The interface unit 440 exchanges information with the interface unit 340 on the first unit 1001 side. In the present embodiment, when the first unit 1001 and the second unit 1002 are directly connected, the USB connector 294 functions as the interface unit 440. When the first unit 1001 and the second unit 1002 are not directly connected, the antenna 295 functions as the interface unit 440. However, the method of exchanging information by the interface unit 440 is not limited to this.

  The control unit 450 controls operations of the display unit 410, the storage unit 430, and the interface unit 440 based on an instruction received by the input unit 420. Control unit 450 includes an input processing unit 452, a display control unit 456, and a program execution unit 458.

  The input processing unit 452 transmits the signal from the input unit 420 to the program execution unit 458 or the interface unit 440. The input processing unit 452 includes a panel input processing unit 453 and a mode setting unit 454.

  The panel input processing unit 453 processes a signal from the panel input unit 422. For example, the panel input processing unit 453 creates an input history 432 (handwritten character data 432a, illustration data 432b, etc.) based on the signal history. Details of the operation of the panel input processing unit 453 will be described later.

  The mode setting unit 454 sets the operation mode of the panel input processing unit 453 based on a predetermined signal from the input unit 420 (such as a signal generated by pressing the center key 242). Details of the operation of the mode setting unit 454 will be described later.

  The display control unit 456 controls the operation of the display unit 410 based on the display data 433. For example, the display control unit 456 displays a screen (operation screen) created as a result of the execution of the program 434 on the display unit 410.

  The program execution unit 458 executes the program 434 based on an instruction received from the input unit 420 or the like. Specifically, the CPU 210 that executes the program 434 using the RAM 271 as a working memory corresponds to the program execution unit 458.

<Outline of operation>
(Mouse mode and tablet mode)
The electronic device 100 issues an instruction for the operation of the application in response to an instruction input to the liquid crystal panel 240, that is, an object (such as a finger 900 or a stylus 950) touching the liquid crystal panel 240. The electronic device 100 (more specifically, the panel input processing unit 453) has two operation modes of “mouse mode” and “tablet mode”. The electronic device 100 operates according to these two operation modes.

  In the mouse mode, the electronic device 100 executes the program 334 in response to an input to the liquid crystal panel 240, and an image created by the executed program 334 (hereinafter referred to as “program operation screen”) is displayed on the liquid crystal panel 140. To display.

  Specifically, in the mouse mode, electronic device 100 moves the cursor in the operation screen of liquid crystal panel 140 in real time according to the change in the input position to liquid crystal panel 240. Here, the “cursor” refers to a display that indicates an input position of a character, a figure, a display object, or the like.

  Further, in the mouse mode, electronic device 100 creates a command for instructing a predetermined operation to program 334 of first unit 1001 in response to a predetermined input to liquid crystal panel 240. For example, when the electronic device 100 determines that the liquid crystal panel 240 has received an input corresponding to a click, a double click, a drag, or the like, the electronic device 100 executes the program 334 according to the input. The operation performed by electronic device 100 at this time is determined by program 334.

  That is, when electronic device 100 is in the mouse mode, the user can use liquid crystal panel 240 as a touch pad. Hereinafter, for the sake of simplicity, the operation of the program 334 according to the movement of the cursor position and the predetermined input to the liquid crystal panel 240 will be collectively referred to as “mouse operation”.

  In the tablet mode, electronic device 100 executes program 434 (or program 334) in response to an input to liquid crystal panel 240, and displays an operation screen of the executed program on liquid crystal panel 240. In addition, electronic device 100 creates a command for a program for displaying an operation screen on liquid crystal panel 240. For example, when an operation button is displayed on the liquid crystal panel 240, the electronic device 100 executes the program 434 and performs an operation corresponding to the operation button on the touched liquid crystal panel 240. That is, when the electronic device 100 is in the tablet mode, the user can use the liquid crystal panel 240 as a touch screen.

  Electronic device 100 switches the operation mode based on a predetermined instruction. In the present embodiment, electronic device 100 switches the operation mode in response to pressing of center key 242.

  However, the operation mode switching instruction is not limited to pressing the center key 242. Electronic device 100 may switch the operation mode in response to pressing of operation key 177 other than center key 242. In addition, electronic device 100 may switch the operation mode in accordance with selection of an operation button displayed on liquid crystal panel 140 or liquid crystal panel 240. The electronic device 100 also switches the operation mode depending on the operation state of the electronic device (during startup processing, return from sleep or hibernation). This point will be described in detail later.

  Display on the liquid crystal panel 140 and the liquid crystal panel 240 in the mouse mode and the tablet mode will be described with reference to FIG. FIG. 22 is a diagram for explaining a screen displayed by the electronic device in each of the mouse mode and the tablet mode.

  In the mouse mode, the electronic device 100 displays an operation screen 500 of software such as word processing software or a Web browser on the liquid crystal panel 140. The screen 500 is similar to a display screen by a personal computer that is widely used at present (that is, having one display). However, the contents of the screen 500 are not drawn in FIG. 22 for simplicity. The screen 500 is not limited to this specific example.

  Screen 500 includes a cursor 510. Here, it is assumed that the cursor 510 is a pointer (mouse pointer) that can freely move on the screen 500. However, the cursor 510 is not limited to a pointer. The cursor 510 may be a display that indicates an input position of a character or a display object. The display form of the cursor 510 is not limited to that shown in FIG. Also, electronic device 100 may change the display form of cursor 510 depending on the position pointed to.

  In the mouse mode, electronic device 100 displays screen 600 on liquid crystal panel 240. Hereinafter, the screen displayed on the liquid crystal panel 240 in the mouse mode is also referred to as a “mouse screen”. Referring to FIG. 22, screen 600 includes a guidance display 610.

  Guidance display 610 is a display for describing the operation of electronic device 100 when left click key 241, center key 242, and right click key 243 are pressed. Guidance display 610 includes a left guidance display 612, a center guidance display 614, and a right guidance display 616.

  The left guidance display 612, the center guidance display 614, and the right guidance display 616 are characters that describe the operation of the electronic device 100 when the left click key 241, the center key 242, and the right click key 243 are pressed, respectively. And / or symbols (though such characters and symbols are not shown in FIG. 22).

  The mouse screen may not always display the left guidance display 612, the center guidance display 614, and the right guidance display 616. In the present embodiment, electronic device 100 does not display left guidance display 612, center guidance display 614, and right guidance display 616 on liquid crystal panel 240 when the corresponding key is invalid.

  One specific example of the mouse screen is shown in FIG. Referring to FIG. 23, screen 600 includes a left guidance display 612, a center guidance display 614, and a right guidance display 616.

  In FIG. 23, the left guidance display 612 includes the characters “left click”. When this display is made, electronic device 100 performs a left click operation (decision process or the like) in response to pressing of left click key 241. The left click operation is determined by the running program.

  In FIG. 23, the center guidance display 614 includes the characters “touch screen operation”. This display is made when the electronic device 100 is in the mouse mode. When this display is made, the electronic device 100 shifts to the tablet mode in response to pressing of the center key 242.

  Electronic device 100 displays a character including character “mouse operation” on center guidance display 614 when in tablet mode. When this display is made, electronic device 100 shifts to the mouse mode in response to pressing of center key 242.

  The right guidance display 616 includes the characters “right click”. This indicates that electronic device 100 performs a right-click operation (menu display, etc.) in response to pressing of right-click key 243. The detailed operation of the right click is determined by the running program.

  In this embodiment, the user can set the mouse screen. The user may be able to set an image such as a photograph stored in the electronic device 100 as the wallpaper of the mouse screen. The wallpaper may be an operation screen of an accessory such as a clock or calendar that does not require user operation. Even when the user cannot set the mouse screen, the mouse screen is not limited to that shown in FIG. The wallpaper can also change automatically depending on the state of the electronic device 100.

  Specifically, image data displayed as a mouse screen is stored in a storage device such as the ROM 272 or RAM 271, and the CPU 210 reads the image data from the storage device and displays it on the liquid crystal panel 240 in the mouse mode.

  Returning to FIG. 22, the screen in the tablet mode will be described. In the tablet mode, the liquid crystal panel 140 displays an operation screen 700 of software such as word processing software or a Web browser. The content of the operation screen 700 is the same as that of the operation screen 500.

  In the tablet mode, the electronic device 100 displays the screen 800 on the liquid crystal panel 240. Hereinafter, the screen displayed on the liquid crystal panel 240 in the tablet mode is also referred to as a “tablet screen”. Referring to FIG. 22, screen 800 includes a guidance display 810 and an operation button display 820.

  The guidance display 810 includes a left guidance display 812, a center guidance display 814, and a right guidance display 816 similarly to the guidance display 610 in the mouse mode. These roles and operations are similar to left guidance display 612, center guidance display 614, and right guidance display 616, respectively, and detailed description thereof will not be repeated.

  The operation button display 820 is used for selecting an application. When electronic device 100 detects the contact of an external object (such as finger 900 or stylus 950) in the region corresponding to operation button display 820, electronic device 100 starts a predetermined operation corresponding to the region.

(About the program)
Here, a program to be executed in the electronic device 100 will be described.

  The programs executed by electronic device 100 according to the present embodiment include a program for displaying an operation screen on liquid crystal panel 140 (hereinafter referred to as “main application”) and a program for displaying an operation screen on liquid crystal panel 240. (Hereinafter referred to as “sub-application”).

  As the main application, an application that operates on a current electronic device such as a browser, a dictionary, a book viewer, and a photo viewer can be assumed. Examples of the sub-application include an input pad (handwritten character input pad, hand-drawn illustration input pad, calculator / number input pad, etc.) that uses input to the liquid crystal panel 240, and an application for assisting operation of the main application.

  In the description of the present embodiment, the sub application is assumed to be independent of the main application. The main application is stored in the storage unit 330 of the first unit 1001. On the other hand, the sub application is stored in the storage unit 430 of the second unit 1002.

  Thus, the main application is separated from the sub application. Therefore, a general-purpose application that operates on other electronic devices can be used as the main application. In this case, the specification of data exchange with the main application of the sub application is matched with the specification of the main application. For example, the instruction of the mouse operation by the sub application is matched with the operation instruction from the conventional touch pad or mouse.

  Furthermore, in the present embodiment, the program execution unit 358 that executes the main application and the program execution unit 458 that executes the sub-application are independent. In this way, it is possible to reduce the load on the processor (CPU 110 in this embodiment) that executes the main application. In particular, when the performance of the CPU of the electronic device 100 is low, dividing the program execution unit in this way is effective.

  By providing a control unit and a storage unit in each of the first unit 1001 and the second unit 1002 as in the present embodiment, data exchange between the first unit 1001 and the second unit 1002 can be reduced, and processing can be performed. Can be speeded up.

  However, the sub application is not necessarily independent of the main application. That is, the same program may function as both the main application and the sub application. Specifically, a part of the program may create a screen to be displayed on the liquid crystal panel 140, and another part of the program may create a screen to be displayed on the liquid crystal panel 240.

  Further, the main application and the sub application may be executed by the same processor. In this case, the processor that executes the application controls the operations of both the liquid crystal panel 140 and the liquid crystal panel 240.

(Sub application)
In the present embodiment, electronic device 100 executes one of a plurality of sub-applications in the tablet mode. One of the sub-applications is a “home application” for determining an application to be executed.

  The home application is a launcher for selecting an application. The home application displays on the liquid crystal panel 240 a screen for selecting one sub-application from the plurality of sub-applications (hereinafter “home menu screen”).

  FIG. 24 shows a specific example of the home menu screen. Referring to FIG. 24, the home menu screen includes a guidance display 810, operation button displays 820a to 820i, and a mouse disabling display 830. When the operation button displays 820a to 820i are touched, the home application calls a sub application corresponding to the operation button displays 820a to 820i. The mouse disabled display 830 is a display for indicating that the electronic device 100 does not perform a mouse operation by an input to the liquid crystal panel 240. This display helps the user to accurately understand the operation of the electronic device 100 in the tablet mode. It also helps the user to distinguish between mouse mode and tablet mode.

  In the present embodiment, it is assumed that electronic device 100 can customize the home menu screen based on a user instruction. Even if the home menu screen cannot be customized, the configuration of the home menu screen is not limited to that shown in FIG. For example, the number and arrangement of operation button displays 820 are not limited to those shown in FIG. The mouse disabled display 830 is not limited to that shown in FIG. Alternatively, the mouse disabled display 830 may not be included in the home menu screen.

  It is assumed that the default sub application in the tablet mode is a home application. That is, when the electronic device 100 operates in the tablet mode for the first time after activation (after power is turned on), the home application is executed.

  The transition of the screen 800 displayed on the liquid crystal panel 240 in the tablet mode will be described with reference to FIG. FIG. 25 is a transition diagram of a screen 800 displayed on the liquid crystal panel 240 in the tablet mode.

  Referring to FIG. 25, when the user selects operation button display 820 (hand-drawn illustration) on home menu screen 800a, electronic device 100 executes the hand-drawn illustration application and displays screen 800b on liquid crystal panel 240. The screen 800b is an operation screen for a hand-drawn illustration application. On the screen 800b shown in FIG. 25, a picture of a dog input during execution of the handwritten illustration application is drawn in the handwriting input frame.

  When the user selects operation button display 820 (home) on screen 800b, electronic device 100 displays window 800d on liquid crystal panel 240. Here, the electronic device 100 may display the window 800d exclusively with the screen 800b, or may display the window 800d superimposed on the screen 800b. The window 800d includes text for inquiring whether to save the created hand-drawn illustration, and operation button displays 820 of “Yes”, “No”, and “Cancel”.

  When “Yes” in the window 800d is selected, the electronic device 100 displays a hand-drawn illustration in the input frame before the window 800d is displayed in the electronic device 100 or an external storage device (such as a hard disk or a flash memory). To store. In addition, the electronic device 100 displays the home menu screen 800a on the liquid crystal panel 240.

  When “No” in the window 800d is selected, the electronic device 100 displays the home menu screen 800a on the liquid crystal panel 240. In this case, electronic device 100 does not store the hand-drawn illustration made in the input frame before display of window 800d in the storage device.

  When “cancel” of the window 800d is selected, the electronic device 100 displays the screen 800b before the window 800d is displayed on the liquid crystal panel 240. That is, if the user selects cancel, the user can continue to create a hand-drawn illustration in the hand-drawn input frame.

  When the user selects the operation button display 820e (Internet) on the home menu screen 800a, the electronic device 100 displays on the liquid crystal panel 140 an operation screen 800c for selecting a homepage list for starting a Web browser.

  When the user selects operation button display 820 (Home) on screen 800c, electronic device 100 displays liquid crystal panel 240 on home menu screen 800a. In this case, electronic device 100 does not perform the storage process of the input data. Therefore, electronic device 100 does not display an inquiry screen such as window 800d.

<Switching mode>
Hereafter, the operation of the electronic device 100 when switching between the mouse mode and the tablet mode will be described in detail with reference to FIG. FIG. 26 is a diagram for explaining the operation of the electronic device 100 when switching between the mouse mode and the tablet mode.

  In the present embodiment, electronic device 100 basically switches the operation mode in response to pressing of center key 242 as described above. The electronic device 100 can transition from the tablet mode to the mouse mode, or can transition from the mouse mode to the tablet mode.

  In the present embodiment, electronic device 100 changes the processing method of input to liquid crystal panel 240 when shifting from the tablet mode to the mouse mode. That is, electronic device 100 treats an input to liquid crystal panel 240 as a mouse operation instruction rather than an operation instruction to a sub application. In addition, the electronic device 100 displays a mouse screen on the liquid crystal panel 240 when shifting to the mouse mode.

  However, the electronic device 100 keeps running the application that has been executed in the tablet mode even after shifting to the mouse mode. Thus, the electronic device 100 can smoothly display the screen on the liquid crystal panel 240 when shifting from the mouse mode to the tablet mode. This is because the startup time of the sub application is unnecessary.

  Further, by continuing to operate the application executed in the tablet mode, the operability when the user performs a mouse operation temporarily with the liquid crystal panel 140 during the tablet mode is improved. For example, consider that the electronic device 100 transitions from the tablet mode to the mouse mode and then transitions back to the tablet mode. Since the electronic device 100 operates as described above, the electronic device 100 displays the operation screen of the same sub application on the liquid crystal panel 240 before and after the transition to the mouse mode. Therefore, the user can continue to use the sub-application before the mouse operation after performing the mouse operation.

  The electronic device 100 displays the operation screen of the main application on the liquid crystal panel 140 not only in the mouse mode but also in the tablet mode. Therefore, electronic device 100 can allow the user to operate the sub application without impairing the visibility of the operation screen of the main application.

(Mode at the start of operation)
In particular, switching of operation modes related to the start of operation of electronic device 100 will be described. In the present embodiment, the operation of the electronic device 100 is roughly classified into (i) activation from a power-off state (hereinafter referred to as normal activation) and (ii) activation from a power saving state (hereinafter referred to as “power activation state”) , Called resumption).

  Here, the “power-off state” is a state in which the operation of each part of the electronic device 100 (excluding a part necessary for starting up the electronic device 100) is stopped. The “power saving state” is a state in which a part of the operation of the electronic device 100 is stopped.

  The power saving state includes a “standby state”, a “hibernation state”, and a “hybrid sleep state” that is a combination of the standby state and the hibernation state.

  When receiving the instruction to shift to the standby state, the electronic device 100 stores the data being worked in the RAM 171. In addition, the electronic device 100 stops supplying power to portions other than portions necessary for restarting operations (such as the power supply circuit 192, the power supply detection unit 193, and the RAM 171).

  When receiving the instruction to shift to the hibernation state, the electronic device 100 stores the data being worked on the hard disk 170. In addition, the electronic device 100 stops supplying power to parts other than those necessary for restarting the operation (such as the power circuit 192 and the power detection unit 193).

  When the hybrid sleep process for shifting to the hybrid sleep state is started, the electronic device 100 first stores data in operation in the memory. At the same time, the electronic device 100 copies the data stored in the memory from the memory to the hard disk when a predetermined time elapses from the instruction in the sleep state.

  However, the types of power saving states are not limited to those described above. Moreover, the electronic device 100 does not necessarily need to prepare all these power saving processes.

  First, (i) the operation mode in normal activation will be described with reference to FIG. FIG. 27 is a diagram schematically showing an operation mode at the normal startup.

  At the normal startup, the electronic device 100 first performs a boot process for starting an OS (Operating System). During the boot process, the electronic device 100 displays a boot screen 2501 on the liquid crystal panel 140. During the boot process, the operation mode of the electronic device 100 is the mouse mode. The electronic device 100 displays a predetermined mouse screen (hereinafter referred to as a fixed screen) 2502 on the liquid crystal panel 240. The fixed screen 2502 does not include a guidance display. This is because mode switching cannot be executed during the boot process.

  When the boot process is completed, the electronic device 100 displays a login screen 2503 on the liquid crystal panel 140. Also at this time, the electronic device 100 displays the fixed screen 2504.

  When the login is completed, the electronic device 100 displays a desktop screen 2505 on the liquid crystal panel 140. In addition, the electronic device 100 displays a mouse screen 2506 on the liquid crystal panel 240. At this stage, the electronic device 100 can switch from the mouse mode to the tablet mode. Accordingly, the mouse screen 2506 includes a guidance display, similar to the mouse screen 600 shown in FIG.

  It is assumed that the default sub application in the tablet mode is a home application. That is, when the electronic device 100 operates in the tablet mode for the first time after activation (after power is turned on), the home application is executed. This operation is not limited to starting from power-off, but is the same when the electronic device 100 is restarted.

  Next, (ii) the operation mode in return will be described with reference to FIG. FIG. 28 is a diagram schematically showing the operation mode at the time of return.

  When resuming from the power saving state, the electronic device 100 first reads data relating to the work state stored in a storage device such as a memory or a hard disk. During this time, the electronic device 100 displays a resuming screen 2601 on the liquid crystal panel 140. The resuming screen 2601 may be omitted. During this time, the operation mode of the electronic device 100 is the mouse mode. The electronic device 100 displays a fixed screen 2602 on the liquid crystal panel 240.

  When the reading of the work state is completed, the electronic device 100 displays a login screen 2603 on the liquid crystal panel 140. Also at this time, the electronic device 100 displays the fixed screen 2604. Depending on the setting in the electronic device 100, this login screen may not be displayed. In this case, the screen automatically shifts to the next login completion screen.

  When the login is completed, the electronic device 100 displays again the display screen 2605 displayed on the liquid crystal panel 140 immediately before shifting to the power saving state on the liquid crystal panel 140 based on the read work state. In addition, the electronic device 100 displays a display screen 2606 based on the sub application that was operating immediately before the transition to the power saving state on the liquid crystal panel 240.

  Electronic device 100 may always display a mouse screen on liquid crystal panel 140 after the login is completed. When it is expected that the first operation performed by the user after the restart is a mouse operation, the operability can be improved by this processing.

  Further, in electronic device 100, when the OS is started, during recovery from the sleep state, or in a standby state when switching a login user (for example, a screen prompting input of login information of a new user is displayed on liquid crystal panel 140). When there is a high possibility that the electronic device 100 will be operated with the mouse next time, such as during a screen saver display on the liquid crystal panel 140), a mouse screen is displayed on the liquid crystal panel 240. However, it may be switched to the mouse mode.

  The electronic device 100 operates in the first mode when starting up or returning from a specific power supply state by operating in the mouse mode when the OS is activated or returning from the sleep state. In addition, the electronic device 100 operates in the first mode while in the specific operation state by operating in the mouse mode during the standby state for switching the login user or during the screen saver display period.

<Operation in tablet mode>
From here, the operation of the electronic device 100 in the tablet mode will be described in more detail with reference to FIG. FIG. 29 is a diagram for describing an operation of electronic device 100 in the tablet mode.

  In the tablet mode, the electronic device 100 roughly divides the home application, executes the input pad application, or executes the sub-screen utilization software.

  The electronic device 100 displays the home menu screen 800a on the liquid crystal panel 240 during execution of the home application. When the electronic device 100 receives a touch on a predetermined position (indicated by the operation button display) on the liquid crystal panel 240 during execution of the home application, the electronic device 100 calls the input pad. In addition, when the electronic device 100 receives a touch on a predetermined position on the liquid crystal panel 240 during execution of the home application, the electronic device 100 starts execution of the sub-screen utilization software.

  When the electronic device 100 calls the input pad, the electronic device 100 displays an input screen on the liquid crystal panel 240. Referring to FIG. 29, in the present embodiment, there are three types of input pads, that is, a handwritten character input pad, a hand-drawn illustration input pad, and a calculator / number input pad. Screens 2702 to 2704 displayed on liquid crystal panel 240 are operation screens for a handwritten character input pad, a handwritten illustration input pad, and a calculator / numeric input pad, respectively. Details of each input pad will be described later.

  In addition, when electronic device 100 receives a predetermined instruction during execution of the input pad, electronic device 100 activates the home application. In the present embodiment, the operation screen of the input pad includes a home button, and electronic device 100 ends the input pad and starts a home application in response to a touch on the home button.

  Referring to FIG. 29, the sub-screen utilization software in this embodiment includes a two-screen utilization guide, the Internet, a dictionary, a book, a photo, and a game.

  The “2-screen usage guide” is an online manual. When the electronic device 100 executes the two-screen utilization guide, the electronic device 100 acquires manual data from the HDD 170 or an external server, and displays the manual on the liquid crystal panel 240 (or the liquid crystal panel 140 or both the liquid crystal panel 140 and the liquid crystal panel 240). .

  “Internet” is software for activating a Web browser and calling various homepage screens, and is hereinafter referred to as Web page calling software. Details of the operation of this software will be described later.

  “Dictionary” is software for calling an electronic dictionary, and is hereinafter referred to as dictionary calling software. In the present embodiment, this software calls one electronic dictionary from a plurality of electronic dictionaries. Details of the operation of this software will be described later.

  “Book” is software for selecting an electronic book (for example, a book in XMDF (Mobile Document Format) format) to be viewed with a book viewer. “Photo” is software for displaying a photo slideshow.

  “Game” is game software for displaying a game screen on the liquid crystal panel 240. In the present embodiment, it is assumed that the “game” is a game utilizing the touch operation of the liquid crystal panel 240.

  When the electronic device 100 receives a predetermined instruction during execution of the sub-screen utilization software, the electronic device 100 ends the sub-screen utilization software and starts a home application. In the present embodiment, the operation screen of the sub screen utilization software includes a home button, and electronic device 100 activates a home application in response to a touch on the home button.

(Handwriting input pad)
The operation of electronic device 100 that executes the handwritten character input pad (specifically, CPU 210 that executes the application) will be described with reference to FIG. FIG. 30 is a diagram illustrating an example of a screen (character input screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the handwritten character input pad.

  Referring to FIG. 30, the character input screen includes a center guidance display 814, a mouse disabled display 830, a home button 840, a text box 2801, a paste button 2802, a candidate area 2803, a backward button 2804, handwriting. An area 2805, a recognition mode switching button 2806, a recognition button 2807, and an erase button 2808 are included.

  A text box 2801 displays characters that have been confirmed as a result of handwritten character recognition. The text box 2801 can display up to 10 confirmed characters. However, the maximum number of characters displayed in the text box 2801 is not limited to this.

  When the paste button 2802 is pressed when there is a character string (one or more characters) in the text box 2801, the electronic device 100 displays the character string on the active application displayed on the liquid crystal panel 140. Send. When the character string is transmitted, the contents of the text box 2801 are automatically cleared (without a special operation from the user).

  The paste button 2802 functions as an Enter key when there is no character string in the text box 2801. For example, the user can press the paste button 2802 to transmit a character string to the search box, and then press the paste button 2802 again to cause the application to execute a search. If there is no character string in text box 2801 (or paste button 2802 functions as an Enter key), electronic device 100 changes the character displayed on paste button 2802 to “Enter”.

  The candidate area 2803 displays input recognition candidates. In the present embodiment, the candidate area 2803 displays a maximum of five recognition candidates in order from the first candidate. However, the maximum number of candidates displayed in the candidate area 2803 is not limited to this.

  In the present embodiment, electronic device 100 automatically adds the first candidate (the first character in candidate area 2803) to text box 2801. In addition, electronic device 100 can change the added character in accordance with selection of a candidate in candidate area 2803. Since there is a high possibility that the first candidate is a character to be input by the user, the number of operations by the user can be reduced by automatically adding the first candidate to the text box 2801.

  When the backward button 2804 is pressed, the electronic device 100 erases the last character of the character string in the text box 2801. If there is no character string in text box 2801, electronic device 100 does not perform an operation associated with pressing back button 2804.

  The handwriting area 2805 accepts input from the outside. Electronic device 100 creates handwritten character data 432 a corresponding to the history of input to handwriting area 2805 and stores it in storage unit 430. For example, the electronic device 100 creates, as handwritten character data 432a, all coordinates that have been input within a predetermined time, or coordinates at the start and end of input that are temporally continuous within a predetermined time. In addition, electronic device 100 displays a graphic corresponding to coordinates (or handwritten character data 432a) for which input has been received in handwritten area 2805.

  In the present embodiment, handwriting area 2805 includes two regions (area 2805a and area 2805b). Electronic device 100 creates handwritten character data 432a for each of area 2805a and area 2805b.

  The recognition mode switching button 2806 switches the recognition mode for handwriting input. In the present embodiment, there are two recognition modes, “automatic mode” and “manual mode”. It is assumed that the recognition mode when the handwriting input pad is first activated is the automatic mode.

  In the “automatic mode”, electronic device 100 automatically starts character recognition for input to handwriting area 2805 after a predetermined time has elapsed since pen-up (end of input to handwriting area 2805). The automatic mode has an advantage that the number of user operations can be reduced. In the automatic mode, electronic device 100 may automatically start character recognition after a predetermined time from the start of input to another event, for example, handwriting area 2805, instead of pen-up.

  In the “manual mode”, the electronic device 100 does not start character recognition until the recognition button 2807 is pressed. The manual mode has an advantage that the user can calmly input characters.

  When the recognition button 2807 is pressed, the electronic device 100 starts character recognition of the handwritten character data 432a based on the input to the input area 2805. In the automatic mode, if the recognition button 2807 is pressed before the recognition opening time, the electronic device 100 starts character recognition of the handwritten character data 432a.

  When delete button 2808 is pressed, electronic device 100 deletes the graphic and handwritten character data 432a displayed in input area 2805. The erase button 2805 is used when a user rewrites a character handwritten before character recognition.

  The usage method of a handwritten character input pad is demonstrated with reference to FIG. 31 and FIG. FIG. 31 and FIG. 32 are diagrams for explaining the operation of electronic device 100 when the handwritten character input pad is used. Here, a description will be given of an operation example in the case where the user searches for “Fugu” using the liquid crystal panel 240 while using an application (such as a Web browser) that displays a screen including a search box.

  FIG. 31A shows a screen displayed on the liquid crystal panel 140 and the liquid crystal panel 240 in the mouse mode. At this time, the liquid crystal panel 240 displays a mouse screen 2920. The liquid crystal panel 140 displays a main application operation screen 2910. Operation screen 2910 includes a search box 2912 and a search start button 2914. It is assumed that the search box 2912 is activated by the mouse operation accompanying the movement of the finger 900 on the liquid crystal panel 240.

  FIG. 31B shows a display screen 2930 of the liquid crystal panel 240 after a mode switching instruction (specifically, pressing of the center key 242) in the state shown in FIG.

Here, display screen 2930 is a home menu screen.
FIG. 31C shows the display on the liquid crystal panel 240 after the button for calling the handwritten character input pad (encircled in FIG. 31B) is pressed in the state shown in FIG. A screen 2940 is shown. Display screen 2940 is a character input screen.

  FIG. 31D shows a display screen 2950 of the liquid crystal panel 240 when a handwriting input is made on the display screen 2940 by the stylus 950. In the input area of the display screen 2950, a graphic 2952 corresponding to handwriting input is displayed.

  FIG. 31 (e) shows a display screen 2960 of the liquid crystal panel 240 when the stylus 950 shown in FIG. 31 (d) is separated from the liquid crystal panel 240. Candidate characters corresponding to handwriting input are displayed in the candidate area of the display screen 2950. In the text box, the first candidate character 2962 (here, the character “kawa”) is displayed.

  FIG. 31F shows a display screen 2970 of the liquid crystal panel 240 when the first candidate character 2962 is confirmed. Character 2972 is a confirmed character.

  FIG. 32G is a diagram showing the display screen 3010 of the liquid crystal panel 240 in a state where the user performs handwriting input with the stylus 950 after the time shown in FIG. Display screen 3010 includes a graphic 3012 corresponding to handwriting input.

  FIG. 32H shows a display screen 3020 of the liquid crystal panel 240 when the stylus 950 shown in FIG. Candidate characters corresponding to handwriting input are displayed in the candidate area of the display screen 2950. In the text box, the first candidate character 3022 (here, the character “knee”) is added to the right side of the already determined character “river”.

  FIG. 32I shows a display screen 3030 of the liquid crystal panel 240 when the character 3032 (“pig”) in the candidate area is pressed by the stylus 950 in the state shown in FIG. As the character 3032 is pressed, the character “knee” displayed in the text box changes to “pig” (character 3034).

  FIG. 32J shows the display screen 3040 of the liquid crystal panel 140 and the display screen 3050 of the liquid crystal panel 240 after the character 3034 (“pig”) is confirmed. Display screen 3040 includes a search box 3042 and a search start button 3044. After the character 3034 is confirmed, when the paste button in the display screen 3050 is pressed by the stylus 950, the character string “Kawaguchi” in the text box is input to the active search box 3042. Accordingly, the search string 3042 displays the character string “Kawaguchi”.

  FIG. 32 (k) shows the display screen 3060 of the liquid crystal panel 140 and the display screen 3070 of the liquid crystal panel 240 after the Enter button (paste button) is pressed after the state shown in FIG. 32 (j). Pressing the Enter button in the display screen 3070 has the same effect as pressing the search start key 3044. In other words, the main application executes a search for the character string “Kawaguchi” in the search box 3042 in response to pressing of the Enter button, and displays a display screen 3060 indicating the search result on the liquid crystal panel 140.

  In the present embodiment, when the text box is full (the maximum number of characters that can be input is input in the text box), electronic device 100 does not accept any more handwriting. The operation of electronic device 100 when the text box is full will be described with reference to FIG. FIG. 33 is a diagram for explaining the operation of electronic device 100 when the text box is full.

  FIG. 33A shows a state where the user is trying to input to the input area with the stylus 950 when the text box is already full (10 characters are included in the text box). At this time, a character string 3112 (a character string of 10 characters “Aiue Okeki Keko”) is displayed in the text box of the display screen 3110 of the liquid crystal screen n240.

  When the stylus 950 contacts the liquid crystal panel 240, the electronic device 100 displays a screen 3120 including a warning display 3122 on the liquid crystal panel 240 (see FIG. 33B). The warning display 3122 includes a character string that prompts the user to confirm the character, that is, press the paste button. Here, the character string included in the warning display 3122 is “Please touch the paste button”, but the character string is not limited to this.

  After the warning display 3122 is displayed, the liquid crystal panel 240 displays a display screen 3130 similar to the display screen 3110 (see FIG. 33C). While the display screen 3130 is displayed, the user can continue character input by pressing the paste button to confirm the character string or by pressing the backward button to delete the already input character. For example, the electronic device 100 automatically changes from the state shown in FIG. 33B to FIG. 33C after a predetermined time has elapsed from the display of the warning display 3122 or in response to some instruction to the liquid crystal panel 240. ) State.

  In the present embodiment, handwritten character data 432a is temporarily stored in RAM 271 or the like, and electronic device 100 discards handwritten character data 432a when the handwritten character input pad ends. Therefore, when the handwritten character input pad is called again, the user can input a new character.

  However, the electronic device 100 may hold the handwritten character data 432a so that the user can perform handwriting input from the last time when the handwriting input pad is used again. In this case, electronic device 100 displays a graphic corresponding to handwritten character data 432a on liquid crystal panel 240 based on handwritten character data 432a when the handwritten character input pad is resumed.

  The user may be able to select which of the above two operations (whether or not the handwritten character data 432a is held) is performed on the handwritten character input pad. In this case, the user can appropriately determine which of the above two operations is performed by the handwritten character input pad according to the usage mode of the handwritten character input.

(Hand-drawn illustration input pad)
The operation of electronic device 100 that executes the hand-drawn illustration input pad (specifically, CPU 210 that executes the application) will be described with reference to FIG. FIG. 34 is a diagram illustrating an example of a screen (illustration input screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the hand-drawn illustration input pad.

  Referring to FIG. 34, the illustration input screen includes a center guidance display 814, a mouse disabled display 830, a home button 840, a drawing area 3201, an undo button 3202, a pen / ruler / eraser button 3203, Pen thickness button 3204, pen color button 3205, stamp button 3206, frame button 3207, delete all button 3208, screen capture button 3209, mail attachment button 3210, file save button 3211, and paste button 3212 Including.

  The drawing area 3201 accepts input from the outside. The electronic device 100 creates the illustration data 432b based on the input to the drawing area 3201 and the drawing settings (input tool, pen thickness, pen color, etc.) and stores the illustration data 432b in the storage unit 430. Similar to the handwritten character data 432a, the illustration data 432b includes all the coordinates that have been input within a predetermined time, or the coordinates at the start and end of input that are temporally continuous within the predetermined time. The illustration data 432b further includes data related to drawing settings (input tool, pen thickness, pen color, etc.).

  In the present embodiment, the ratio of the horizontal length to the vertical length of the drawing area 3201 is 4: 3. This is because the photograph is also processed using the handwritten illustration input pad. However, the aspect ratio of the drawing area 3201 is not limited to this.

  An undo button 3202 is a button for canceling the previous input operation to the drawing area 3201. The electronic device 100 manages the inputs recorded in the input history 432 in order of time, and when the undo button 3202 is pressed, the previous input is deleted from the input history 432. At the same time, the corresponding drawing portion is deleted from the drawing area 3201.

  A pen / ruler / eraser button 3203 is a button for selecting a drawing tool for the drawing area 3201. In response to pressing of the pen / ruler / eraser button 3203, the electronic device 100 switches the drawing tool in the order of pen, ruler, eraser, pen,.

  The pen thickness button 3204 is a button for setting the pen thickness. In response to pressing of the pen thickness button 3204, the electronic device 100 changes the setting of the thickness of a line drawn according to the input to the drawing area 3201 when the pen tool is selected. Alternatively, electronic device 100 may display on LCD panel 240 a screen for allowing the user to set the thickness of the line when pen thickness button 3204 is pressed.

  A pen color button 3205 is a button for setting a pen color. In response to pressing of the pen color button 3205, the setting of the color of the line to be drawn is changed according to the input to the drawing area 3201 when the pen tool is selected. Alternatively, electronic device 100 may display on LCD panel 240 a screen for allowing the user to set a line color when pen color button 3205 is pressed.

  The stamp button 3206 is a button for causing a stamp to be drawn in the drawing area 3201 in response to an input to the drawing area 3201.

  The frame button 3207 is a button for adding a frame such as a decorative frame to the illustration drawn in the drawing area 3201.

  The delete all button 3208 is a button for deleting all the illustration data 432b. By pressing this button, the user can set the drawing area 3201 to the state (plain color) when the hand-drawn illustration input pad is activated.

  A screen capture button 3209 is a button for displaying a part of the screen displayed on the liquid crystal panel 140 in the drawing area 3201. The mail attachment button 3210 is a button for attaching the illustration data 432b to an electronic mail. The file save button 3211 is a button for saving the illustration data 432b in a designated storage area. The area for storing the illustration data 432b may be fixed or may be designated by the user.

  The paste button 3212 is a button for sending the illustration data 432b to the active main application. When the paste button 3212 is pressed, the electronic device 100 provides the active main application with the illustration data 432b created by the hand-drawn illustration input pad. For example, the user can insert an illustration into a document being created by the main application using the hand-drawn illustration input pad.

  In the present embodiment, as described with reference to FIG. 25, when the hand-drawn illustration input pad receives an instruction to end the hand-drawn illustration input pad, it determines whether or not the created hand-drawn illustration data 432b needs to be saved. Inquire. However, this inquiry is not essential. The hand-drawn illustration input pad may automatically discard the hand-drawn illustration data 432b created so far at the end.

(Calculator / Number input pad)
The operation of electronic device 100 that executes the calculator / number input pad (specifically, CPU 210 that executes the application) will be described with reference to FIG. FIG. 35 is a diagram illustrating an example of a screen (calculator screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the calculator / number input pad.

  Referring to FIG. 35, the calculator screen includes a center guidance display 814, a mouse disabled display 830, a home button 840, a number box 3301, a pasting button 3302, a number button 3303, and a function button 3304.

  The number box 3301 displays the input number or the number of the calculation result. It is assumed that the maximum number of numbers that can be displayed in the number box 3301 is eight. However, the maximum number is not limited to eight.

  When paste button 3302 is pressed, electronic device 100 transmits the number displayed in number box 3301 to the active application displayed on liquid crystal panel 140.

  A number button 3303 is a button for inputting a number in the number box 3301. The function button 3304 is a button for instructing a predetermined calculation such as four arithmetic operations. Since the operation of electronic device 100 when numeric button 3303 and function button 3304 are pressed is the same as the operation of a normal calculator or a calculator application, detailed description thereof will not be repeated here.

(the Internet)
The operation of electronic device 100 (specifically, CPU 210 that executes an application) that executes Web page calling software that is one of the sub-screen utilization software will be described with reference to FIG. FIG. 36 is a diagram illustrating an example of a screen (Internet screen) displayed on the liquid crystal panel 240 by the electronic device 100 while the Web page calling software is activated.

  Referring to FIG. 36, the Internet screen includes a guidance display 810 (left guidance display 812, center guidance display 814, and right guidance display 816), a plurality of operation button displays 820, a mouse disabled display 830, and a home button 840. including.

  Each of the operation button displays 820 corresponds to a web page to be called. Each operation button display 820 includes characters representing the name of the corresponding Web page (such as “Internet 2” in the figure). When electronic device 100 detects a touch on an area corresponding to operation button display 820, electronic device 100 activates a web browser and displays the selected web page on liquid crystal panel 140 (or liquid crystal panel 240).

  Note that the web page calling software may display a scroll bar on the liquid crystal panel 240. An example of an Internet screen including a scroll bar is shown in FIG. Referring to FIG. 37, the Internet screen includes a scroll bar 3500. When the user drags the slider 3502 in the scroll bar 3500, the Web page calling software scrolls the Internet screen.

(dictionary)
The operation of electronic device 100 (specifically, CPU 210 that executes an application) that executes dictionary calling software, which is one of the sub-screen utilization software, will be described with reference to FIG. FIG. 38 is a diagram illustrating an example of a screen (dictionary selection screen) displayed on the liquid crystal panel 240 by the electronic device 100 during activation of the dictionary calling software.

  Referring to FIG. 38, the dictionary selection screen includes a guidance display 810 (a left guidance display 812, a center guidance display 814, and a right guidance display 816), a plurality of operation button displays 820, a mouse disabled display 830, and a home button 840. Including.

  Each of the operation button displays 820 corresponds to the electronic dictionary to be called. Each operation button display 820 includes characters representing the name of the corresponding electronic dictionary (such as “English-Japanese dictionary” in the figure). When electronic device 100 detects a touch on an area corresponding to operation button display 820, electronic device 100 activates the electronic dictionary and displays the screen of the activated electronic dictionary on liquid crystal panel 140 (or liquid crystal panel 240).

  The dictionary calling software may display a scroll bar on the liquid crystal panel 240. An example of a dictionary selection screen including a scroll bar is shown in FIG. Referring to FIG. 39, the dictionary selection screen includes a scroll bar 3700. When the user drags slider 3702 in scroll bar 3700, the dictionary calling software scrolls the dictionary selection screen.

<Process flow>
(Basic flow)
With reference to FIG. 40, the flow of processing performed by electronic device 100 according to the present embodiment will be described. FIG. 40 is a diagram illustrating a flow of processing performed by the electronic device 100 in a flowchart format. In FIG. 40, the processing performed by the control unit 350 in the first unit 1001 and the processing performed by the control unit 450 in the second unit 1002 are collectively shown.

  In step S <b> 101, the control unit 350 and the control unit 450 perform a normal activation process or a resumption process when receiving a normal activation instruction or a resumption instruction of the electronic device 100.

  Normal startup is startup from a power-off state, as already described. Examples of the normal startup process performed by the control unit 350 include a boot process and a boot screen display on the liquid crystal panel 140. The normal startup process performed by the control unit 450 includes displaying a boot screen on the liquid crystal panel 240.

  Control unit 350 and control unit 450 handle pressing of a predetermined button (such as power switch 191) in the power-off state as a normal activation instruction. One control unit (control unit 450 or 350) accepts a normal activation instruction, performs its own normal activation process, and instructs the other control unit (control unit 350 or 450) to perform a normal activation process. The structure to give may be sufficient.

  As already described, the restart is a start from the power saving state. The restart process performed by the control unit 350 includes reading of a work state stored in the RAM 171 or the HDD 170 or displaying a restarting screen on the liquid crystal panel 140. The restart process performed by the control unit 450 includes reading of a work state stored in the RAM 271 or the HDD 170 or displaying a restarting screen on the liquid crystal panel 240.

  The control unit 350 and the control unit 450 handle pressing of a predetermined button (such as the power switch 191) in the power saving state or touching the liquid crystal panel 240 as a normal activation instruction. One control unit (450 or 350) may perform the restart process in response to an instruction from the other control unit (350 or 450) that has received the restart instruction.

  In step S103, the mode setting unit 454 included in the control unit 450 determines the operation mode to be either the mouse mode or the tablet mode.

  In the present embodiment, the operation mode for normal activation is determined to be the mouse mode. In the case of normal activation, mode setting unit 454 sets mode data 437 in storage unit 430 to data representing a mouse mode.

  The mode setting unit 454 determines the mode data 437 and the mode data 437 before the power saving state based on the operation mode when restarting. In this case, the control unit 450 stores the mode data 437 before the power saving state in the RAM 271 or the like when shifting to the power saving state or at the time of restarting. Alternatively, the mode setting unit 454 may always set the mode data 437 to data representing the mouse mode when resuming. In this case, the controller 450 does not have to perform the storage process of the mode data 437 before the power saving state.

  In step S105, the input processing unit 452 determines whether the operation mode is the mouse mode. That is, the input processing unit 452 determines whether the operation mode is the mouse mode based on the mode data 437.

  If the operation mode is the mouse mode (YES in step S105), control unit 350 and control unit 450 proceed to the mouse mode operation in step S107. If the operation mode is not the mouse mode (NO in step S105), control unit 350 and control unit 450 proceed to the tablet mode operation in step S113.

  In step S107, control unit 350 and control unit 450 perform a mouse mode operation. That is, the control unit 350 and the control unit 450 control each unit of the electronic device 100 so that the input to the liquid crystal panel 240 causes the mouse operation of the main application. Details of the mouse mode operation will be described later.

  In step S109, the mode setting unit 454 determines whether a mode switching instruction has been accepted. Specifically, mode setting unit 454 determines whether a signal in response to pressing of center key 242 has been received. However, the mode switching instruction is not limited to pressing the center key 242.

  If there is a mode switching instruction during the mouse mode (YES in step S109), control unit 450 executes a switching process from the mouse mode to the tablet mode in step S111. When there is no mode switching instruction (NO in step S109), control unit 350 and control unit 450 repeat the processing from step S107 (mouse mode operation).

  In step S111, the control unit 450 executes a switching process from the mouse mode to the tablet mode. For example, in step S111, the control unit 450 displays the operation screen of the sub application on the liquid crystal panel 240 and switches the operation of the panel input processing unit 453. Details of the switching process from the mouse mode to the tablet mode will be described later. After step S111 ends, control unit 350 and control unit 450 proceed to the tablet mode operation of step S113.

  In step S113, control unit 350 and control unit 450 perform a tablet mode operation. That is, the control unit 350 and the control unit 450 control each unit of the electronic device 100 so that the sub application operates in response to an input to the liquid crystal panel 240. Details of the tablet mode operation will be described later.

  In step S115, mode setting unit 454 determines whether a mode switching instruction has been accepted. Specifically, mode setting unit 454 determines whether a signal in response to pressing of center key 242 has been received. However, the mode switching instruction is not limited to pressing the center key 242.

  If there is a mode switching instruction during the tablet mode (YES in step S115), control unit 450 executes a switching process from the tablet mode to the mouse mode in step S117. When there is no mode switching instruction (NO in step S115), control unit 350 and control unit 450 repeat the processing from step S113 (tablet mode operation).

  In step S117, the control unit 450 executes switching processing from the tablet mode to the mouse mode. For example, the control unit 450 displays the mouse screen on the liquid crystal panel 240 and switches the operation of the panel input processing unit 453 in step S117. Details of the switching process from the tablet mode to the mouse mode will be described later. After completion of step S117, control unit 350 and control unit 450 proceed to the mouse mode operation of step S107.

  Note that the control unit 350 and the control unit 450 perform a power-off process or a power-saving state transition process when receiving a power-off instruction or a power-saving state transition instruction. These processes are interrupt processes, and are performed after any of the steps in FIG. However, these processes are not shown in FIG.

(Mouse mode operation)
The mouse mode operation in step S107 in FIG. 40 will be described in detail with reference to FIG. FIG. 41 is a flowchart illustrating the processing flow of the mouse mode operation.

  First, the operation of the control unit 350 on the first unit 1001 side will be described. The operation flow of the control unit 350 is shown on the left side in FIG.

  In step S <b> 201, the control unit 350 acquires coordinate data from the interface unit 340. The coordinate data is transmitted from the control unit 450 on the second unit 1002 side to the interface unit 440 via the interface unit 440.

  In step S203, control unit 350 determines the cursor position based on the coordinate data. More specifically, the program execution unit 358 included in the control unit 350 performs the process of step S203. The program execution unit 358 executes the program 334 to determine the cursor position.

  In step S <b> 205, the control unit 350 acquires a command from the interface unit 340. This command is transmitted from the control unit 450 on the second unit 1002 side to the interface unit 340 via the interface unit 440.

  In step S207, control unit 350 performs an application operation according to the command. Specifically, program execution unit 358 executes program 334 to perform application operations. The program execution unit 358 determines an application operation based on the application type, the cursor position, and the command type.

  The application operation is similar to the operation associated with a mouse click of a currently popular application. The application operation includes, for example, selection or activation of a file or folder at the cursor position, or execution of processing according to a button (minimum (maximum) button, close button, etc.) at the cursor position.

  Next, the operation of the control unit 450 on the second unit 1002 side will be described. The flow of the operation of the control unit 450 is shown on the right side in FIG.

  In step S301, the panel input processing unit 453 included in the input processing unit 452 of the control unit 450 acquires a scan image from the liquid crystal panel 240 (panel input unit 422).

  In step S303, the panel input processing unit 453 calculates coordinate data specifying the input position to the liquid crystal panel 240 based on the scanned image acquired in step S301.

  In step S305, the panel input processing unit 453 controls the interface unit 440 and transmits coordinate data to the interface unit 340 on the first unit 1001 side.

  In step S307, the panel input processing unit 453 determines whether a predetermined scan cycle time has elapsed since the execution of step S301. If the scan cycle time has elapsed (YES in step S307), panel input processing unit 453 repeats the processing from step S301 (scanned image acquisition). If the scan cycle time has not elapsed (NO in step S307), control unit 450 proceeds to the process in step S309.

  In step S309, the input processing unit 452 determines whether there has been a click operation. Specifically, the input processing unit 452 determines whether the left click key 241 or the right click key 243 has been pressed. Note that the input processing unit 452 may determine that the tap operation on the liquid crystal panel 240 is a click operation. Specifically, the input processing unit 452 determines that a tap operation has been performed when the detection of an external object is started and ended in a specific area in the liquid crystal panel 240 within a predetermined short time.

  If there is no click operation (NO in step S309), control unit 450 repeats the processing from step S307. If there is a click operation (YES in step S309), control unit 450 proceeds to the process of step S311.

  In step S311, the input processing unit 452 controls the interface unit 440, and transmits a command to the interface unit 340 on the first unit 1001 side according to the click operation in step S309.

  Here, it is assumed that the input processing unit 452 determines the type of command according to the type of click operation. For example, the input processing unit 452 transmits different commands when the left click key 241 is pressed and when the right click key 243 is pressed.

  In the above, the command transmission by the click operation has been described. However, the operation that causes the command transmission is not limited to the click operation. For example, the input processing unit 452 may transmit a command corresponding to a double click, drag, multi-touch, gesture operation, or the like.

(Tablet mode operation)
The tablet mode operation in step S109 of FIG. 40 will be described in detail with reference to FIG. FIG. 42 is a flowchart illustrating the processing flow of the tablet mode operation.

  First, the operation of the control unit 350 on the first unit 1001 side will be described. The operation flow of the control unit 350 is shown on the left side in FIG.

  In step S <b> 401, the control unit 350 acquires data from the interface unit 340. The “data” here is created by the control unit 450 on the second unit 1002 side executing the sub-application.

  Specifically, the “data” is, for example, character (or number) data or illustration data. The data may be a command. For example, when “Enter” on the operation screen of the handwritten character input pad is touched, the control unit 450 creates a command.

  In step S403, control unit 350 performs an application operation corresponding to the data. Specifically, program execution unit 358 executes program 334 to perform application operations. The program execution unit 358 processes the data acquired in step S401 by the main application being executed. When step S403 ends, the control unit 350 returns to the process of step S401.

  Next, the operation of the control unit 450 on the second unit 1002 side will be described. The operation flow of the controller 450 is shown on the right side in FIG.

  In step S501, the panel input processing unit 453 included in the input processing unit 452 of the control unit 450 acquires a scan image from the liquid crystal panel 240 (panel input unit 422).

  In step S503, the panel input processing unit 453 calculates coordinate data specifying the input position to the liquid crystal panel 240 based on the scanned image acquired in step S501.

  In step S505, the panel input processing unit 453 determines whether a predetermined scan cycle time has elapsed since the execution of step S501. When the scan cycle time has elapsed (YES in step S505), panel input processing unit 453 repeats the processing from step S501 (scanned image acquisition). If the scan cycle time has not elapsed (NO in step S505), control unit 450 proceeds to the process in step S507.

  In step S507, control unit 450 performs a sub-application operation. The sub application operation includes operation execution by time count. In step S507, the control unit 450 may control the interface unit 440 to transmit data to the first unit 1001 side.

  In step S <b> 509, the control unit 450 stores data (referred to as “operation element”) for determining the operation of the application being executed in the storage unit 430. In the present embodiment, the operation parameter 435 including the elapsed time from the event and the input history 432 correspond to the operation elements. Note that the control unit 450 performs the processing in step S509 at a predetermined time interval, when changing an operating element, when saving an operating element (for example, when saving an illustration), and the like.

(Mode switching: mouse mode to tablet mode)
Mode switching (from mouse mode to tablet mode) in step S107 in FIG. 40 will be described in detail with reference to FIG. FIG. 43 is a flowchart illustrating the process flow of the mode switching (from the mouse mode to the tablet mode) operation.

  In step S601, the mode setting unit 454 included in the control unit 450 loads the operation definition of the immediately preceding application stored in the storage unit 430.

  Here, the “immediate application” is a sub application that was last operated in the tablet mode before the mode setting unit 454 received a mode switching instruction to the tablet mode. In the present embodiment, since the sub application continues to operate even in the mouse mode, the immediately preceding application is the same as the sub application operating at the time of mode switching instruction.

  In step S603, the program execution unit 458 included in the control unit 450 resumes the execution of the operation of the sub application that has been executed just before based on the operation definition loaded in step S601. Then, the program execution unit 458 controls the display control unit 456 to display the operation screen of the sub application on the display unit 410.

  However, the program execution unit 458 may perform the process of step S601 at any time during the process of step S603. That is, during the execution of the immediately preceding application, the operation definition stored in the storage unit 430 may be read as necessary, and the sub application may be executed based on the read data.

  In step S605, the panel input processing unit 453 included in the control unit 450 switches the processing method of the signal from the panel input unit 422. That is, the panel input processing unit 453 converts the signal from the panel input unit 422 into an operation instruction of the sub application.

(Mode switching: tablet mode to mouse mode)
From here, the mode switching (from tablet mode to mouse mode) in step S113 of FIG. 40 will be described in detail.

  FIG. 44 is a flowchart showing a first processing flow of mode switching (tablet mode to mouse mode) operation.

  The electronic device 100 is configured to load the operation definition of the immediately preceding application when the operation mode is switched from the mouse mode to the tablet mode. In this case, on the liquid crystal panel 240 immediately after switching to the tablet mode, a screen that was displayed immediately before switching to the mouse mode during the previous tablet mode execution is displayed.

  The operation definition is stored in the storage unit 430 including the RAM 271 and the like when the operation mode is switched from the tablet mode to the mouse mode by pressing the center key 242 or the like.

  Specifically, at the time of such mode switching, the program execution unit 458 displays, in step S701, information that identifies the application that is being executed (immediate application) and the operation content at that point in time for the operation definition. Is stored in the RAM 271.

  In step S <b> 703, when the display control unit 456 included in the control unit 450 receives a signal indicating that a mode switching instruction has been received from the mode setting unit 454, the display control unit 450 transmits a mouse to the liquid crystal panel 240 based on the display data 433. Display the mode screen.

  In step S <b> 705, the panel input processing unit 453 included in the control unit 450 switches the processing method of the signal from the panel input unit 422. That is, the panel input processing unit 453 converts the signal from the panel input unit 422 into a sub-application operation instruction.

  The operation definition stored as described above is read and loaded when the operation mode is switched from the mouse mode to the tablet mode (step S601 in FIG. 43). If the operation definition is not stored (initialized state), the home application is activated in step S601.

  The stored operation definition is preferably initialized when the power of the electronic device 100 is turned off or when the electronic device 100 is restarted. As a result, when the tablet mode is executed for the first time after the electronic device 100 is normally activated, restarted, or restarted, the home menu is first displayed on the liquid crystal panel 240.

  In the electronic device 100, it is preferable that the operation definition is initialized when the home application is activated by, for example, touching the home button after the sub-application is executed. In this case, if the operation mode is switched to the mouse mode at this time, the home application display screen is displayed when the tablet mode is restored. This is because the home application is activated in step S601 if the action definition has been initialized. Therefore, when the home application is activated, it is not necessary to store the current tablet mode operation definition when switching to the mouse mode.

<Processing related to the sub application “book”>
In the present embodiment, in electronic device 100, when the operation mode is switched from the mouse mode to the tablet mode, information corresponding to the operation content in the previous tablet mode is displayed on liquid crystal panel 240.

  For example, in the sub application “book”, the history information of the electronic book selected to be viewed by the book viewer in the sub application so far is stored, and the display based on the history information is displayed on the liquid crystal panel 240. Do. The operation according to the sub application “book” will be described below.

  Referring to FIG. 45, when the user selects operation button display 820x ("book" of the sub screen utilization software) on home menu screen 800a, electronic device 100 executes "book" of the sub application and displays screen 800e. Displayed on the liquid crystal panel 240. A screen 800e is an operation screen of this application.

  The screen 800e includes a left guidance display, a center guidance display, and a right guidance display (a left guidance display 812, a center guidance display 814, a right guidance display 816, such as FIG. 37).

  Further, on the screen 800e, characters “book” indicating the sub-application during execution are displayed in the upper center of the screen 800, and characters “title order” and “history order display” are displayed on the left side. ing.

  Further, in the center of the screen 800e, characters of the name of each electronic book such as “electronic book 5”, “electronic book 4”, and “electronic book 1” indicating the name of the electronic book to be viewed in the book viewer are displayed. A softkey that displays a column is displayed. On the screen 800e, the soft keys representing the respective electronic books are arranged in order from the top according to the history selected in the “book” application.

  In this case, in the electronic device 100, as shown in FIG. 46, book information 432c is stored as input data 431 of the storage unit 430.

  The book information 432c includes “book name information” shown in Table 1 and “book history information” shown in Table 2.

  The book name information is information in which information for specifying an electronic book that can be selected as a browsing target in the book viewer is arranged in the order of the names, for example.

  In the “book” application, the book history information is information in which information for specifying an electronic book selected as an object to be browsed in the book viewer is arranged in the newly selected order.

  As an operation screen of the “book” application, by default, information on each electronic book is displayed as a soft key in the order according to the book history information described above.

  The information displayed in each soft key may be the name of the electronic book as shown on the screen 800e, or may be an image corresponding to the electronic book.

  On the operation screen of the “book” application, the display arrangement of the soft keys of each electronic book can be changed between one corresponding to the book name information and one corresponding to the book history information.

  That is, on the operation screen, character strings of “title order” and “history order display” are displayed, but when the soft keys are displayed in an arrangement corresponding to the book name information, “title order” is Although displayed as a simple character string, “history order display” is displayed as a soft key. When the “history order display” soft key is selected, the display arrangement of the soft keys of each electronic book is changed according to the book history information. When the soft keys are displayed in an arrangement corresponding to the book history report, “title order” is displayed as a soft key, and “history order display” is displayed as a simple character string. When the “Title Order” soft key is selected and operated, the display arrangement of the soft keys of each electronic book is changed according to the book name information.

  A soft key “XXXXX books” is displayed at the top of the screen 800e. The soft key is operated to connect the electronic device 100 on the network to a site where the electronic book can be downloaded. When the “◯◯◯◯ books” soft key is operated, data for connecting to the site is transmitted from the control unit 450 to the first unit 1001 (step S507). As a result, the electronic device 100 is connected to the site and the liquid crystal panel 140 displays the homepage screen of the site.

  On the other hand, when a soft key for selecting an electronic book in the screen 800e is operated, the book viewer is activated from the control unit 450 to the first unit 1001, and the electronic book corresponding to the selected soft key is viewed. Data for this is transmitted (step S507). As a result, in the electronic device 100, the selected electronic book viewing screen of the book viewer is displayed on the liquid crystal panel 140. In this case, for example, control unit 450 switches the operation to the mouse mode, and accepts operations such as page turning for the book viewer to liquid crystal panel 240.

  The book history information is updated each time an electronic book is selected in the “book” application. For example, when an electronic book “electronic book 6” is selected from the state shown in Table 2, the book history information is updated to that shown in Table 3. In response to this, on the operation screen of the “book” application, the arrangement of the soft keys corresponding to the electronic book is changed as displayed on the screen 800f of FIG.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  DESCRIPTION OF SYMBOLS 100 Electronic device, 100A, 100B Case, 100C Hinge, 100D Concave part, 101,104 Main body apparatus, 102,102A, 103 Display apparatus, 130,230 Driver, 131 Operation signal line drive circuit, 132 Data signal line drive circuit, 133 Photosensor drive circuit, 134 switch, 135 amplifier, 140, 140A, 240 liquid crystal panel, 141 pixel circuit, 141b, 141g, 141r subpixel circuit, 143 electrode pair, 143a pixel electrode, 143b counter electrode, 144 photosensor circuit, 145 , 145b, 145g, 145r photodiode, 146 capacitor, 151A active matrix substrate, 151B counter substrate, 152 liquid crystal layer, 153b, 153g, 153r color filter, 157 data signal line, 161 Polarizing filter, 162 Glass substrate, 163 Light shielding film, 164 Alignment film, 173 Memory card reader / writer, 174,274 External communication unit, 175 Microphone, 176 Speaker, 177 Operation key, 179,279 Backlight, 180,280 Image processing Engine, 181, 281 Driver control unit, 182, 282 Timer, 183, 283 Signal processing unit, 191 Power switch, 192 Power circuit, 193, 293 Power detection unit, 194, 294 connector, 195, 295 Antenna, 196 connector, 297 Signal strength detection unit, 310, 410 display unit, 320, 420 input unit, 330, 430 storage unit, 340, 440 interface unit, 350, 450 control unit, 432c book history, 900 fingers, 1001, 1001A 1st unit, 1002 2nd unit.

Claims (6)

First display means;
A second display means;
Storage means;
Control means for controlling the display mode of the first and second display devices,
The second display means is a display-integrated tablet capable of receiving input from the outside,
The control means includes
A first mode in which a screen created by processing executed in response to an input to the tablet is displayed on the first display means, and a screen created by processing executed in response to an input to the tablet is displayed in the second mode. It is possible to operate in the second mode displayed on the means,
When switching the operation mode from the second mode to the first mode, operation information that is information for specifying the content of the operation in the second mode is stored in the storage unit,
An electronic apparatus that causes the second display unit to display information corresponding to the operation information stored in the storage unit when switching the operation mode from the first mode to the second mode.   When the control unit switches the mode to operate from the first mode to the second mode, the control unit displays a screen including an arrangement of soft keys as information according to the operation information stored in the storage unit. The electronic device according to claim 1, wherein the electronic device is displayed on a second display unit. In the second mode, content can be selected,
The storage means stores, as the operation information, information relating to a history of content selection in the second mode,
The electronic device according to claim 2, wherein the soft key is a soft key for selecting content arranged in correspondence with information on a history of selection of the content stored in the storage unit. The storage means stores information specifying content that can be selected in the second mode in an order according to the name,
The control means receives an input of information for selecting whether to arrange the soft keys in correspondence with information on a history of selection of the content or in an order according to the name. The electronic device described. First display means, second display means comprising a display-integrated tablet capable of accepting external input, operation means, and control means for controlling display modes of the first and second display devices An electronic device control method comprising:
Operating in a first mode in which a screen created by processing executed in response to an input to the tablet is displayed on the first display means;
Operating in a second mode for causing the second display means to display a screen created by processing executed in response to an input to the tablet;
Storing the operation information, which is information for specifying the content of the operation in the second mode, in the storage means when switching the operation mode from the second mode to the first mode;
And displaying the information corresponding to the operation information stored in the storage means on the second display means when switching the operation mode from the first mode to the second mode. Device control method. First display means, second display means comprising a display-integrated tablet capable of accepting external input, operation means, and control means for controlling display modes of the first and second display devices A computer-readable control program executed in an electronic device comprising:
In the electronic device,
Operating in a first mode in which a screen created by processing executed in response to an input to the tablet is displayed on the first display means;
Operating in a second mode for causing the second display means to display a screen created by processing executed in response to an input to the tablet;
Storing the operation information, which is information for specifying the content of the operation in the second mode, in the storage means when switching the operation mode from the second mode to the first mode;
Causing the second display unit to display information corresponding to the operation information stored in the storage unit when switching the mode operating from the first mode to the second mode. Electronic equipment control program.

Images