JP2011008425A - Electronic device, operation mode setting method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic device which achieves intuitive operation for a user.SOLUTION: The electronic device 7000 includes a display device 103 scanning an object on a display screen of a liquid crystal panel through the display screen; a specifying part 340 specifying the shape of a contact region on the display screen with which the object is in contact, based on data obtained by scanning; and a setting part 314 setting the operation mode of the electronic device to one of a plurality of operation modes based on the shape of the contact region.

Description

  The present invention relates to an electronic device, an operation mode setting method for the electronic device, and a program for controlling the electronic device. In particular, the present invention relates to an electronic device that accepts touch input, an operation mode setting method for the electronic device, and a program for controlling the electronic device.

  Conventionally, a notebook personal computer is usually provided with a touch pad. The user can move the mouse cursor displayed on the screen by sliding a finger on the touch pad.

  Patent Document 1 discloses a portable personal computer having a liquid crystal display panel on the back of the touch pad as a notebook personal computer having the touch pad. The portable personal computer includes function buttons.

  When the function button is operated, the portable personal computer displays a menu for selecting a function on the liquid crystal display panel. When the user selects, for example, a numeric keypad from the menu, the portable personal computer displays the numeric keypad on the liquid crystal panel. The user can cause the portable personal computer to execute the calculation by operating the numeric keypad via the touch pad. On the other hand, when the function button is not operated, the user can perform the same input as the conventional one via the touch pad. For example, the user can move the mouse cursor.

  Patent Document 2 discloses a touch panel type portable information processing device as a notebook personal computer including the touch pad. The portable information processing apparatus detects a contact area with the touch panel. Further, the portable information processing apparatus determines whether the touch is performed with the touch pen or the finger based on the detected touch area and a parameter represented by a preset area. When it is determined that the mobile information processing apparatus is touched with a finger, the input coordinate data (for example, a numeric keypad) is displayed in a size that can be input with the finger.

Conventionally, coordinate input devices such as a touch panel and a digitizer are known.
Patent Document 3 discloses a configuration having a touch panel function and a digitizer function as the coordinate input device. More specifically, the coordinate input device disclosed in Patent Document 3 constitutes a touch panel in which two optical scanning detection units detect a position indicated on the input plate of the coordinate input device. Further, the two vibration sensors constitute a digitizer that obtains coordinate values with high accuracy based on the vibration applied to the input plate.

  In a state where the coordinate input has been performed, the coordinate input device drives only one of the optical scanning detection units to detect the size of an object that approaches or touches the input plate. If the detected size is smaller than a predetermined value (size about the fingertip), the coordinate input device determines that touch input has occurred and drives the two optical scanning detection units to perform touch input. Get the coordinate value of. On the other hand, if the detected size is larger than a predetermined value (a size larger than the hand fist), the coordinate input device determines that pen input is being performed, and performs coordinate detection using two vibration sensors.

JP 2000-339097 A Japanese Patent Laid-Open No. 9-231006 Japanese Patent Laid-Open No. 10-254623

  However, the portable personal computer of Patent Document 1 needs to include the above-described function buttons. In addition, since the operation mode of the portable personal computer is switched at least by turning on / off the function button, the user needs to be aware of the operation mode of the operation mode of the portable personal computer.

  In the portable information processing apparatus disclosed in Patent Document 2, for example, when the user touches with a finger, the user can perform only an operation of touching input coordinate data (touch input operation). Further, even when the user touches with a touch pen, the user can perform only a touch input operation.

  Further, since the coordinate input device of Patent Document 3 performs processing while paying attention only to the size of the detected object, it cannot perform processing according to how the finger touches. That is, the process performed by the coordinate input device is not different, regardless of the posture in which the user touches the finger. Further, even when the user uses a touch pen, if the palm of the hand is not in proximity or in contact, the coordinate input device determines that the touch input is performed with a finger.

  The present invention has been made in view of the above problems, and an object thereof is to provide an electronic device capable of realizing an intuitive operation for a user, an operation mode setting method for the electronic device, and a program. .

  According to one aspect of the present invention, an electronic device is an electronic device including a first display screen, and scanning means for scanning an object on the first display screen via the first display screen; Based on the obtained data, the specifying means for specifying the shape of the contact area on the first display screen in contact with the object, and the operation mode of the electronic device based on the shape of the contact area are set to a plurality of operation modes. Setting means for setting any one of the operation modes.

  Preferably, the setting means sets the operation mode based on the major axis length and minor axis length of the ellipse while processing the shape as an ellipse.

  Preferably, the electronic device further includes an input device including a first display including a first display screen, an input key group including a plurality of input keys, and a second display including a second display screen. The first display and the input key group are arranged on a plane defined by the X axis and the Y axis perpendicular to the X axis. The first display is arranged on the Y axis negative direction side with respect to the input key group. The second display is arranged on the Y axis positive direction side with respect to the input key group. Characters assigned to the input keys are displayed on at least one of the plurality of input keys. The character stands upright from the Y-axis negative direction side toward the Y-axis positive direction side.

  Preferably, the electronic device further includes display control means for displaying an image on each of the first display screen and the second display screen. The electronic device executes a first operating system and a second operating system. The display control means displays an image based on the first operating system on the first display screen, and displays an image based on the second operating system on the second display screen.

  Preferably, the electronic device is an input device including a display including a first display screen and an input key group including a plurality of input keys. The display and the input key group are arranged on a plane defined by the X axis and a Y axis perpendicular to the X axis. The display is arranged on the Y axis negative direction side with respect to the input key group. Characters assigned to the input keys are displayed on at least one of the plurality of input keys. The character stands upright from the Y-axis negative direction side toward the Y-axis positive direction side.

  Preferably, the electronic device is a display including a first display screen, and further includes display control means for displaying a keyboard image showing a software keyboard on the first display screen.

  Preferably, the electronic device further includes a display including a first display screen and a keyboard including an input key group including a plurality of input keys, and the keyboard is placed on the first display screen.

  Preferably, the electronic device includes a display device including the first display screen, and a light amount determination unit that determines whether the incident light amount of the scattered light from the external light source with respect to the first display screen is larger than a predetermined value. Further prepare. The display device includes a light source that irradiates light from the first display screen to the outside of the display device inside the display device. When it is determined that the amount of incident light is greater than a predetermined value, the scanning unit performs a scan using scattered light, and when it is determined that the amount of incident light is not greater than the predetermined value, the display device Scanning is performed using light emitted from a light source that emits light to the outside.

  Preferably, the electronic device determines whether the incident light amount of the scattered light from the external light source with respect to the display device including the first display screen and the first display screen is greater than a predetermined value. And a number measuring means for measuring the number of shadows formed on the first display screen by the scattered light based on the data obtained by scanning in a state where the object is in contact with the first display screen. The number determining means for determining whether or not the number of shadows is greater than a predetermined value, and a first light source for irradiating the first display screen with light outside the display device. The display device includes a second light source that irradiates light from the first display screen to the outside of the display device inside the display device. When it is determined that the amount of incident light is greater than a predetermined value, the scanning unit uses light emitted from the first light source when it is determined that the number of shadows is greater than a predetermined value. When scanning is performed and it is determined that the number of shadows is not greater than the predetermined value, scanning is performed using scattered light, and the amount of incident light is determined not to be greater than the predetermined value The scanning using the light emitted from the second light source is performed.

  According to another aspect of the present invention, an operation mode setting method is an operation mode setting method in an electronic device having a display screen, the step of scanning an object on the display screen via the display screen, The step of identifying the shape of the contact area on the display screen in contact with the object based on the data obtained by the above, and the operation mode of the electronic device based on the shape of the contact area, among the plurality of operation modes And setting to any one of the operation modes.

  When the further another situation of this invention is followed, a program is a program which controls the operation mode in the electronic device provided with the display screen. The program scans an object on the display screen through the display screen, and specifies the shape of the contact area on the display screen in contact with the object based on the data obtained by the scan; Based on the shape of the contact area, the electronic device is caused to execute the step of setting the operation mode of the electronic device to one of a plurality of operation modes.

  An intuitive operation for the user can be realized.

It is the figure which showed the external appearance of the electronic device. It is a block diagram showing the hardware constitutions of an electronic device. It is the figure which showed the structure of the liquid crystal panel of an electronic device, and the peripheral circuit of the said liquid crystal panel. It is sectional drawing of a liquid crystal panel and a backlight. It is the figure which showed the timing chart at the time of operating an optical sensor circuit. It is sectional drawing of a liquid crystal panel and a backlight, Comprising: It is the figure which showed the structure which a photodiode receives the light from a backlight in the case of a scan. It is the figure which showed schematic structure of the command. It is a figure for demonstrating the command of classification "000". FIG. 10 is a diagram for explaining a command of type “001”. It is a figure for demonstrating the command of classification "010". FIG. 10 is a diagram for explaining a command of type “011”. It is a figure for demonstrating the command of classification "100". FIG. 10 is a diagram for explaining a command of type “101”. It is the figure which showed schematic structure of the response data. It is the figure which showed the image obtained by scanning a finger | toe. It is a circuit diagram of another liquid crystal panel with a built-in photosensor. It is sectional drawing which showed the structure which a photodiode receives external light in the case of a scan. It is a block diagram showing the hardware constitutions of the modification of an electronic device. It is the figure which showed the case where a finger | toe was made to contact the display screen of a liquid crystal panel in the state which fell the user's finger | toe with respect to the liquid crystal panel. It is the figure which showed the case where a finger | toe was made to contact the display screen of a liquid crystal panel in the state which stood the user's finger with respect to the liquid crystal panel. FIG. 20 is a diagram specifically illustrating “pointer operation” when a finger touches the display screen of the liquid crystal panel in the state of FIG. 19. FIG. 21 is a diagram specifically illustrating a “touch operation” when a finger touches the display screen of the liquid crystal panel in the state of FIG. 20. It is the figure explaining pointer operation and touch operation in one figure. It is the figure which showed the block of the electronic device. It is the flowchart which showed the flow of the process performed with an electronic device. It is the flowchart which showed the flow of the process of an electronic device when three threshold values regarding a contact angle are provided. It is the flowchart which showed the flow of the process performed with an electronic device in more detail than FIG. It is the flowchart which showed the flow of the process performed with an electronic device in more detail than FIG. It is a figure for demonstrating a part of process shown with the flowchart of FIG. It is a figure for demonstrating the scan image obtained when it scans using the light source which irradiates the infrared rays (internal light) incorporated in the liquid crystal panel. It is a figure for demonstrating a contact area | region and a detection area | region when it scans using infrared rays and a finger is made to contact the display screen of a liquid crystal panel with contact angle (theta) 1. FIG. It is a figure for demonstrating the said contact area | region and said detection area | region when it scans using infrared rays and a finger is made to contact the display screen of a liquid crystal panel with contact angle (theta) 2. FIG. It is a figure for demonstrating the contact angle of a finger | toe. It is a figure for demonstrating an example of the detection method of specific contact angle (theta). It is a figure for demonstrating the scan image obtained when it scans using the external light which is scattered light. It is the figure which showed the external appearance of the electronic device. It is a figure for demonstrating the scan image obtained when it scans using the light from the light source of FIG. It is a figure for demonstrating the process of an electronic device at the time of using a direction judgment part. It is a figure for demonstrating the structure using the detected contact angle (theta) and the determination result by the direction determination part. It is the flowchart which showed the flow of the selection process of the scanning method by electronic equipment. It is a figure for demonstrating the new process in an electronic device. It is a figure for demonstrating a part of process shown by the flowchart of FIG. 27 and FIG. It is a figure for demonstrating the further new process in an electronic device. It is the figure which showed the process of the electronic device at the time of using an input process part. It is the flowchart which showed the flow of the process performed with an electronic device. It is the figure which showed the external appearance of the modification of an electronic device. It is a figure showing operation of a user's finger. It is a figure for demonstrating user operation with respect to an electronic device. It is a figure for demonstrating the process in an electronic device. It is a figure for demonstrating the other process in an electronic device. It is a figure for demonstrating the further another process in an electronic device. It is a figure for demonstrating the further another process in an electronic device. It is a figure for demonstrating the further another process in an electronic device. It is the figure which showed the external appearance of the electronic device. It is a figure showing "pointer operation" when a finger contacts the display screen of the liquid crystal panel at a contact angle θ1. FIG. 10 is a diagram showing a “touch operation” when a finger touches the display screen of the liquid crystal panel at a contact angle θ2. It is the figure which showed the external appearance of the electronic device. It is a figure showing "pointer operation" when a finger contacts the display screen of the liquid crystal panel at a contact angle θ1. FIG. 10 is a diagram showing a “touch operation” when a finger touches the display screen of the liquid crystal panel at a contact angle θ2. It is the figure which showed the structure of the communication system containing an electronic device. It is the figure which showed the external appearance of the electronic device. FIG. 62 is a diagram illustrating a configuration of a conventional personal computer, and is a comparative diagram for comparison with FIG. 61. It is a figure for demonstrating the case where a finger is made to contact in the state which laid down with respect to the display screen of a liquid crystal panel. FIG. 64 is a diagram for explaining a case where a finger is brought into contact with the display screen in a state where the finger is slightly raised from FIG. It is a figure for demonstrating the case where a finger | toe is made to contact a display screen in the state which stood up a little from Fig.64 (a). FIG. 66 is a diagram for describing a case where a finger is brought into contact with the display screen in a state where the finger is slightly raised from FIG. It is the figure which showed the block of the electronic device. It is a figure for demonstrating the scan image obtained when it scans using the light source which irradiates the infrared rays (internal light) incorporated in the liquid crystal panel. It is a figure for demonstrating the scan image obtained when it scans using the light source which irradiates the infrared rays incorporated in the liquid crystal panel. It is the figure which showed the block of the electronic device. It is the figure which showed the external appearance of the electronic device. It is the figure which showed the image image | photographed with the camera. It is the figure which showed the block of the electronic device. It is a figure for demonstrating the calculation method of contact angle (theta) in a stereoscopic display device.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<< About electronic equipment with photosensor liquid crystal >>
An outline of an electronic device including the photosensor liquid crystal will be described below. Hereinafter, a case where the electronic apparatus has two screens will be described as an example, but the present invention is not limited to this. For example, as will be described later, the electronic device may be one screen.

<1. Appearance of electronic equipment>
FIG. 1 is a diagram illustrating an appearance of electronic device 100 according to the present embodiment. Referring to FIG. 1, electronic device 100 includes a first housing 100A and a second housing 100B.

  The first casing 100A and the second casing 100B are foldably connected by a hinge 100C. The first housing 100A includes a liquid crystal panel 140 with a built-in optical sensor. The second housing 100B includes an optical sensor built-in liquid crystal panel 240. As described above, the electronic device 100 includes two optical sensor built-in liquid crystal panels.

  When the user uses the electronic device 100, the liquid crystal panel 240 is arranged on the user side rather than the operation keys 177. In the following, for convenience of explanation, the liquid crystal panel 240 and the operation key 177 will be described as being arranged on a plane defined by the X axis and the Y axis perpendicular to the X axis.

  The liquid crystal panel 240 is arranged on the Y axis negative direction side with respect to the operation key 177. The liquid crystal panel 140 is arranged on the Y axis positive direction side with respect to the operation keys 177. More specifically, the liquid crystal panel 140 is arranged on the Y axis positive direction side with respect to the operation key 177 in a state where the electronic device 100 is opened (that is, not in a folded state). More specifically, in a state where the angle between the first casing 100A and the second casing 100B is approximately 90 degrees or more, the liquid crystal panel 140 is arranged on the Y axis positive direction side with respect to the operation key 177. . In addition, it can be said that the liquid crystal panel 140 is arranged on the side opposite to the liquid crystal panel 240 with respect to the operation key 177 in this state. Characters assigned to the input keys are displayed on at least one of the plurality of input keys in the operation key 177. For example, alphabets and kana are displayed as the characters. Further, the above-mentioned characters are erect from the Y-axis negative direction side toward the Y-axis positive direction side. For example, when the operation key 177 is a QWERTY type keyboard, the input key indicating “Z”, the input key indicating “A”, and the input indicating “Q” from the Y-axis negative direction to the Y-axis positive direction The keys are arranged in that order.

  The electronic device 100 is configured as a portable device having a display function such as a PDA (Personal Digital Assistant), a notebook personal computer, a portable telephone, and an electronic dictionary.

<2. Hardware configuration>
Next, an aspect of a specific configuration of the electronic device 100 will be described with reference to FIG. FIG. 2 is a block diagram illustrating a hardware configuration of electronic device 100.

  Electronic device 100 includes a first unit 1001 and a second unit 1002. The second unit 1002 is detachably connected to the first unit 1001 from the electronic device 100. The first unit 1001 includes a main body device 101 and a display device 102. The second unit 1002 includes a display device 103 and a main device 104.

  The first housing 100A includes a display device 102. Second housing 100B includes main device 101. The second housing 100B includes a second unit 1002.

(About the first unit)
The main unit 101 includes a CPU (Central Processing Unit) 110, a RAM (Random Access Memory) 171, a ROM (Read-Only Memory) 172, a memory card reader / writer 173, an external communication unit 174, a microphone 175, A speaker 176, an operation key 177, a power switch 191, a power circuit 192, a power detection unit 193, a USB (Universal Serial Bus) connector 194, an antenna 195, and a LAN (Local Area Network) connector 196 are included. . Each component (110, 171 to 177, 193) is mutually connected by a data bus DB1. A memory card 1731 is attached to the memory card reader / writer 173.

  CPU 110 executes a program. The operation key 177 receives an instruction input from the user of the electronic device 100. The RAM 171 stores data generated by the execution of the program by the CPU 110 or data input via the operation keys 177 in a volatile manner. The ROM 172 stores data in a nonvolatile manner. The ROM 172 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory.

  The external communication unit 174 communicates with other electronic devices. Specifically, the external communication unit 174 communicates with, for example, the second unit 1002 via the USB connector 194. The external communication unit 174 performs wireless communication with the second unit 1002 via the antenna 195, for example. Further, the external communication unit 174 performs wired communication with other electronic devices via the LAN connector 196.

  Note that the main device 101 may communicate with other electronic devices by wireless communication other than Bluetooth (registered trademark). For example, the external communication unit 174 may perform wireless communication with other electronic devices connected to the LAN via a wireless LAN antenna (not shown). Alternatively, wireless communication may be performed with another electronic device via an infrared port (not shown).

The power switch 191 is a switch for starting up the electronic device 100.
When the power switch 191 is turned on, the power supply circuit 192 supplies power to each component connected to the data bus DB1 and the display device 102 via the power detection unit 193. When the power switch 191 is turned on, the power circuit 192 supplies power to the external communication unit 174 without going through the power detection unit 193.

  The power supply detection unit 193 detects the output from the power supply circuit 192. In addition, the power supply detection unit 193 sends information (for example, a voltage value and a current value) regarding the detected output to the CPU 110.

  The USB connector 194 is used to connect the first unit 1001 to the second unit 1002. Note that the main device 101 may include other USB connectors in addition to the USB connector 194.

  The first unit 1001 transmits data to the second unit 1002 via the USB connector 194. The first unit 1001 receives data from the second unit 1002 via the USB connector 194. Further, the first unit 1001 supplies power to the second unit 1002 via the USB connector 194.

  The antenna 195 is used for communication according to the Bluetooth (registered trademark) standard between the first unit 1001 and another communication apparatus (for example, the second unit 1002). The LAN connector 196 is used to connect the electronic device 100 to the LAN.

  The display device 102 includes a driver 130, an optical sensor built-in liquid crystal panel 140 (hereinafter referred to as a liquid crystal panel 140), an internal IF 178, a backlight 179, and an image processing engine 180.

  The driver 130 is a drive circuit for driving the liquid crystal panel 140 and the backlight 179. Various drive circuits included in the driver 130 will be described later.

  The liquid crystal panel 140 is a device having a liquid crystal display function and a photosensor function. That is, the liquid crystal panel 140 can perform image display using liquid crystal and sensing using an optical sensor. Details of the liquid crystal panel 140 will be described later.

  An internal IF (Interface) 178 mediates exchange of data between the main device 101 and the display device 102.

  The backlight 179 is a light source disposed on the back surface of the liquid crystal panel 140. The backlight 179 irradiates the back surface with uniform light.

  The image processing engine 180 controls the operation of the liquid crystal panel 140 via the driver 130. Here, the control is performed based on various data sent from the main apparatus 101 via the internal IF 178. Note that the various data includes commands to be described later. Further, the image processing engine 180 processes data output from the liquid crystal panel 140 and sends the processed data to the main apparatus 101 via the internal IF 178. Further, the image processing engine 180 includes a driver control unit 181, a timer 182, and a signal processing unit 183.

  The driver control unit 181 controls the operation of the driver 130 by sending a control signal to the driver 130. In addition, the driver control unit 181 analyzes a command transmitted from the main device 101. Then, the driver control unit 181 sends a control signal based on the analysis result to the driver 130. Details of the operation of the driver 130 will be described later.

The timer 182 generates time information and sends the time information to the signal processing unit 183.
The signal processing unit 183 receives data output from the optical sensor. Here, since the data output from the optical sensor is analog data, the signal processing unit 183 first converts the analog data into digital data. Further, the signal processing unit 183 performs data processing on the digital data according to the content of the command sent from the main device 101. Then, the signal processing unit 183 sends data (hereinafter referred to as response data) including data after the above data processing and time information acquired from the timer 182 to the main body device 101. The signal processing unit 183 includes a RAM (not shown) that can store a plurality of scan data, which will be described later, continuously.

  The command includes a sensing command for instructing sensing by the optical sensor. Details of the sensing command and the response data will be described later (FIGS. 7, 8, and 14).

  Note that the timer 182 is not necessarily provided in the image processing engine 180. For example, the timer 182 may be provided outside the image processing engine 180 in the display device 102. Alternatively, the timer 182 may be provided in the main body device 101. Further, the microphone 175 and the speaker 176 are not always provided in the electronic device 100, and may be configured so as not to include either or both of the microphone 175 and the speaker 176 depending on the embodiment of the electronic device 100.

  Here, the display device 102 includes a system liquid crystal. The system liquid crystal is a device obtained by integrally forming peripheral devices of the liquid crystal panel 140 on the glass substrate of the liquid crystal panel 140. In the present embodiment, the driver 130 (excluding the circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are integrally formed on the glass substrate of the liquid crystal panel 140. Note that the display device 102 is not necessarily configured using the system liquid crystal, and the driver 130 (excluding a circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are included in the glass substrate. Other substrates may be configured.

(About the second unit)
The second unit 1002 receives power supply from the first unit 1001. Specifically, the second unit 1002 is supplied with power from the power supply circuit 192 of the first unit 1001 by connecting a USB connector 294 described later and the USB connector 194 of the first unit 1001.

  The main body device 104 includes a CPU 210, a RAM 271, a ROM 272, an external communication unit 274, a power supply detection unit 293, a USB connector 294, an antenna 295, and a signal strength detection unit 297. Each component (210, 271, 272, 274, 293) is connected to each other by a data bus DB2.

  The CPU 210 executes a program. The RAM 271 stores data generated by the execution of the program by the CPU 210 in a volatile manner. The ROM 272 stores data in a nonvolatile manner. The ROM 272 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory.

  The external communication unit 274 communicates with other electronic devices. Specifically, the external communication unit 274 communicates with, for example, the first unit 1001 via the USB connector 294. The external communication unit 274 communicates with the first unit 1001 through the antenna 295, for example.

  Note that main device 104 may communicate with another electronic device (for example, first unit 1001) by wireless communication other than Bluetooth (registered trademark). For example, the external communication unit 274 may perform wireless communication with other electronic devices via an infrared port (not shown).

  The signal strength detection unit 297 detects the strength of the signal received via the antenna 295. Then, the signal strength detection unit 297 sends the detected strength to the external communication unit 274.

  The USB connector 294 is used to connect the second unit 1002 to the first unit 1001.

  The second unit 1002 transmits data to the first unit 1001 via the USB connector 294. The second unit 1002 receives data from the first unit 1001 via the USB connector 294. Furthermore, the second unit 1002 receives power supply from the first unit 1001 via the USB connector 294 as described above. The second unit 1002 stores the electric power supplied from the first unit 1001 in a battery (not shown).

  The antenna 295 is used for communication between the second unit 1002 and the first unit 1001, for example, in accordance with the Bluetooth (registered trademark) standard.

  The power detection unit 293 detects the power supplied via the USB connector 294. In addition, the power supply detection unit 293 sends information about the detected power to the CPU 210.

Further, the main device 104 may have a function of performing infrared communication.
The display device 103 includes a driver 230, an optical sensor built-in liquid crystal panel 240 (hereinafter referred to as “liquid crystal panel 240”), an internal IF 278, a backlight 279, and an image processing engine 280. The image processing engine 280 includes a driver control unit 281, a timer 282, and a signal processing unit 283.

  The display device 103 has the same configuration as the display device 102. That is, the driver 230, the liquid crystal panel 240, the internal IF 278, the backlight 279, and the image processing engine 280 have the same configuration as the driver 130, the liquid crystal panel 140, the internal IF 178, the backlight 179, and the image processing engine 180 in the display device 102. Have each. The driver control unit 281, the timer 282, and the signal processing unit 283 have the same configurations as the driver control unit 181, the timer 182, and the signal processing unit 183 in the display device 102, respectively. Therefore, description of each functional block included in display device 103 will not be repeated.

  By the way, the processing in the electronic device 100 is realized by each hardware and software executed by the CPU 110. Such software may be stored in the ROM 172 in advance. The software may be stored in a memory card 1731 or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by the memory card reader / writer 173 or other reading device, or downloaded via the external communication unit 174 or communication IF (not shown), and then temporarily stored in the ROM 172. Is done. The software is read from the ROM 172 by the CPU 110 and stored in the RAM 171 in the form of an executable program. CPU 110 executes the program.

  Each component constituting the main device 101 of the electronic device 100 shown in FIG. 2 is a general one. Therefore, it can be said that the essential part of the present invention is the software stored in the RAM 171, the ROM 172, the memory card 1731 and other storage media, or the software downloadable via the network. Since the hardware operation of main device 101 of electronic device 100 is well known, detailed description will not be repeated.

  The storage medium is not limited to a memory card, but is a CD-ROM, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile). Disc)), IC (Integrated Circuit) cards (excluding memory cards), optical cards, mask ROM, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory), flash ROM, and other semiconductor memories, etc. It may be a medium to be used.

  The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

<3. Configuration and drive of photosensor built-in liquid crystal panel>
Next, the configuration of the liquid crystal panel 140 and the configuration of peripheral circuits of the liquid crystal panel 140 will be described. FIG. 3 is a diagram showing a configuration of the liquid crystal panel 140 and peripheral circuits of the liquid crystal panel 140.

  Referring to FIG. 3, the liquid crystal panel 140 includes a pixel circuit 141, an optical sensor circuit 144, a scanning signal line Gi, a data signal line SRj, a data signal line SGj, a data signal line SBj, and a sensor signal line. SSj, sensor signal line SDj, read signal line RWi, and reset signal line RSi are included. Note that i is a natural number satisfying 1 ≦ i ≦ m, and j is a natural number satisfying 1 ≦ j ≦ n.

  2 includes a scanning signal line driving circuit 131, a data signal line driving circuit 132, an optical sensor driving circuit 133, a switch 134, as peripheral circuits of the liquid crystal panel 140. And an amplifier 135.

  The scanning signal line drive circuit 131 receives the control signal TC1 from the driver control unit 181 shown in FIG. The scanning signal line drive circuit 131 applies a predetermined voltage in order from the scanning signal line G1 to each scanning signal line (G1 to Gm) based on the control signal TC1. More specifically, the scanning signal line driving circuit 131 sequentially selects one scanning signal line from the scanning signal lines (G1 to Gm) per unit time, and a TFT (to be described later) with respect to the selected scanning signal line. A voltage capable of turning on the gate of the thin film transistor 142 is applied (hereinafter referred to as a high level voltage). Note that a low level voltage is applied to a scanning signal line that is not selected without applying a high level voltage.

  The data signal line driving circuit 132 receives image data (DR, DG, DB) from the driver control unit 181 shown in FIG. The data signal line driving circuit 132 applies a voltage corresponding to one row of image data to the 3n data signal lines (SR1 to SRn, SG1 to SGn, SB1 to SBn) for each unit time. Apply sequentially.

  Note that although a driving method called a so-called line-sequential method has been described here, the driving method is not limited to this.

  The pixel circuit 141 is a circuit for setting the luminance (transmittance) of one pixel. Further, m × n pixel circuits 141 are arranged in a matrix. More specifically, m pixel circuits 141 are arranged in the vertical direction in FIG. 3 and n pixel circuits in the horizontal direction.

  The pixel circuit 141 includes an R subpixel circuit 141r, a G subpixel circuit 141g, and a B subpixel circuit 141b. Each of these three circuits (141r, 141g, 141b) includes a TFT 142, a pair of electrode pairs 143 including a pixel electrode and a counter electrode, and a capacitor (not shown).

  In addition, CMOS (Complementary Metal Oxide Semiconductor) capable of forming n-type transistors and p-type transistors can be realized, and the movement speed of carriers (electrons or holes) is several hundreds compared to amorphous silicon thin film transistors (a-Si TFTs). For example, a polycrystalline silicon thin film transistor (p-Si TFT) is used as the TFT 142 in the display device 102 because it is twice as fast. Note that the TFT 142 will be described as an n-channel field effect transistor. However, the TFT 142 may be a p-type channel field effect transistor.

  The source of the TFT 142 in the R subpixel circuit 141r is connected to the data signal line SRj. The gate of the TFT 142 is connected to the scanning signal line Gi. Further, the drain of the TFT 142 is connected to the pixel electrode of the electrode pair 143. A liquid crystal is disposed between the pixel electrode and the counter electrode. The G sub-pixel circuit 141g and the B sub-pixel circuit 141b have the same configuration as the R sub-pixel circuit 141r except that the data signal line to which the source of each TFT 142 is connected is different. Therefore, description of these two circuits (141g, 141b) will not be repeated.

  Here, setting of luminance in the pixel circuit 141 will be described. First, the high level voltage is applied to the scanning signal line Gi. By the application of the high level voltage, the gate of the TFT 142 is turned on. In this manner, with the gate of the TFT 142 turned on, a specified voltage (voltage corresponding to image data for one pixel) is applied to each data signal line (SRj, SGj, SBj). Thereby, a voltage based on the designated voltage is applied to the pixel electrode. As a result, a potential difference is generated between the pixel electrode and the counter electrode. Based on this potential difference, the liquid crystal responds and the luminance of the pixel is set to a predetermined luminance. Note that the potential difference is held by the capacitor (auxiliary capacitor) (not shown) until the scanning signal line Gi is selected in the next frame period.

  The optical sensor drive circuit 133 receives the control signal TC2 from the driver control unit 181 shown in FIG.

  Then, the optical sensor drive circuit 133 sequentially selects one signal line from the reset signal lines (RS1 to RSm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, the voltage VDDR that is higher than usual is applied. Note that a voltage VSSR lower than the voltage applied to the selected reset signal line is kept applied to the unselected reset signal line. For example, the voltage VDDR may be set to 0V and the voltage VSSR may be set to −5V.

  Further, the optical sensor driving circuit 133 sequentially selects one signal line from the readout signal lines (RW1 to RWm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, a voltage VDD higher than usual is applied. Note that the voltage VSSR is applied to the read signal line that is not selected. For example, the value of VDD may be set to 8V.

  The timing for applying the voltage VDDR and the timing for applying the voltage VDD will be described later.

  The optical sensor circuit 144 includes a photodiode 145, a capacitor 146, and a TFT 147. In the following description, it is assumed that the TFT 147 is an n-type channel field effect transistor. However, the TFT 147 may be a p-type channel field effect transistor.

  The anode of the photodiode 145 is connected to the reset signal line RSi. On the other hand, the cathode of the photodiode 145 is connected to one electrode of the capacitor 146. The other electrode of the capacitor 146 is connected to the read signal line RWi. Hereinafter, a connection point between the photodiode 145 and the capacitor 146 is referred to as a node N.

  The gate of the TFT 147 is connected to the node N. The drain of the TFT 147 is connected to the sensor signal line SDj. Further, the source of the TFT 147 is connected to the sensor signal line SSj. Details of sensing using the optical sensor circuit 144 will be described later.

  The switch 134 is a switch provided to switch whether or not to apply a predetermined voltage to the sensor signal lines (SD1 to SDn). The switching operation of the switch 134 is performed by the optical sensor driving circuit 133. Note that the voltage applied to the sensor signal lines (SD1 to SDn) when the switch 134 is turned on will be described later.

  The amplifier 135 amplifies the voltage output from each sensor signal line (SS1 to SSn). The amplified voltage is sent to the signal processing unit 183 shown in FIG.

  The image processing engine 180 controls the timing for displaying an image on the liquid crystal panel 140 using the pixel circuit 141 and the timing for sensing using the optical sensor circuit 144.

  FIG. 4 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179. Referring to FIG. 4, liquid crystal panel 140 includes an active matrix substrate 151A, a counter substrate 151B, and a liquid crystal layer 152. The counter substrate 151B is disposed to face the active matrix substrate 151A. The liquid crystal layer 152 is sandwiched between the active matrix substrate 151A and the counter substrate 151B. The backlight 179 is disposed on the opposite side of the liquid crystal layer 152 with respect to the active matrix substrate 151A.

  The active matrix substrate 151 </ b> A includes a polarizing filter 161, a glass substrate 162, a pixel electrode 143 a that constitutes the electrode pair 143, a photodiode 145, a data signal line 157, and an alignment film 164. Further, although not shown in FIG. 4, the active matrix substrate 151A includes the capacitor 146, the TFT 147, the TFT 142, and the scanning signal line Gi shown in FIG.

  In the active matrix substrate 151A, the polarizing filter 161, the glass substrate 162, the pixel electrode 143a, and the alignment film 164 are arranged in this order from the backlight 179 side. The photodiode 145 and the data signal line 157 are formed on the liquid crystal layer 152 side of the glass substrate 162.

  The counter substrate 151B includes a polarizing filter 161, a glass substrate 162, a light shielding film 163, color filters (153r, 153g, 153b), a counter electrode 143b constituting the electrode pair 143, and an alignment film 164.

  In the counter substrate 151B, the alignment film 164, the counter electrode 143b, the color filters (153r, 153g, 153b), the glass substrate 162, and the polarizing filter 161 are arranged in this order from the liquid crystal layer 152 side. The light shielding film 163 is formed in the same layer as the color filters (153r, 153g, 153b).

  The color filter 153r is a filter that transmits light having a red wavelength. The color filter 153g is a filter that transmits light having a green wavelength. The color filter 153b is a filter that transmits light having a blue wavelength. Here, the photodiode 145 is arranged at a position facing the color filter 153b.

  The liquid crystal panel 140 displays an image by blocking external light or light emitted from a light source such as a backlight 179 or transmitting the light. Specifically, the liquid crystal panel 140 changes the direction of liquid crystal molecules in the liquid crystal layer 152 by applying a voltage between the pixel electrode 143a and the counter electrode 143b, thereby blocking or transmitting the light. However, since the light cannot be completely blocked by the liquid crystal alone, a polarizing filter 161 that transmits only light having a specific polarization direction is provided.

  Note that the position of the photodiode 145 is not limited to the above position, and may be provided at a position facing the color filter 153r or a position facing the color filter 153g.

  Here, the operation of the optical sensor circuit 144 will be described. FIG. 5 is a diagram showing a timing chart when the optical sensor circuit 144 is operated. In FIG. 5, a voltage VINT indicates a potential at the node N in the photosensor circuit 144. A voltage VPIX is an output voltage from the sensor signal line SSj shown in FIG. 3 and is a voltage before being amplified by the amplifier 135.

  The following description will be divided into a reset period for resetting the optical sensor circuit 144, a sensing period for sensing light using the optical sensor circuit 144, and a readout period for reading the sensed result.

  First, the reset period will be described. In the reset period, the voltage applied to the reset signal line RSi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDDR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR). As described above, by applying the high-level voltage to the reset signal line RSi, a current starts to flow in the forward direction (from the anode side to the cathode side) of the photodiode 145. As a result, the voltage VINT which is the potential of the node N has a value represented by the following expression (1). In Equation (1), the forward voltage drop amount in the photodiode 145 is Vf.

VINT = VSSR + | VDDR−VSSR | −Vf (1)
Therefore, the potential of the node N is a value smaller by Vf than the voltage VDDR as shown in FIG.

  Here, since the voltage VINT is not more than the threshold value for turning on the gate of the TFT 147, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change. Further, a difference corresponding to the voltage VINT occurs between the electrodes of the capacitor 146. For this reason, the capacitor 146 accumulates charges corresponding to the difference.

  Next, the sensing period will be described. In the sensing period following the reset period, the voltage applied to the reset signal line RSi instantaneously switches from the high level (voltage VDDR) to the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR).

  Thus, by changing the voltage applied to the reset signal line RSi to the low level, the potential of the node N becomes higher than the voltage of the reset signal line RSi and the voltage of the read signal line RWi. For this reason, in the photodiode 145, the voltage on the cathode side becomes higher than the voltage on the anode side. That is, the photodiode 145 is in a reverse bias state. In such a reverse bias state, when the photodiode 145 receives light from the light source, current starts to flow from the cathode side to the anode side of the photodiode 145. As a result, as shown in FIG. 5, the potential of the node N (that is, the voltage VINT) becomes lower with the passage of time.

  Since the voltage VINT continues to decrease in this way, the gate of the TFT 147 does not turn on. Therefore, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change.

  Next, the reading period will be described. In the readout period following the sensing period, the voltage applied to the reset signal line RSi is kept at the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDD). Here, the voltage VDD is higher than the voltage VDDR.

  Thus, by applying a high level voltage instantaneously to the read signal line RWi, the potential of the node N is raised through the capacitor 146 as shown in FIG. Note that the increase width of the potential of the node N is a value corresponding to the voltage applied to the read signal line RWi. Here, since the potential of the node N (that is, the voltage VINT) is raised to a threshold value that turns on the gate of the TFT 147, the gate of the TFT 147 is turned on.

  At this time, if a constant voltage is applied in advance to the sensor signal line SDj (see FIG. 3) connected to the drain side of the TFT 147, the sensor signal line SSj connected to the source side of the TFT 147 will cause the VPIX in FIG. As shown in the graph, a voltage corresponding to the potential of the node N is output.

  Here, when the amount of light received by the photodiode 145 (hereinafter referred to as the amount of received light) is small, the slope of the straight line shown in the VINT graph of FIG. 5 becomes gentle. As a result, the voltage VPIX is higher than when the amount of received light is large. As described above, the optical sensor circuit 144 changes the value of the voltage output to the sensor signal line SSj in accordance with the amount of light received by the photodiode 145.

  By the way, in the above, the operation | movement was demonstrated paying attention to one optical sensor circuit 144 among the m * n optical sensor circuits which exist. Below, operation | movement of each photosensor circuit in the liquid crystal panel 140 is demonstrated.

  First, the optical sensor drive circuit 133 applies a predetermined voltage to all n sensor signal lines (SD1 to SDn). Next, the photosensor drive circuit 133 applies a voltage VDDR that is higher than normal to the reset signal line RS1. The other reset signal lines (RS2 to RSm) and read signal lines (RW1 to RWm) are kept in a state where a low level voltage is applied. As a result, the n photosensor circuits in the first row in FIG. 3 enter the reset period described above. Thereafter, the n photosensor circuits in the first row enter a sensing period. Further, thereafter, the n photosensor circuits in the first row enter a reading period.

  Note that the timing at which a predetermined voltage is applied to all n sensor signal lines (SD1 to SDn) is not limited to the above timing, and may be any timing that is applied at least before the readout period. .

  When the readout period of the n photosensor circuits in the first row is completed, the photosensor drive circuit 133 applies a voltage VDDR that is higher than usual to the reset signal line RS2. That is, the reset period of the n photosensor circuits in the second row starts. When the reset period ends, the n photosensor circuits in the second row enter a sensing period, and thereafter enter a reading period.

  Thereafter, the processing described above is sequentially performed on the n photosensor circuits in the third row, the n photosensor circuits in the fourth row,..., The n photosensor circuits in the m row. As a result, the sensing result of the first row, the sensing result of the second row,..., The sensing result of the m-th row are output in this order from the sensor signal lines (SS1 to SSn).

  In the display device 102, sensing is performed for each row as described above, and a sensing result is output from the liquid crystal panel 140 for each row. For this reason, hereinafter, the data after the signal processing unit 183 performs the above-described data processing on the data regarding the voltage for m rows from the first row to the m-th row output from the liquid crystal panel 140, This is called “scan data”. That is, scan data refers to image data obtained by scanning a scan target (for example, a user's finger). An image displayed based on the scan data is referred to as a “scanned image”. Further, in the following, sensing is referred to as “scan”.

  In the above description, the configuration in which scanning is performed using all the m × n photosensor circuits has been described as an example. However, the configuration is not limited thereto. Scanning may be performed on a partial region of the surface of the liquid crystal panel 140 using a photosensor circuit selected in advance.

  In the following, it is assumed that the electronic device 100 can take either of the two configurations. Further, switching between the components is assumed to be performed by a command sent from the main device 101 based on an input via the operation key 177 or the like. Note that when scanning is performed on a partial area on the surface of the liquid crystal panel 140, the image processing engine 180 sets a scan target area. The setting of the area may be configured to be specified by the user via the operation key 177.

  As described above, when scanning is performed on a part of the surface of the liquid crystal panel 140, there are the following modes of use for displaying an image. The first is a mode in which an image is displayed in a surface area other than the partial area (hereinafter referred to as a scan area). The second is a mode in which no image is displayed in the surface area other than the scan area. Which mode is used is based on a command sent from the main apparatus 101 to the image processing engine 180.

  FIG. 6 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179, showing a configuration in which the photodiode 145 receives light from the backlight 179 during scanning.

  Referring to FIG. 6, when the user's finger 900 is in contact with the surface of the liquid crystal panel 140, a part of the light emitted from the backlight 179 is part of the user's finger 900 (abbreviated in the contacted area). Reflected on the plane). The photodiode 145 receives the reflected light.

  Even in a region where the finger 900 is not in contact, part of the light emitted from the backlight 179 is reflected by the user's finger 900. Even in this case, the photodiode 145 receives the reflected light. However, since the finger 900 is not in contact with the surface of the liquid crystal panel 140 in the region, the amount of light received by the photodiode 145 is smaller than the region in which the finger 900 is in contact. Note that when the light emitted from the backlight 179 is reflected further away, the amount of light received by the photodiode 145 becomes smaller. For this reason, when the user's finger 900 or the like is not in contact with or close to the liquid crystal panel 140, the amount of received light is extremely low, or the light reflected by the finger 900 cannot be received.

  Here, by turning on the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the amount of light reflected by the user's finger 900 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), and the liquid crystal panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter, also referred to as a reflected image) obtained by reflecting light with the finger 900.

Note that examples of the scan object other than the finger 900 include a stylus.
By the way, in the present embodiment, a liquid crystal panel is described as an example of the display device of electronic device 100, but another panel such as an organic EL (Electro-Luminescence) panel is used instead of the liquid crystal panel. Also good.

<4. About Data>
Next, commands exchanged between the first unit 1001 and the second unit 1002 and commands exchanged between the main device 101 and the display device 102 in the first unit 1001 will be described.

  FIG. 7 is a diagram showing a schematic configuration of a command. Referring to FIG. 7, the command includes header DA01, first field DA02, second field DA03, third field DA04, fourth field DA05, fifth field DA06, and spare data area DA07. including.

  FIG. 8 is a diagram for explaining a command of type “000” (that is, a sensing command). The CPU 110 sends a command of type “000” (hereinafter referred to as “first command”) from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 110 sends the first command from the main body device 101 to the display device 102. In the following, the case where the CPU 110 sends the first command from the main unit 101 of the first unit 1001 to the second unit 1002 will be described as an example.

  The CPU 110 writes the command type (“000”), the command transmission destination, and the like in the header DA01. The CPU 110 writes the value of the timing whose number is “1” in the first field DA02. The CPU 110 writes the value of the data type with the number “2” in the second field DA03. The CPU 110 writes the value of the reading method whose number is “3” in the third field DA04. The CPU 110 writes the value of the image gradation number “4” in the fourth field DA05. The CPU 110 writes the value of the resolution with the number “5” in the fifth field DA06.

  The first command in which “00” is set in the first field DA02 requests the image processing engine 280 to transmit scan data at that time. That is, the sensing first command requests transmission of scan data obtained by scanning using the optical sensor circuit of the liquid crystal panel 240 after the image processing engine 280 receives the first command. Further, the first command in which “01” is set in the first field DA02 requests transmission of scan data when the scan result is changed. Further, the first command in which “10” is set in the first field DA02 requests transmission of scan data at regular intervals.

  The first command in which “001” is set in the second field DA03 requests transmission of the coordinate value of the center coordinate in the partial image. In addition, the first command in which “010” is set in the second field DA03 requests transmission of only the partial image whose scan result has changed. Note that the change in the scan result indicates that the previous scan result is different from the current scan result. Furthermore, the first command in which “100” is set in the second field DA03 requests transmission of the entire image.

  Here, the “whole image” is an image generated by the image processing engine 280 based on the voltage output from each optical sensor circuit when scanning using m × n optical sensor circuits. . The “partial image” is a part of the entire image. The reason why the partial image is requested to transmit only the partial image whose scan result has changed will be described later.

  The coordinate value and the partial image or the entire image may be requested at the same time. Further, in the case of a configuration in which a partial area on the surface of the liquid crystal panel 240 is scanned, the entire image is an image corresponding to the area to be scanned.

  The first sensing command in which “00” is set in the third field DA04 requests to scan with the backlight 279 turned on. In addition, the first command in which “01” is set in the third field DA04 requests that the backlight 279 be turned off to perform scanning. Note that the configuration for scanning with the backlight 279 off is described later (FIG. 17). Furthermore, the first command in which “10” is set in the third field DA04 requests scanning using both reflection and transmission. Note that using both reflection and transmission refers to scanning a scanning object by switching between a method of turning on and scanning the backlight 279 and a method of turning off the backlight and scanning.

  The first command in which “00” is set in the fourth field DA05 requests monochrome binary image data. Further, the first command in which “01” is set in the fourth field DA05 requests multi-gradation image data. Further, the first command in which “10” is set in the fourth field DA05 requests RGB color image data.

  The first command in which “0” is set in the fifth field DA06 requests image data with high resolution. The first command in which “1” is set in the fifth field DA06 requests image data with a low resolution.

  In addition to the data shown in FIG. 8, the first command includes designation of a region to be scanned (region of pixels driving the optical sensor circuit 144), timing for scanning, timing for lighting the backlight 179, and the like. Is described.

  Note that the image processing engine 280 analyzes the content of the first command, and sends back data (that is, response data) according to the analysis result to the main device 101.

  FIG. 9 is a diagram for explaining a command of type “001” (hereinafter referred to as “second command”). The CPU 110 sends a second command from the main unit 101 of the first unit 1001 to the second unit 1002.

  The CPU 110 writes the command type (“001”), the command transmission destination, and the like in the header DA01. The CPU 110 writes the value of the display request with the number “1” in the first field DA02. CPU 110 writes information on the number / type of number “2” in second field DA03. The CPU 110 writes the value of the display range whose number is “3” in the third field DA04. The CPU 110 writes information related to the image data with the number “4” in the fourth field DA05.

  The second command in which “001” is set in the first field DA02 requests the image processing engine 280 to display an image on the liquid crystal panel 240 (sub-screen). The second command in which “010” is set in the first field DA02 requests the image processing engine 280 to display an icon on the liquid crystal panel 240. Further, the second command in which “011” is set in the first field DA02 requests the image processing engine 280 to display the handwriting area on the liquid crystal panel 240.

  The second field DA03 stores the number of images to be displayed on the liquid crystal panel 240 and a number for designating the type of handwriting language. The image processing engine 280 performs processing according to the number of images or the type of language.

  The second command in which “01” is set in the third field DA04 requests the image processing engine 280 to designate the display range on the liquid crystal panel 240 by coordinates. Further, the second command in which “10” is set in the third field DA04 requests the image processing engine 280 to set the display range on the liquid crystal panel 240 to the entire display area.

  The fourth field DA05 stores image data to be displayed on the liquid crystal panel 240 and position information for displaying the image data. The image processing engine 280 performs processing for displaying the image data at a position specified by the position information.

  FIG. 10 is a diagram for explaining a command of type “010” (hereinafter referred to as “third command”). The CPU 110 sends the third command from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 210 sends a third command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPUs 110 and 210 write the command type (“001”), the command transmission destination, and the like in the header DA01. The CPUs 110 and 210 write the value of the OS (Operating System) processing request with the number “1” in the first field DA02. The CPUs 110 and 210 write the value of the OS information with the number “2” in the second field DA03.

  The third command in which “01” or “10” is set in the first field DA02 is transmitted from the second unit 1002 to the first unit 1001.

  The third command in which “01” is set in the first field DA02 requests the first unit 1001 to transmit information indicating the type of OS of the first unit 1001 (main device). The third command in which “10” is set in the first field DA02 requests the first unit 1001 to start the OS specified by the OS information.

  The third command in which “000”, “001”, or “010” is set in the second field DA03 is transmitted from the second unit 1002 to the first unit 1001.

  The third command in which “000” is set in the second field DA03 does not request the activation of the OS in the first unit 1001. The third command in which “001” is set in the second field DA03 indicates that the second unit 1002 has selected to start the first OS. Furthermore, the third command in which “010” is set in the second field DA03 indicates that the second unit 1002 has selected to start the second OS.

  FIG. 11 is a diagram for explaining a command of type “011” (hereinafter referred to as “fourth command”). The CPU 210 sends a fourth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPU 210 writes the command type (“011”), the command transmission destination, and the like in the header DA01. The CPU 210 writes information related to the activated application with the number “1” in the first field DA02. The CPU 210 writes the startup information with the number “2” in the second field DA03.

  In the first field DA02, information specifying an application to be activated in the first unit 1001 is stored. The second field DA03 stores information used at the time of activation setting and information used after activation.

  FIG. 12 is a diagram for explaining a command of type “100” (hereinafter referred to as “fifth command”). The CPU 210 sends the fifth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPU 210 writes the command type (“100”), the command transmission destination, and the like in the header DA01. The CPU 210 writes information related to the reception request with the number “1” in the first field DA02. The CPU 210 writes information relating to the number “2” in the second field DA03. The CPU 210 writes information related to the file with the number “3” in the third field DA04.

  The fifth command in which “01” is set in the first field DA02 requests the first unit 1001 to receive a file. In the second field DA03, the number of files transmitted from the second unit 1002 to the first unit 1001 is stored. Further, a file transmitted from the second unit 1002 to the first unit 1001 is stored in the third field DA04.

  FIG. 13 is a diagram for explaining a command of type “101” (hereinafter referred to as “sixth command”). The CPU 110 sends the sixth command from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 210 sends the sixth command from the main body device 104 of the second unit 1002 to the first unit 1001.

  The CPUs 110 and 210 write the command type (“101”), the command transmission destination, and the like in the header DA01. The CPUs 110 and 210 write the value of the communication type with the number “1” in the first field DA02. The CPUs 110 and 210 write the value of the connection destination with the number “2” in the second field DA03. The CPUs 110 and 210 write the value of the transfer destination with the number “3” in the third field DA04. The CPUs 110 and 210 write the value of the signal strength acquisition timing number “4” in the fourth field DA05.

  The sixth command in which “001” is set in the first field DA02 requests the counterpart device to perform infrared communication. The sixth command in which “010” is set in the first field DA02 requests the counterpart device to perform wireless communication by Bluetooth (registered trademark). Furthermore, the sixth command in which “011” is set in the first field DA02 requests the counterpart device to perform LAN communication.

  The sixth command in which “000” is set in the second field DA03 indicates that there is no information for designating the connection destination of communication.

  Further, the sixth command in which “001” is set in the second field DA03 is transmitted by the first unit 1001 to the device to which the first unit 1001 is connected. Such a sixth command requests transmission of information related to a device to which the first unit 1001 is connected.

  Further, the sixth command in which “010” is set in the second field DA03 is transmitted by the second unit 1002 to the first unit 1001 to which the second unit 1002 is connected. Such a sixth command requests transmission of information regarding the first unit 1001 to which the second unit 1002 is connected.

  Further, the sixth command in which “011” is set in the second field DA03 is transmitted by the second unit 1002 to the first unit 1001 to which the second unit 1002 is connected. Such a sixth command requests that information regarding the second unit 1002 be set as connection destination device information.

  Further, the sixth command in which “100” is set in the second field DA03 is transmitted by the first unit 1001 to a device to which the first unit 1001 is connected (for example, the second unit 1002). Such a sixth command requests that information regarding the first unit 1001 be set as connection destination device information.

  The sixth command in which “000” is set in the third field DA04 indicates that it does not have information specifying the transfer destination of data (for example, a file).

  The sixth command in which “001” is set in the third field DA04 is transmitted by the first unit 1001 to the data transfer destination device. Such a sixth command requests transmission of information relating to the data transfer destination device.

  Further, the sixth command in which “010” is set in the third field DA04 is transmitted by the second unit 1002 to the first unit 1001 that is the data transfer destination. Such a sixth command requests transmission of information related to the first unit 1001 of the data transfer destination.

  The sixth command in which “011” is set in the third field DA04 is transmitted by the second unit 1002 to the first unit 1001 that is the data transfer destination. Such a sixth command requests that information regarding the second unit 1002 be set as device information of the data transfer source.

  Furthermore, the sixth command in which “100” is set in the third field DA04 is transmitted by the first unit 1001 to the data transfer destination device (for example, the second unit 1002). Such a sixth command requests that information regarding the first unit 1001 be set as device information of the data transfer source.

  The sixth command in which “00”, “01”, “10”, or “11” is set in the fourth field DA05 is transmitted by the first unit 1001 to the second unit 1002.

  The sixth command in which “00” is set in the fourth field DA05 does not request the second unit 1002 to transmit data indicating the signal strength. In addition, the sixth command in which “01” is set in the fourth field DA05 requests the signal strength detection unit 297 to transmit data indicating the signal strength at that time. Furthermore, the sixth command in which “10” is set in the fourth field DA05 requests transmission of data indicating the signal strength when the signal strength is changed. Further, the sixth command in which “11” is set in the fourth field DA05 requests the transmission of data indicating the signal strength at regular intervals.

  FIG. 14 is a diagram showing a schematic configuration of response data. The response data is data corresponding to the content of the first command (sensing command).

  When the first command is transmitted from the main device 101 to the second unit 1002, the CPU 210 transmits response data from the display device 103 to the first unit 1001. When the first command is transmitted from the main device 101 to the display device 102 of the first unit 1001, the image processing engine 180 transmits response data from the image processing engine 180 to the main device 101. Hereinafter, a case where the first command is transmitted from the main device 101 to the second unit 1002 will be described as an example.

  Referring to FIG. 14, the response data includes a header data area DA11, a coordinate data area DA12, a time data area DA13, and an image data area DA14. Here, the value of the center coordinates of the partial image is written in the data area DA12 indicating the coordinates. The time information acquired from the timer 282 of the image processing engine 280 is written in the data area indicating the time. Further, image data (that is, scan data) after being processed by the image processing engine 280 is written in the data area indicating the image.

  FIG. 15 is a diagram illustrating an image obtained by scanning the finger 900 (that is, a scanned image). Referring to FIG. 15, an image of a region W1 surrounded by a thick solid line is an entire image, and an image of a region P1 surrounded by a broken line is a partial image. The center point C1 of the cross indicated by a thick line is the center coordinate.

  In the present embodiment, a pixel (that is, a predetermined gradation or a pixel having a photosensor circuit which is a rectangular region and whose output voltage from the sensor signal line SSj is equal to or higher than a predetermined value). An area including all of the pixels having a luminance equal to or higher than a predetermined luminance is set as a partial image area.

  The center coordinates are coordinates determined in consideration of the gradation of each pixel in the partial image area. Specifically, for each pixel in the partial image, the center coordinates are determined by performing weighting processing based on the gradation of the pixel and the distance between the pixel and the rectangular center point (that is, the centroid). Is done. That is, the center coordinates do not necessarily match the centroid of the partial image.

  However, the position of the center coordinates is not necessarily limited to the above, and the center coordinates may be the coordinates of the centroid or the coordinates near the centroid.

  When “001” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the center coordinate in the data area DA12 indicating the coordinates. In this case, the image processing engine 280 does not write image data in the data area DA14 indicating an image. After writing the value of the center coordinate, the image processing engine 280 sends response data including the value of the center coordinate to the main body device 104. The main device 104 sends response data including the value of the center coordinate to the main device 101 of the first unit 1001. As described above, when “001” is set in the data area indicating the data type, the first command does not request output of the image data but requests output of the value of the center coordinate.

  When “010” is set in the data area indicating the data type of the first command, the image processing engine 280 stores the image data of the partial image whose scan result has changed in the data area DA14 indicating the image. Write. In this case, the image processing engine 280 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 280 writes the image data of the partial image whose scan result has changed, and then sends response data including the image data of the partial image to the main body device 104. The main device 104 sends response data including image data of the partial image to the main device 101 of the first unit 1001. Thus, when “010” is set in the data area indicating the data type, the first command does not request the output of the value of the center coordinate, and the image data of the partial image whose scan result has changed. Request output.

  As described above, the reason for requesting transmission of only the partial image whose scan result has changed is that the scan data of the partial image area of the scan data is more important than the scan data of the other area. This is because the data is high, and the scan data in the region corresponding to the region of the partial image in the scan data is likely to change due to the contact state with the scan object such as the finger 900.

  When “011” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the center coordinate in the data area DA12 indicating the coordinate and also indicates the data indicating the image. The image data of the partial image whose scan result has changed is written in the area DA14. Thereafter, the image processing engine 280 sends response data including the value of the center coordinate and the image data of the partial image to the main body device 104. The main device 104 sends response data including the value of the center coordinate and the image data of the partial image to the main device 101 of the first unit 1001. Thus, when “011” is set in the data area indicating the data type, the first command outputs the value of the center coordinate and the output of the image data of the partial image whose scan result has changed. Request.

  Further, when “100” is set in the data area indicating the data type of the first command, the image processing engine 280 displays the entire image in the data area DA14 indicating the image of the response data shown in FIG. Write image data. In this case, the image processing engine 280 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 280 writes the image data of the whole image and then sends response data including the image data of the whole image to the main body device 104. The main device 104 sends response data including image data of the entire image to the main device 101 of the first unit 1001. As described above, when “100” is set in the data area indicating the data type, the first command requests the output of the image data of the entire image without requesting the output of the center coordinate value. .

  If “101” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the central coordinate in the data area DA12 indicating the coordinate and also indicates the data indicating the image. The image data of the entire image is written in the area DA14. Thereafter, the image processing engine 280 sends response data including the value of the center coordinate and the image data of the entire image to the main body device 104. The main device 104 sends response data including the value of the center coordinate and the image data of the entire image to the main device 101 of the first unit 1001. As described above, when “101” is set in the data area indicating the data type, the first command requests the output of the value of the center coordinate and the output of the image data of the entire image.

<5. About First Modification of Configuration>
By the way, the structure of the liquid crystal panel 140 is not limited to the structure shown in FIG. Hereinafter, a liquid crystal panel having a mode different from that in FIG. 3 will be described.

  FIG. 16 is a circuit diagram of a photosensor built-in liquid crystal panel 140A according to the different embodiment. Referring to FIG. 16, photosensor built-in liquid crystal panel 140A (hereinafter referred to as liquid crystal panel 140A) includes three photosensor circuits (144r, 144g, 144b) in one pixel. Thus, the liquid crystal panel 140A is different from the liquid crystal panel 140 including one photosensor circuit in one pixel in that the liquid crystal panel 140A includes three photosensor circuits (144r, 144g, and 144b) in one pixel. The configuration of the optical sensor circuit 144 is the same as that of each of the three optical sensor circuits (144r, 144g, 144b).

  Further, the three photodiodes (145r, 145g, 145b) in one pixel are arranged at positions facing the color filter 153r, the color filter 153g, and the color filter 153b, respectively. Therefore, the photodiode 145r receives red light, the photodiode 145g receives green light, and the photodiode 145b receives blue light.

  In addition, since the liquid crystal panel 140 includes only one photosensor circuit 144 in one pixel, two data signal lines for the TFT 147 disposed in one pixel are a sensor signal line SSj and a sensor signal line SDj. Met. However, since the liquid crystal panel 140A includes three photosensor circuits (144r, 144g, 144b) in one pixel, there are six data signal lines for TFTs (147r, 147g, 147b) arranged in one pixel. It becomes.

  Specifically, the sensor signal line SSRj and the sensor signal line SDRj are arranged corresponding to the TFT 147r connected to the cathode of the photodiode 145r arranged at a position facing the color filter 153r. A sensor signal line SSGj and a sensor signal line SDGj are arranged corresponding to the TFT 147g connected to the cathode of the photodiode 145g arranged at a position facing the color filter 153g. Further, a sensor signal line SSBj and a sensor signal line SDBj are disposed corresponding to the TFT 147b connected to the cathode of the photodiode 145b disposed at a position facing the color filter 153b.

  In such a liquid crystal panel 140A, the white light emitted from the backlight 179 passes through the three color filters (153r, 153g, 153b), and red, green, and blue are displayed on the surface of the liquid crystal panel 140A. It mixes and becomes white light. Here, when white light is reflected by the scan object, a part of the white light is absorbed by the pigment on the surface of the scan object, and a part is reflected. And the reflected light permeate | transmits three color filters (153r, 153g, 153b) again.

  At this time, the color filter 153r transmits light having a red wavelength, and the photodiode 145r receives light having the red wavelength. The color filter 153g transmits light having a green wavelength, and the photodiode 145g receives light having the green wavelength. The color filter 153b transmits light having a blue wavelength, and the photodiode 145b receives light having the blue wavelength. That is, the light reflected by the scan object is separated into three primary colors (R, G, B) by the three color filters (153r, 153g, 153b), and each photodiode (145r, 145g, 145b) The light of the color corresponding to is received.

  When a part of white light is absorbed by the pigment on the surface of the scan target, the amount of light received by each photodiode (145r, 145g, 145b) is different for each photodiode (145r, 145g, 145b). For this reason, the output voltages of the sensor signal line SSRj, the sensor signal line SSGj, and the sensor signal line SSBj are different from each other.

  Therefore, the image processing engine 180 determines the R gradation, the G gradation, and the B gradation according to each output voltage, so that the image processing engine 180 sends the RGB color image to the main body device 101. Can send.

  As described above, when the electronic apparatus 100 includes the liquid crystal panel 140A, the scan target can be scanned in color.

  Next, a scanning method different from the above-described scanning method (that is, the method of scanning the reflected image in FIG. 6) will be described with reference to FIG.

  FIG. 17 is a cross-sectional view showing a configuration in which a photodiode receives external light during scanning. As shown in the figure, part of the external light is blocked by the finger 900. Therefore, the photodiode disposed under the surface region of the liquid crystal panel 140 that is in contact with the finger 900 can hardly receive external light. In addition, although the photodiodes disposed under the surface area where the shadow of the finger 900 is formed can receive a certain amount of external light, the amount of external light received is small compared to the surface area where no shadow is formed.

  Here, by turning off the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the position of the finger 900 with respect to the surface of the liquid crystal panel 140 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), and the liquid crystal panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter also referred to as a shadow image) obtained by blocking external light with the finger 900.

  Further, the display device 102 may be configured to perform scanning by turning on the backlight 179 and then performing scanning again by turning off the backlight 179. Alternatively, the display device 102 may be configured such that after the backlight 179 is turned off and the scan is performed, the backlight 179 is turned on and the scan is performed again.

  In this case, since two scanning methods are used together, two scan data can be obtained. Therefore, it is possible to obtain a highly accurate result as compared with the case of scanning using only one scanning method.

<6. About display device>
The operation of the display device 103 is controlled in accordance with a command (for example, a first command) from the main body device 101 as in the operation of the display device 102. The display device 103 has the same configuration as the display device 102. Therefore, when the display device 103 receives the same command from the main body device 101 as the display device 102, the display device 103 performs the same operation as the display device 102. For this reason, description of the configuration and operation of the display device 103 will not be repeated.

  Note that the main device 101 can send commands having different commands to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform different operations. Further, the main device 101 may send a command to either the display device 102 or the display device 103. In this case, only one display device performs an operation according to the command. Further, the main device 101 may send a command having the same command to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform the same operation.

  Note that the size of the liquid crystal panel 140 of the display device 102 and the size of the liquid crystal panel 240 of the display device 103 may be the same or different. Further, the resolution of the liquid crystal panel 140 and the resolution of the liquid crystal panel 240 may be the same or different.

<7. Regarding Second Modification of Configuration>
In the present embodiment, a configuration in which electronic device 100 includes a liquid crystal panel with a built-in optical sensor such as liquid crystal panel 140 and liquid crystal panel 240 will be described. However, only one liquid crystal panel has a built-in photosensor. It may be.

  FIG. 18 is a block diagram illustrating a hardware configuration of electronic device 1300. Similar to the electronic device 100, the electronic device 1300 includes a first housing 100A and a second housing 100B. Referring to FIG. 18, electronic device 1300 includes a first unit 1001A and a second unit 1002. The first unit 1001A includes a main body device 101 and a display device 102A. The second unit 1002 includes a main device 104 and a display device 103.

  The display device 102A includes a liquid crystal panel that does not include a photosensor (that is, a liquid crystal panel having only a display function). The electronic device 1300 is different from the electronic device 100 in which the first unit 1001A includes a liquid crystal panel that does not include a photosensor, and the first unit 1001 includes a liquid crystal panel 240 that includes a photosensor. Such an electronic device 1300 performs the above-described sensing using the display device 103 of the second unit 1002.

  Further, the first unit 1001 may include, for example, a resistive film type or capacitive type touch panel, instead of the liquid crystal panel 140 incorporating the photosensor.

  In this embodiment, the display device 102 includes the timer 182 and the display device 103 includes the timer 282. However, the display device 102 and the display device 103 may share one timer. .

  In the present embodiment, electronic device 100 is described as a foldable device, but electronic device 100 is not necessarily limited to a foldable device. For example, the electronic device 100 may be a sliding device configured such that the first housing 100A slides with respect to the second housing 100B.

  Since electronic device 100 according to the present embodiment is configured as described above, second unit 1002 is detachable from first unit 1001 via USB connectors 194 and 294.

  And the electronic device 100 which concerns on this Embodiment can exhibit the following functions, for example at the time of power activation. First, when the user depresses the power switch 191 of the first unit 1001, the first unit 1001 activates BIOS (Basic Input / Output System) by using the power from the power supply circuit 192.

  The second unit 1002 acquires power from the first unit 1001 via the USB connectors 194 and 294. The second unit 1002 can transmit and receive data to and from the first unit 1001 by using the power. At this time, the CPU 210 of the second unit 1002 can display the OS (Operation System) type on the liquid crystal panel 240 in a selectable manner by using the power from the USB connectors 194 and 294.

  The user selects an OS to be started up via the liquid crystal panel 240. The CPU 210 transmits a command (for example, a “first OS” command shown in FIG. 10) specifying the OS to be activated to the first unit 1001 via the USB connectors 194 and 294 according to the user's selection. . The first unit 1001 starts the OS in response to the command.

  Further, for example, the second unit 1002 transmits / receives data to / from an external mobile phone or the like via the antenna 295. The CPU 210 of the second unit 1002 acquires photographic image data and corresponding thumbnail data from an external mobile phone via the antenna 295, and stores the photographic image data and corresponding thumbnail data in the RAM 271 or the like. The CPU 210 reads the thumbnail data from the RAM 271 and causes the liquid crystal panel 240 to display a thumbnail image of the photo in a selectable manner.

  In response to a selection command from the outside, the CPU 210 causes the liquid crystal panel 240 to display a photographic image. Alternatively, the CPU 210 displays a photographic image on the liquid crystal panel 140 or the display device 102A via the USB connector 294.

  By the way, in the above, the case where scanning was performed using the backlight 179 was described as one scanning method. However, as a scanning method, the following methods are further exemplified. First, a light source that emits infrared light is built in the liquid crystal panels 140 and 240. For example, the electronic device 100 is configured to include one light source in association with one pixel. The electronic device 100 performs scanning by controlling on / off of the infrared irradiation. That is, electronic device 100 performs the same control as the scan using light from backlight 179 by irradiating infrared rays instead of irradiating light (for example, white light) from backlight 179. Also by such a method, the same effect as the case where the backlight 179 is used can be acquired.

<8. Hardware Configuration of Electronic Device 100>
The processing in the electronic device 100 is realized by each hardware and software executed by the CPUs 110 and 210. Such software may be stored in advance in the RAM 171 or a hard disk (not shown). The software may be stored in a memory card 1731 or a DVD-ROM (not shown) or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by a reading device such as the memory card reader / writer 173 or downloaded via the communication IF 8180 and then temporarily stored in the RAM 171 or the hard disk. The software is read from the RAM 171 and the hard disk by the CPU 110. CPU 110 executes the program.

  Each component constituting the main device 101, 104 of the electronic device 100 is a general one. Therefore, it can be said that the essential part of the present invention is the software stored in the RAM 171 or the hard disk, the memory card 1731 and other storage media, or the software downloadable via the network.

  Recording media are not limited to memory cards, DVD-ROMs, CD-ROMs, FDs (Flexible Disks), and hard disks, but are also magnetic tapes, cassette tapes, optical disks (MO (Magnetic Optical Discs) / MDs (Mini Discs) / Semiconductor memory such as DVD (Digital Versatile Disc), IC (Integrated Circuit) card, optical card, mask ROM, EPROM (Electronically Programmable Read-Only Memory), EEPROM (Electronically Erasable Programmable Read-Only Memory), flash ROM, etc. It may be a medium that carries the program in a fixed manner.

  The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

<< First Specific Implementation Example (2 Screens) >>
Hereinafter, for convenience of explanation, a specific implementation example will be described by taking as an example the case where the electronic device 1300 (see FIG. 18) is used. In addition, since it is the same as that of the case where the electronic device 1300 is used when the electronic device 100 is used, description is not repeated.

<1. Outline of Operation of Electronic Device 1300>
An outline of the operation of the electronic apparatus 1300 will be described with reference to FIGS. When the electronic device 1300 detects that an object has contacted the display screen of the liquid crystal panel 240, the electronic device 1300 sets the operation mode of the electronic device 1300 to an operation mode corresponding to the contact angle of the object. Hereinafter, a case where the object is a user's finger will be described as an example. The object is not limited to a finger, and may be, for example, a stylus pen or a writing instrument.

  FIG. 19 is a diagram showing a case where the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 while the user's finger 900 is lying on the liquid crystal panel 240. FIG. 19A is a perspective view when a user's finger 900 is brought into contact with the user's finger laid down. FIG. 19B is a cross-sectional view taken along line XIX-XIX in FIG. FIG. 19C is a diagram showing a scan image obtained with the finger 900 lying on the screen.

  When the electronic device 1300 detects that the finger 900 has touched the display screen of the liquid crystal panel 240 in the state of FIG. 19, the electronic device 1300 sets the operation mode of the electronic device 1300 to an operation mode that processes the contact of the finger 900 as a pointer operation. To do. Note that “pointer operation” refers to an operation of moving the mouse cursor. More specifically, when the electronic device 1300 detects that the finger 900 has touched the display screen of the liquid crystal panel 240 in the state of FIG. 19, the electronic device 1300 continuously receives coordinate input and sets the display position of the mouse cursor according to the input. change.

  FIG. 20 is a diagram illustrating a case where the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 while the user's finger 900 is raised with respect to the liquid crystal panel 240. FIG. 20A is a perspective view when the user's finger 900 is brought into contact with the finger 900 in an upright state. FIG. 20B is a cross-sectional view taken along line XX-XX in FIG. FIG. 20C is a diagram showing a scan image obtained with the finger 900 raised.

  When the electronic device 1300 detects that the finger 900 has touched the display screen of the liquid crystal panel 240 in the state of FIG. 20, the electronic device 1300 sets the operation mode of the electronic device 1300 to an operation mode that processes the touch of the finger 900 as a touch operation. To do. The “touch operation” refers to, for example, an operation for selecting an object (for example, a button) displayed on the screen.

  More specifically, when the electronic device 1300 detects that the finger 900 has touched the display screen of the liquid crystal panel 240 in the state of FIG. 20, the electronic device 1300 displays an image associated with the area touched by the finger 900. While specifying, the process which the said image shows is performed. For example, when the finger 900 touches an icon (image) indicating overwriting saving of a file created by the document creation program in the state of FIG. 20, the electronic device 1300 executes overwriting saving of the file.

  As described above, the electronic device 1300 processes the operation mode of the electronic device 1300 as an operation mode for processing the contact as a pointer operation and a touch operation in accordance with the contact angle of the finger 900 with respect to the display screen of the liquid crystal panel 240. Set to one of the operating modes. That is, electronic device 1300 sets the operation mode of electronic device 1300 to an operation mode corresponding to the contact angle of finger 900.

  FIG. 21 is a diagram specifically showing the “pointer operation” when the finger 900 touches the display screen of the liquid crystal panel 240 in the state of FIG. FIG. 21A is a diagram illustrating the movement of the finger 900. FIG. 21B is a diagram for explaining the movement of the mouse cursor.

  Referring to FIG. 21A, the user moves the finger 900 to the liquid crystal panel 240 as indicated by an arrow 2101 at a contact angle within a predetermined first range (hereinafter referred to as “contact angle θ1”). Is moved on the display screen. Here, the “contact angle” refers to an angle formed between the normal of the display screen and the finger 900 when the finger 900 is in contact with the display screen of the liquid crystal panel 240. The “first range” is, for example, not less than 45 degrees and not more than 90 degrees. The “contact angle” may be defined as an angle formed between the display screen and the finger 900 when the finger 900 is in contact with the display screen of the liquid crystal panel 240.

  Here, when the user moves the finger 900 as indicated by the arrow 2101 as described above, the electronic device 1300 displays the mouse displayed on the display screen of the display device 102A as shown in FIG. The cursor 2103 is moved in the direction of the arrow 2102 based on the movement.

  FIG. 22 is a diagram specifically showing the “touch operation” when the finger 900 touches the display screen of the liquid crystal panel 240 in the state of FIG. 20. FIG. 22A shows the movement of the finger 900. FIG. 21B is a diagram for explaining processing associated with the touch operation.

  Referring to FIG. 22A, the user moves the finger 900 in the direction of the arrow 2201, and the liquid crystal has a contact angle within a predetermined second range (hereinafter referred to as “contact angle θ2”). Assume that the finger 900 is brought into contact with the display screen of the panel 240. The “second range” is, for example, not less than 0 degrees and less than 45.

  Here, when the user performs an operation of bringing the finger 900 into contact with the button icon 2202 at the contact angle θ2, the electronic device 1300 performs processing according to the selection of the button icon 2202 as illustrated in FIG. Execute.

  FIG. 23 is a diagram illustrating the pointer operation and the touch operation in one drawing. Referring to FIG. 23, when the user moves finger 900 in the direction of arrow 2301 at contact angle θ1 on the display screen of liquid crystal panel 240, electronic device 1300 displays a mouse displayed on display device 102A. The cursor 2303 is moved in the direction of the arrow 2302.

  On the other hand, when the user brings the finger 900 into contact with the display screen of the liquid crystal panel 240 at the contact angle θ2, the electronic device 1300 performs processing associated with selection of an image displayed on the liquid crystal panel 240. For example, when the electronic device 1300 displays a website on the liquid crystal panel 240 and the selected image is an image for specifying a link destination image, the displayed image is used as the link destination image. Switch.

  With the above configuration, by using the liquid crystal panel 240, the user can immediately switch whether the input through the liquid crystal panel 240 is a pointer operation or a touch operation. As described above, the electronic device 1300 can realize an intuitive operation for the user.

  In addition, the user does not need to touch the display screen of the display device 102A, which is the main screen, with the finger 900 when switching operations. Further, by using the electronic device 1300, the user can perform work on a plane parallel to the work surface of the desk on which the electronic device 1300 is placed. Therefore, the electronic device 1300 is convenient for the user.

  In the above description, examples of the pointer operation and the touch operation have been described. However, the present invention is not limited to the operation. In the above description, the electronic apparatus 1300 is described as having a configuration in which the operation mode is set according to the contact angle of the finger 900 with respect to the display screen of the liquid crystal panel 240. However, the electronic device 1300 is not necessarily required to use the contact angle. Hereinafter, first, the configuration in which the electronic device 1300 uses the contact angle will be described in more detail. Next, a configuration in which the electronic device 1300 uses a contact angle will be described.

<2. Functional block>
FIG. 24 is a diagram illustrating a block of the electronic device 1300. Referring to FIG. 24, electronic device 1300 includes display device 102 </ b> A, display device 103, storage device 390, and control unit 300. The storage device 390 is, for example, a RAM 171, a ROM 172, a RAM 271, or a ROM 272. The control unit 300 includes a data acquisition unit 310, a region detection unit 311, a distance calculation unit 312, an angle detection unit 313, a setting unit 314, a direction determination unit 315, a light amount determination unit 316, and a number measurement unit 317. A number determining unit 318, an input processing unit 319, and a display control unit 320.

  The display device 103 includes the liquid crystal panel 240 as described above. That is, the display device 103 scans the finger 900 on the display screen of the liquid crystal panel 240 via the display screen.

  The data acquisition unit 310 instructs the display device 103 to execute the scan. In addition, the data acquisition unit 310 acquires scan data obtained by the scan from the display device 103.

  Here, data indicating the shape of the finger 900 in the scan data is referred to as “shape data”. The shape data includes first element data indicating the shape of the contact area in contact with the finger 900 on the display screen of the liquid crystal panel 240 and second element data indicating the shape of the vicinity area of the contact area. It is out. Hereinafter, a region obtained by combining the contact region and the nearby region is referred to as a “detection region”. Note that specific examples of the contact area, the vicinity area, and the detection area will be described later (FIG. 30 and the like).

  The area detection unit 311 detects the detection area and the contact area based on the shape data. The distance calculation unit 312 calculates the distance between the center point of the contact area and the point farthest from the center point in the detection area.

  The angle detection unit 313 detects a contact angle based on the shape data. More specifically, the angle detection unit 313 detects the contact angle based on at least the data indicating the detection area and the first element data indicating the contact area. More specifically, the angle detection unit 313 includes the calculated distance, which is data based on the data indicating the detection area and the first element data indicating the contact area, and a first threshold (δpn) described later. The contact angle is detected based on the second threshold value (p (xpk) −δppk) (see FIG. 34). A specific contact angle detection method of the angle detection unit 313 will be described later.

  The setting unit 314 sets the operation mode of the electronic device 1300 to one of a plurality of operation modes based on the shape data indicating the shape of the finger 900 in the scan data. For example, the setting unit 314 sets the operation mode of the electronic device 1300 to an operation mode that accepts a touch on the display screen of the liquid crystal panel 240 as a “pointer operation” or an operation mode that accepts the contact as a “touch operation”. More specifically, the setting unit 314 sets the operation mode of the electronic device 1300 based on the contact angle detected by the angle detection unit 313.

  The display control unit 320 displays various images on the display device 102A and the display device 103. For example, the display control unit 320 displays the website on the display devices 102A and 103.

  The direction determination unit 315 will be described later in the section “<6. Finger direction>”. The light quantity determination unit 316, the number measurement unit 317, and the number determination unit 318 will be described in the section “<7. Selection of scan method>” described later. The input processing unit 319 will be described later in “<9. Character input>”.

<3. Control structure>
(1) A control structure of the electronic device 1300 will be described. FIG. 25 is a flowchart showing a flow of processing executed by the electronic device 1300.

  Referring to FIG. 25, electronic device 1300 detects whether or not finger 900 has touched the display screen of liquid crystal panel 240 in step S2. When electronic device 1300 determines that finger 900 has touched the display screen (YES in step S2), in step S4, whether or not the contact angle of finger 900 with respect to the display screen is equal to or greater than a predetermined threshold θth. Judging.

  The threshold value θth is, for example, a value that distinguishes the first range and the second range described above. In the example described above, the threshold θth is 45 degrees. On the other hand, when electronic device 1300 determines that finger 900 is not in contact with the display screen (NO in step S2), the process returns to step S2.

  If the contact angle θ of the finger 900 is equal to or greater than the threshold θth (YES in step S4), the electronic device 1300 changes the operation mode of the electronic device 1300 to an operation mode that processes the contact of the finger 900 as a pointer operation in step S6. Set. That is, the electronic device 1300 performs a pointer operation. On the other hand, when the contact angle θ of the finger 900 is less than the threshold θth (NO in step S4), the electronic device 1300 performs an operation of processing the operation mode of the electronic device 1300 as a touch operation in step S8. Set to mode. That is, the electronic device 1300 performs a touch operation.

  As described above, the electronic device 1300 detects the contact angle θ and executes processing according to the detection result.

  FIG. 26 is a flowchart illustrating a process flow of the electronic device 1300 when three threshold values related to the contact angle are provided. Referring to FIG. 26, electronic device 1300 detects whether or not finger 900 has touched the display screen of liquid crystal panel 240 in step S12. When electronic device 1300 determines that finger 900 has touched the display screen (YES in step S12), in step S14, whether or not contact angle θ of finger 900 with respect to the display screen is equal to or greater than a predetermined threshold value α. Determine whether. On the other hand, when electronic device 1300 determines that finger 900 is not in contact with the display screen (NO in step S2), the process returns to step S12.

  When the contact angle θ of the finger 900 is equal to or greater than the threshold value α (YES in step S14), electronic device 1300 sets the operation mode of electronic device 1300 to the operation mode for executing operation A in step S16. On the other hand, in the electronic device 1300, when the contact angle θ of the finger 900 is less than the threshold α (NO in step S14), in step S18, the contact angle θ of the finger 900 with respect to the display screen is greater than or equal to a predetermined threshold β. Judge whether there is. Note that β <α.

  When the contact angle θ of the finger 900 is equal to or larger than the threshold β (YES in step S18), electronic device 1300 sets the operation mode of electronic device 1300 to the operation mode for executing operation B in step S20. On the other hand, in the electronic device 1300, when the contact angle θ of the finger 900 is less than the threshold β (NO in step S18), in step S22, the contact angle θ of the finger 900 with respect to the display screen is greater than or equal to a predetermined threshold γ. Judge whether there is. Note that γ <β.

  If the contact angle θ of the finger 900 is equal to or greater than the threshold γ (YES in step S22), electronic device 1300 sets the operation mode of electronic device 1300 to the operation mode for executing operation C in step S24. On the other hand, when the contact angle θ of the finger 900 is less than the threshold γ (NO in step S22), electronic device 1300 sets the operation mode of electronic device 1300 to the operation mode for executing operation D in step S26.

  As described above, the electronic device 1300 can process the contact of the finger 900 as three or more different operations by setting two or more thresholds related to the contact angle.

  (2) FIG. 27 is a flowchart showing the flow of processing executed by the electronic device 1300 in more detail than FIG. Referring to FIG. 27, electronic device 1300 detects whether or not finger 900 has touched the display screen of liquid crystal panel 240 in step S102. When electronic device 1300 determines that finger 900 has touched the display screen (YES in step S102), in step S104, whether or not the contact angle of finger 900 with respect to the display screen is equal to or greater than a predetermined threshold θth. Judging.

  If the contact angle θ of the finger 900 is equal to or greater than the threshold θth (YES in step S104), electronic device 1300 performs a pointer operation in step S106. On the other hand, when the contact angle θ of the finger 900 is less than the threshold θth (NO in step S104), electronic device 1300 performs a touch operation in step S112. Here, the threshold value θth is a common value in steps S104, S110, and S116. However, for example, 45 degrees in step S104, 30 degrees in step S110, 30 degrees in step S116, and 60 degrees in step S116. The threshold value θth may be used. By using different threshold values θth in this way, electronic device 1300 can “become unable to shift to a different operation mode unless a larger change is made once the operation mode is determined”. Note that the processing described above is the same as the processing described based on FIG.

  In step S <b> 108, electronic device 1300 detects whether or not finger 900 has left the display screen of liquid crystal panel 240. When electronic device 1300 determines that finger 900 has left the display screen (YES in step S108), it returns the process to step S102. On the other hand, when electronic device 1300 determines that finger 900 is not separated from the display screen (NO in step S108), in step S110, the contact angle of finger 900 with respect to the display screen is equal to or greater than a predetermined threshold θth. Determine whether or not.

  If the contact angle θ of the finger 900 is equal to or greater than the threshold θth (YES in step S110), electronic device 1300 returns the process to step S106. On the other hand, when the contact angle θ of the finger 900 is less than the threshold θth (NO in step S110), electronic device 1300 advances the process to step S112.

  In step S <b> 114, electronic device 1300 detects whether finger 900 has left the display screen of liquid crystal panel 240. If electronic device 1300 determines that finger 900 has left the display screen (YES in step S114), it returns the process to step S102. On the other hand, when electronic device 1300 determines that finger 900 is not separated from the display screen (NO in step S114), in step S116, the contact angle of finger 900 with respect to the display screen is equal to or greater than a predetermined threshold θth. Determine whether or not.

  When the contact angle θ of the finger 900 is equal to or greater than the threshold θth (YES in step S116), electronic device 1300 advances the process to step S106. On the other hand, when the contact angle θ of the finger 900 is less than the threshold θth (NO in step S116), electronic device 1300 returns the process to step S112.

  (3) FIG. 28 is a flowchart showing the flow of processing executed by the electronic device 1300 in more detail than FIG. FIG. 28 is a flowchart showing a more practical processing flow than FIG.

  Referring to FIG. 28, electronic device 1300 detects whether or not finger 900 has touched the display screen of liquid crystal panel 240 in step S202. When electronic device 1300 determines that finger 900 has touched the display screen (YES in step S202), in step S204, timer 282 is reset (t = 0), and time measurement is started using timer 282. To do. In step S206, electronic device 1300 determines whether or not the contact angle of finger 900 with respect to the display screen is equal to or greater than a predetermined threshold value θth.

  If the contact angle θ of the finger 900 is equal to or greater than the threshold θth (YES in step S206), electronic device 1300 performs a pointer operation in step S208. Specifically, the control unit 300 of the electronic device 1300 moves the pointer according to the moving amount and moving direction of the finger 900. On the other hand, when the contact angle θ of the finger 900 is less than the threshold θth (NO in step S206), electronic device 1300 performs a touch operation in step S218. Specifically, the display device 103 outputs the coordinates touched by the finger 900 to the control unit 300, but the control unit 300 does not finalize the touch operation at this time.

  In step S210, electronic device 1300 detects whether or not finger 900 has left the display screen of liquid crystal panel 240. When electronic device 1300 determines that finger 900 has moved away from the display screen (YES in step S210), electronic device 1300 determines in step S214 whether or not measurement time t is equal to or greater than a predetermined threshold Tth.

  If electronic device 1300 determines that measurement time t is equal to or greater than a predetermined threshold Tth (Yes in step S214), it returns the process to step S202. On the other hand, when electronic device 1300 determines that measurement time t is less than a predetermined threshold Tth (No in step S214), electronic device 1300 performs pointer determination processing in step S216. Here, the pointer confirmation process is a process executed when a click operation is accepted. That is, the electronic device 1300 sets the operation mode of the electronic device 1300 to an operation mode in which the touch operation of the finger 900 is processed as a click operation.

  When electronic device 1300 determines that finger 900 is not separated from the display screen (NO in step S210), in step S212, whether or not the contact angle of finger 900 with respect to the display screen is equal to or greater than a predetermined threshold θth. Determine whether. If the contact angle θ of the finger 900 is equal to or greater than the threshold θth (YES in step S212), electronic device 1300 returns the process to step S208. On the other hand, electronic device 1300 advances the process to step S218 when contact angle θ of finger 900 is less than threshold value θth (NO in step S212).

  In step S220, electronic device 1300 detects whether or not finger 900 has left the display screen of liquid crystal panel 240. When electronic device 1300 determines that finger 900 has moved away from the display screen (YES in step S220), electronic device 1300 determines in step S224 whether or not measurement time t is equal to or greater than a predetermined threshold Tth.

  When electronic device 1300 determines that measurement time t is equal to or greater than a predetermined threshold Tth (Yes in step S224), it returns the process to step S202. On the other hand, when electronic device 1300 determines that measurement time t is less than a predetermined threshold Tth (No in step S224), electronic device 1300 performs a touch confirmation process in step S226. Here, the touch confirmation process is an output process of coordinate values for the operating system and application software of the electronic device 1300. For example, when an icon is displayed at a position corresponding to the coordinates, the touch confirmation process is the same as the process executed when a click operation is accepted.

  When electronic device 1300 determines that finger 900 is not separated from the display screen (NO in step S220), in step S222, whether or not the contact angle of finger 900 with respect to the display screen is equal to or greater than a predetermined threshold θth. Determine whether. If the contact angle θ of the finger 900 is equal to or greater than the threshold θth (YES in step S222), electronic device 1300 advances the process to step S208. On the other hand, when the contact angle θ of the finger 900 is less than the threshold θth (NO in step S222), electronic device 1300 returns the process to step S218.

<4. Processing according to elapsed time>
FIG. 29 is a diagram for explaining a part of the processing shown in the flowchart of FIG. FIG. 29A is a diagram for explaining a case where the contact angle θ is equal to or larger than the threshold value θth. FIG. 29B is a diagram for describing a case where the contact angle θ is less than the threshold θth.

  Referring to FIG. 29A, state A indicates a state where finger 900 is not in contact with the display screen of liquid crystal panel 240. That is, the state A shows a case where NO is determined in step S202 in FIG. A state B indicates a state in which the finger 900 is in contact with the display screen at a contact angle θ1 equal to or greater than the threshold θth. That is, the state B indicates the state of step S208 in FIG. The state C indicates a state in which the finger 900 is released from the display screen at less than the threshold value Tth after the time measurement is started. That is, the state C shows a state determined as NO in step S214 of FIG.

  When transitioning to the state C, the electronic device 1300 determines that the user's operation is a click operation, as described above. When the user releases the finger 900 from the display screen after the time Tth has elapsed since the start of time measurement, the electronic device 1300 determines that the user's operation is an interruption of the pointer operation.

  Referring to FIG. 29 (b), state A shows a state where finger 900 is not in contact with the display screen of liquid crystal panel 240. That is, the state A shows a case where NO is determined in step S202 in FIG. A state B indicates a state in which the finger 900 is in contact with the display screen at a contact angle θ2 that is less than the threshold θth. That is, the state B indicates the state of step S218 in FIG. The state C indicates a state in which the finger 900 is released from the display screen at less than the threshold value Tth after the time measurement is started. That is, the state C indicates a state where NO is determined in step S224 in FIG.

  When transitioning to the state C, the electronic device 1300 determines that the user's operation is a touch operation (key touch operation) as described above. When the user releases the finger 900 from the display screen after the time Tth has elapsed since the start of time measurement, the electronic device 1300 determines that the user's operation is an interruption of the touch operation.

  As described above, the electronic device 1300 determines whether the user operation is a pointer operation or a touch operation according to the contact angle θ when the finger 900 touches the display screen of the liquid crystal panel 240. Thereafter, the electronic device 1300 measures the time until the finger 900 is not in contact with the display screen. If the measured time is less than the threshold value Tth, electronic device 1300 determines that the user's operation is a click operation at the pointer position during the pointer operation, and the user's operation is a key (button etc.) touch operation during the touch operation. Judge that there is.

  If the above configuration is applied, a drag operation (an operation of moving the mouse while pressing the click button) by double-clicking (pressing and releasing twice) or tapping (pressing twice or not releasing the second time) can be realized. .

<5. Contact angle detection method>
A specific method for detecting the contact angle will be described based on three scanning methods.

(1) Internal Light Utilization Method FIG. 30 shows a scan image (based on scan data) obtained when scanning is performed using a light source that emits infrared light (internal light) built in the liquid crystal panel 240 as described above. It is a figure for demonstrating (image). FIG. 30A is a diagram showing a state in which the finger 900 is in contact with the display screen of the liquid crystal panel 240. Here, the figure of the liquid crystal panel 240 in FIG. 30A is a cross-sectional view taken along line XXX-XXX in FIG. FIG. 30B is a diagram showing an overall image of a scan image obtained when the finger 900 touches the display screen in the state of FIG.

  Referring to FIG. 30, when finger 900 touches the display screen of liquid crystal panel 240, area detection unit 311 of electronic device 1300 detects contact area 3001 and detection area 3002 based on scan data (shape data). To do. The area detection unit 311 also detects a neighboring area that is a difference area between the contact area 3001 and the detection area 3002. A specific method for detecting each area will be described later (FIG. 34).

  In FIGS. 30A and 30B, the white portion is the contact region 3001. Further, the portion where the finger 900 is floating from the display screen has a color (an intermediate color (gray) between white and black) corresponding to the distance between the finger 900 and the display screen. Other portions are black (uniform). Note that the detectable range 3003 is a range in which the voltage based on the infrared light reflected by the finger 900 does not fall below a certain value (detection lower limit value). In other words, the “detectable range” is a range in which infrared rays reflected by the finger 900 can be detected by a predetermined amount or more. The detectable range 3003 is, for example, 0 to 3 cm. In FIG. 30B, for the convenience of explanation, the arrow direction is the short direction of the display screen of the liquid crystal panel 240.

  FIG. 31 is a diagram for explaining the contact area and the detection area when scanning is performed using infrared rays and the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 at a contact angle θ1. is there.

  FIG. 31A is a diagram illustrating a state in which the finger 900 is in contact with the display screen of the liquid crystal panel 240 at the contact angle θ1. More specifically, FIG. 31A is a diagram in which the contact angle θ in FIG. 30A is θ1. FIG. 31B is a diagram showing the received light voltage obtained in the state of FIG. That is, FIG. 31B shows the voltage output from the liquid crystal panel 240 of the display device 103 in association with the position on the display screen of the liquid crystal panel 240 when the finger 900 is scanned using infrared rays. It is a graph. More specifically, FIG. 31 (b) is a diagram showing the voltage at a position on the display screen corresponding to FIG. 31 (a).

  Referring to FIG. 31, since the contact angle θ1 of the finger 900 is equal to or greater than the threshold θth, the contact area 3101 and the detection area 3102 are wider than in the case of FIG. 32 described later. Voltage δpn is a detection lower limit value. The detectable range 3103 is a range in which the detected voltage is equal to or higher than the voltage δpn.

  FIG. 32 is a diagram for explaining the contact area and the detection area when scanning is performed using infrared rays and the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 at a contact angle θ2. is there.

  FIG. 32A is a diagram showing a state in which the finger 900 is in contact with the display screen of the liquid crystal panel 240 at the contact angle θ2. More specifically, FIG. 32A is a diagram in which the contact angle θ in FIG. 30A is θ2. FIG. 32B is a diagram showing the received light voltage obtained in the state of FIG. That is, FIG. 32B shows the voltage output from the liquid crystal panel 240 of the display device 103 in association with the position on the display screen of the liquid crystal panel 240 when the finger 900 is scanned using infrared rays. It is a graph. More specifically, FIG. 32B is a diagram showing a voltage at a position on the display screen corresponding to the cross-sectional view of FIG.

  Referring to FIG. 32, since contact angle θ2 of finger 900 is less than threshold value θth, contact region 3201 and detection region 3202 are narrower than contact region 3101 and detection region 3102 in the case of FIG. 31, respectively. The detectable range 3203 is a range in which the detected voltage is equal to or higher than the voltage δpn. That is, the detectable range 3203 is the same range as the detectable range 3201 in FIG.

  From the above, the contact area and the detection area become narrower as the contact angle θ of the finger 900 with respect to the display screen becomes smaller. The electronic device 1300 detects the contact angle using such a phenomenon. Before describing a specific contact angle detection method, the contact angle of the finger 900 will be described in more detail.

  FIG. 33 is a diagram for explaining the contact angle of the finger 900. Referring to FIG. 33, the contact angle of finger 900 should be originally obtained from the center of gravity line Lg of finger 900. In other words, the contact angle of the finger 900 is strictly not the angle θ but the angle λ. However, since infrared rays are reflected from the surface of the finger 900, the electronic device 1300 cannot detect the center of gravity line. The “center of gravity line” is a line that connects the center of gravity of a cross section perpendicular to the direction in which the finger 900 extends.

  Therefore, it is conceivable to use the angle τ shown in FIG. 33 instead of the angle λ. However, when the pressure (finger pressure) of the finger 900 increases, the contact area 3001 expands. For this reason, the angle τ changes depending on the pressure of the finger 900. For this reason, it is difficult to use the angle τ.

  As described above, the electronic device 1300 calculates the contact angle (that is, the angle θ) using the center of the contact region 3001 and the maximum value of the detectable range 3003 (the distance corresponding to the detection lower limit value δpn). By adopting such a calculation method, the contact angle θ can be calculated without depending on the finger pressure of the finger 900, so that the contact angle θ can be easily calculated. In addition, when the user changes the tilt of the finger 900, the change in the contact angle θ can be detected with high accuracy.

  FIG. 34 is a diagram for explaining an example of a specific method for detecting the contact angle θ. Further, the graph showing the voltage in FIG. 34 is the same as the graph showing the voltage in FIG. Referring to FIG. 34, electronic device 1300 calculates contact angle θ using the following equation (2).

θ = arctan ((Xn_max− (Xp_max + Xp_min) / 2) / (p (Xpk) −δppk−δpn)) (2)
Here, p (Xpk) is the maximum value of the received light voltage p (X). Further, δppk is a predetermined value. The contact area 3401 is an area between the position Xp_max and the position Xp_min. The detection area 3402 is an area between the position Xn_max and the position Xn_min.

  The light reception voltage p (x) is a value obtained by subtracting the calibration value cal from the detection voltage v (x) of the photosensor. The calibration value may be different for each optical sensor.

  Further, when the finger is close to the periphery of the touch pad, the position Xn_max may not exist in the area where the photosensor is arranged. In such a case, the electronic device 1300 may perform a process of increasing the detection lower limit voltage δpn or decreasing the illuminance of infrared rays.

  In the above description, the voltage p (x) is described as an analog quantity, but a quantized digital value obtained by AD converting the analog quantity may be used as the voltage p (x). In the case of binarization, the density may be in a minute range with respect to the detection region 3402. That is, as shown in FIG. 15, when the scanned image is converted into binary values of white and black, the processing may be performed as follows.

  When the electronic device 1300 converts an image to white or black with a certain luminance as a boundary, for example, how many white pixels or black pixels exist in 25 (5 × 5) pixels. The luminance of the 5 × 5 region can be measured depending on whether or not (that is, the density). The electronic device 1300 treats the luminance as the luminance of the central pixel in the 5 × 5 region. In addition, the luminance of a pixel adjacent to the center pixel is obtained by the same process as that for the center pixel.

  However, in the adjacent pixels, 20 (4 × 5) pixels overlap with the central pixel, and thus the obtained luminance is luminance that blurs the image. For this reason, if the area for obtaining the luminance is enlarged, sufficient resolution as an image cannot be obtained. Therefore, it is necessary to make the area for obtaining the luminance minute. In addition, if the area for obtaining the luminance is too small, the luminance resolution is lowered. For this reason, the area for obtaining the luminance needs to be an appropriate size.

  As another example, the electronic device 1300 may be a method that responds to the received light voltage when converting to binary. For example, when the electronic device 1300 converts to a binary value, the second threshold value (p (xpk) −δppk) is simply converted into a binary value according to the threshold value, and the first threshold value (δpn) is Edge extraction is performed to convert the boundary into binary values. The electronic device 1300 can obtain the center point for the second threshold or the position Xn_max for the first threshold from the image.

  34, the method of detecting the contact angle θ using the received light voltage has been described. However, the contact angle θ may be calculated using received light power instead of the received light voltage. Further, in Equation (2), instead of p (Xpk), δppk, and δpn, the square root of p (Xpk), the square root of δppk, and the square root of δpn may be used, respectively.

  Alternatively, as shown in FIG. 74, in the stereoscopic display liquid crystal in which the microlens array 740 is arranged on the display surface of the liquid crystal panel 240 (hereinafter referred to as “liquid crystal surface”), the ± Light (information) is detected with an incident angle of η degrees. This incident angle ± η is equal to the parallax angle ± η given by the microlens 741 at the time of display output of the stereoscopic display liquid crystal. For example, two liquid crystal pixels are arranged for one microlens 741 with respect to the parallax angle ± η. One pixel performs display for the left eye, the other pixel performs display for the right eye, and each performs display output at a different angle toward the eye 749. Such liquid crystal pixels and microlenses 741 are arranged on a plane, and the entire screen is displayed in three dimensions. FIG. 74 is a diagram for explaining a method of calculating the contact angle θ in the stereoscopic display device. Hereinafter, as an example, a configuration in which two photodiodes 145 are arranged corresponding to one microlens 741 and a parallax angle for stereoscopic display is ± 3 degrees, that is, a total of 6 degrees will be described.

  In this case, it can be considered that one of the adjacent optical sensors has an inclination of −3 degrees with respect to the axis, and the other has an inclination of +3 degrees with respect to the axis. That is, the two optical sensors have a relative inclination of 6 degrees. More detailed description is as follows. One optical sensor detects light incident on the microlens 741 at an incident angle of 3 degrees, whereas the other optical sensor detects light incident on the microlens 741 at an incident angle of −3 degrees. The incident angle of light received by the other optical sensor and the other optical sensor is different by 6 degrees.

  Then, using information detected with an inclination of −3 degrees and information detected with an inclination of +3 degrees, two points on the X axis are obtained as points of the same finger 900. Therefore, the electronic device 1300 obtains the distance p (X) using the fact that the distance p (X) from the liquid crystal surface to the point of the finger 900 is proportional to the difference between the two obtained distances. Also good.

  Specifically, when the distance between the finger 900 and the liquid crystal surface at Xn_max is p (Xn_max), the position of Xn_max + p (Xn_max) × tan (η) and Xn_max + p (Xn_max) × tan (−η) The same point of the finger 900 can be detected at the position of. The electronic device 1300 finds two points (the above two positions) having the same reflection amount on the X axis. If the two points are X1 and X2, respectively, the distance p (X) is {(| X1-X2 |) / 2} / tan (η).

  The electronic device 1300 obtains not only the distance between the finger 900 and the liquid crystal surface at Xn_max but also the distance between the finger 900 and the liquid crystal surface at each position X. In this case, the vertical axis of FIG. 34 is the distance p (x) instead of the voltage p (x), and the graph of the detected distance p (x) is a graph having reverse characteristics to that of FIG. That is, the distance of the contact area is shorter than the detection area (that is, the vicinity area) excluding the non-contact area and the contact area. The detection area is an area where the distance p (x) is shorter than a predetermined threshold value Dth1. The contact area is an area where the distance p (x) is shorter than the threshold value Dth2 (threshold value Dth1> threshold value Dth2). Based on such a graph, the electronic device 1300 can obtain the contact angle θ as described in FIG.

  As described above, the electronic apparatus 1300 detects the light incident on the microlens 741 at two different incident angles (η, −η) by the adjacent optical sensors. In the electronic device 1300, the distance between the region (minute region) on the surface of the finger 900 and the liquid crystal surface is determined for each region, the distance between two points on the liquid crystal surface corresponding to the region, and the two incident angles. Calculated from (η, −η). The incident angle is an angle determined in advance by the characteristics of the microlens 741. After calculating the distance, the electronic device 1300 is farthest from the center point among the threshold value corresponding to the first threshold value, the threshold value corresponding to the second threshold value, the center point of the contact area, and the detection area. Based on the distance from the point, the contact angle θ is calculated.

  Further, the finger 900 does not necessarily make contact in a state parallel to the longitudinal direction or the short direction of the display screen of the liquid crystal panel 240. Therefore, when calculating the contact angle θ, the contact angle θ is also calculated using the light reception voltage p (x) corresponding to the longitudinal direction and the light reception voltage p (x) corresponding to the short direction. May be.

  As described in “<2. Function block>” above, “δpn” is the first threshold value, and “p (xpk) −δppk” is the second threshold value. The distance Di calculated by the distance calculation unit 312 is expressed by the following equation (3).

Di = Xn_max− (Xp_max−Xp_min) / 2 (3)
(2) External light (scattered light) utilization method FIG. 35 is a diagram for explaining a scan image (image based on scan data) obtained when scanning is performed using external light (see FIG. 17) that is scattered light. FIG. FIG. 35A is a diagram showing a state in which the finger 900 is in contact with the display screen of the liquid crystal panel 240. Here, the figure of the liquid crystal panel 240 in FIG. 35A is a cross-sectional view taken along line XXXV-XXXV in FIG. FIG. 35B is a diagram showing an entire image of a scan image obtained when the finger 900 contacts the display screen in the state of FIG.

  Referring to FIG. 35, when finger 900 touches the display screen of liquid crystal panel 240, area detection unit 311 of electronic device 1300 detects contact area 3501 and detection area 3502 based on scan data (shape data). To do. The area detection unit 311 also detects a neighboring area that is a difference area between the contact area 3501 and the detection area 3502.

  Note that in FIGS. 35A and 35B, the black portion is the contact region 3501. Further, the portion where the finger 900 is floating from the display screen has a color (an intermediate color (gray) between white and black) corresponding to the distance between the finger 900 and the display screen. Other portions are white. The detectable range 3503 is, for example, 0 to 1 cm.

  In addition, the detected voltage p (x) graph is a graph having characteristics opposite to those in FIG. That is, the voltage in the contact area is lower than the detection area (that is, the vicinity area) excluding the non-contact area and the contact area. In this case, the detection area is an area where the voltage is lower than a predetermined threshold. Further, the contact area is an area where the voltage is lower than a threshold smaller than the predetermined threshold. The same applies to the external light (direct) utilization method described later.

(3) External Light (Direct) Utilization Method FIG. 36 is a diagram illustrating an appearance of the electronic device 1300A. Referring to FIG. 36, electronic device 1300A has a configuration in which light source 3601 and light source 3602 are further added to the configuration of electronic device 1300. The electronic device 1300A controls on / off of the light sources 3601 and 3602.

  The light source 3601 and the light source 3602 irradiate the display screen of the liquid crystal panel 240 with light. Hereinafter, a method of performing scanning using light (direct light) emitted from the light sources 3601 and 3602 will be described.

  Hereinafter, the electronic apparatus 1300 will be described as a configuration capable of executing the scan method illustrated in FIGS. 6 and 30 and the scan method illustrated in FIGS. 17 and 35. In addition, when using the said system, the scattered light mentioned above is not necessarily required.

  FIG. 37 is a diagram for explaining a scan image (an image based on scan data) obtained when scanning is performed using light from the light sources 3601 and 3602 in FIG. FIG. 37A shows a state in which the finger 900 is in contact with the display screen of the liquid crystal panel 240. Here, the figure of the liquid crystal panel 240 in FIG. 37A is a cross-sectional view taken along line XXXVII-XXXVII in FIG. FIG. 37B is a diagram showing an overall image of a scan image obtained when the finger 900 touches the display screen in the state of FIG.

  Referring to FIG. 37, when finger 900 touches the display screen of liquid crystal panel 240, area detection unit 311 of electronic apparatus 1300A detects contact area 3701 and detection area 3702 based on scan data (shape data). To do. The area detection unit 311 also detects a neighboring area that is a difference area between the contact area 3701 and the detection area 3702. Here, the detection region 3702 is a shadow of the finger 900 due to scattered light. The detectable range 3703 is, for example, 0 to 1 cm.

  Furthermore, electronic device 1300A detects a shadow 3704 of finger 900 by light source 3601 and a shadow 3705 of finger 900 by light source 3602. Electronic device 1300A calculates contact angle θ using at least one of shadow 3704 and shadow 3705. Hereinafter, a method for calculating the contact angle θ will be described.

  Electronic device 1300A calculates contact angle θ using angle φ1 formed by shadow 3704 and the longitudinal side of the display screen of liquid crystal panel 240. Specifically, electronic device 1300A calculates contact angle θ by the following equation (4).

θ = (π−φ1) / 2 (4)
Moreover, you may obtain | require a contact angle as follows. Electronic device 1300A calculates an angle φ2 formed by shadow 3704 and shadow 3705. And electronic device 1300A calculates contact angle theta by the following formulas (5).

θ = (π−φ2) / 2 (5)
By using such a scanning method, the electronic device 1300A can calculate the contact angle θ of the finger 900 even when there is no light source for irradiating scattered light and the illuminance of the scattered light is small. Become.

<6. Finger direction>
Next, a configuration in which electronic device 1300 performs processing using direction determination unit 315 will be described (see FIG. 24). Hereinafter, the coordinate axes of the orthogonal coordinate system on the display screen of the liquid crystal panel 240 are referred to as an X axis and a Y axis. For convenience of explanation, the X axis is the axis in the longitudinal direction of the display screen, and the Y axis is the axis in the short direction of the display screen. The X axis may be the longitudinal axis, and the Y axis may be the short axis. Furthermore, in a state where the finger 900 is in contact with the display screen, a direction in which the finger 900 points toward the display screen is referred to as a “contact direction”.

  Based on the shape data described above, the direction determination unit 315 determines whether the contact direction is a first direction having a positive component of the X axis or a second direction having a negative component of the X axis. Determine if there is.

  The setting unit 314 sets the operation mode of the electronic device 1300 to one of a plurality of operation modes based on the determination result of the direction determination unit 315. When the contact direction is the first direction having a positive component of the X axis, for example, the setting unit 314 sets the operation mode of the electronic device 1300 to an operation mode that processes the contact as a pointer operation. On the other hand, when the contact direction is the second direction having a negative component of the X axis, for example, the setting unit 314 sets the operation mode of the electronic device 1300 to an operation mode that processes the contact as a touch operation. .

  FIG. 38 is a diagram for explaining processing of the electronic device 1300 when the direction determination unit 315 is used. FIG. 38A is a diagram illustrating a case where the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 at the contact angle θ2, and a state where the finger 901 is brought into contact with the display screen at the contact angle θ1. FIG. 38B is a diagram illustrating a case where the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 at the contact angle θ1 and a state where the finger 901 is brought into contact with the display screen at the contact angle θ2.

  Referring to FIG. 38A, when the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 at the contact angle θ2, the contact direction of the finger 900 is the second direction having a negative component of the X axis. Therefore, the electronic device 1300 sets the operation mode to an operation mode that processes the contact as a touch operation. Further, when the finger 901 is brought into contact with the display screen of the liquid crystal panel 240 at the contact angle θ1, the contact direction of the finger 901 is the first direction having a positive component of the X axis. The operation mode is set to an operation mode for processing the contact as a pointer operation.

  Referring to FIG. 38B, when the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 at the contact angle θ1, the contact direction of the finger 900 is the second direction having a negative component of the X axis. Therefore, the electronic device 1300 sets the operation mode to an operation mode that processes the contact as a touch operation. In addition, when the finger 901 is brought into contact with the display screen of the liquid crystal panel 240 at the contact angle θ2, the contact direction of the finger 901 is the first direction having a positive component of the X axis. The operation mode is set to an operation mode for processing the contact as a pointer operation.

  As described above, when the direction determination unit 315 is used, the electronic device 1300 sets the operation mode of the electronic device 1300 based on the contact direction of the fingers 900 and 901 regardless of the contact angle θ.

  Also, the user can use, for example, the left hand finger (finger 901 in FIG. 38) for the pointer operation and the right hand finger (finger 900 in FIG. 38) for the touch operation. Further, by using the finger 900 and the finger 901, the user can simultaneously perform a pointer operation and a touch operation.

  In the above description, the configuration in which the setting unit 314 sets the operation mode based on the determination result by the direction determination unit 315 has been described. However, the configuration of the setting unit 314 is not limited to such a configuration, and the setting unit 314 may have the following configuration. That is, the setting unit 314 sets the operation mode of the electronic device 1300 to one of a plurality of operation modes based on the detected contact angle θ and the determination result of the direction determination unit 315. Hereinafter, a specific example will be described.

  FIG. 39 is a diagram for explaining a configuration using the detected contact angle θ and the determination result by the direction determination unit 315. FIG. 39A shows a case where the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 at the contact angle θ1 and a state where the finger 901 is brought into contact with the display screen at the contact angle θ1. FIG. 39B is a diagram illustrating a case where the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 at the contact angle θ2 and a state where the finger 901 is brought into contact with the display screen at the contact angle θ2.

  Referring to FIG. 39 (a), the contact direction of finger 900 and the contact direction of finger 901 are the same even though the contact angle of finger 900 and the contact angle of finger 901 are both θ1 (first range). Therefore, the electronic device 1300 operates in different operation modes depending on whether the finger 900 is in contact with the finger 901 or not.

  For example, when the electronic device 1300 causes the finger 900 to contact the display screen at the contact angle θ1, the electronic device 1300 changes the operation mode to the contact mode because the contact direction of the finger 900 is the second direction. The operation mode to be processed as a pointer operation on the display device 102A (main screen) is set. On the other hand, when the electronic device 1300 touches the display screen with the finger 901 at the contact angle θ1, the electronic device 1300 changes the operation mode to the contact mode because the contact direction of the finger 901 is the first direction. The operation mode to be processed as a pointer operation on the display device 103 (sub screen) is set.

  Next, referring to FIG. 39 (b), the contact direction of the finger 900 and the contact angle of the finger 901 are both in spite of the contact angle of the finger 900 and the contact angle of the finger 901 being θ2 (second range). Since the contact direction is different, the electronic device 1300 operates in different operation modes depending on whether the finger 900 is in contact with the finger 901 or not.

  For example, when the electronic device 1300 causes the finger 900 to contact the display screen at the contact angle θ2, the electronic device 1300 changes the operation mode to the contact mode because the contact direction of the finger 900 is the second direction. The operation mode is set to be processed as a touch operation on the display device 102A (main screen). On the other hand, when the electronic device 1300 touches the display screen with the finger 901 at the contact angle θ2, the electronic device 1300 changes the operation mode to the contact mode because the contact direction of the finger 901 is the first direction. The operation mode to be processed as a touch operation on the display device 103 (sub screen) is set.

  As described above, in the electronic device 1300, when the contact angle θ of the finger 900 (right hand finger) is equal to or larger than the threshold θth, when the contact angle θ of the finger 900 is less than the threshold θth, the contact angle of the finger 901 (left hand finger). Different processing is performed in four cases, where θ is equal to or greater than the threshold θth and the contact angle θ of the finger 901 is less than the threshold θth.

  Therefore, the user can set the operation mode of the electronic device 1300 to the operation mode desired by the user by adjusting the contact direction and the contact angle of the fingers 900 and 901.

<7. Select scanning method>
Depending on the environment in which electronic device 1300A is used, it may be preferable for electronic device 1300 to execute the scanning method shown in FIG. 30 from the viewpoint of accurately detecting contact angle θ. Alternatively, it may be preferable from the same viewpoint that the electronic device 1300A executes the scan method shown in FIG. Furthermore, it may be preferable from the same viewpoint that the electronic apparatus 1300A executes the scan method shown in FIG.

  Therefore, a configuration will be described in which electronic device 1300A (see FIG. 36) selects an optimal scan method from the scan method shown in FIG. 30, the scan method shown in FIG. 35, and the scan method shown in FIG. .

  The electronic device 1300A uses a light amount determination unit 316, a number measurement unit 317, and a number determination unit 318 when selecting a scan method (see FIG. 24).

  The light amount determination unit 316 determines whether the incident light amount of the scattered light with respect to the display screen of the liquid crystal panel 240 is larger than a predetermined value.

  In the state where the finger 900 is in contact with the display screen of the liquid crystal panel 240, the number measuring unit 317 converts the number of shadows formed on the display screen by the scattered light into the data obtained by the scan. Measure based on.

  The number determination unit 318 determines whether or not the number of shadows measured by the number measurement unit 317 is greater than a predetermined value.

  When it is determined that the amount of incident light is larger than the predetermined value, if it is determined that the number of shadows is larger than the predetermined value, the display device 103 is irradiated from the light sources 3601 and 3602. Scan using light. On the other hand, if it is determined in the above case that the number of shadows is not greater than the predetermined value, the display device 103 performs scanning using the scattered light.

  Further, when it is determined that the amount of incident light is not larger than a predetermined value, the display device 103 emits red light emitted from a light source (for example, an infrared LED) that irradiates the infrared light incorporated in the display device 103. Scan using external light.

  FIG. 40 is a flowchart showing the flow of a scanning method selection process performed by electronic device 1300A. Referring to FIG. 40, electronic device 1300A sets the infrared LED to an off state in step S302. In step S304, electronic device 1300A measures the amount of incident scattered light. In step S306, electronic device 1300A determines whether or not the measured incident light amount is equal to or greater than a predetermined value (specified value).

  If electronic device 1300A determines that the amount of incident light is greater than or equal to the specified value (YES in step S306), it sets the infrared LED to the on state in step S312. That is, the electronic device 1300A emits infrared rays. On the other hand, when electronic device 1300A determines that the amount of incident light is less than the prescribed value (NO in step S306), it sets the infrared LED to the on state in step S308. Then, in step S310, electronic device 1300A selects the internally reflected light method (scan method shown in FIG. 30).

  In step S314, the electronic device 1300A specifies the darkest place (that is, the position where the voltage is low) based on the scan data (see FIG. 35B). That is, electronic device 1300A specifies the point with the lowest voltage in the contact area (hereinafter referred to as “contact point”). In step S316, electronic device 1300A measures the number of shadows coming out of the contact point. That is, electronic device 1300A measures the number of shadows that finger 900 forms on the display screen due to the scattered light, based on the scan data.

  In step S318, electronic device 1300A determines whether or not the number of measured shadows is equal to or greater than the predetermined value (specified value). If electronic device 1300A determines that the number of shadows is equal to or greater than the specified value (Yes in step S318), it sets the infrared LED to the off state in step S324. That is, the electronic device 1300A stops the infrared irradiation. In step S326, the electronic apparatus 1300A selects the external incident light method using the light sources 3601 and 3602 (scan method shown in FIG. 37).

  On the other hand, when electronic device 1300A determines that the number of shadows is less than the specified value (No in step S318), it sets the infrared LED to the off state in step S320. Then, the electronic device 1300A selects the external incident light method using the scattered light (scan method shown in FIG. 35).

  Note that the electronic device 1300A may be configured to perform continuous scanning using the above three methods, and to select a method with less variation (deviation) of each scan data detected in each method.

  In the above description, the configuration in which electronic device 1300A selects one scan method from three scan methods has been described as an example, but the present invention is not limited to this. The electronic device 1300A may be configured to select one scan method from two scan methods. For example, the configuration of the electronic device 1300A may be configured as follows.

  That is, when the electronic device 1300A determines that the incident light amount is greater than a predetermined value, the electronic device 1300A performs scanning using scattered light, and determines that the incident light amount is not greater than the predetermined value. A scan using light emitted from the infrared LED is performed. That is, the configuration of the electronic device 1300A may be configured to select one scan method from the scan method illustrated in FIG. 30 and the scan method illustrated in FIG.

<8. Click processing / drag processing>
(1) FIG. 41 is a diagram for explaining new processing in the electronic device 1300. FIG. 41A is a comparison diagram with respect to FIG. FIG. 41B is a diagram illustrating a case where the contact angle θ of the finger 900 with respect to the display screen of the liquid crystal panel 240 is changed. Here, the process relating to FIG. 41B corresponds to the new process.

  Referring to FIG. 41 (a), when the finger 900 touches the display screen of the liquid crystal panel 240 at the contact angle θ2 (when transitioning from the state A to the state B), as described above, the electronic device 1300 is an electronic device. The operation mode of the device 1300 is set to an operation mode for processing the contact as a touch operation.

  Referring to FIG. 41 (b), when the user touches the finger 900 with the display screen of the liquid crystal panel 240 at the contact angle θ1 (state A), the pointer operation is executed as described above. Further, when the contact angle of the finger 900 is set to the contact angle θ2 by the user standing the finger 900 while maintaining the contact (state B), the electronic device 1300 determines that the click operation is performed and executes the click process.

  As described above, when the previous operation is a pointer operation, electronic device 1300 determines that the contact operation at contact angle θ2 is a click operation. On the other hand, if there is no previous operation, electronic device 1300 determines that the contact at contact angle θ2 is a touch operation.

  With the configuration described above, the user can shift the processing of the electronic device 1300 from the pointer operation to the click operation by raising the finger 900 that has been laid down.

  (2) FIG. 42 is a diagram for explaining a part of the processing shown in the flowcharts of FIGS. 27 and 28. Fig.42 (a) is a figure for demonstrating a part of process shown by the flowchart of FIG. FIG. 42B is a diagram for explaining a part of the processing shown in the flowchart of FIG.

  Referring to FIG. 42A, state A indicates the state of step S208 in FIG. Further, the state B indicates a state in which the electronic device 1300 determines NO in step S202 after determining YES in step S214 in FIG. That is, the state B is a diagram showing a state immediately after the user touches the display screen with the finger 900 over the threshold value Tth as a pointer operation. Furthermore, the state C shows the state of step S208 in FIG. The state D is a diagram showing a state in which the electronic device 1300 executes the process (click operation) in step S216 after determining NO in step S214 in FIG.

  Referring to FIG. 42B, state A shows the state of step S106 in FIG. The state B indicates a state in which the electronic device 1300 has determined NO in step S102 after determining YES in step S108 of FIG. That is, the state B is a diagram showing a state where the user releases the finger 900 from the display screen after the pointer operation is performed. State C is a diagram illustrating a state in which, after state B, electronic device 1300 performs the process of step S112 after determining NO in step S104.

  (3) FIG. 43 is a diagram for explaining further new processing in the electronic device 1300. FIG. 43A is a comparison diagram with respect to FIG. FIG. 43B is a diagram showing a case where the finger 900 is slid on the display screen after the contact angle θ of the finger 900 with respect to the display screen of the liquid crystal panel 240 is changed. Here, the process related to FIG. 43B corresponds to the new process.

  Referring to FIG. 43A, when the user touches the display screen of the liquid crystal panel 240 with the contact angle θ1 (state A), the pointer operation is executed as described above. Furthermore, the contact angle of the finger 900 is set to the contact angle θ2 when the user stands the finger 900 while maintaining the contact (state B). Furthermore, when a certain time elapses while maintaining the contact state (state C), the electronic device 1300 determines that the click operation is performed, and executes a click process. That is, FIG. 43A also shows the processing of FIG. 41B in more detail.

  Referring to FIG. 43 (b), state A and state B are the same as state A and state B in FIG. 43 (a), respectively. Here, after the state B shown in FIG. 43B, when the user slides the finger 900 on the display screen (state C), the electronic device 1300 can be operated by the drag operation instead of the click operation of the user. Judge that there is.

  As described above, when the electronic device 1300 raises the finger 900 from the state where the finger 900 is laid down and further moves the finger 900 on the display screen, the electronic device 1300 executes the drag process.

<9. Character input>
Next, a configuration in which the electronic device 1300 performs processing using the input processing unit 319 will be described (see FIG. 24). FIG. 44 is a diagram illustrating processing of the electronic device 1300 when the input processing unit 319 is used. FIG. 44A shows an image displayed on the liquid crystal panel 240. FIG. 44B shows a plurality of contact states between the display screen of the liquid crystal panel 240 and the finger 900.

  Referring to FIG. 44, display control unit 320 displays a plurality of operation keys for data input on the display screen of liquid crystal panel 240. In the electronic device 1300, a plurality of characters are assigned in advance to the operation keys.

  When the finger 900 touches the display area in which one operation key of the plurality of operation keys is displayed, the setting unit 314 selects the finger 900 from among the plurality of characters assigned to the one operation key. The operation mode of electronic device 1300 is set to an operation mode in which characters corresponding to the contact angle are handled as input candidates.

  The input processing unit 319 processes the input candidate as a confirmed character based on the finger 900 leaving the display area. This will be specifically described below.

  When the finger 900 touches the icon (operation key) 4401, as shown in FIG. 44B, the electronic device 1300 displays a plurality of characters (for example, D, E, F, d, e) assigned to the icon 4401. , F), the operation mode of electronic device 1300 is set to an operation mode in which a character (for example, E) corresponding to the contact angle of finger 900 is handled as an input candidate. Then, the input processing unit 319 processes the input candidate as a confirmed character based on the finger 900 moving away from the display area at the contact angle.

  With such a configuration, the user first selects an operation key with the finger 900, and then changes the tilt of the finger 900 so that characters desired for the user (including numbers and symbols) can be displayed on the electronic device 1300. Can be entered.

  Note that the display control unit 320 sequentially displays characters according to the contact angle on the display screen of the liquid crystal panel 240 until the input characters are determined. Thereby, the user can visually recognize the character currently selected by the user.

<10. OS selection>
Next, the electronic device 1300 includes an OS on the main device side (first unit 1001A) (hereinafter referred to as “main OS”) and an OS on the sub device side (second unit 1002) (hereinafter referred to as “sub OS”). An example of processing that can be executed by the electronic device 1300 will be described.

  Based on the contact angle of the finger 900 with respect to the display screen of the liquid crystal panel 240, the electronic device 1300 determines whether the input instruction by the touch of the finger 900 is an instruction for the main OS or an instruction for the sub OS.

  FIG. 45 is a flowchart showing the flow of processing performed in electronic device 1300. Referring to FIG. 45, electronic device 1300 detects whether or not finger 900 has touched the display screen of liquid crystal panel 240 in step S402.

  If electronic device 1300 determines that finger 900 has touched the display screen (YES in step S402), in step S404, whether or not the contact angle of finger 900 with respect to the display screen is equal to or greater than a predetermined threshold θth. Judging.

  If the contact angle θ of the finger 900 is equal to or greater than the threshold θth (YES in step S404), the electronic device 1300 changes the operation mode of the electronic device 1300 to an operation mode that processes the contact as an operation on the main OS in step S406. Set. On the other hand, when the contact angle θ of the finger 900 is less than the threshold θth (NO in step S404), the electronic device 1300 operates to process the operation mode of the electronic device 1300 as an operation on the sub OS in step S408. Set to mode.

  With such a configuration, the user can give instructions to a plurality of OSs using a single user input interface.

  Alternatively, the main OS can handle the pointer operation and the sub OS can perform the touch operation. In the above description, an example in which two OSs exist is described, but the present invention is not limited to this. For example, an instruction may be individually input from one interface to three or more OSs according to the contact angle.

  Further, the above-described processing can also be applied when two OSs are mounted on one unit (for example, the first unit 1001A). Further, the above-described processing can be applied even in a configuration in which two OSs operate in common in both units (1001A, 1002).

  Further, the main OS and the sub OS may be the same type of OS or different types of OS.

<11. Modified example of electronic device>
FIG. 46 is a diagram illustrating an appearance of a modified example of the electronic device 1300. Referring to FIG. 46, electronic device 4600 includes a display device 102A, a display device 103A, and operation keys 177. Display device 103 </ b> A includes a liquid crystal panel 240. Electronic device 4600 is different from electronic device 1300 in that the size of the display screen of liquid crystal panel 240 is larger than the size of liquid crystal panel 240 of electronic device 1300.

  FIG. 47 is a diagram showing the operation of the user's finger 900. FIG. 21A shows the movement of the finger 900 at the contact angle θ1. FIG. 21B is a diagram illustrating the movement of the finger 900 at the contact angle θ2.

  Referring to FIG. 47A, it is assumed that the user moves the finger 900 on the display screen of the liquid crystal panel 240 as indicated by an arrow 4701 at the contact angle θ1. In this case, for example, electronic device 4600 moves the mouse cursor displayed on the display screen of display device 102A in the direction of arrow 2102 based on the movement. That is, the electronic device 4600 performs a pointer operation.

  Referring to FIG. 47 (b), it is assumed that the user moves finger 900 in the direction of arrow 4702 and causes finger 900 to contact the display screen of liquid crystal panel 240 at contact angle θ2. In this case, the electronic device 4600 performs a touch operation.

  Note that the electronic device 4600 is not configured to switch the operation between the pointer operation and the touch operation in accordance with the contact angle θ, but the electronic device 4600 has an instruction to the main OS and the sub OS according to the contact angle θ. In the case of a configuration that switches operation between instructions, electronic device 4600 performs the following processing.

  It is assumed that the user moves the finger 900 on the display screen of the liquid crystal panel 240 at the contact angle θ1. In this case, the electronic device 4600 sets the operation mode of the electronic device 4600 to an operation mode for processing the contact as an operation on the main OS. On the other hand, it is assumed that the user moves the finger 900 on the display screen of the liquid crystal panel 240 at the contact angle θ2. In this case, the electronic device 4600 sets the operation mode of the electronic device 4600 to an operation mode in which the contact is processed as an operation on the sub OS.

  FIG. 48 is a diagram for explaining a user operation on the electronic device 4600. Referring to FIG. 48, when the user touches the display screen of liquid crystal panel 240 with finger 900 (for example, the finger of the right hand) at contact angle θ 1 and moves finger 900 in the direction of arrow 4801, electronic device 4600 is displayed. Moves the mouse cursor 4806 of the display device 102A in the direction of the arrow 4805.

  When the user touches the display screen of the liquid crystal panel 240 with the finger 901 (for example, the finger of the left hand) at the contact angle θ 2 and moves the finger 900 in the direction of the arrow 4802, the electronic device 4600 displays the liquid crystal panel 240. The touch operation is performed on the operation key 4803 corresponding to the part touched by the finger 901 among the plurality of operation keys (software keys) displayed in FIG.

  Furthermore, when the pen 4810 is brought into contact with the display screen of the liquid crystal panel 240 at the contact angle θ1 instead of the fingers 900 and 901 and the pen 4810 is moved in the direction of the arrow 4804, the electronic device 4600 displays the display device. The mouse cursor 4806 of 102A is moved in the direction corresponding to the arrow 4804.

As described above, the electronic device 4600 executes the same processing as the electronic device 1300.
By the way, the display screen of the liquid crystal panel 240 of the electronic device 4600 is wider than the display screen of the liquid crystal panel 240 of the electronic device 1300. In addition, the display screen of the liquid crystal panel 240 of the electronic device 4600 has the same size as the display screen of the display device 102A. When the electronic device 4600 adopts such a configuration, the electronic device 4600 has the following advantages.

  When a person reads a book or writes a character on a notebook on a desk, the book or Norton page is placed in a portion parallel to the desk and close to the human body. Therefore, when the liquid crystal panel 240 is arranged in front of the operation key 177 (on the user side) as in the electronic device 4600, when the electronic device is not used (reading a book or writing characters in a notebook as described above). ) Is close to usability.

  Further, the electronic device 4600 may be configured such that the liquid crystal panel is slid to a position below the operation key 177 so that all or a part of the display screen of the liquid crystal panel 240 can be placed invisible by the operation key 177. Good.

<12. Illustration drawing>
FIG. 49 is a diagram for describing processing in electronic device 4600. The storage device 390 of the electronic device 4600 stores graphic software. Referring to FIG. 49, display control unit 320 displays a user interface screen during execution of the graphic software on the display screen of display device 102 </ b> A and the display screen of liquid crystal panel 240. More specifically, when the user interface screen becomes active on the display screen of the display device 102A, the display control unit 320 displays the user interface screen on the display screen of the liquid crystal panel 240.

  When the user touches the finger 900 with the display screen of the liquid crystal panel 240 at a contact angle θ2, the electronic device 4600 sets the operation mode of the electronic device 4600 to an operation mode that processes the contact as a touch operation on the liquid crystal panel 240. To do. In this case, the display control unit 320 performs a drawing process based on the touch operation on the display screen of the display device 102A. Thereby, an image 4903 based on the locus 4901 by the touch operation is drawn on the display screen of the display device 102A. More specifically, when the touch operation is an operation on a drawing area on the user interface screen, the display control unit 320 performs a drawing process based on the touch operation on at least the display screen of the display device 102A.

  49 is a diagram showing a case where the drawing process based on the touch operation is performed on the display screen of the liquid crystal panel 240 and the display screen of the display device 102A.

  When the contact time is shorter than a predetermined time, the electronic device 4600 does not perform the drawing process and selects the object (for example, text or graphic) displayed at the touched position. Process as. In this case, electronic device 4600 also processes the same object displayed on display device 102A as the selected object.

  Further, when the user inputs a character with the operation key 177 in a state where the object is selected, the character is displayed in the object. As a result, for example, when the object is text, the user can perform additional character input or character overwrite input on the text.

  On the other hand, when the user touches the display screen of the liquid crystal panel 240 with the finger 901 at the contact angle θ1, for example, the electronic device 4600 sets the operation mode of the electronic device 4600 as the pointer operation on the display screen of the display device 102A. Set the operation mode to be processed. Specifically, it is assumed that the user moves the finger 901 on the display screen of the liquid crystal panel 240 as indicated by an arrow 4902 at the contact angle θ1. In this case, for example, electronic device 4600 moves mouse cursor 4905 displayed on the display screen of display device 102A in the direction of arrow 4904 based on the movement. That is, the electronic device 4600 performs a pointer operation.

<13. Print image>
FIG. 50 is a diagram for explaining another process in the electronic device 4600. The storage device 390 of the electronic device 4600 stores application software. Referring to FIG. 50, display control unit 320 displays a user interface screen at the time of execution of the application software on the display screen of display device 102A and accepts a predetermined process at the time of execution of the application software. Then, a print image corresponding to the user interface screen is displayed on the display screen of the liquid crystal panel 240. More specifically, electronic device 4600 displays a print image of an application that is active on the display screen of display device 102A on the display screen of liquid crystal panel 240.

  When the user touches the finger 900 with the display screen of the liquid crystal panel 240 at a contact angle θ2, the electronic device 4600 sets the operation mode of the electronic device 4600 to an operation mode that processes the contact as a touch operation on the liquid crystal panel 240. To do. In this case, electronic device 4600 performs a process of selecting an object corresponding to the touch position from the displayed objects.

  For example, when the electronic device 4600 performs a process of selecting one of the objects 5011, 5012, and 5013 included in the menu button 5010, the electronic apparatus 4600 executes a process defined in association with the selected object.

  When electronic device 4600 performs processing for selecting text as an object, when the user inputs a character using operation key 177, the character is displayed in the object. As a result, for example, when the object is text, the user can perform additional character input or character overwrite input on the text.

  On the other hand, when the user touches the display screen of the liquid crystal panel 240 with the finger 901 at the contact angle θ1, for example, the electronic device 4600 sets the operation mode of the electronic device 4600 as the pointer operation on the display screen of the display device 102A. Set the operation mode to be processed. Specifically, it is assumed that the user moves the finger 901 on the display screen of the liquid crystal panel 240 as indicated by an arrow 5002 at the contact angle θ1. In this case, for example, electronic device 4600 moves mouse cursor 5004 displayed on display screen of display device 102A in the direction of arrow 5003 based on the movement. That is, the electronic device 4600 performs a pointer operation.

<14. Portal site>
FIG. 51 is a diagram for explaining still another process in the electronic device 4600. FIG. 51A is a diagram illustrating an example of a display image displayed on the display screen of the display device 102A and the display screen of the liquid crystal panel 240. FIG. FIG. 51B is a diagram showing another example of a display image displayed on the display screen of the display device 102A and the display screen of the liquid crystal panel 240.

  The storage device 390 of the electronic device 4600 stores web browser software. Referring to FIG. 51, display control unit 320 displays a preset portal site on the display screen of liquid crystal panel 240 based on the activation of the web browser. The “portal site” is a site that is displayed first when the web browser software is started.

  When the user touches the finger 900 with the display screen of the liquid crystal panel 240 at a contact angle θ2, the electronic device 4600 sets the operation mode of the electronic device 4600 to an operation mode that processes the contact as a touch operation on the liquid crystal panel 240. To do. In this case, based on the fact that the image indicating the link destination in the portal site is selected by the touch operation, the display control unit 320, as shown in FIG. 51A, the link destination specified by the image indicating the link destination. The website is displayed on the display screen of the display device 102A.

  On the other hand, when the user touches the display screen of the liquid crystal panel 240 with the finger 901 at the contact angle θ1, for example, the electronic device 4600 sets the operation mode of the electronic device 4600 as the pointer operation on the display screen of the display device 102A. Set the operation mode to be processed. That is, electronic device 4600 moves mouse cursor 5102. Note that the mouse cursor 5102 moves in a direction corresponding to the direction of the arrow 5111 as shown in FIG.

  In this case, the display control unit 320 changes the display screen of the liquid crystal panel 240 from the portal site to an image (for example, a web page) that is active on the display screen of the display device 102A as shown in FIG. Site). More specifically, the display control unit 320 switches the display screen of the liquid crystal panel 240 from the portal site to the reduced image of the active image. When the user stops the pointer operation by releasing the finger 901 from the display screen, the display control unit 320 displays the portal site on the display screen of the liquid crystal panel 240 again. When the pointer is operated, a click operation is performed based on the contact time between the display screen of the liquid crystal panel 240 and the finger 900 (see FIG. 42A).

  Further, as shown in FIG. 51B, in a state where the finger 901 is in contact with the display screen of the liquid crystal panel 240 at the contact angle θ1, the user displays the other finger 900 on the liquid crystal panel 240 at the contact angle θ2. When touching the screen, electronic device 4600 performs the following processing. That is, when the touch position is an image (object) that specifies a link destination, the electronic device 4600 displays an image to be displayed in an active window on the display screen of the display device 102A from the original image. Switch to the next image.

  As described above, the electronic device 4600 can execute a pointer operation on the display screen of the display device 102A and a screen switching process on the display screen based on the touch operation by a user operation on the display screen of the liquid crystal panel 240. Further, an active window of the display device 102 </ b> A can be displayed on the display screen of the liquid crystal panel 240 by a user operation on the display screen of the liquid crystal panel 240.

  Further, as described above, the electronic device 1300 causes the link site to be displayed when the finger 900 contacts the image (object) indicating the link destination displayed on the liquid crystal panel 240 at the contact angle θ2 (that is, the touch operation). Is displayed on the display screen of the display device 102A. The display screen of the liquid crystal panel 240 is smaller than the display screen of the display device 102A. For this reason, the smaller the region in which the user touches the finger 900, the more the user must stand the finger 900 to be touched. Therefore, when the user selects an object with the finger 900 on the display screen of the liquid crystal panel 240, the user unconsciously raises the finger 900. Thus, the electronic device 1300 can be said to be a highly convenient device for the user because the user can perform intuitive operations.

<15. Media Player>
FIG. 52 is a diagram for explaining still another process in electronic device 4600. The storage device 390 of the electronic device 4600 stores media player software. Referring to FIG. 52, display control unit 320 performs processing for setting a display area for displaying video on the display screen of display device 102A based on the activation of the media player. Further, the display control unit 320 displays an image including a software key for controlling the display of the video (hereinafter also referred to as “controller image”) on the display screen of the liquid crystal panel 240.

  When the user touches the finger 900 with the display screen of the liquid crystal panel 240 at a contact angle θ2, the electronic device 4600 sets the operation mode of the electronic device 4600 to an operation mode that processes the contact as a touch operation on the liquid crystal panel 240. To do. In this case, electronic device 4600 executes processing associated with button 5205 based on, for example, selection of button (image) 5205 as the software key by a touch operation in the direction of arrow 5201. In the case of FIG. 52, the electronic device 4600 is displayed on the display device 102A and executes a fast-forward process of the video. In addition, the electronic device 4600 executes playback, stop, recording reservation from the program guide, and display of the recorded program according to the touch position.

  Note that when the user is touching the finger 900 with the display screen of the liquid crystal panel 240 at the contact angle θ2, the electronic device 4600 may not display the mouse cursor 5203.

  On the other hand, when the user touches the display screen of the liquid crystal panel 240 with the finger 901 at the contact angle θ1, for example, the electronic device 4600 sets the operation mode of the electronic device 4600 as the pointer operation on the display screen of the display device 102A. Set the operation mode to be processed. That is, electronic device 4600 moves mouse cursor 5203. Note that the mouse cursor 5203 moves in a direction corresponding to the direction of the arrow 5202, for example. In this case, for example, when the mouse cursor 5203 moves to the position of the person in the video, the display control unit 320 displays an image introducing the person on the display screen of the display device 102A based on the additional information added in advance to the video data. Display. For example, the display control unit 320 displays the name of the person on the display screen.

  Note that the video data may include Web content and Web links. In this case, the electronic device 4600 can display Web content and a Web link by moving the mouse cursor 5203. The controller image may be realized by a web page existing on the network.

  As described above, the electronic device 4600 uses the controller image displayed on the display screen of the liquid crystal panel 240 to perform display control of the video displayed on the display device 102A, thereby allowing the user to intuitively and easily. Operation becomes possible.

<16. Controller>
FIG. 53 is a diagram for explaining still another process in electronic device 4600. The storage device 390 of the electronic device 4600 stores image data indicating a plurality of appliances different from the electronic device 4600 and application software that controls the operation of each appliance. Referring to FIG. 53, when the application software is activated, display control unit 320 displays images indicating the plurality of appliances on the display screen of display device 102A based on the image data.

  For example, when the user causes the finger 901 to contact the display screen of the liquid crystal panel 240 at the contact angle θ1, the electronic device 4600 processes the operation mode of the electronic device 4600 as a pointer operation on the display screen of the display device 102A. Set the operation mode. That is, electronic device 4600 moves mouse cursor 5303. Note that the mouse cursor 5303 moves in a direction corresponding to the direction of the arrow 5302, for example.

  Here, when the pointer operation is shorter than a predetermined time, electronic device 4600 processes the pointer operation as a click operation. For example, when the mouse cursor 5303 is positioned on the air conditioner image 5304, the electronic device 4600 selects the air conditioner image 5304 by the processing. Note that the present invention is not limited thereto, and the electronic device 4600 may be configured to select the air conditioner image 5304 when the mouse cursor 5303 is positioned on the air conditioner image 5304 for a certain period of time.

  The display control unit 320 causes the display screen of the liquid crystal panel 240 to display an image including software keys for controlling the operation of the electrical appliance selected by the click operation or the like on the display screen of the display device 102A. For example, when the air conditioner image 5304 is selected, the controller image of the air conditioner is displayed on the display screen of the display control unit 320 and the liquid crystal panel 240.

  Here, when the user touches the finger 900 with the display screen of the liquid crystal panel 240 at the contact angle θ2, the electronic device 4600 operates to process the operation mode of the electronic device 4600 as a touch operation on the liquid crystal panel 240. Set to mode. In this case, electronic device 4600 executes a process associated with the button based on, for example, selection of a button that is the software key by a touch operation in the direction of arrow 5301.

  Moreover, when performing temperature setting, you may make the structure of the electronic device 4600 as follows. That is, the user slides the finger 900 on the display screen of the liquid crystal panel 240 (specifically, on the temperature setting object) at a contact angle θ2 to slide the finger 900 on the display screen. The device 4600 adjusts the temperature depending on the amount and direction of sliding.

  The controller image may be realized by a web page existing on the network.

  As described above, the electronic device 4600 can execute control of a plurality of electrical appliances on the display screen of the liquid crystal panel 240. That is, the electronic device 4600 functions as a remote controller that controls a plurality of electrical appliances.

<< Second specific implementation example (one screen) >>
<1. Basic configuration>
In the above “<< first specific implementation example (2 screens) >>”, the electronic devices 1300, 1300A, and 4600 having two display screens have been described. Hereinafter, an electronic apparatus including one display screen that performs the operation mode switching process according to the contact angle described above will be described.

  FIG. 54 is a diagram illustrating an appearance of the electronic device 5400. Referring to FIG. 54, electronic device 5400 includes a liquid crystal panel 240. The electronic device 5400 has a configuration similar to that of the second unit 1002 (see FIG. 18). Therefore, the description of the hardware configuration of the electronic device 5400 will not be repeated below. Note that the electronic device 5400 does not include a hardware keyboard (operation keys).

  FIG. 55 is a diagram illustrating a “pointer operation” when the finger 900 is in contact with the display screen of the liquid crystal panel 240 at a contact angle θ1. FIG. 55A shows the movement of the finger 900. FIG. 55B is a diagram for explaining the movement of the mouse cursor.

  Referring to FIG. 55A, assume that the user moves the finger 900 on the display screen of the liquid crystal panel 240 as indicated by an arrow 5501 at the contact angle θ1. In this case, as shown in FIG. 55B, electronic device 5400 moves mouse cursor 5503 displayed on the display screen of liquid crystal panel 240 in the direction of arrow 5502 based on the movement.

  FIG. 56 is a diagram illustrating a “touch operation” when the finger 900 contacts the display screen of the liquid crystal panel 240 at a contact angle θ2. FIG. 56A shows the movement of the finger 900. FIG. 56B is a diagram for explaining processing associated with the touch operation.

  Referring to FIG. 56 (a), it is assumed that the user moves finger 900 in the direction of arrow 5601 and causes finger 900 to contact the display screen of liquid crystal panel 240 at contact angle θ2. In this case, the electronic device 5400 executes processing associated with selection of the button icon 5601 as shown in FIG. This figure shows processing in which the electronic device 5400 sets the number of ramen orders to “6”.

<2. Hardware keyboard with LCD>
FIG. 57 is an external view of the electronic device 5700. Referring to FIG. 57, electronic device 5700 includes a liquid crystal panel 240 and operation keys 177A that are hardware. The electronic device 5700 has a configuration similar to that of the second unit 1002 (see FIG. 18), except that the operation key 177A is provided. The operation keys include a plurality of input keys.

  Further, when the user uses the electronic device 5700, the liquid crystal panel 240 is arranged on the user side rather than the operation keys 177A. In the following description, for convenience of explanation, it is assumed that the liquid crystal panel 240 and the operation key 177A are arranged on a plane defined by the X axis and the Y axis perpendicular to the X axis.

  The liquid crystal panel 240 is arranged on the Y axis negative direction side with respect to the operation key 177A. Characters assigned to the input keys are displayed on at least one of the plurality of input keys in the operation key 177A. For example, alphabets and kana are displayed as the characters. Further, the above-mentioned characters are erect from the Y-axis negative direction side toward the Y-axis positive direction side. For example, when the operation key 177A is a QWERTY type keyboard, the input key indicating “Z”, the input key indicating “A”, and the input indicating “Q” from the Y-axis negative direction to the Y-axis positive direction The keys are arranged in that order.

  Electronic device 5700 displays a keyboard image showing a software keyboard on the display screen of liquid crystal panel 240, for example.

  FIG. 58 is a diagram illustrating a “pointer operation” when the finger 900 is in contact with the display screen of the liquid crystal panel 240 at the contact angle θ1. FIG. 58A is a diagram showing the movement of the finger 900. FIG. 58B is a diagram for explaining the movement of the mouse cursor.

  Referring to FIG. 58A, it is assumed that the user moves the finger 900 on the display screen of the liquid crystal panel 240 as indicated by an arrow 5801 at the contact angle θ1. In this case, as shown in FIG. 58B, electronic device 5700 moves mouse cursor 5803 displayed on the display screen of liquid crystal panel 240 in the direction of arrow 5802 based on the movement.

  FIG. 59 is a diagram illustrating a “touch operation” when the finger 900 is in contact with the display screen of the liquid crystal panel 240 at a contact angle θ2. FIG. 59A shows the movement of the finger 900. FIG. FIG. 59B is a diagram for explaining processing associated with the touch operation.

  Referring to FIG. 59 (a), assume that the user moves finger 900 in the direction of arrow 5901 and touches the finger 900 on the display screen of liquid crystal panel 240 at contact angle θ2. In this case, the electronic device 5700 executes processing associated with the selection of the object as shown in FIG. This figure shows processing in which the electronic device 5700 scrolls a part of the screen.

  FIG. 60 is a diagram illustrating a configuration of a communication system including the electronic device 5700. Referring to FIG. 60, a communication system 6000 includes an electronic device 5700 and three electronic devices 6001, 6002, and 6003.

  The electronic device 5700 can communicate with the three electronic devices 6001, 6002, and 6003 by the external communication unit 274 (see FIG. 18). Electronic devices 6001, 6002, and 6003 each include a display device. The electronic devices 6001, 6002, and 6003 have the same hardware configuration as that of an ordinary personal computer except that the electronic devices 6001, 6002, and 6003 are not provided with hardware operation keys (keyboards).

  In the communication system 6000, an instruction (control command) input by the user is transmitted to the three electronic devices 6001, 6002, and 6003 using the electronic device 5700 that is an input device. At that time, electronic device 5700 determines the transmission destination of the instruction according to contact angle θ of finger 900. Specifically, it is as follows.

  When the contact angle θ of the finger 900 is equal to or greater than the threshold A, the electronic device 5700 transmits an input instruction for the contact to the electronic device 6001. When the contact angle θ of the finger 900 is less than the threshold value A and greater than or equal to the threshold value B (A> B), the electronic device 5700 transmits an input instruction for the contact to the electronic device 6002. When the contact angle θ of the finger 900 is less than the threshold value B, the electronic device 5700 transmits an input instruction for the contact to the electronic device 6003.

  That is, the external communication unit 274 transmits a control command to an electronic device specified as a communication target in the operation mode set based on the contact angle θ.

  With such a configuration, the user can perform an operation on the electronic device according to the contact angle of the finger 900 among the electronic devices 6001, 6002, and 6003. Therefore, the user can operate the plurality of electronic devices 6001, 6002, and 6003 with one input device (electronic device 5700).

  Note that the electronic devices 6001, 6002, and 6003 may be simple displays that do not include an OS. Further, the communication between the electronic device 5700 and the electronic devices 6001, 6002, and 6003 may be wireless communication as shown in FIG. 60, or wired communication.

  Further, for example, when the contact angle θ of the finger 900 is equal to or smaller than the threshold C (B> C), the electronic device 5700 may process an input instruction by the contact as an instruction to the electronic device 5700.

<3. Configuration in which a keyboard is arranged on the display screen>
FIG. 61 is a diagram illustrating an appearance of the electronic device 6100. FIG. Referring to FIG. 61, electronic device 6100 includes a keyboard 6101, a cable 6102, and a display device 6103.

  The keyboard 6101 is a hardware keyboard including an input key group including a plurality of input keys. The cable 6102 is a communication cable. The keyboard 6101 is placed on the display screen of the display device 6103. Specifically, the keyboard 6101 is placed at a position on the back side when the user uses the electronic device 6100. The keyboard 6101 and the display device 6103 are communicably connected via a cable 6102.

  The display device 6103 includes a liquid crystal panel 240. The display device 6103 has a configuration similar to that of the electronic device 5400 (see FIG. 54) except that an input instruction is received from a keyboard 6101 which is an input device. Therefore, the description of the hardware configuration and the like of display device 6103 will not be repeated.

  The electronic device 6100 displays the image 6120 on the display screen of the liquid crystal panel 240 so that the orientation of the image 6120 is on the keyboard 6101 side. Examples of the image 6120 include a document file.

  The electronic device 6100 also executes the operation mode switching process according to the contact angle described above. Referring to FIG. 61, for example, assume that the user moves finger 900 on the display screen of liquid crystal panel 240 at contact angle θ1. In this case, electronic device 6100 moves mouse cursor 6112 displayed on the display screen of liquid crystal panel 240, for example, in the direction of arrow 6110 based on the movement. Further, it is assumed that the user moves the finger 900 in the direction of the arrow 6111 and makes the finger 900 contact the display screen of the liquid crystal panel 240 at the contact angle θ2. In this case, electronic device 6100 executes processing (for example, document selection) associated with object selection as a touch operation.

  Note that the electronic device 6100 is preferably configured to determine an area where the keyboard 6101 is placed by scanning and to display the image 6120 in an area excluding the area.

  FIG. 62 is a diagram showing a configuration of a conventional personal computer, and is a comparative diagram for comparison with FIG. Referring to FIG. 62, personal computer 6200 includes a keyboard 6201, a display 6202, and a mouse 6203. The personal computer 6200 is placed on a desk together with the document 6299. The document 6299 is placed on the user side of the keyboard 6201.

  61 displays an image 6120 in front of the keyboard 6101 (on the user side). Therefore, when the user of the electronic device 6100 performs an operation on the image 6102, the user can perform the operation with a feeling similar to that of performing work on a document as shown in FIG.

<< Other Modifications >>
<1. Use of fingerprint>
Next, a configuration of electronic device 6700 (see FIG. 67) that performs the above-described operation mode setting using a fingerprint will be described. First, the correspondence between the contact angle of the finger 900 and the scanned image obtained in the contact state of the finger 900 will be described.

  FIG. 63 is a diagram for explaining a case where the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 while being laid. FIG. 63A is a diagram showing a state in which the finger 900 is in contact with the display screen of the liquid crystal panel 240. 63A is a cross-sectional view taken along line LXIII-LXIII in FIG. 63B. FIG. 63B is a diagram showing an overall image of a scan image obtained when the finger 900 touches the display screen in the state of FIG. That is, FIG. 63B is a fingerprint image of the finger 900. Since the finger 900 is laid down as shown in FIG. 63 (a), the electronic device 6700 can scan almost all the fingerprints at the fingertips as shown in FIG. 63 (b).

  FIG. 64 is a diagram for explaining a case where the finger 900 is brought into contact with the display screen in a state where the finger 900 is slightly raised from FIG. FIG. 64A is a diagram showing a state in which the finger 900 is in contact with the display screen of the liquid crystal panel 240. 64A is a cross-sectional view taken along line LXIV-LXIV in FIG. 64B. FIG. 64B is a diagram showing an entire image of a scan image obtained when the finger 900 contacts the display screen in the state of FIG. Since the finger 900 is slightly raised as shown in FIG. 64A, the electronic device 6700 can scan a part of the fingerprint at the fingertip (here, 80%) as shown in FIG. 64B.

  FIG. 65 is a diagram for explaining a case where the finger 900 is brought into contact with the display screen in a state where the finger 900 is slightly raised from FIG. FIG. 65A is a diagram showing a state in which the finger 900 is in contact with the display screen of the liquid crystal panel 240. Moreover, the figure of the liquid crystal panel 240 of Fig.65 (a) is LXV-LXV arrow directional cross-sectional view of FIG.65 (b). FIG. 65B is a diagram showing an entire image of a scan image obtained when the finger 900 touches the display screen in the state of FIG. As shown in FIG. 65 (a), since the finger 900 is raised slightly more than in FIG. 64 (a), the electronic device 6700 has a narrower range than that in FIG. 64 (b) (here, as shown in FIG. 65 (b)). Then, it is possible to scan 60% of the fingertip fingerprint).

  FIG. 66 is a diagram for explaining a case where the finger 900 is brought into contact with the display screen in a state where the finger 900 is slightly raised from FIG. FIG. 66A shows a state in which the finger 900 is in contact with the display screen of the liquid crystal panel 240. 66 (a) is a cross-sectional view taken along line LXVI-LXVI in FIG. 66 (b). FIG. 66 (b) is a diagram showing an overall image of a scan image obtained when the finger 900 touches the display screen in the state of FIG. 66 (a). As shown in FIG. 66 (a), since the finger 900 is raised slightly more than that in FIG. 65 (a), the electronic device 6700 has a narrower range (here, as shown in FIG. 66 (b) than in FIG. 65 (b). Then, 50% of the fingerprint of the fingertip) can be scanned.

  As described above, as the user raises the finger 900, the range of the user's fingerprint that can be scanned by the electronic device 6700 changes. The electronic device 6700 switches the operation mode of the electronic device 6700 using the phenomenon.

  FIG. 67 is a diagram illustrating a block of the electronic device 6700. 67, electronic device 6700 includes display device 102A, display device 103, control unit 300A, and storage device 390. The control unit 300A includes a data acquisition unit 310, a setting unit 314, a direction determination unit 315, an input processing unit 319, a display control unit 320, and a ratio calculation unit 330.

  The storage device 390 stores a reference image indicating a fingerprint in advance. The reference image is an image obtained by performing the above scan when a portion of the finger 900 beyond the first joint is brought into contact with the display screen in a state parallel to the display screen of the liquid crystal panel 240. Hereinafter, the direction indicated by the fingertip when the finger 900 is projected onto the display screen of the liquid crystal panel 240 is referred to as “fingertip direction”.

  The ratio calculation unit 330 calculates the ratio of the fingerprint area of the scanned finger in the fingertip direction to the fingerprint area in the reference image. Specifically, the ratio calculating unit 330 is 100%, 80%, 60%, as shown in FIGS. 63 (b), 64 (b), 65 (b), and 66 (b), for example. A ratio such as 50% is calculated.

  The setting unit 314 sets the operation mode of the electronic device 6700 to one of a plurality of operation modes based on the calculated ratio. For example, when the ratio is 80% or more, the setting unit 314 sets the operation mode of the electronic device 6700 to an operation mode that processes the contact of the finger 900 as a pointer operation. In addition, when the ratio is less than 80%, the setting unit 314 sets the operation mode of the electronic device 6700 to an operation mode in which contact with the finger 900 is processed as a touch operation.

  As described above, since the electronic device 6700 uses the fingerprint, it is not necessary to calculate the contact angle θ unlike the electronic device 1300.

  The reference image is preferably a fingerprint image obtained by scanning a user's fingerprint in advance. This is because the electronic device 6700 can obtain a highly accurate result in the calculation of the ratio.

<2. Other configurations that do not detect contact angle>
Next, another configuration that does not detect the contact angle θ as in the configuration using the fingerprint will be described.

  FIG. 68 is a diagram for explaining a scan image obtained when scanning is performed using a light source that emits infrared light (internal light) built in the liquid crystal panel 240. FIG. 68A shows a state in which the finger 900 is brought into contact with the display screen of the liquid crystal panel 240 at the contact angle θ1. Here, the figure of the liquid crystal panel 240 in FIG. 68A is a cross-sectional view taken along line LXVIII-LXVIII in FIG. FIG. 68B is a diagram showing an overall image of a scan image obtained when the finger 900 touches the display screen in the state of FIG.

  Referring to FIG. 68, the white portion is a contact area 6801. Further, a portion (detection region 6802) in a state where the finger 900 is floating from the display screen has a color (an intermediate color (gray) between white and black) corresponding to the distance between the finger 900 and the display screen. Other portions are black (uniform). The detectable range 6803 is, for example, 0 to 3 cm.

  As shown in FIG. 68 (b), the shape of the contact region 6801 is elliptical. Further, since the finger 900 is in contact with the display screen of the liquid crystal panel 240 while lying, the fingertip direction is the major axis direction of the ellipse.

  FIG. 69 is a diagram for explaining a scan image obtained when scanning is performed using a light source that emits infrared rays built in the liquid crystal panel 240. FIG. 69A shows a state in which the finger 900 is in contact with the display screen of the liquid crystal panel 240 at the contact angle θ2. Here, the figure of the liquid crystal panel 240 in FIG. 69A is a cross-sectional view taken along line LXIV-LXIV in FIG. FIG. 69B is a diagram showing an entire image of a scan image obtained when the finger 900 contacts the display screen in the state of FIG.

  Referring to FIG. 69, the white part is a contact area 6901. Further, a portion (detection region 6902) where the finger 900 is floating from the display screen has a color (an intermediate color (gray) between white and black) corresponding to the distance between the finger 900 and the display screen. Other portions are black (uniform). The detectable range 6903 is, for example, 0 to 3 cm.

  As shown in FIG. 69 (b), the shape of the contact region 6901 is elliptical. In addition, as shown in FIG. 69 (a), the finger 900 is in a state of standing more than the state of FIG. 68 (a). For this reason, the length of the major axis of the ellipse shown in FIG. 69 (b) is shorter than the length of the major axis of the ellipse shown in FIG. 68 (b).

  As can be seen from FIGS. 69 (b) and 68 (b), when the finger 900 is transitioned from the state in which the finger 900 is laid down to the state in which the finger 900 is raised, the amount of change in the length of the major axis is increased. In comparison, the amount of change in the length of the short axis is small.

  As described above, the elliptical shapes of the contact areas 6801 and 6901 change according to the contact angle θ of the finger 900. Thus, the configuration of an electronic device that switches the operation mode based on a change in the elliptical shape will be described.

  FIG. 70 is a diagram illustrating a block of the electronic device 7000. Referring to FIG. 70, electronic device 7000 includes display device 102A, display device 103, control unit 300B, and storage device 390. The control unit 300B includes a data acquisition unit 310, a setting unit 314, a direction determination unit 315, a light amount determination unit 316, a number measurement unit 317, a number determination unit 318, an input processing unit 319, and a display control unit 320. And a specifying unit 340.

  The identifying unit 340 identifies the shape of the contact area on the display screen of the liquid crystal panel 240 with which the finger 900 is in contact based on the scan data.

  The setting unit 314 sets the operation mode of the electronic device 7000 to one of a plurality of operation modes based on the specified shape of the contact area. More specifically, the setting unit 314 processes the identified shape as an elliptical shape, and sets the operation mode of the electronic device 7000 based on the major axis length and minor axis length of the ellipse. More detailed description is as follows.

  The setting unit 314 stores, as comparison data, data related to the length of the minor axis of the ellipse and the length of the major axis in the storage device 390 in advance. Further, the data is associated with the operation mode. Then, the setting unit 314 sets the operation mode by comparing the data with the length of the major axis calculated by the setting unit 314 and the length of the minor axis.

  The setting unit 314 may be configured to set the operation mode of the electronic device 7000 based on, for example, the ratio of the length of the major axis to the length of the minor axis.

<3. Shooting with camera>
Next, a configuration for detecting the contact angle θ using an image of the finger 900 taken by the camera will be described.

  71 is a diagram illustrating an appearance of the electronic device 1300B. Referring to FIG. 71, electronic device 1300B has a configuration in which electronic device 1300 includes camera 6901. The camera 6901 is attached to the side surface of the display device 102A. The control unit 300 controls the camera 6901. The camera 6901 captures an image including the display screen of the liquid crystal panel 240.

  FIG. 72 is a view showing an image photographed by the camera 6901. Note that the camera 6901 may continuously capture images while the electronic device 1300B is activated (that is, in a state where the power is turned on). Alternatively, images may be continuously captured when the electronic device 1300B is being activated and the electronic device 1300B is not in the sleep state. Alternatively, electronic device 1300B may perform shooting using a trigger such as an object such as finger 900 touching the display screen of liquid crystal panel 240.

  The control unit 300 detects the contact angle θ of the finger 900 based on the image captured by the camera 6901. More specifically, the storage device 390 of the electronic device 1300B stores a plurality of reference images associated with information indicating the contact angle in advance. When the image is captured, the control unit 300 determines which of the reference images is the image. Further, the control unit 300 determines that the contact angle associated with the reference image determined to be close to the captured image is the contact angle of the finger 900. Since the processing of electronic device 1300B after determining contact angle θ is the same as the processing of electronic device 1300, description thereof will not be repeated.

  In addition, information indicating the contact angle is not necessarily associated with the reference image, and identification information other than the information indicating the contact angle may be associated. In this case, the control unit 300 may select one operation mode from a plurality of operation modes based on the identification information associated with the reference image determined to be close to the captured image.

<4. Capacitance method>
In the above description, the configuration in which each of the electronic devices 1300, 1300A, 1300B, and 4600 uses the optical sensor built-in liquid crystal panel 240 has been described. Hereinafter, a configuration in which an electronic device includes a capacitive touch panel instead of the optical sensor built-in liquid crystal panel 240 will be described.

  FIG. 73 is a diagram illustrating a block of the electronic device 7300. Referring to FIG. 73, electronic device 7300 includes display device 102A, display device 103B, control unit 300C, and storage device 390. The display device 103B is a capacitive touch panel capable of multipoint input.

  The control unit 300C includes a capacity change detection unit 351, a shape calculation unit 352, a distance calculation unit 312, an angle detection unit 313, a setting unit 314, a direction determination unit 315, an input processing unit 319, and a display control unit. 320.

  The capacitance change detection unit 351 detects a change in capacitance when a finger touches the touch panel.

  Based on the detection result by the capacitance change detection unit 351, the shape calculation unit 352 calculates the shape of the region on the touch panel where the finger 900 is in contact and the region where the finger 900 is close to the touch panel.

  The setting unit 314 sets the operation mode of the electronic device 7300 to one of a plurality of operation modes based on the shape calculated by the shape calculation unit 352. More specifically, the setting unit 314 sets the operation mode of the electronic device 7300 based on the calculation result by the distance calculation unit 312 and the detection result by the angle detection unit 313 (see FIG. 34).

  As described above, when the electronic device 7300 detects that the finger 900 has touched the display screen of the liquid crystal panel 240 in the same manner as the other electronic devices 1300, 1300A, 1300B, and 4600, the operation mode of the electronic device 7300 is changed to The operation mode according to the contact angle of the finger 900 can be set.

<< Other >>
(1) In the above description, various application examples have been described. However, if an electronic device can set which function can be turned on and off, the above-described function can be realized by one electronic device.

  (2) Each electronic device 1300, 1300A, 1300B, 4600 may be configured to include the display device 102 instead of the display device 102A.

  (3) By using each electronic device 1300A, 1300B, 4600, 5700, 6100, 6700, 7000, 7300, the user can perform work on a plane parallel to the work surface of the desk on which each electronic device is placed. Can be done. Therefore, each electronic device is convenient for the user.

  (4) The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  DESCRIPTION OF SYMBOLS 100 Electronic device, 100A housing | casing, 100B housing | casing, 101 main body apparatus, 102 display apparatus, 102A display apparatus, 103 display apparatus, 103A display apparatus, 103B display apparatus, 104 main body apparatus, 140 liquid crystal panel, 177 operation key, 177A operation Key, 240 liquid crystal panel, 274 external communication unit, 300 control unit, 300A control unit, 300B control unit, 300C control unit, 310 data acquisition unit, 311 area detection unit, 312 distance calculation unit, 313 angle detection unit, 314 setting unit 315 Direction determination unit, 316 Light amount determination unit, 317 Number measurement unit, 318 Number determination unit, 319 Input processing unit, 320 Display control unit, 330 Ratio calculation unit, 340 Identification unit, 351 Capacity change detection unit, 352 Shape calculation unit 390 storage devices, 900 fingers, 901 fingers, 300 electronic equipment, 1300A electronic equipment, 1300B electronic equipment, 3001 contact area, 3002 detection area, 3101 contact area, 3102 detection area, 3201 contact area, 3202 detection area, 3401 contact area, 3402 detection area, 3501 contact area, 3502 detection Area 3601 light source 3602 light source 3701 contact area 3702 detection area 3704 shadow 3705 shadow 4600 electronic device 4810 pen 5400 electronic device 5700 electronic device 6000 communication system 6001 electronic device 6002 electronic device 6003 Electronic device, 6100 Electronic device, 6101 Keyboard, 6103 Display device, 6700 Electronic device, 6801 Contact area, 6802 Detection area, 6901 Contact area, 6901 Camera, 690 Detection area, 7000 electronic devices.

Claims (11)

An electronic device having a first display screen,
Scanning means for scanning an object on the first display screen via the first display screen;
Based on the data obtained by the scan, a specifying means for specifying the shape of the contact area on the first display screen in contact with the object;
An electronic device comprising: setting means for setting an operation mode of the electronic device to one of a plurality of operation modes based on the shape of the contact area.   The electronic device according to claim 1, wherein the setting unit sets the operation mode based on a length of a major axis and a length of a minor axis of the ellipse while processing the shape as an ellipse. The electronic device is
An input device including a first display including the first display screen and an input key group including a plurality of input keys;
A second display including a second display screen,
The first display and the input key group are arranged on a plane defined by an X axis and a Y axis perpendicular to the X axis,
The first display is arranged on the Y axis negative direction side from the input key group,
The second display is arranged on the Y axis positive direction side from the input key group,
Characters assigned to the input keys are displayed on at least one of the input keys.
The electronic device according to claim 1, wherein the character is erected from the Y-axis negative direction side toward the Y-axis positive direction side. The electronic device is
A display control means for displaying an image on each of the first display screen and the second display screen;
Executing a first operating system and a second operating system;
The said display control means displays the image based on the said 1st operating system on the said 1st display screen, and displays the image based on the said 2nd operating system on the said 2nd display screen. Electronics. The electronic device is
An input device including a display including the first display screen and an input key group including a plurality of input keys;
The display and the input key group are arranged on a plane defined by an X axis and a Y axis perpendicular to the X axis.
The display is arranged on the Y axis negative direction side from the input key group,
Characters assigned to the input keys are displayed on at least one of the input keys.
The electronic device according to claim 1, wherein the character is erected from the Y-axis negative direction side toward the Y-axis positive direction side. The electronic device is
A display including the first display screen;
The electronic device according to claim 1, further comprising display control means for displaying a keyboard image showing a software keyboard on the first display screen. The electronic device further includes a display including the first display screen and a keyboard including an input key group including a plurality of input keys.
The electronic device according to claim 1, wherein the keyboard is placed on the first display screen. The electronic device is
A display device including the first display screen;
A light amount judging means for judging whether an incident light amount of scattered light from the external light source with respect to the first display screen is larger than a predetermined value;
The display device includes a light source that irradiates light from the first display screen to the outside of the display device inside the display device,
The scanning means includes
When it is determined that the amount of incident light is greater than the predetermined value, the scan is performed using the scattered light, and it is determined that the amount of incident light is not greater than the predetermined value. 3. The electronic apparatus according to claim 1, wherein the scanning is performed using light emitted from a light source that emits light to the outside of the display device. The electronic device is
A display device including the first display screen;
A light amount determination means for determining whether or not an incident light amount of scattered light from an external light source with respect to the first display screen is larger than a predetermined value;
Number of shadows that the object forms on the first display screen due to the scattered light in a state where the object is in contact with the first display screen based on data obtained by the scan Measuring means;
Number determining means for determining whether the number of shadows is greater than a predetermined value;
A first light source that emits light to the first display screen outside the display device;
The display device includes a second light source that irradiates light from the first display screen to the outside of the display device inside the display device,
The scanning means includes
When it is determined that the amount of incident light is greater than the predetermined value, the light emitted from the first light source is used when it is determined that the number of shadows is greater than the predetermined value. And when the number of shadows is determined not to be greater than the predetermined value, the scan is performed using the scattered light,
3. The electronic device according to claim 1, wherein when the incident light amount is determined not to be larger than the predetermined value, the scanning using the light emitted from the second light source is performed. An operation mode setting method in an electronic device having a display screen,
Scanning an object on the display screen via the display screen;
Identifying the shape of the contact area on the display screen in contact with the object, based on the data obtained by the scan;
And setting an operation mode of the electronic device to one of a plurality of operation modes based on the shape of the contact area. A program for controlling an operation mode in an electronic device having a display screen,
The program is
Scanning an object on the display screen via the display screen;
Identifying the shape of the contact area on the display screen in contact with the object, based on the data obtained by the scan;
A program for causing the electronic device to execute an operation mode of the electronic device based on the shape of the contact area as one of a plurality of operation modes.