JP2010224635A - Display device, display method and display program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To simplify the editing operation of image through a touch panel by a user. <P>SOLUTION: The display device 100 includes: a touch panel 140, an acquisition means 111 for acquiring a contact area through the touch panel 140; a first determination means 112 for determining whether the shape of the contact area satisfies a predetermined condition; and a display control means 114 for displaying an image in a position corresponding to a first contact area in which the predetermined condition is not satisfied through the touch panel, and erasing image within a second contact area wherein the predetermined condition is satisfied. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a display device, a display method, and a display program capable of receiving an instruction from a user via a touch panel and displaying an image via the touch panel.

  Conventionally, there has been known a display device that can receive an instruction from a user via a touch panel and display an image or text via the touch panel. When the user inputs handwriting on the touch panel via a stylus pen or the like, the display device displays a line image (handwritten character or the like) corresponding to the handwriting input via the touch panel. The following techniques are disclosed as techniques for editing an image by bringing a pen or a finger into contact with a touch panel.

  For example, JP-A-5-224853 (Patent Document 1) discloses an electronic board. According to Japanese Patent Application Laid-Open No. 5-224853 (Patent Document 1), an electronic board includes display means for displaying graphics and documents, editing means for editing and creating graphic information and document information to be displayed on the display means, Graphic information and document information to be displayed on the display means and display input / output management means for managing the display position information; input means for designating a part of the graphic or document displayed on the display means; Position detecting means for detecting the indicated position. The input means has a function of editing processing according to the type of display object to be input, and the display object discrimination means for determining the type of display object to be input by the input means during the input operation, and the input operation The input means includes an edit function selecting means for selecting an edit processing function corresponding to the type of display object to be input.

  Japanese Patent Laid-Open No. 7-175587 (Patent Document 2) discloses an information processing apparatus. According to Japanese Patent Application Laid-Open No. 7-175587 (Patent Document 2), a flat display is fitted on a desk to display data. A file as a real object is placed on the desk. A camera is installed on the desk, and the displayed data, real objects, and the state of the operator's hand are imaged. The processing device understands the operator's intention from the movement of the hand and performs an operation in accordance with the intention. For example, when the operator moves the data in the file direction by hand, the data is moved and displayed in the file direction, and when approaching a certain distance, the data is filed in a storage device (not shown). For example, instead of a file, a FAX transmitter is on the desk, and when the data is moved in the direction of the FAX transmitter and superimposed, the data is automatically FAXed.

  Japanese Patent Laying-Open No. 2004-199181 (Patent Document 3) discloses a coordinate input device and a display device. According to Japanese Patent Laying-Open No. 2004-199181 (Patent Document 3), when the display screen is touched by a pen mouse or the like possessed by the user, the coordinates of the touched portion are detected. When an operation of touching the pen mouse by the user is performed, not only the tip of the pen mouse touches but also the back of the hand holding the pen mouse may touch. In such a case, the calculation unit obtains a movement amount of each touch portion within a predetermined time, selects a touch portion having a large movement amount, and outputs coordinate data of the selected touch portion.

Japanese Patent Laid-Open No. 5-224853 Japanese Patent Laid-Open No. 7-175589 JP 2004-199181 A

  However, in a display device having a conventional touch panel, when a user inputs a handwritten image, for example, an item (such as an icon) such as “Draw a line” or “Erase with an eraser” is selected once, and then the item is selected. Corresponding operations were input to the display device via the touch panel. That is, an input operation for handwriting input is complicated.

  The present invention has been made to solve such a problem, and an object thereof is to simplify an image editing operation by a user via a touch panel.

  According to one aspect of the present invention, a display device is provided. The display device includes a touch panel, an acquisition unit that acquires the contact area via the touch panel, a first determination unit that determines whether the shape of the contact area satisfies a predetermined condition, and a predetermined condition via the touch panel. Display control means for displaying an image at a position corresponding to the first contact area that does not satisfy the condition, and erasing the image in the second contact area that satisfies the predetermined condition.

  Preferably, the first determination means determines whether or not the area of the contact region is equal to or greater than a predetermined value, and when the area of the contact region is equal to or greater than the predetermined value, the shape of the contact region satisfies a predetermined condition. If it is determined that the contact area is smaller than a predetermined value, it is determined that the shape of the contact area does not satisfy the predetermined condition.

  Preferably, when the acquisition unit acquires the second contact area, the display device has a predetermined time from when the acquisition unit acquires the latest first contact area until the acquisition of the second contact area. Second determination means for determining whether the time is less than or not is further provided. The display control means corresponds to the latest first contact area when the time from the acquisition means acquiring the latest first contact area to the acquisition of the second contact area is less than a predetermined time. When a series of images including images is erased, and the time from when the acquisition unit acquires the latest first contact area until the second contact area is acquired is a predetermined time or more, the second contact area Delete the image inside.

  Preferably, the touch panel includes a plurality of photosensor circuits that generate a first electrical signal in response to incident light and a plurality of pixel circuits that emit light in response to a second electrical signal. The acquisition unit generates image data as a contact area based on the first electrical signals from the plurality of photosensor circuits. The display control means causes the plurality of pixel circuits to display an image by inputting a second electrical signal corresponding to the image to the plurality of pixel circuits.

  Preferably, the acquisition unit acquires a series of contact areas as handwritten input via the touch panel. The series of images corresponding to the series of contact areas is a line image.

  When another situation of this invention is followed, the display method in a display apparatus provided with a touchscreen and a calculation process part is provided. The display method includes a step in which the arithmetic processing unit acquires the contact area via the touch panel, a step in which the arithmetic processing unit determines whether or not the shape of the contact area satisfies a predetermined condition, and the arithmetic processing unit includes: A step of displaying an image at a position corresponding to the first contact area that does not satisfy the predetermined condition via the touch panel; and a step of the operation processing unit erasing the image in the second contact area that satisfies the predetermined condition. Is provided.

  If another situation of this invention is followed, the display program for displaying an image on a display apparatus provided with a touch panel and a calculation process part will be provided. The display program causes the arithmetic processing unit to acquire a contact area via the touch panel, to determine whether the shape of the contact area satisfies a predetermined condition, and does not satisfy the predetermined condition via the touch panel. A step of displaying an image at a position corresponding to the first contact area and a step of erasing the image in the second contact area satisfying a predetermined condition are executed.

  As described above, according to the present invention, the image editing operation by the user via the touch panel is simplified.

It is a 1st image figure which shows the operation | movement outline | summary of the electronic device 100 which concerns on this Embodiment. It is a 2nd image figure which shows the operation | movement outline | summary of the electronic device 100 which concerns on this Embodiment. It is a block diagram showing the hardware constitutions of an electronic device. It is the figure which showed the structure of the touchscreen, and the peripheral circuit of the said touchscreen. It is sectional drawing of a touchscreen and a backlight. It is the figure which showed the timing chart at the time of operating an optical sensor circuit. It is sectional drawing which showed the structure which a photodiode receives the light from a backlight in the case of a scan. It is the figure which showed schematic structure of the sensing command. It is the figure which showed the value of the data in each area | region of a sensing command, and the meaning content which the said value shows. It is the figure which showed schematic structure of the response data. It is the figure which showed the image obtained by scanning a finger | toe. It is a circuit diagram which shows the modification of the liquid crystal panel with a built-in optical sensor. It is sectional drawing which showed the structure which a photodiode receives external light in the case of a scan. It is a block diagram which shows the function structure of the electronic device which concerns on this Embodiment. It is an image figure which shows the electronic device and produced | generated image data when two or more contact areas are detected in the touch panel and the acquisition part. It is an image figure which shows the process sequence of the display process in the electronic device which concerns on this Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<Schematic Configuration of Electronic Device 100>
First, with reference to FIG. 1, a schematic configuration of an electronic device 100 having an optical sensor built-in liquid crystal panel 140 (hereinafter also referred to as a touch panel 140) according to the present embodiment as an example of a display device having a touch panel will be described. . The electronic device 100 is realized by an information device such as a PDA (Personal Digital Assistant), a personal computer, a mobile phone, or an electronic dictionary.

  FIG. 1 is a first conceptual diagram showing an outline of operation of electronic device 100 according to the present embodiment. More specifically, FIG. 1A is an image diagram when the first handwritten image is input to touch panel 140 by stylus pen 99. FIG. 1B is an image diagram when a second handwritten image is input to touch panel 140 by stylus pen 99. FIG. 1C is a first image diagram when input to the touch panel 140 is completed. FIG. 1D is an image diagram when the finger 900 touches the first handwritten image on the touch panel 140. FIG. 1E is a second image diagram when the input to the touch panel 140 is completed.

  Electronic device 100 includes a touch panel 140 and operation keys 177. Touch panel 140 receives a handwriting input command from the user and displays a handwritten image corresponding to the handwriting input command. Touch panel 140 according to the present embodiment can detect a contact area between an external object and touch panel 140. The touch panel may be the optical sensor built-in liquid crystal panel 140 according to the present embodiment, or may be a capacitive touch panel or an infrared touch panel capable of detecting a large number of contact positions. The operation key 177 also receives an operation command.

<Outline of Operation of Electronic Device 100>
Next, an outline of operation of electronic device 100 according to the present embodiment will be described. First, referring to FIG. 1A, the user slides stylus pen 99 on touch panel 140. The electronic device 100 detects a slide operation (first handwriting input command) of the stylus pen 99 and displays a first handwritten image (first line image) corresponding to the slide operation on the touch panel 140.

  Referring to FIG. 1B, the user slides stylus pen 99 on touch panel 140 again. Electronic device 100 detects a slide operation (second handwriting input command) of stylus pen 99 and displays a second handwritten image (second line image) corresponding to the slide operation on touch panel 140.

  As a result, referring to FIG. 1C, electronic device 100 displays the first handwritten image and the second handwritten image on touch panel 140.

  On the other hand, referring to FIG. 1D, in a state where electronic device 100 displays the first handwritten image on touch panel 140, the user touches the first handwritten image with finger 900. Referring to FIG. 1E, electronic device 100 erases a portion (part of a line image) corresponding to an area (contact area) touched by user's finger 900 from the first handwritten image. That is, electronic device 100 causes touch panel 140 to display the first handwritten image other than the area touched by user's finger 900.

  As described above, electronic device 100 according to the present embodiment draws a handwritten image based on an area (first contact area) where stylus pen 99 touches touch panel 140. Then, based on the area (second contact area) where the finger 900 touches the touch panel 140, the electronic device 100 deletes a portion located in the second contact area from the already drawn image.

  As a result, the user does not need to select an icon for erasing the handwritten image (such as an eraser icon) or reselect an icon for drawing the handwritten image (such as a pen icon). In other words, the user can input a drawing command and an erasing command to the electronic device 100 without selecting an icon again. That is, an input operation by the user via the touch panel 140 can be further simplified.

  Furthermore, electronic device 100 according to the present embodiment also has the following functions. FIG. 2 is a second image diagram showing an outline of the operation of electronic device 100 according to the present embodiment. More specifically, FIG. 2A is an image diagram when a first handwritten image is input to touch panel 140 by stylus pen 99. FIG. 2B is an image diagram when the finger 900 touches the first handwritten image on the touch panel 140 within a predetermined time after the input of the first handwritten image. FIG. 1C is a first image diagram when input to the touch panel 140 is completed. FIG. 1D is an image diagram when the finger 900 touches the first handwritten image on the touch panel 140 after a predetermined time has elapsed after the input of the first handwritten image. FIG. 1E is a second image diagram when the input to the touch panel 140 is completed.

  First, referring to FIG. 2A, the user slides stylus pen 99 on touch panel 140. The electronic device 100 detects a slide operation (first handwriting input command) of the stylus pen 99 and displays a first handwritten image (first line image) corresponding to the slide operation on the touch panel 140.

  Referring to FIG. 1B, the user touches the first handwritten image with finger 900 immediately after lifting stylus pen 99 from touch panel 140. That is, the user touches the first handwritten image with the finger 900 before a predetermined time (for example, 1 second) elapses after it is detected that the stylus pen 99 is lifted from the touch panel 140. Referring to FIG. 1C, electronic device 100 erases the first handwritten image.

  On the other hand, referring to FIG. 1D, the user touches the first handwritten image with finger 900 after elapse of a predetermined time after lifting stylus pen 99 from touch panel 140. That is, the user touches the first handwritten image with the finger 900 after a predetermined time has elapsed since it was detected that the stylus pen 99 was lifted from the touch panel 140. Referring to FIG. 1E, electronic device 100 erases a portion of the first handwritten image that is located in a contact area (second area) between finger 900 and touch panel 140.

  Thus, electronic device 100 according to the present embodiment erases the entire first handwritten image when finger 900 touches the first handwritten image immediately after the first handwritten image is input. On the other hand, electronic device 100 erases only the image in the contact area of finger 900 when finger 900 touches the first handwritten image after a predetermined time has elapsed since the first handwritten image was input.

  Thus, the user can delete the entire first handwritten image without touching the entire first handwritten image with the finger 900. That is, the input operation by the user via the touch panel can be simplified.

Hereinafter, a configuration for realizing such a function will be described in detail.
<Hardware Configuration of Electronic Device 100>
Next, an aspect of a specific configuration of electronic device 100 according to the present embodiment will be described. FIG. 3 is a block diagram showing a hardware configuration of electronic device 100 according to the present embodiment. Referring to FIG. 3, electronic device 100 includes a main device 101 and a display device 102 as main components.

  The main unit 101 includes a CPU (Central Processing Unit) 110, a RAM (Random Access Memory) 171, a ROM (Read-Only Memory) 172, a memory card reader / writer 173, a communication unit 174, a microphone 175, and a speaker. 176 and operation keys 177 are included. Each component is connected to each other by a data bus DB1. A memory card 1731 is attached to the memory card reader / writer 173.

  CPU 110 executes a program. The operation key 177 receives an instruction input from the user of the electronic device 100. The RAM 171 stores data generated by the execution of the program by the CPU 110 or data input via the operation keys 177 in a volatile manner. The ROM 172 stores data in a nonvolatile manner. The ROM 172 is a ROM capable of writing and erasing, such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory. The communication unit 174 performs wireless communication with other electronic devices (not shown). Although not shown in FIG. 3, the electronic device 100 may include an interface (IF) for connecting to another electronic device by wire.

  The display device 102 includes a driver 130, an optical sensor built-in liquid crystal panel 140 (touch panel 140), an internal IF 178, a backlight 179, and an image processing engine 180.

  The driver 130 is a drive circuit for driving the touch panel 140 and the backlight 179. Various drive circuits included in the driver 130 will be described later.

  The touch panel 140 is a device having a liquid crystal display function and an optical sensor function. That is, the touch panel 140 can perform image display using liquid crystal and sensing using an optical sensor. Details of the touch panel 140 will be described later.

  An internal IF (Interface) 178 mediates exchange of data between the main device 101 and the display device 102.

  The backlight 179 is a light source disposed on the back surface of the touch panel 140. The backlight 179 irradiates the back surface with uniform light.

  The image processing engine 180 controls the operation of the touch panel 140 via the driver 130. Here, the control is performed based on various data sent from the main apparatus 101 via the internal IF 178. Note that the various data includes commands to be described later. Further, the image processing engine 180 processes the data output from the touch panel 140 and sends the processed data to the main body device 101 via the internal IF 178. Further, the image processing engine 180 includes a driver control unit 181, a timer 182, and a signal processing unit 183.

  The driver control unit 181 controls the operation of the driver 130 by sending a control signal to the driver 130. In addition, the driver control unit 181 analyzes a command transmitted from the main device 101. Then, the driver control unit 181 sends a control signal based on the analysis result to the driver 130. Details of the operation of the driver 130 will be described later.

The timer 182 generates time information and sends the time information to the signal processing unit 183.
The signal processing unit 183 receives data output from the optical sensor. Here, since the data output from the optical sensor is analog data, the signal processing unit 183 first converts the analog data into digital data. Further, the signal processing unit 183 performs data processing on the digital data according to the content of the command sent from the main device 101. Then, the signal processing unit 183 sends data (hereinafter referred to as response data) including data after the above data processing and time information acquired from the timer 182 to the main unit 101. The signal processing unit 183 includes a RAM (not shown) that can store a plurality of scan data, which will be described later, continuously.

  The command includes a sensing command for instructing sensing by the optical sensor. Details of the sensing command and the response data will be described later (FIGS. 8 to 10).

  Note that the timer 182 is not necessarily provided in the image processing engine 180. For example, the timer 182 may be provided outside the image processing engine 180 in the display device 102. Alternatively, the timer 182 may be provided in the main body device 101. Further, the microphone 175 and the speaker 176 are not always included in the electronic device 100, and may be configured so as not to include either or both of the microphone 175 and the speaker 176 depending on the embodiment of the electronic device 100.

  Here, the display device 102 includes a system liquid crystal. The system liquid crystal is a device obtained by integrally forming peripheral devices of the touch panel 140 on the glass substrate of the touch panel 140. In the present embodiment, driver 130 (excluding a circuit for driving backlight 179), internal IF 178, and image processing engine 180 are integrally formed on the glass substrate of touch panel 140. Note that the display device 102 is not necessarily configured using a system liquid crystal, and the driver 130 (excluding a circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are included in the glass substrate. Other substrates may be configured.

  By the way, the processing in the electronic device 100 is realized by each hardware and software executed by the CPU 110. Such software may be stored in the ROM 172 in advance. The software may be stored in a memory card 1731 or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by the memory card reader / writer 173 or other reading device, or downloaded via the communication unit 174 or communication IF (not shown), and then temporarily stored in the ROM 172. The The software is read from the ROM 172 by the CPU 110 and stored in the RAM 171 in the form of an executable program. CPU 110 executes the program.

  Each component constituting the main device 101 of the electronic device 100 shown in FIG. 3 is a general one. Therefore, it can be said that the essential part of the present embodiment is software stored in the RAM 171, ROM 172, memory card 1731 or other storage medium, or software downloadable via a network. Since the hardware operation of main device 101 of electronic device 100 is well known, detailed description will not be repeated.

  The storage medium is not limited to a memory card, but is a CD-ROM, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile). Disc)), IC (Integrated Circuit) cards (excluding memory cards), optical cards, mask ROM, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory), flash ROM, and other semiconductor memories, etc. It may be a medium to be used.

  The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

<Configuration and Drive of Touch Panel 140>
Next, the configuration of the touch panel 140 and the configuration of peripheral circuits of the touch panel 140 will be described. FIG. 4 is a diagram illustrating a configuration of the touch panel 140 and peripheral circuits of the touch panel 140.

  Referring to FIG. 4, touch panel 140 includes pixel circuit 141, photosensor circuit 144, scanning signal line Gi, data signal line SRj, data signal line SGj, data signal line SBj, and sensor signal line SSj. Sensor signal line SDj, readout signal line RWi, and reset signal line RSi. Note that i is a natural number satisfying 1 ≦ i ≦ m, and j is a natural number satisfying 1 ≦ j ≦ n.

  The driver 130 of the display device 102 illustrated in FIG. 3 includes, as peripheral circuits of the touch panel 140, a scanning signal line driving circuit 131, a data signal line driving circuit 132, a photosensor driving circuit 133, a switch 134, and an amplifier. 135.

  The scanning signal line drive circuit 131 receives the control signal TC1 from the driver control unit 181 shown in FIG. The scanning signal line drive circuit 131 applies a predetermined voltage in order from the scanning signal line G1 to each scanning signal line (G1 to Gm) based on the control signal TC1. More specifically, the scanning signal line driving circuit 131 sequentially selects one scanning signal line from the scanning signal lines (G1 to Gm) per unit time, and a TFT (to be described later) with respect to the selected scanning signal line. A voltage capable of turning on the gate of the thin film transistor 142 is applied (hereinafter referred to as a high level voltage). Note that a low level voltage is applied to a scanning signal line that is not selected without applying a high level voltage.

  The data signal line driving circuit 132 receives image data (DR, DG, DB) from the driver control unit 181 shown in FIG. The data signal line driving circuit 132 applies a voltage corresponding to one row of image data to the 3n data signal lines (SR1 to SRn, SG1 to SGn, SB1 to SBn) for each unit time. Apply sequentially.

  Note that although a driving method called a so-called line-sequential method has been described here, the driving method is not limited to this.

  The pixel circuit 141 is a circuit for setting the luminance (transmittance) of one pixel. Further, m × n pixel circuits 141 are arranged in a matrix. More specifically, m pixel circuits 141 are arranged in the vertical direction in FIG. 4 and n in the horizontal direction.

  The pixel circuit 141 includes an R subpixel circuit 141r, a G subpixel circuit 141g, and a B subpixel circuit 141b. Each of these three circuits (141r, 141g, 141b) includes a TFT 142, a pair of electrode pairs 143 including a pixel electrode and a counter electrode, and a capacitor (not shown).

  In addition, CMOS (Complementary Metal Oxide Semiconductor) that can make n-type transistors and p-type transistors can be realized, and the movement speed of carriers (electrons or holes) is several hundreds compared to amorphous silicon thin film transistors (a-Si TFTs). For example, a polycrystalline silicon thin film transistor (p-Si TFT) is used as the TFT 142 in the display device 102 because it is twice as fast. Note that the TFT 142 will be described as an n-channel field effect transistor. However, the TFT 142 may be a p-type channel field effect transistor.

  The source of the TFT 142 in the R subpixel circuit 141r is connected to the data signal line SRj. The gate of the TFT 142 is connected to the scanning signal line Gi. Further, the drain of the TFT 142 is connected to the pixel electrode of the electrode pair 143. A liquid crystal is disposed between the pixel electrode and the counter electrode. The G sub-pixel circuit 141g and the B sub-pixel circuit 141b have the same configuration as the R sub-pixel circuit 141r except that the data signal line to which the source of each TFT 142 is connected is different. Therefore, description of these two circuits (141g, 141b) will not be repeated.

  Here, setting of luminance in the pixel circuit 141 will be described. First, the high level voltage is applied to the scanning signal line Gi. By the application of the high level voltage, the gate of the TFT 142 is turned on. In this manner, with the gate of the TFT 142 turned on, a specified voltage (voltage corresponding to image data for one pixel) is applied to each data signal line (SRj, SGj, SBj). Thereby, a voltage based on the designated voltage is applied to the pixel electrode. As a result, a potential difference is generated between the pixel electrode and the counter electrode. Based on this potential difference, the liquid crystal responds and the luminance of the pixel is set to a predetermined luminance. Note that the potential difference is held by the capacitor (auxiliary capacitor) (not shown) until the scanning signal line Gi is selected in the next frame period.

  The optical sensor drive circuit 133 receives the control signal TC2 from the driver control unit 181 shown in FIG.

  Then, the optical sensor drive circuit 133 sequentially selects one signal line from the reset signal lines (RS1 to RSm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, the voltage VDDR that is higher than usual is applied. Note that a voltage VSSR lower than the voltage applied to the selected reset signal line is kept applied to the unselected reset signal line. For example, the voltage VDDR may be set to 0V and the voltage VSSR may be set to −5V.

  Further, the optical sensor driving circuit 133 sequentially selects one signal line from the readout signal lines (RW1 to RWm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, a voltage VDD higher than usual is applied. Note that the voltage VSSR is applied to the read signal line that is not selected. For example, the value of VDD may be set to 8V.

  The timing for applying the voltage VDDR and the timing for applying the voltage VDD will be described later.

  The optical sensor circuit 144 includes a photodiode 145, a capacitor 146, and a TFT 147. In the following description, it is assumed that the TFT 147 is an n-type channel field effect transistor. However, the TFT 147 may be a p-type channel field effect transistor.

  The anode of the photodiode 145 is connected to the reset signal line RSi. On the other hand, the cathode of the photodiode 145 is connected to one electrode of the capacitor 146. The other electrode of the capacitor 146 is connected to the read signal line RWi. Hereinafter, a connection point between the photodiode 145 and the capacitor 146 is referred to as a node N.

  The gate of the TFT 147 is connected to the node N. The drain of the TFT 147 is connected to the sensor signal line SDj. Further, the source of the TFT 147 is connected to the sensor signal line SSj. Details of sensing using the optical sensor circuit 144 will be described later.

  The switch 134 is a switch provided to switch whether or not to apply a predetermined voltage to the sensor signal lines (SD1 to SDn). The switching operation of the switch 134 is performed by the optical sensor driving circuit 133. Note that the voltage applied to the sensor signal lines (SD1 to SDn) when the switch 134 is turned on will be described later.

  The amplifier 135 amplifies the voltage output from each sensor signal line (SS1 to SSn). The amplified voltage is sent to the signal processing unit 183 shown in FIG.

  Note that the image processing engine 180 controls the timing at which an image is displayed on the touch panel 140 using the pixel circuit 141 and the timing at which sensing is performed using the optical sensor circuit 144.

  FIG. 5 is a cross-sectional view of the touch panel 140 and the backlight 179. Referring to FIG. 5, touch panel 140 includes an active matrix substrate 151 </ b> A, a counter substrate 151 </ b> B, and a liquid crystal layer 152. The counter substrate 151B is disposed to face the active matrix substrate 151A. The liquid crystal layer 152 is sandwiched between the active matrix substrate 151A and the counter substrate 151B. The backlight 179 is disposed on the opposite side of the liquid crystal layer 152 with respect to the active matrix substrate 151A.

  The active matrix substrate 151 </ b> A includes a polarizing filter 161, a glass substrate 162, a pixel electrode 143 a that constitutes the electrode pair 143, a photodiode 145, a data signal line 157, and an alignment film 164. Further, although not shown in FIG. 5, the active matrix substrate 151A includes the capacitor 146, the TFT 147, the TFT 142, and the scanning signal line Gi shown in FIG.

  In the active matrix substrate 151A, the polarizing filter 161, the glass substrate 162, the pixel electrode 143a, and the alignment film 164 are arranged in this order from the backlight 179 side. The photodiode 145 and the data signal line 157 are formed on the liquid crystal layer 152 side of the glass substrate 162.

  The counter substrate 151B includes a polarizing filter 161, a glass substrate 162, a light shielding film 163, color filters (153r, 153g, 153b), a counter electrode 143b constituting the electrode pair 143, and an alignment film 164.

  In the counter substrate 151B, the alignment film 164, the counter electrode 143b, the color filters (153r, 153g, 153b), the glass substrate 162, and the polarizing filter 161 are arranged in this order from the liquid crystal layer 152 side. The light shielding film 163 is formed in the same layer as the color filters (153r, 153g, 153b).

  The color filter 153r is a filter that transmits light having a red wavelength. The color filter 153g is a filter that transmits light having a green wavelength. The color filter 153b is a filter that transmits light having a blue wavelength. Here, the photodiode 145 is arranged at a position facing the color filter 153b.

  The touch panel 140 displays an image by blocking or transmitting light emitted from a light source such as external light or a backlight 179. Specifically, the touch panel 140 changes the direction of the liquid crystal molecules of the liquid crystal layer 152 by applying a voltage between the pixel electrode 143a and the counter electrode 143b, thereby blocking or transmitting the light. However, since the light cannot be completely blocked by the liquid crystal alone, a polarizing filter 161 that transmits only light having a specific polarization direction is provided.

  Note that the position of the photodiode 145 is not limited to the above position, and may be provided at a position facing the color filter 153r or a position facing the color filter 153g.

  Here, the operation of the optical sensor circuit 144 will be described. FIG. 6 is a diagram illustrating a timing chart when the optical sensor circuit 144 is operated. In FIG. 6, a voltage VINT indicates a potential at the node N in the photosensor circuit 144. A voltage VPIX is an output voltage from the sensor signal line SSj shown in FIG. 4 and is a voltage before being amplified by the amplifier 135.

  The following description will be divided into a reset period for resetting the optical sensor circuit 144, a sensing period for sensing light using the optical sensor circuit 144, and a readout period for reading the sensed result.

  First, the reset period will be described. In the reset period, the voltage applied to the reset signal line RSi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDDR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR). As described above, by applying the high-level voltage to the reset signal line RSi, a current starts to flow in the forward direction (from the anode side to the cathode side) of the photodiode 145. As a result, the voltage VINT which is the potential of the node N has a value represented by the following expression (1). In Equation (1), the forward voltage drop amount in the photodiode 145 is Vf.

VINT = VSSR + | VDDR−VSSR | −Vf (1)
Therefore, the potential of the node N is a value smaller by Vf than the voltage VDDR as shown in FIG.

  Here, since the voltage VINT is not more than the threshold value for turning on the gate of the TFT 147, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change. Further, a difference corresponding to the voltage VINT occurs between the electrodes of the capacitor 146. For this reason, the capacitor 146 accumulates charges corresponding to the difference.

  Next, the sensing period will be described. In the sensing period following the reset period, the voltage applied to the reset signal line RSi instantaneously switches from the high level (voltage VDDR) to the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR).

  Thus, by changing the voltage applied to the reset signal line RSi to the low level, the potential of the node N becomes higher than the voltage of the reset signal line RSi and the voltage of the read signal line RWi. For this reason, in the photodiode 145, the voltage on the cathode side becomes higher than the voltage on the anode side. That is, the photodiode 145 is in a reverse bias state. In such a reverse bias state, when the photodiode 145 receives light from the light source, current starts to flow from the cathode side to the anode side of the photodiode 145. As a result, as shown in FIG. 6, the potential of the node N (that is, the voltage VINT) becomes lower with the passage of time.

  Since the voltage VINT continues to decrease in this way, the gate of the TFT 147 does not turn on. Therefore, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change.

  Next, the reading period will be described. In the readout period following the sensing period, the voltage applied to the reset signal line RSi is kept at the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDD). Here, the voltage VDD is higher than the voltage VDDR.

  Thus, by applying a high level voltage instantaneously to the read signal line RWi, the potential of the node N is raised through the capacitor 146 as shown in FIG. Note that the increase width of the potential of the node N is a value corresponding to the voltage applied to the read signal line RWi. Here, since the potential of the node N (that is, the voltage VINT) is raised to a threshold value that turns on the gate of the TFT 147, the gate of the TFT 147 is turned on.

  At this time, if a constant voltage is applied in advance to the sensor signal line SDj (see FIG. 4) connected to the drain side of the TFT 147, the sensor signal line SSj connected to the source side of the TFT 147 will cause the VPIX in FIG. As shown in the graph, a voltage corresponding to the potential of the node N is output.

  Here, if the amount of light received by the photodiode 145 (hereinafter referred to as the amount of received light) is small, the slope of the straight line shown in the VINT graph of FIG. 6 becomes gentle. As a result, the voltage VPIX is higher than when the amount of received light is large. As described above, the optical sensor circuit 144 changes the value of the voltage output to the sensor signal line SSj in accordance with the amount of light received by the photodiode 145.

  By the way, in the above, the operation | movement was demonstrated paying attention to one optical sensor circuit 144 among the m * n optical sensor circuits which exist. Below, operation | movement of each photosensor circuit in the touch panel 140 is demonstrated.

  First, the optical sensor drive circuit 133 applies a predetermined voltage to all n sensor signal lines (SD1 to SDn). Next, the photosensor drive circuit 133 applies a voltage VDDR that is higher than normal to the reset signal line RS1. The other reset signal lines (RS2 to RSm) and read signal lines (RW1 to RWm) are kept in a state where a low level voltage is applied. As a result, the n photosensor circuits in the first row in FIG. 4 enter the reset period described above. Thereafter, the n photosensor circuits in the first row enter a sensing period. Further, thereafter, the n photosensor circuits in the first row enter a reading period.

  Note that the timing at which a predetermined voltage is applied to all n sensor signal lines (SD1 to SDn) is not limited to the above timing, and may be any timing that is applied at least before the readout period. .

  When the readout period of the n photosensor circuits in the first row is completed, the photosensor drive circuit 133 applies a voltage VDDR that is higher than usual to the reset signal line RS2. That is, the reset period of the n photosensor circuits in the second row starts. When the reset period ends, the n photosensor circuits in the second row enter a sensing period, and thereafter enter a reading period.

  Thereafter, the processing described above is sequentially performed on the n photosensor circuits in the third row, the n photosensor circuits in the fourth row,..., The n photosensor circuits in the m row. As a result, the sensing result of the first row, the sensing result of the second row,..., The sensing result of the m-th row are output in this order from the sensor signal lines (SS1 to SSn).

  In the display device 102, sensing is performed for each row as described above, and a sensing result is output from the touch panel 140 for each row. For this reason, hereinafter, the data after the signal processing unit 183 performs the above-described data processing on the data related to the voltage for m rows from the first row to the m-th row output from the touch panel 140 will be referred to as “ This is referred to as “scan data”. That is, scan data refers to image data obtained by scanning a scan target (for example, a user's finger). An image displayed based on the scan data is referred to as a “scanned image”. Further, in the following, sensing is referred to as “scan”.

  In the above description, the configuration in which scanning is performed using all m × n photosensor circuits has been described as an example, but the present invention is not limited to this. Scanning may be performed on a partial area of the surface of the touch panel 140 using a photosensor circuit selected in advance.

  In the following, it is assumed that the electronic device 100 can take either of the two configurations. Further, switching between the components is assumed to be performed by a command sent from the main device 101 based on an input via the operation key 177 or the like. Note that when scanning is performed on a partial area on the surface of the touch panel 140, the image processing engine 180 sets a scan target area. The setting of the area may be configured to be specified by the user via the operation key 177.

  As described above, when scanning is performed on a part of the surface of the touch panel 140, there are the following modes of use for displaying an image. The first is a mode in which an image is displayed in a surface area other than the partial area (hereinafter referred to as a scan area). The second is a mode in which no image is displayed in the surface area other than the scan area. Which mode is used is based on a command sent from the main apparatus 101 to the image processing engine 180.

  FIG. 7 is a cross-sectional view of the touch panel 140 and the backlight 179, and shows a configuration in which the photodiode 145 receives light from the backlight 179 during scanning.

  Referring to FIG. 7, when the user's finger 900 is in contact with the surface of the touch panel 140, a part of the light emitted from the backlight 179 is the user's finger 900 (substantially flat) in the contact area. ). The photodiode 145 receives the reflected light.

  Even in a region where the finger 900 is not in contact, part of the light emitted from the backlight 179 is reflected by the user's finger 900. Even in this case, the photodiode 145 receives the reflected light. However, since the finger 900 is not in contact with the surface of the touch panel 140 in the region, the amount of light received by the photodiode 145 is smaller than the region in which the finger 900 is in contact. Note that the photodiode 145 cannot receive most of the light emitted from the backlight 179 that does not reach the user's finger 900.

  Here, by turning on the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the amount of light reflected by the user's finger 900 from the sensor signal line SSj. it can. In this manner, by controlling the turning on and off of the backlight 179, the touch panel 140 has a touch position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), and the surface of the touch panel 140. The voltage output from the sensor signal lines (SS1 to SSn) changes depending on the direction of the finger 900 with respect to the.

  As described above, the display device 102 can scan an image (hereinafter, also referred to as a reflected image) obtained by reflecting light with the finger 900. Note that examples of the scan target other than the finger 900 include a stylus pen 99 and the like.

  By the way, in this Embodiment, although the touch panel was mentioned as an example and demonstrated as a display apparatus of the electronic device 100, other panels, such as an organic EL (Electro-Luminescence) panel, may be used instead of a touch panel. .

<About data>
Next, the sensing command will be described. In the display device 102, the image processing engine 180 analyzes the content of the sensing command and sends back data (that is, response data) according to the analysis result to the main body device 101.

  FIG. 8 is a diagram showing a schematic configuration of a sensing command. Referring to FIG. 8, the sensing command includes a header data area DA01, a timing data area DA02, a data type data area DA03, a reading method data area DA04, and image gradation data. An area DA05, a data area DA06 indicating resolution, and a spare data area DA07 are included.

  FIG. 9 is a diagram showing data values in each area of the sensing command and the meaning contents indicated by the values.

  Referring to FIG. 9, a sensing command in which “00” is set in the data area indicating the timing requests image processing engine 180 to transmit scan data at that time. That is, the sensing command requests transmission of scan data obtained by scanning using the optical sensor circuit 144 after the image processing engine 180 receives the sensing command. A sensing command in which “01” is set in the data area indicating the timing requests transmission of scan data when the scan result has changed. Furthermore, a sensing command in which “10” is set in the data area indicating the timing requests transmission of scan data at regular intervals.

  A sensing command in which “001” is set in the data area indicating the data type requests transmission of the coordinate value of the center coordinate in the partial image. In addition, a sensing command in which “010” is set in the data area indicating the data type requests transmission of only a partial image whose scan result has changed. Note that the change in the scan result indicates that the previous scan result is different from the current scan result. Further, a sensing command in which “100” is set in the data area indicating the data type requests transmission of the entire image.

  Here, the “entire image” is an image generated by the image processing engine 180 based on the voltage output from each photosensor circuit when scanning using m × n photosensor circuits. . The “partial image” is a part of the entire image. The reason why the partial image is requested to transmit only the partial image whose scan result has changed will be described later.

  The coordinate value and the partial image or the entire image may be requested at the same time. Further, in the case of a configuration in which a partial area on the surface of the touch panel 140 is scanned, the entire image is an image corresponding to the area to be scanned.

  A sensing command in which “00” is set in the data area indicating the reading method requests to scan with the backlight 179 on. A sensing command in which “01” is set in the data area indicating the reading method requests scanning with the backlight 179 off. A configuration for scanning with the backlight 179 off is described later (FIG. 13). Furthermore, a sensing command in which “10” is set in the data area indicating the reading method requests scanning using both reflection and transmission. Note that the combined use of reflection and transmission refers to scanning a scanning object by switching between a method of turning on and scanning the backlight 179 and a method of turning off and scanning the backlight.

  A sensing command in which “00” is set in the data area indicating the image gradation requests monochrome image data of black and white. For example, the CPU 110 may request the image processing engine 180 for binary image data as data indicating the contact area. The image processing engine 180 may implement the acquisition unit 111 described later. A sensing command in which “01” is set in the data area indicating the image gradation requests multi-gradation image data. Furthermore, a sensing command in which “10” is set in the data area indicating the image gradation requests RGB color image data.

  A sensing command in which “0” is set in the data area indicating the resolution requests image data with a high resolution. A sensing command in which “1” is set in the data area indicating the resolution requests image data with a low resolution.

  In addition to the data shown in FIGS. 8 and 9, the sensing command includes designation of a region to be scanned (region of a pixel that drives the optical sensor circuit 144), timing for scanning, and timing for lighting the backlight 179. Etc. are described.

  FIG. 10 is a diagram showing a schematic configuration of response data. The response data is data corresponding to the content of the sensing command, and is data that the image processing engine 180 of the display device 102 transmits to the main body device 101.

  Referring to FIG. 10, the response data includes a header data area DA11, a coordinate data area DA12, a time data area DA13, and an image data area DA14. Here, the value of the center coordinates of the partial image is written in the data area DA12 indicating the coordinates. In addition, time information acquired from the timer 182 of the image processing engine 180 is written in the data area indicating the time. Further, image data (that is, scan data) after being processed by the image processing engine 180 is written in the data area indicating the image.

  FIG. 11 is a diagram illustrating an image obtained by scanning the finger 900 (that is, a scanned image). Referring to FIG. 11, an image of a region W1 surrounded by a thick solid line is an entire image, and an image of a region P1 surrounded by a broken line is a partial image. The center point C1 of the cross indicated by a thick line is the center coordinate.

  In the present embodiment, a pixel (that is, a predetermined gradation or a pixel having a photosensor circuit which is a rectangular region and whose output voltage from the sensor signal line SSj is equal to or higher than a predetermined value). An area including all of the pixels having a luminance equal to or higher than a predetermined luminance is set as a partial image area.

  The center coordinates are coordinates determined in consideration of the gradation of each pixel in the partial image area. Specifically, for each pixel in the partial image, the center coordinates are determined by performing a weighting process based on the gradation of the pixel and the distance between the pixel and the center point of the rectangle (that is, the centroid). Is done. That is, the center coordinates do not necessarily match the centroid of the partial image.

  However, the position of the center coordinates is not necessarily limited to the above, and the center coordinates may be the coordinates of the centroid or the coordinates near the centroid.

  When “001” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the value of the central coordinate in the data area DA12 indicating the coordinate. In this case, the image processing engine 180 does not write image data in the data area DA14 indicating an image. After writing the value of the center coordinate, the image processing engine 180 sends response data including the value of the center coordinate to the main body device 101. As described above, when “001” is set in the data area indicating the data type, the sensing command requests the output of the value of the center coordinate without requesting the output of the image data.

  Further, when “010” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the image data of the partial image whose scan result has changed in the data area DA14 indicating the image. . In this case, the image processing engine 180 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 180 writes the image data of the partial image whose scan result has changed, and then sends response data including the image data of the partial image to the main body device 101. As described above, when “010” is set in the data area indicating the data type, the sensing command does not request the output of the value of the center coordinate, and the image data of the partial image whose scan result has changed. Request output.

  As described above, the reason for requesting transmission of only the partial image whose scan result has changed is that the scan data of the partial image area of the scan data is more important than the scan data of the other area. This is because the data is high, and the scan data in the region corresponding to the region of the partial image in the scan data is likely to change due to the contact state with the scan object such as the finger 900.

  When “011” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the value of the central coordinate in the data area DA12 indicating the coordinate and also displays the data area indicating the image. The image data of the partial image whose scan result has changed is written in DA14. Thereafter, the image processing engine 180 sends response data including the value of the center coordinate and the image data of the partial image to the main body device 101. As described above, when “011” is set in the data area indicating the data type, the sensing command requests the output of the value of the center coordinate and the output of the image data of the partial image whose scan result has changed. To do.

  If “100” is set in the data area indicating the data type of the sensing command, the image processing engine 180 displays the image of the entire image in the data area DA14 indicating the image of the response data shown in FIG. Write data. In this case, the image processing engine 180 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. After writing the image data of the entire image, the image processing engine 180 sends response data including the image data of the entire image to the main body device 101. As described above, when “100” is set in the data area indicating the data type, the sensing command requests the output of the image data of the entire image without requesting the output of the value of the center coordinate.

  When “101” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the value of the central coordinate in the data area DA12 indicating the coordinate and also displays the data area indicating the image. Write the image data of the entire image to DA14. Thereafter, the image processing engine 180 sends response data including the value of the center coordinate and the image data of the entire image to the main body device 101. As described above, when “101” is set in the data area indicating the data type, the sensing command requests the output of the value of the center coordinate and the output of the image data of the entire image.

<About modification>
By the way, the structure of the touch panel 140 is not limited to the structure shown in FIG. Below, the touch panel of the aspect different from FIG. 4 is demonstrated.

  FIG. 12 is a circuit diagram of a photosensor built-in liquid crystal panel 140A according to the above different embodiment. Referring to FIG. 12, photosensor built-in liquid crystal panel 140A (touch panel 140A) includes three photosensor circuits (144r, 144g, 144b) in one pixel. As described above, the touch panel 140A is different from the touch panel 140 including one photosensor circuit in one pixel in that the touch panel 140A includes three photosensor circuits (144r, 144g, 144b) in one pixel. The configuration of the optical sensor circuit 144 is the same as that of each of the three optical sensor circuits (144r, 144g, 144b).

  Further, the three photodiodes (145r, 145g, 145b) in one pixel are arranged at positions facing the color filter 153r, the color filter 153g, and the color filter 153b, respectively. Therefore, the photodiode 145r receives red light, the photodiode 145g receives green light, and the photodiode 145b receives blue light.

  In addition, since the touch panel 140 includes only one photosensor circuit 144 in one pixel, two data signal lines for the TFT 147 disposed in one pixel are a sensor signal line SSj and a sensor signal line SDj. there were. However, since the touch panel 140A includes three photosensor circuits (144r, 144g, 144b) in one pixel, there are six data signal lines for TFTs (147r, 147g, 147b) arranged in one pixel. Become.

  Specifically, the sensor signal line SSRj and the sensor signal line SDRj are arranged corresponding to the TFT 147r connected to the cathode of the photodiode 145r arranged at a position facing the color filter 153r. A sensor signal line SSGj and a sensor signal line SDGj are arranged corresponding to the TFT 147g connected to the cathode of the photodiode 145g arranged at a position facing the color filter 153g. Further, a sensor signal line SSBj and a sensor signal line SDBj are disposed corresponding to the TFT 147b connected to the cathode of the photodiode 145b disposed at a position facing the color filter 153b.

  In such a touch panel 140A, the white light emitted from the backlight 179 passes through the three color filters (153r, 153g, 153b), and on the surface of the touch panel 140A, red, green, and blue are mixed and white. It becomes light. Here, when white light is reflected by the scan object, a part of the white light is absorbed by the pigment on the surface of the scan object, and a part is reflected. And the reflected light permeate | transmits three color filters (153r, 153g, 153b) again.

  At this time, the color filter 153r transmits light having a red wavelength, and the photodiode 145r receives light having the red wavelength. The color filter 153g transmits light having a green wavelength, and the photodiode 145g receives light having the green wavelength. The color filter 153b transmits light having a blue wavelength, and the photodiode 145b receives light having the blue wavelength. That is, the light reflected by the scan object is separated into three primary colors (R, G, B) by three color filters (153r, 153g, 153b), and each photodiode (145r, 145g, 145b) The light of the color corresponding to is received.

  When a part of white light is absorbed by the pigment on the surface of the scan target, the amount of light received by each photodiode (145r, 145g, 145b) is different for each photodiode (145r, 145g, 145b). For this reason, the output voltages of the sensor signal line SSRj, the sensor signal line SSGj, and the sensor signal line SSBj are different from each other.

  Therefore, the image processing engine 180 determines the R gradation, the G gradation, and the B gradation according to each output voltage, so that the image processing engine 180 sends the RGB color image to the main body device 101. Can send.

  As described above, when the electronic apparatus 100 includes the touch panel 140A, the scan target can be scanned in color.

  Next, a scanning method different from the scanning method described with reference to FIG. 7 (that is, the method of scanning the reflected image) will be described with reference to FIG.

  FIG. 13 is a cross-sectional view showing a configuration in which a photodiode receives external light during scanning. Referring to FIG. 13, part of the external light is blocked by finger 900. Therefore, the photodiode disposed under the surface area of the touch panel 140 that is in contact with the finger 900 can hardly receive external light. In addition, although the photodiodes disposed under the surface area where the shadow of the finger 900 is formed can receive a certain amount of external light, the amount of external light received is small compared to the surface area where no shadow is formed.

  Here, by turning off the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the position of the finger 900 with respect to the surface of the touch panel 140 from the sensor signal line SSj. . In this manner, by controlling the backlight 179 to be turned on and off, the touch panel 140 has a touch position of the finger 900, a contact area of the finger 900 (determined by the pressing force of the finger 900), and the surface of the touch panel 140. The voltage output from the sensor signal lines (SS1 to SSn) changes depending on the direction of the finger 900 with respect to the.

  As described above, the display device 102 can scan an image (hereinafter also referred to as a shadow image) obtained by blocking external light with the finger 900.

  Further, the display device 102 may be configured such that after scanning with the backlight 179 lit, the backlight 179 is turned off and scanning is performed again. Alternatively, the display device 102 may be configured to perform scanning by turning off the backlight 179 and then performing scanning again by turning on the backlight 179.

  In this case, since two scanning methods are used together, two scan data can be obtained. Therefore, it is possible to obtain a highly accurate result as compared with the case of scanning using only one scanning method.

<Functional Configuration of Electronic Device 100>
Next, a functional configuration of electronic device 100 according to the present embodiment will be described. FIG. 14 is a block diagram showing a functional configuration of electronic device 100 according to the present embodiment.

  Referring to FIG. 14, electronic device 100 includes an acquisition unit 111, an area determination unit 112, a time determination unit 113, and a display control unit 114. In addition, the electronic device 100 includes a RAM 171 and a touch panel 140 as shown in FIG.

  First, each of the plurality of optical sensor circuits 144 receives incident light and generates an electrical signal corresponding to the incident light. The plurality of photosensor circuits 144 as a whole input image data corresponding to the scenery input to the touch panel 140 to the acquisition unit 111 via the image processing engine 180 (FIG. 3) or the like.

  Each of the plurality of pixel circuits 141 emits visible light to the outside based on the image data from the display control unit 114 via the image processing engine 180. More specifically, the plurality of pixel circuits 141 as a whole emit light from the backlight 179 based on image data (electrical signals) from the display control unit 114 via the image processing engine 180 (FIG. 3) or the like. Display handwritten images and text while using.

  The RAM 171 stores the image data 171A from the image processing engine 180 and the acquisition time 171B when the image data 171A is acquired. For example, the RAM 171 stores image data indicating a contact area at each sensor timing from the image processing engine 180 and second image data indicating a handwritten image corresponding to a series of the contact areas. The RAM 171 stores the time when the input of a series of contact areas is completed (interrupted), that is, the time when the CPU 110 acquires image data in which no contact areas exist.

  Below, the handwritten image corresponding to a series of the said contact area is also called a stroke. In the present embodiment, a handwritten image (stroke) corresponding to a series of the contact areas is a single line image. More specifically, the stroke refers to the touch panel 140 of the stylus pen 99 or the finger 900 from when the stylus pen 99 or the finger 900 contacts the touch panel 140 to when the stylus pen 99 or the finger 900 is lifted from the touch panel 140. The trajectory on the surface.

  The acquisition unit 111, the area determination unit 112, the time determination unit 113, and the display control unit 114 are functions realized by the CPU 110 or the image processing engine 180. More specifically, each function of the CPU 110 is a function realized by the CPU 110 executing a control program stored in the RAM 171 or the like and controlling each hardware shown in FIG.

  The acquisition unit 111 is realized by the CPU 110, for example. The acquisition unit 111 generates “binary image data” based on electrical signals input from the plurality of optical sensor circuits 144 via the image processing engine 180. For example, the acquisition unit 111 generates “binary image data” based on color image data or multivalued image data from the image processing engine. Alternatively, the acquisition unit 111 acquires “binary image data” directly from the image processing engine 180.

  “Binary image data” indicates a contact area. In the present embodiment, “binary image data” indicates a contact area of an external object (such as the stylus pen 99 or the finger 900) with respect to the touch panel 140 as “black pixel” and a non-contact area as “white pixel”. .

  Alternatively, the acquisition unit 111 may acquire data indicating a boundary between the contact area and the non-contact area as a contact area instead of the binary image data based on the image data from the image processing engine 180.

  The acquisition unit 111 may be realized by, for example, the image processing engine 180 illustrated in FIG. That is, the acquisition unit 111 realized by the image processing engine 180 generates binary image data based on the electrical signal from the optical sensor circuit 144, and the binary image data is generated by the CPU 110. You may deliver to the time judgment part 113, the display control part 114, etc.

  As described above, the acquisition unit 111 temporarily stores binary image data (contact area) in the RAM 171. For example, the acquisition unit 111 updates the binary image data in the RAM 171 to the latest binary image data based on the image data from the image processing engine 180. Alternatively, the acquisition unit 111 may sequentially accumulate binary image data indicating the contact area in the RAM 171 only during a period in which the contact area is detected.

  The area determination unit 112 determines whether or not the shape of the contact area satisfies a predetermined condition based on the data indicating the contact area from the acquisition unit 111. The area determining unit 112 according to the present embodiment calculates the number of black pixels (or white pixels) based on the binary image data from the acquiring unit 111. The area determination unit 112 acquires the number of pixels as the area of the contact region. The area determination unit 112 reads a first predetermined value (first threshold value) from the RAM 171. The area determining unit 112 determines whether or not the area of the contact area is equal to or greater than a first predetermined value.

  The area determination unit 112 determines that the finger 900 has touched the touch panel 140 when the area of the contact region is equal to or greater than the first predetermined value. The area determination unit 112 delivers “finger image data” to the time determination unit 113 for each binary image data when the area of the contact region is equal to or greater than the first predetermined value. Each of the “finger image data” indicates the center coordinates of the contact area and the boundary between the contact area and the non-contact area.

  When the electronic device 100 does not have the function of the time determination unit 113 described later, the area determination unit 112 may directly transfer the finger image data to the display control unit 114 as an area erase command described later.

  On the other hand, the area determination unit 112 determines that the stylus pen 99 has touched the touch panel 140 when the area of the contact region is less than the first predetermined value. When the area of the contact area is less than the first predetermined value, the area determination unit 112 delivers “handwritten image data” to the time determination unit 113 and the display control unit 114 for each binary image data. Each of the “handwritten image data” indicates the center coordinates of the contact area.

  The area determination unit 112 reads the second predetermined value (second threshold value) from the RAM 171, and determines the area of the contact region based on the binary image data and the second predetermined value. It may be determined whether or not a predetermined value of 2 or more. Then, when the area of the contact area is equal to or larger than the second predetermined value, the area determination unit 112 may ignore the contact area. That is, it may be considered that no touch operation has been performed on the touch panel 140. This is because when the area of the contact area is equal to or larger than the second predetermined value, there is a high possibility that the “palm” or the like has touched the touch panel 140.

  Further, the area determination unit 112 may determine whether or not the shape of the contact region satisfies a predetermined condition based on another determination criterion. For example, the RAM 171 may store image data indicating a predetermined shape. Then, the area determination unit 112 may perform pattern matching on the shape of the contact region and a predetermined shape. That is, the area determination unit 112 determines that the shape of the contact area satisfies the predetermined condition when the shape of the contact area matches the predetermined shape, and the shape of the contact area does not match the predetermined shape In addition, it may be determined that the shape of the contact area does not satisfy a predetermined condition.

  Based on the handwritten image data sequentially input from the area determining unit 112, the time determining unit 113 stores the time (first acquisition time) at which the latest contact operation is completed in the RAM 171. For example, when the time determination unit 113 stops receiving handwritten image data from the area determination unit 112, the time determination unit 113 stores the time when the last handwritten image is received in the RAM 171.

  In the present embodiment, the image processing engine 180 measures the acquisition time using a timer 182 or a clock. Then, the image processing engine 180 passes the acquisition time to the CPU 110 together with the binary image data.

  The time determination unit 113 receives the finger image data from the area determination unit 112 and stores the time (second acquisition time) at which the finger image data was acquired in the RAM 171. The time determination unit 113 determines whether or not the time from the first acquisition time to the second acquisition time is less than a predetermined time.

  When the time from the first acquisition time to the second acquisition time is equal to or greater than a predetermined time, the time determination unit 113 delivers the finger image data to the display control unit 114 as an area erase command. On the other hand, when the time from the first acquisition time to the second acquisition time is equal to or longer than a predetermined time, the time determination unit 113 delivers a stroke deletion command to the display control unit 114.

  The display control unit 114 is realized by the CPU 110 or the image processing engine 180. The display control unit 114 displays a handwritten image on the touch panel 140 based on the handwritten image data from the area determining unit 112.

  More specifically, the display control unit 114 acquires the center coordinates of the contact area for each handwritten image data. The display control unit 114 may calculate the center coordinates of the contact area based on each handwritten image data from the area determination unit 112, or the center from the image processing engine 180 via the acquisition unit 111 or the area determination unit 112. You may acquire a coordinate.

  The display control unit 114 acquires the locus of the contact area between the touch panel 140 and the stylus pen 99 (or the finger 900) based on the center coordinates of the contact area related to each sensor timing. The display control unit 114 stores the locus of the contact area as a line image (one stroke) in the RAM 171 and causes the touch panel 140 to display the line image.

  Based on the finger image data (area deletion command) from time determination unit 113, display control unit 114 causes touch panel 140 to display a screen from which the image in the contact area has been deleted. That is, the display control unit 114, based on the finger image data sequentially passed from the time determination unit 113, among the handwritten images (line images) drawn on the touch panel 140, the part (line image Delete some).

  For example, the display control unit 114 erases the line segment in the contact area based on the data indicating the boundary between the contact area and the non-contact area from the time determination unit 113. Alternatively, the display control unit 114 deletes a line segment located within a predetermined radius from the center of the contact area based on the center coordinates of the contact area from the time determination unit 113. Specifically, the display control unit 114 updates the image data 171A in the RAM 171 based on the finger image data sequentially delivered from the time determination unit 113 and causes the touch panel 140 to display the updated line image.

  The display control unit 114 causes the touch panel 140 to display an image from which the latest line image has been deleted based on the stroke deletion command from the time determination unit 113. That is, the display control unit 114 erases the line image last drawn on the touch panel 140 based on the stroke erasure command passed from the time determination unit 113. In other words, the display control unit 114 updates the image data 171 </ b> A in the RAM 171 based on the stroke erasure command passed from the time determination unit 113.

  As described above, since electronic device 100 according to the present embodiment switches between a drawing command and an erasing command according to the shape of the contact area, an image editing operation by the user via the touch panel is simplified. For example, the electronic device 100 uses the touch panel 140 to draw a line image in response to a contact operation with an index finger (a contact area is narrow), and the line image according to a contact operation with a thumb (a contact area is wide). It is also possible to erase part or all of. Conversely, the electronic device 100 draws a line image according to the contact operation with the thumb (wide contact area) and erases part or all of the line image according to the contact operation with the index finger (narrow contact area). You may do.

  Note that the area determination unit 112 according to the present embodiment is configured so that the touch panel 140 and the acquisition unit 111 detect two or more contact areas, that is, the stylus pen 99 (or the finger 900) and the “palm” (hand 901). ) And the touch panel 140 function as follows. That is, the area determination unit 112 determines whether or not the contact area satisfies a predetermined condition based on the area of the contact area having the smallest area in the image data and the first predetermined value. Then, the time determination unit 113 generates handwritten image data, finger image data (area deletion command), or a stroke deletion command based on the determination result.

  FIG. 15 is an image diagram showing electronic device 100 and generated image data when two or more contact points are detected by touch panel 140 and acquisition unit 111. More specifically, FIG. 15A is an image diagram of electronic device 100 when stylus pen 99 and the palm are in contact with touch panel 140. FIG. 15B is an image diagram of image data acquired when the stylus pen 99 and the palm touch the touch panel 140. FIG. 15C is an image diagram of the electronic device 100 when the finger 900 and the palm are in contact with the touch panel 140. FIG. 15D is an image diagram of image data acquired when the finger 900 and the palm are in contact with the touch panel 140.

  As shown in FIG. 15A, when the stylus pen 99 and the palm touch the touch panel 140, the touch panel 140 and the acquisition unit 111 acquire the image data shown in FIG. At this time, as shown in FIG. 15B, the image data indicates a contact area T1 between the stylus pen 99 and the touch panel 140, and a contact area T2 between the palm and the touch panel 140. Since the area of the contact area T1 is smaller than that of the contact area T2, the area determination unit 112 according to the present embodiment generates handwritten image data regarding the contact area T1.

  As shown in FIG. 15C, when the finger 900 and the palm touch the touch panel 140, the touch panel 140 and the acquisition unit 111 acquire the image data shown in FIG. At this time, as shown in FIG. 15D, the image data indicates a contact area T3 between the stylus pen 99 and the touch panel 140, and contact areas T4 and T5 between the palm and the touch panel 140. The area determination unit 112 according to the present embodiment generates finger image data for the contact region T3 because the contact region T3 has a smaller area than the contact regions T4 and T5.

<Display processing>
Next, display processing in electronic apparatus 100 according to the present embodiment will be described with reference to FIGS. FIG. 16 is an image diagram showing a processing procedure of display processing in electronic device 100 according to the present embodiment.

  In the present embodiment, the plurality of photosensor circuits 144 included in the touch panel 140 always generates an electrical signal according to incident light. The plurality of optical sensor circuits 144 input an electrical signal (image data) corresponding to incident light to the CPU 110 via the image processing engine 180. That is, the CPU 110 functioning as the acquisition unit 111 acquires electrical signals (image data) from the plurality of optical sensor circuits 144 via the image processing engine 180.

  CPU 110 determines whether or not a contact area has been detected based on image data from image processing engine 180 (step S102). CPU110 repeats the process from step S102, when a contact area is not detected (when it is NO in step S102).

  When CPU 110 functioning as area determination unit 112 detects a contact area (YES in step S102), CPU 110 determines whether or not the contact area indicates contact between finger 900 and touch panel 140 (step S104). . CPU 110 functioning as display control unit 114 does not indicate contact between finger 900 and touch panel 140, that is, if the contact area indicates contact between stylus pen 99 and touch panel 140 (YES in step S104). If present, a handwritten image is displayed on the touch panel 140 based on the contact area (first contact area).

  The CPU 110 determines whether or not the contact of the stylus pen 99 with the touch panel 140 has ended (step S108). CPU110 repeats the process from step S106, when the contact with the touch panel 140 of the stylus pen 99 is not complete | finished (when it is NO in step S108). CPU110 memorize | stores the time which contact was complete | finished in RAM171, when the contact with respect to the touchscreen 140 of the stylus pen 99 is complete | finished (when it is YES in step S108) (step S110).

  That is, when the area of the contact area is less than a predetermined value, the CPU 110 is displayed on the touch panel 140 based on the image data sequentially input from the image processing engine 180 until the touch of the stylus pen 99 with the touch panel 140 is completed. Update the line image. CPU110 repeats the process from step S102 after step S110.

  On the other hand, when the contact area indicates the contact between the finger 900 and the touch panel 140 (YES in step S104), the CPU 110 performs the current contact operation within a predetermined time from the time when the previous contact operation is completed. It is determined whether it has been received (step S112). When the present contact operation is performed after a predetermined time has elapsed from the time when the previous contact operation was completed (YES in step S112), CPU 110 causes touch panel 140 to display an image in the contact area (one line image). Part) is deleted (step S114). That is, CPU 110 causes touch panel 140 to display an image obtained by deleting the image in the contact area.

  CPU 110 determines whether or not the contact of finger 900 on touch panel 140 has ended (step S116). CPU110 repeats the process from step S114, when the contact with the touch panel 140 of the finger | toe 900 does not complete | finish (when it is NO in step S116). CPU110 repeats the process from step S102, when the contact with the touch panel 140 of the finger | toe 900 is complete | finished (when it is YES in step S116).

  That is, when the area of the contact area is equal to or greater than a predetermined value, the CPU 110 determines the line on the touch panel 140 based on the image data sequentially input from the image processing engine 180 until the contact of the finger 900 with the touch panel 140 is completed. Continue erasing the image in the contact area of the image.

  When the current contact operation is performed within a predetermined time from the time when the previous contact operation is completed (NO in step S112), CPU 110 performs a series of handwritten images (line lines) last drawn on touch panel 140. The image is erased (step S118). That is, CPU 110 causes touch panel 140 to display a screen from which the latest line image has been deleted. CPU110 repeats the process from step S102.

<Other embodiments>
It goes without saying that the present invention can also be applied to a case where it is achieved by supplying a program to a system or apparatus. Then, a storage medium storing a program represented by software for achieving the present invention is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the program code stored in the storage medium It is possible to enjoy the effects of the present invention also by reading and executing.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card (IC memory card), ROM (mask ROM, flash) EEPROM, etc.) can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code However, it is needless to say that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Further, after the program code read from the storage medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  99 stylus pen, 100 electronic device, 101 main unit, 102 display device, 110 CPU, 111 acquisition unit, 112 area determination unit, 113 time determination unit, 114 display control unit, 130 driver, 131 scanning signal line drive circuit, 132 data Signal line driving circuit, 133 photosensor driving circuit, 134 switch, 135 amplifier, 140, 140A liquid crystal panel with built-in photosensor, 141 pixel circuit, 141b, 141g, 141r subpixel circuit, 143 electrode pair, 143a pixel electrode, 143b counter electrode 144 photosensor circuit, 145, 145b, 145g, 145r photodiode, 146 capacitor, 151A active matrix substrate, 151B counter substrate, 152 liquid crystal layer, 153b, 153g, 153r color filter, 57 Data signal line, 161 Polarizing filter, 162 Glass substrate, 163 Shading film, 164 Alignment film, 171 RAM, 171A Image data, 171B Acquisition time, 173 Memory card reader / writer, 174 Communication device, 175 Microphone, 176 Speaker, 177 Operation Key, 179 backlight, 180 image processing engine, 181 driver control unit, 182 timer, 183 signal processing unit, 900 finger, 901 hand, 1731 memory card, DB1, DB2 data bus.

Claims (7)

A touch panel;
Obtaining means for obtaining a contact area via the touch panel;
First determination means for determining whether or not the shape of the contact area satisfies a predetermined condition;
Display control means for displaying an image at a position corresponding to the first contact area that does not satisfy the predetermined condition via the touch panel, and erasing the image in the second contact area that satisfies the predetermined condition; A display device. The first determination means includes
Determining whether the area of the contact area is a predetermined value or more;
When the area of the contact area is equal to or greater than a predetermined value, it is determined that the shape of the contact area satisfies the predetermined condition,
The display device according to claim 1, wherein when the area of the contact region is less than a predetermined value, it is determined that the shape of the contact region does not satisfy the predetermined condition. When the acquisition unit acquires the second contact area, the time from when the acquisition unit acquires the latest first contact area to the acquisition of the second contact area is less than a predetermined time. A second determining means for determining whether or not
The display control means includes
When the time from when the acquisition unit acquires the latest first contact area to the acquisition of the second contact area is less than the predetermined time, it corresponds to the latest first contact area. Erasing a series of images including the images,
When the time from when the acquisition unit acquires the latest first contact area to the acquisition of the second contact area is equal to or longer than the predetermined time, the image in the second contact area is The display device according to claim 1, wherein the display device is erased. The touch panel
A plurality of photosensor circuits for generating a first electrical signal in response to incident light;
A plurality of pixel circuits that emit light in response to a second electrical signal;
The acquisition means generates image data as the contact area based on the first electrical signals from the plurality of photosensor circuits,
4. The display control unit according to claim 1, wherein the display control unit causes the plurality of pixel circuits to display the image by inputting the second electric signal corresponding to the image to the plurality of pixel circuits. 5. The display device described. The acquisition means acquires a series of the contact areas as handwritten input via the touch panel,
The display device according to claim 1, wherein the series of images corresponding to the series of contact areas is a line image. A display method in a display device including a touch panel and an arithmetic processing unit,
The arithmetic processing unit acquiring a contact area via the touch panel;
The arithmetic processing unit determining whether or not the shape of the contact area satisfies a predetermined condition;
The arithmetic processing unit displaying an image at a position corresponding to a first contact area that does not satisfy the predetermined condition via the touch panel;
And a step of erasing the image in the second contact area satisfying the predetermined condition. A display program for displaying an image on a display device including a touch panel and an arithmetic processing unit,
In the arithmetic processing unit,
Obtaining a contact area via the touch panel;
Determining whether the shape of the contact area satisfies a predetermined condition;
Displaying an image at a position corresponding to the first contact area that does not satisfy the predetermined condition via the touch panel;
And a step of erasing the image in the second contact area that satisfies the predetermined condition.

Images