JP2010129051A - Object display device, object display method, and object display program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object display device that allows a user to easily move an object displayed on a display panel. <P>SOLUTION: The object display device 100 is provided with: a display control means 116 for displaying the object X on a first display panel 140; an acceptance means 111 for accepting specification of a first position Y in the first display panel via the first display panel; a first determination means 112 for determining whether or not the first position is within a display area of the object; and a second determination means 113 for determining whether or not the first position is within a first area to be preset in the first display panel. The display control means 116 redisplays the object on a second display panel when the first determination means determines that the first position is within the display area and the second determination means determines that the first position is within the first area. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an object display device, an object display method, and an object display program for moving an object such as an icon displayed on a display panel according to a command from a user, and in particular, is displayed on a first display panel. The present invention relates to an object display device, an object display method, and an object display program that can move an existing object to a second display panel.

  Electronic devices (object display devices) that can display objects such as icons are known. Such an object display device has a display panel with a tablet function or a display panel with a built-in optical sensor, displays an object through the display panel, and applies an object to the object through the display panel. Some accept various commands.

For example, Japanese Patent Application Laid-Open No. 6-44001 (Patent Document 1) discloses a display control device. According to Japanese Patent Laid-Open No. 6-44001 (Patent Document 1), an icon displayed on a display screen of a display device is moved according to an instruction with a touch pen of a coordinate input device. When the moving direction and moving speed of the icon are detected at predetermined time intervals and the moving instruction is finished, that is, when the touch pen leaves the icon (tablet surface), if the moving speed of the icon at that time is equal to or higher than the predetermined speed, It is further moved by a predetermined distance in the moving direction.
Japanese Patent Laid-Open No. 6-44001

  As described above, regarding an object display device that displays an object via a display panel and receives various commands for the object via the display panel, there is a technique for a user to easily move the object on the display panel. It has been demanded. In particular, in an object display device having a plurality of display panels, it is difficult to move an object beyond the display panel by a touch operation (drag and drop operation) or the like, so the object displayed on the first display panel There is a strong demand for a technique for easily moving the display to the second display panel.

  The present invention has been made to solve such a problem, and an object of the present invention is to make it easier for a user to display an object displayed on the first display panel when the user has a plurality of display panels. An object display device, an object display method, and an object display program are provided.

  According to an aspect of the present invention, an object display device is provided. The object display device includes first and second display panels, display control means for displaying an object on the first display panel, and a first position in the first display panel via the first display panel. An accepting means for accepting designation, a first judging means for judging whether or not the first position is within the display area of the object, and a first position in which the first position is preset in the first display panel. 2nd judgment means which judges whether it is in the area of this. The display control means determines that the first determination means determines that the first position is within the display area, and the second determination means determines that the first position is within the first area. Redisplay the object on the second display panel.

  Preferably, the accepting unit accepts designation of a second position in the second display panel via the second display panel. The first determination unit determines whether or not the second position is within the display area of the object. The second determination means determines whether or not the second position is in a second area preset in the second display panel. The display control means determines that the first determination means determines that the second position is within the display area, and the second determination means determines that the second position is within the second area. Redisplay the object on the first display panel.

  Preferably, the object display device further includes third determination means for determining whether or not the object is displayed across the first display panel and the second display panel. The display control unit redisplays the object when the third determination unit determines that the object is displayed across the first display panel and the second display panel.

  Preferably, the object display device determines that the first determination means determines that the first position is within the display area, and the second determination means determines that the first position is within the first area. In this case, the apparatus further includes fourth determination means for determining whether or not the designation of the first position is canceled. The display control means redisplays the object when the fourth determination means determines that the designation of the first position has been cancelled.

  Preferably, the first display panel includes a plurality of photo sensor circuits that generate an input signal according to incident light and a plurality of pixel circuits that emit light representing an object according to an output signal. The accepting means accepts designation of the first position based on input signals from the plurality of photosensor circuits. The display control means displays an object on the first display panel by outputting an output signal to the plurality of pixel circuits.

  If another situation of this invention is followed, the object display method in the object display apparatus which has a 1st and 2nd display panel and an arithmetic processing part will be provided. The object display method includes a step in which the arithmetic processing unit accepts designation of the first position in the first display panel via the first display panel, and the arithmetic processing unit has a display area in which the first position is an object. A step of determining whether the first position is within a first area preset in the first display panel, a step of determining whether the first position is in the first display panel, and a calculation A step of re-displaying the object on the second display panel when the processing unit determines that the first position is within the display area and the first position is within the first area; Is provided.

  If another situation of this invention is followed, the object display program for displaying an object on the object display apparatus which has a 1st and 2nd display panel and an arithmetic processing part will be provided. The object display program receives the designation of the first position in the first display panel via the first display panel, and whether or not the first position is within the object display area. Determining whether the first position is within a first area preset in the first display panel, and determining that the first position is within the display area. When the determination is made and it is determined that the first position is within the first area, the step of redisplaying the object on the second display panel is executed.

  As described above, according to the object display device having a plurality of display panels according to the present invention, the object display device and the object that allow the user to easily move the object displayed on the first display panel to the second display panel A display method and an object display program are provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[Embodiment 1]
<Overall Configuration of Electronic Device 100>
First, an overall configuration of electronic apparatus 100 that is an example of the object display device according to the present embodiment will be described. In this embodiment, an icon indicating an image file, a document file, or the like will be described as an example of an object displayed on the display panel. However, the object displayed on the display panel may be a window for various applications.

  FIG. 1 is a first schematic diagram showing an outline of the operation of electronic device 100 according to the present embodiment. More specifically, FIG. 1A is an image diagram illustrating the electronic device 100 when the user selects the icon X on the first display panel 140, and FIG. 1B is a diagram illustrating the first display panel of the user. FIG. 1C is an image diagram showing the electronic device 100 when the icon X is moved into a predetermined area within 140, and FIG. 1C shows the electronic device 100 when the second display panel 240 displays the icon X again. It is an image figure shown.

  1A to 1C, electronic device 100 includes a first housing 100A and a second housing 100B. The first casing 100A and the second casing 100B are foldably connected by a hinge 100C. The first housing 100A includes a first photosensor built-in liquid crystal panel 140 (hereinafter also referred to as a first display panel 140). The second housing 100B includes a second photosensor built-in liquid crystal panel 240 (hereinafter also referred to as a second display panel 240). As described above, electronic device 100 according to the present embodiment includes two liquid crystal panels with a built-in optical sensor. The electronic device 100 is configured as a portable device having a display function such as a PDA (Personal Digital Assistant), a notebook personal computer, a portable telephone, and an electronic dictionary.

  Electronic device 100 according to the present embodiment displays an object such as icon X on at least one of first and second display panels 140 and 240. The electronic device 100 receives various commands for the object from the user via the first and second display panels 140 and 240. For example, the electronic device 100 accepts a movement command (drag and drop command) for the icon X displayed on the first display panel 140 and moves the icon X within the first display panel, X is displayed again on the second display panel 240.

<Outline of Operation of Electronic Device 100>
Next, with reference to FIG. 1A to FIG. 1C, an outline of operation of electronic device 100 according to the present embodiment will be described.

  Referring to FIG. 1A, first, a user is displayed on first display panel 140 of electronic apparatus 100 according to the present embodiment via his / her finger 900 or a stylus pen (not shown). Touch icon X. As a result, the icon X is selected (held).

  More specifically, the electronic device 100 receives the designation of the first position (designated position Y) via the first display panel 140, and thereby the icon X based on the designated position Y and the display position of the icon X. The selection command is accepted. As a result, the selection flag corresponding to the icon X is turned on in the electronic device 100, and the first display panel 140 displays the selected icon X in a manner different from other objects. For example, the first display panel 140 changes the color of the selected icon X to a darker or lighter color than icons indicating other similar files. When the user releases the finger 900 from the first display panel 140, the selection state of the icon X may be released, or the selection state may continue until another icon is selected.

  Referring to FIG. 1B, the user holds the finger 900 or the stylus pen in the state where the icon X is selected, that is, the finger 900 or the stylus pen is in contact with the first display panel 140. Slide in the direction of the display panel 240. As a result, the icon X moves on the first display panel 140 toward the second display panel 240. That is, the user can freely move the icon X in the first display panel 140 (drag the icon X freely).

  When the finger 900 is located in the first area 140Z (lower part of the first display panel 140) of the first display panel 140, that is, when the designated position Y is located in the first area 140Z, FIG. As shown, the electronic device 100 redisplays the icon X on the second display panel 240. More specifically, the electronic device 100 resets the display position of the icon X in the second display panel 240.

  Thus, in electronic device 100 according to the present embodiment, a user can easily move an object displayed on one of first display panel 140 and second display panel 240 to the other. it can. In particular, electronic device 100 according to the present embodiment allows a user to easily display an object displayed on first display panel 140 using one finger 900 without using another input device or the like. It can be moved to the second display panel 240. Hereinafter, a configuration for realizing such a function will be described in detail.

  In this embodiment, the case where the electronic apparatus 100 includes the optical sensor built-in liquid crystal panels 140 and 240 will be described. However, the electronic apparatus 100 as an object display device has a normal touch panel display panel. There may be.

<Hardware Configuration of Electronic Device 100>
Next, an aspect of a specific configuration of the electronic device 100 will be described. FIG. 2 is a block diagram illustrating a hardware configuration of electronic device 100. Referring to FIG. 2, electronic device 100 includes a main device 101, a first display device 102, and a second display device 103 as main components.

  The main unit 101 includes a CPU (Central Processing Unit) 110, a RAM (Random Access Memory) 171, a ROM (Read-Only Memory) 172, a memory card reader / writer 173, a communication unit 174, a microphone 175, and a speaker. 176 and operation keys 177 are included. Each component is connected to each other by a data bus DB1. A memory card 1731 is attached to the memory card reader / writer 173.

  CPU 110 executes a program. The operation key 177 receives an instruction input from the user of the electronic device 100. The RAM 171 stores data generated by the execution of the program by the CPU 110 or data input via the operation keys 177 in a volatile manner. The ROM 172 stores data in a nonvolatile manner. The ROM 172 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory. The communication unit 174 performs wireless communication with other electronic devices (not shown). Although not illustrated in FIG. 2, the electronic device 100 may include an interface (IF) for connecting to another electronic device by wire.

  The display device 102 includes a driver 130, a first optical sensor built-in liquid crystal panel 140, an internal IF 178, a backlight 179, and an image processing engine 180.

  The driver 130 is a drive circuit for driving the liquid crystal panel 140 and the backlight 179. Various drive circuits included in the driver 130 will be described later.

  The liquid crystal panel 140 is a device having a liquid crystal display function and a photosensor function. That is, the liquid crystal panel 140 can perform image display using liquid crystal and sensing using an optical sensor. Details of the liquid crystal panel 140 will be described later.

  An internal IF (Interface) 178 mediates exchange of data between the main device 101 and the display device 102.

  The backlight 179 is a light source disposed on the back surface of the liquid crystal panel 140. The backlight 179 irradiates the back surface with uniform light.

  The image processing engine 180 controls the operation of the liquid crystal panel 140 via the driver 130. Here, the control is performed based on various data sent from the main apparatus 101 via the internal IF 178. Note that the various data includes commands to be described later. Further, the image processing engine 180 processes data output from the liquid crystal panel 140 and sends the processed data to the main apparatus 101 via the internal IF 178. Further, the image processing engine 180 includes a driver control unit 181, a timer 182, and a signal processing unit 183.

  The driver control unit 181 controls the operation of the driver 130 by sending a control signal to the driver 130. In addition, the driver control unit 181 analyzes a command transmitted from the main device 101. Then, the driver control unit 181 sends a control signal based on the analysis result to the driver 130. Details of the operation of the driver 130 will be described later.

The timer 182 generates time information and sends the time information to the signal processing unit 183.
The signal processing unit 183 receives data output from the optical sensor. Here, since the data output from the optical sensor is analog data, the signal processing unit 183 first converts the analog data into digital data. Further, the signal processing unit 183 performs data processing on the digital data in accordance with the content of the command sent from the main device 101. Then, the signal processing unit 183 sends data (hereinafter referred to as response data) including data after the above data processing and time information acquired from the timer 182 to the main unit 101. The signal processing unit 183 includes a RAM (not shown) that can store a plurality of scan data, which will be described later, continuously.

  The command includes a sensing command for instructing sensing by the optical sensor. Details of the sensing command and the response data will be described later (FIGS. 7 to 9).

  Note that the timer 182 is not necessarily provided in the image processing engine 180. For example, the timer 182 may be provided outside the image processing engine 180 in the display device 102. Alternatively, the timer 182 may be provided in the main body device 101. Further, the microphone 175 and the speaker 176 are not always provided in the electronic device 100, and may be configured so as not to include either or both of the microphone 175 and the speaker 176 depending on the embodiment of the electronic device 100.

  Here, the display device 102 includes a system liquid crystal. The system liquid crystal is a device obtained by integrally forming peripheral devices of the liquid crystal panel 140 on the glass substrate of the liquid crystal panel 140. In the present embodiment, the driver 130 (excluding the circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are integrally formed on the glass substrate of the liquid crystal panel 140. Note that the display device 102 is not necessarily configured using a system liquid crystal, and the driver 130 (excluding a circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are included in the glass substrate. Other substrates may be configured.

  The display device 103 includes a driver 230, a second optical sensor built-in liquid crystal panel 240, an internal IF 278, a backlight 279, and an image processing engine 280. The image processing engine 280 includes a driver control unit 281, a timer 282, and a signal processing unit 283.

  The display device 103 has the same configuration as the display device 102. That is, the driver 230, the liquid crystal panel 240, the internal IF 278, the backlight 279, and the image processing engine 280 have the same configuration as the driver 130, the liquid crystal panel 140, the internal IF 178, the backlight 179, and the image processing engine 180 in the display device 102. Have each. The driver control unit 281, the timer 282, and the signal processing unit 283 have the same configurations as the driver control unit 181, the timer 182, and the signal processing unit 183 in the display device 102. Therefore, description of each functional block included in display device 103 will not be repeated.

  By the way, the processing in the electronic device 100 is realized by each hardware and software executed by the CPU 110. Such software may be stored in the ROM 172 in advance. The software may be stored in a memory card 1731 or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by the memory card reader / writer 173 or other reading device, or downloaded via the communication unit 174 or communication IF (not shown), and then temporarily stored in the ROM 172. The The software is read from the ROM 172 by the CPU 110 and stored in the RAM 171 in the form of an executable program. CPU 110 executes the program.

  Each component constituting the main device 101 of the electronic device 100 shown in FIG. 2 is a general one. Therefore, it can be said that the essential part of the present invention is the software stored in the RAM 171, the ROM 172, the memory card 1731 and other storage media, or the software downloadable via the network. Since the hardware operation of main device 101 of electronic device 100 is well known, detailed description will not be repeated.

  The storage medium is not limited to a memory card, but is a CD-ROM, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile). Disc)), IC (Integrated Circuit) cards (excluding memory cards), optical cards, mask ROM, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory), flash ROM, and other semiconductor memories, etc. It may be a medium to be used.

  The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

<Configuration and Drive of Photosensor Built-in Liquid Crystal Panel 140>
Next, the configuration of the liquid crystal panel 140 and the configuration of peripheral circuits of the liquid crystal panel 140 will be described. FIG. 3 is a diagram showing a configuration of the liquid crystal panel 140 and peripheral circuits of the liquid crystal panel 140.

  Referring to FIG. 3, the liquid crystal panel 140 includes a pixel circuit 141, an optical sensor circuit 144, a scanning signal line Gi, a data signal line SRj, a data signal line SGj, a data signal line SBj, and a sensor signal line. SSj, sensor signal line SDj, read signal line RWi, and reset signal line RSi are included. Note that i is a natural number satisfying 1 ≦ i ≦ m, and j is a natural number satisfying 1 ≦ j ≦ n.

  2 includes a scanning signal line driving circuit 131, a data signal line driving circuit 132, an optical sensor driving circuit 133, a switch 134, as peripheral circuits of the liquid crystal panel 140. And an amplifier 135.

  The scanning signal line drive circuit 131 receives the control signal TC1 from the driver control unit 181 shown in FIG. The scanning signal line drive circuit 131 applies a predetermined voltage in order from the scanning signal line G1 to each scanning signal line (G1 to Gm) based on the control signal TC1. More specifically, the scanning signal line driving circuit 131 sequentially selects one scanning signal line from the scanning signal lines (G1 to Gm) per unit time, and a TFT (to be described later) with respect to the selected scanning signal line. A voltage capable of turning on the gate of the thin film transistor 142 is applied (hereinafter referred to as a high level voltage). Note that a low level voltage is applied to a scanning signal line that is not selected without applying a high level voltage.

  The data signal line driving circuit 132 receives image data (DR, DG, DB) from the driver control unit 181 shown in FIG. The data signal line driving circuit 132 applies a voltage corresponding to one row of image data to the 3n data signal lines (SR1 to SRn, SG1 to SGn, SB1 to SBn) for each unit time. Apply sequentially.

  Note that although a driving method called a so-called line-sequential method has been described here, the driving method is not limited to this.

  The pixel circuit 141 is a circuit for setting the luminance (transmittance) of one pixel. Further, m × n pixel circuits 141 are arranged in a matrix. More specifically, m pixel circuits 141 are arranged in the vertical direction in FIG. 3 and n pixel circuits in the horizontal direction.

  The pixel circuit 141 includes an R subpixel circuit 141r, a G subpixel circuit 141g, and a B subpixel circuit 141b. Each of these three circuits (141r, 141g, 141b) includes a TFT 142, a pair of electrode pairs 143 including a pixel electrode and a counter electrode, and a capacitor (not shown).

  In addition, CMOS (Complementary Metal Oxide Semiconductor) capable of forming n-type transistors and p-type transistors can be realized, and the movement speed of carriers (electrons or holes) is several hundreds compared to amorphous silicon thin film transistors (a-Si TFTs). For example, a polycrystalline silicon thin film transistor (p-Si TFT) is used as the TFT 142 in the display device 102 because it is twice as fast. Note that the TFT 142 will be described as an n-channel field effect transistor. However, the TFT 142 may be a p-type channel field effect transistor.

  The source of the TFT 142 in the R subpixel circuit 141r is connected to the data signal line SRj. The gate of the TFT 142 is connected to the scanning signal line Gi. Further, the drain of the TFT 142 is connected to the pixel electrode of the electrode pair 143. A liquid crystal is disposed between the pixel electrode and the counter electrode. The G sub-pixel circuit 141g and the B sub-pixel circuit 141b have the same configuration as the R sub-pixel circuit 141r except that the data signal line to which the source of each TFT 142 is connected is different. Therefore, description of these two circuits (141g, 141b) will not be repeated.

  Here, setting of luminance in the pixel circuit 141 will be described. First, the high level voltage is applied to the scanning signal line Gi. By the application of the high level voltage, the gate of the TFT 142 is turned on. In this manner, with the gate of the TFT 142 turned on, a specified voltage (voltage corresponding to image data for one pixel) is applied to each data signal line (SRj, SGj, SBj). Thereby, a voltage based on the designated voltage is applied to the pixel electrode. As a result, a potential difference is generated between the pixel electrode and the counter electrode. Based on this potential difference, the liquid crystal responds and the luminance of the pixel is set to a predetermined luminance. Note that the potential difference is held by the capacitor (auxiliary capacitor) (not shown) until the scanning signal line Gi is selected in the next frame period.

  The optical sensor drive circuit 133 receives the control signal TC2 from the driver control unit 181 shown in FIG.

  Then, the optical sensor drive circuit 133 sequentially selects one signal line from the reset signal lines (RS1 to RSm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, the voltage VDDR that is higher than usual is applied. Note that a voltage VSSR lower than the voltage applied to the selected reset signal line is kept applied to the unselected reset signal line. For example, the voltage VDDR may be set to 0V and the voltage VSSR may be set to −5V.

  Further, the optical sensor driving circuit 133 sequentially selects one signal line from the readout signal lines (RW1 to RWm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, a voltage VDD higher than usual is applied. Note that the voltage VSSR is applied to the read signal line that is not selected. For example, the value of VDD may be set to 8V.

  The timing for applying the voltage VDDR and the timing for applying the voltage VDD will be described later.

  The optical sensor circuit 144 includes a photodiode 145, a capacitor 146, and a TFT 147. In the following description, it is assumed that the TFT 147 is an n-type channel field effect transistor. However, the TFT 147 may be a p-type channel field effect transistor.

  The anode of the photodiode 145 is connected to the reset signal line RSi. On the other hand, the cathode of the photodiode 145 is connected to one electrode of the capacitor 146. The other electrode of the capacitor 146 is connected to the read signal line RWi. Hereinafter, a connection point between the photodiode 145 and the capacitor 146 is referred to as a node N.

  The gate of the TFT 147 is connected to the node N. The drain of the TFT 147 is connected to the sensor signal line SDj. Further, the source of the TFT 147 is connected to the sensor signal line SSj. Details of sensing using the optical sensor circuit 144 will be described later.

  The switch 134 is a switch provided to switch whether or not to apply a predetermined voltage to the sensor signal lines (SD1 to SDn). The switching operation of the switch 134 is performed by the optical sensor driving circuit 133. Note that the voltage applied to the sensor signal lines (SD1 to SDn) when the switch 134 is turned on will be described later.

  The amplifier 135 amplifies the voltage output from each sensor signal line (SS1 to SSn). The amplified voltage is sent to the signal processing unit 183 shown in FIG.

  The image processing engine 180 controls the timing for displaying an image on the liquid crystal panel 140 using the pixel circuit 141 and the timing for sensing using the optical sensor circuit 144.

  FIG. 4 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179. Referring to FIG. 4, liquid crystal panel 140 includes an active matrix substrate 151A, a counter substrate 151B, and a liquid crystal layer 152. The counter substrate 151B is disposed to face the active matrix substrate 151A. The liquid crystal layer 152 is sandwiched between the active matrix substrate 151A and the counter substrate 151B. The backlight 179 is disposed on the opposite side of the liquid crystal layer 152 with respect to the active matrix substrate 151A.

  The active matrix substrate 151 </ b> A includes a polarizing filter 161, a glass substrate 162, a pixel electrode 143 a that constitutes the electrode pair 143, a photodiode 145, a data signal line 157, and an alignment film 164. Further, although not shown in FIG. 4, the active matrix substrate 151A includes the capacitor 146, the TFT 147, the TFT 142, and the scanning signal line Gi shown in FIG.

  In the active matrix substrate 151A, the polarizing filter 161, the glass substrate 162, the pixel electrode 143a, and the alignment film 164 are arranged in this order from the backlight 179 side. The photodiode 145 and the data signal line 157 are formed on the liquid crystal layer 152 side of the glass substrate 162.

  The counter substrate 151B includes a polarizing filter 161, a glass substrate 162, a light shielding film 163, color filters (153r, 153g, 153b), a counter electrode 143b constituting the electrode pair 143, and an alignment film 164.

  In the counter substrate 151B, the alignment film 164, the counter electrode 143b, the color filters (153r, 153g, 153b), the glass substrate 162, and the polarizing filter 161 are arranged in this order from the liquid crystal layer 152 side. The light shielding film 163 is formed in the same layer as the color filters (153r, 153g, 153b).

  The color filter 153r is a filter that transmits light having a red wavelength. The color filter 153g is a filter that transmits light having a green wavelength. The color filter 153b is a filter that transmits light having a blue wavelength. Here, the photodiode 145 is arranged at a position facing the color filter 153b.

  The liquid crystal panel 140 displays an image by blocking external light or light emitted from a light source such as a backlight 179 or transmitting the light. Specifically, the liquid crystal panel 140 changes the direction of liquid crystal molecules in the liquid crystal layer 152 by applying a voltage between the pixel electrode 143a and the counter electrode 143b, thereby blocking or transmitting the light. However, since the light cannot be completely blocked by the liquid crystal alone, a polarizing filter 161 that transmits only light having a specific polarization direction is provided.

  Note that the position of the photodiode 145 is not limited to the above position, and may be provided at a position facing the color filter 153r or a position facing the color filter 153g.

  Here, the operation of the optical sensor circuit 144 will be described. FIG. 5 is a diagram showing a timing chart when the optical sensor circuit 144 is operated. In FIG. 5, a voltage VINT indicates a potential at the node N in the photosensor circuit 144. A voltage VPIX is an output voltage from the sensor signal line SSj shown in FIG. 3 and is a voltage before being amplified by the amplifier 135.

  The following description will be divided into a reset period for resetting the optical sensor circuit 144, a sensing period for sensing light using the optical sensor circuit 144, and a readout period for reading the sensed result.

  First, the reset period will be described. In the reset period, the voltage applied to the reset signal line RSi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDDR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR). As described above, by applying the high-level voltage to the reset signal line RSi, a current starts to flow in the forward direction (from the anode side to the cathode side) of the photodiode 145. As a result, the voltage VINT which is the potential of the node N has a value represented by the following expression (1). In Equation (1), the forward voltage drop amount in the photodiode 145 is Vf.

VINT = VSSR + | VDDR−VSSR | −Vf (1)
Therefore, the potential of the node N is a value smaller by Vf than the voltage VDDR as shown in FIG.

  Here, since the voltage VINT is not more than the threshold value for turning on the gate of the TFT 147, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change. Further, a difference corresponding to the voltage VINT occurs between the electrodes of the capacitor 146. For this reason, the capacitor 146 accumulates charges corresponding to the difference.

  Next, the sensing period will be described. In the sensing period following the reset period, the voltage applied to the reset signal line RSi instantaneously switches from the high level (voltage VDDR) to the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR).

  Thus, by changing the voltage applied to the reset signal line RSi to the low level, the potential of the node N becomes higher than the voltage of the reset signal line RSi and the voltage of the read signal line RWi. For this reason, in the photodiode 145, the voltage on the cathode side becomes higher than the voltage on the anode side. That is, the photodiode 145 is in a reverse bias state. In such a reverse bias state, when the photodiode 145 receives light from the light source, current starts to flow from the cathode side to the anode side of the photodiode 145. As a result, as shown in FIG. 5, the potential of the node N (that is, the voltage VINT) becomes lower with the passage of time.

  Since the voltage VINT continues to decrease in this way, the gate of the TFT 147 does not turn on. Therefore, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change.

  Next, the reading period will be described. In the readout period following the sensing period, the voltage applied to the reset signal line RSi is kept at the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDD). Here, the voltage VDD is higher than the voltage VDDR.

  Thus, by applying a high level voltage instantaneously to the read signal line RWi, the potential of the node N is raised through the capacitor 146 as shown in FIG. Note that the increase width of the potential of the node N is a value corresponding to the voltage applied to the read signal line RWi. Here, since the potential of the node N (that is, the voltage VINT) is raised to a threshold value that turns on the gate of the TFT 147, the gate of the TFT 147 is turned on.

  At this time, if a constant voltage is applied in advance to the sensor signal line SDj (see FIG. 3) connected to the drain side of the TFT 147, the sensor signal line SSj connected to the source side of the TFT 147 will cause the VPIX in FIG. As shown in the graph, a voltage corresponding to the potential of the node N is output.

  Here, when the amount of light received by the photodiode 145 (hereinafter referred to as the amount of received light) is small, the slope of the straight line shown in the VINT graph of FIG. 5 becomes gentle. As a result, the voltage VPIX is higher than when the amount of received light is large. As described above, the optical sensor circuit 144 changes the value of the voltage output to the sensor signal line SSj in accordance with the amount of light received by the photodiode 145.

  By the way, in the above, the operation | movement was demonstrated paying attention to one optical sensor circuit 144 among the m * n optical sensor circuits which exist. Below, operation | movement of each photosensor circuit in the liquid crystal panel 140 is demonstrated.

  First, the optical sensor drive circuit 133 applies a predetermined voltage to all n sensor signal lines (SD1 to SDn). Next, the photosensor drive circuit 133 applies a voltage VDDR that is higher than normal to the reset signal line RS1. The other reset signal lines (RS2 to RSm) and read signal lines (RW1 to RWm) are kept in a state where a low level voltage is applied. As a result, the n photosensor circuits in the first row in FIG. 3 enter the reset period described above. Thereafter, the n photosensor circuits in the first row enter a sensing period. Further, thereafter, the n photosensor circuits in the first row enter a reading period.

  Note that the timing at which a predetermined voltage is applied to all n sensor signal lines (SD1 to SDn) is not limited to the above timing, and may be any timing that is applied at least before the readout period. .

  When the readout period of the n photosensor circuits in the first row is completed, the photosensor drive circuit 133 applies a voltage VDDR that is higher than usual to the reset signal line RS2. That is, the reset period of the n photosensor circuits in the second row starts. When the reset period ends, the n photosensor circuits in the second row enter a sensing period, and thereafter enter a reading period.

  Thereafter, the processing described above is sequentially performed on the n photosensor circuits in the third row, the n photosensor circuits in the fourth row,..., The n photosensor circuits in the m row. As a result, the sensing result of the first row, the sensing result of the second row,..., The sensing result of the m-th row are output in this order from the sensor signal lines (SS1 to SSn).

  In the display device 102, sensing is performed for each row as described above, and a sensing result is output from the liquid crystal panel 140 for each row. For this reason, hereinafter, the data after the signal processing unit 183 performs the above-described data processing on the data regarding the voltage for m rows from the first row to the m-th row output from the liquid crystal panel 140, This is called “scan data”. That is, scan data refers to image data obtained by scanning a scan target (for example, a user's finger). An image displayed based on the scan data is referred to as a “scanned image”. Further, in the following, sensing is referred to as “scan”.

  Moreover, in the above, although the structure which scans using all the m * n photosensor circuits was mentioned as an example, it is not limited to this. Scanning may be performed on a partial region of the surface of the liquid crystal panel 140 using a photosensor circuit selected in advance.

  In the following, it is assumed that the electronic device 100 can take either of the two configurations. Further, switching between the components is assumed to be performed by a command sent from the main device 101 based on an input via the operation key 177 or the like. Note that when scanning is performed on a partial area on the surface of the liquid crystal panel 140, the image processing engine 180 sets a scan target area. The setting of the area may be configured to be specified by the user via the operation key 177.

  As described above, when scanning is performed on a partial region of the surface of the liquid crystal panel 140, there are the following modes of use for displaying an image. The first is a mode in which an image is displayed in a surface area other than the partial area (hereinafter referred to as a scan area). The second is a mode in which no image is displayed in the surface area other than the scan area. Which mode is used is based on a command sent from the main apparatus 101 to the image processing engine 180.

  FIG. 6 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179, showing a configuration in which the photodiode 145 receives light from the backlight 179 during scanning.

  Referring to FIG. 6, when the user's finger 900 is in contact with the surface of the liquid crystal panel 140, a part of the light emitted from the backlight 179 is part of the user's finger 900 (abbreviated in the contacted area). Reflected on the plane). The photodiode 145 receives the reflected light.

  Even in a region where the finger 900 is not in contact, part of the light emitted from the backlight 179 is reflected by the user's finger 900. Even in this case, the photodiode 145 receives the reflected light. However, since the finger 900 is not in contact with the surface of the liquid crystal panel 140 in the region, the amount of light received by the photodiode 145 is smaller than the region in which the finger 900 is in contact. Note that the photodiode 145 cannot receive most of the light emitted from the backlight 179 that does not reach the user's finger 900.

  Here, by turning on the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the amount of light reflected by the user's finger 900 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter, also referred to as a reflected image) obtained by reflecting light with the finger 900.

Note that examples of the scan object other than the finger 900 include a stylus pen.
By the way, in the present embodiment, a liquid crystal panel is described as an example of the display device of electronic device 100, but another panel such as an organic EL (Electro-Luminescence) panel is used instead of the liquid crystal panel. Also good.

<About data>
Next, the sensing command will be described. In the display device 102, the image processing engine 180 analyzes the content of the sensing command and sends back data (that is, response data) according to the analysis result to the main body device 101.

  FIG. 7 is a diagram showing a schematic configuration of a sensing command. Referring to FIG. 7, the sensing command includes a header data area DA01, a timing data area DA02, a data type data area DA03, a reading method data area DA04, and image gradation data. An area DA05, a data area DA06 indicating resolution, and a spare data area DA07 are included.

  FIG. 8 is a diagram showing data values in each area of the sensing command and the meaning contents indicated by the values.

  Referring to FIG. 8, the sensing command in which “00” is set in the data area indicating the timing requests image processing engine 180 to transmit scan data at that time. That is, the sensing command requests transmission of scan data obtained by scanning using the optical sensor circuit 144 after the image processing engine 180 receives the sensing command. A sensing command in which “01” is set in the data area indicating the timing requests transmission of scan data when the scan result has changed. Furthermore, a sensing command in which “10” is set in the data area indicating the timing requests transmission of scan data at regular intervals.

  A sensing command in which “001” is set in the data area indicating the data type requests transmission of the coordinate value of the center coordinate in the partial image. In addition, a sensing command in which “010” is set in the data area indicating the data type requests transmission of only a partial image whose scan result has changed. Note that the change in the scan result indicates that the previous scan result is different from the current scan result. Further, a sensing command in which “100” is set in the data area indicating the data type requests transmission of the entire image.

  Here, the “entire image” is an image generated by the image processing engine 180 based on the voltage output from each photosensor circuit when scanning using m × n photosensor circuits. . The “partial image” is a part of the entire image. The reason why the partial image is requested to transmit only the partial image whose scan result has changed will be described later.

  The coordinate value and the partial image or the entire image may be requested at the same time. Further, in the case of a configuration in which scanning is performed on a partial area on the surface of the liquid crystal panel 140, the entire image is an image corresponding to the area to be scanned.

  A sensing command in which “00” is set in the data area indicating the reading method requests to scan with the backlight 179 on. A sensing command in which “01” is set in the data area indicating the reading method requests scanning with the backlight 179 off. A configuration for scanning with the backlight 179 off is described later (FIG. 11). Furthermore, a sensing command in which “10” is set in the data area indicating the reading method requests scanning using both reflection and transmission. Note that the combined use of reflection and transmission refers to scanning a scan object by switching between a method in which the backlight 179 is turned on for scanning and a method in which the backlight is turned off for scanning.

  A sensing command in which “00” is set in the data area indicating the image gradation requests monochrome image data of black and white. A sensing command in which “01” is set in the data area indicating the image gradation requests multi-gradation image data. Furthermore, a sensing command in which “10” is set in the data area indicating the image gradation requests RGB color image data.

  A sensing command in which “0” is set in the data area indicating the resolution requests image data with a high resolution. A sensing command in which “1” is set in the data area indicating the resolution requests image data with a low resolution.

  In addition to the data shown in FIG. 7 and FIG. 8, the sensing command includes designation of an area to be scanned (area of pixels that drive the optical sensor circuit 144), timing to perform scanning, and timing to turn on the backlight 179. Etc. are described.

  FIG. 9 is a diagram showing a schematic configuration of response data. The response data is data corresponding to the content of the sensing command, and is data that the image processing engine 180 of the display device 102 transmits to the main body device 101.

  Referring to FIG. 9, the response data includes a header data area DA11, a coordinate data area DA12, a time data area DA13, and an image data area DA14. Here, the value of the center coordinates of the partial image is written in the data area DA12 indicating the coordinates. In addition, time information acquired from the timer 182 of the image processing engine 180 is written in the data area indicating the time. Further, image data (that is, scan data) after being processed by the image processing engine 180 is written in the data area indicating the image.

  FIG. 10 is a diagram illustrating an image obtained by scanning the finger 900 (that is, a scanned image). Referring to FIG. 10, an image of a region W1 surrounded by a thick solid line is an entire image, and an image of a region P1 surrounded by a broken line is a partial image. The center point C1 of the cross indicated by a thick line is the center coordinate.

  In the present embodiment, a pixel (that is, a predetermined gradation or a pixel having a photosensor circuit which is a rectangular region and whose output voltage from the sensor signal line SSj is equal to or higher than a predetermined value). An area including all of the pixels having a luminance equal to or higher than a predetermined luminance is set as a partial image area.

  The center coordinates are coordinates determined in consideration of the gradation of each pixel in the partial image area. Specifically, for each pixel in the partial image, the center coordinate is determined by performing a weighting process based on the gradation of the pixel and the distance between the pixel and the rectangular center point (that is, the centroid). Is done. That is, the center coordinates do not necessarily match the centroid of the partial image.

  However, the position of the center coordinates is not necessarily limited to the above, and the center coordinates may be the coordinates of the centroid or the coordinates near the centroid.

  When “001” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the value of the central coordinate in the data area DA12 indicating the coordinate. In this case, the image processing engine 180 does not write image data in the data area DA14 indicating an image. After writing the value of the center coordinate, the image processing engine 180 sends response data including the value of the center coordinate to the main body device 101. As described above, when “001” is set in the data area indicating the data type, the sensing command requests the output of the value of the center coordinate without requesting the output of the image data.

  Further, when “010” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the image data of the partial image whose scan result has changed in the data area DA14 indicating the image. . In this case, the image processing engine 180 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 180 writes the image data of the partial image whose scan result has changed, and then sends response data including the image data of the partial image to the main body device 101. As described above, when “010” is set in the data area indicating the data type, the sensing command does not request the output of the value of the center coordinate, and the image data of the partial image whose scan result has changed. Request output.

  As described above, the reason for requesting transmission of only the partial image whose scan result has changed is that the scan data of the partial image area of the scan data is more important than the scan data of the other area. This is because the data is high, and the scan data in the region corresponding to the region of the partial image in the scan data is likely to change due to the contact state with the scan object such as the finger 900.

  When “011” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the value of the central coordinate in the data area DA12 indicating the coordinate and also displays the data area indicating the image. The image data of the partial image whose scan result has changed is written in DA14. Thereafter, the image processing engine 180 sends response data including the value of the center coordinate and the image data of the partial image to the main body device 101. As described above, when “011” is set in the data area indicating the data type, the sensing command requests the output of the value of the center coordinate and the output of the image data of the partial image whose scan result has changed. To do.

  If “100” is set in the data area indicating the data type of the sensing command, the image processing engine 180 displays the image of the entire image in the data area DA14 indicating the image of the response data shown in FIG. Write data. In this case, the image processing engine 180 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. After writing the image data of the entire image, the image processing engine 180 sends response data including the image data of the entire image to the main body device 101. As described above, when “100” is set in the data area indicating the data type, the sensing command requests the output of the image data of the entire image without requesting the output of the value of the center coordinate.

  When “101” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the value of the central coordinate in the data area DA12 indicating the coordinate and also displays the data area indicating the image. Write the image data of the entire image to DA14. Thereafter, the image processing engine 180 sends response data including the value of the center coordinate and the image data of the entire image to the main body device 101. As described above, when “101” is set in the data area indicating the data type, the sensing command requests the output of the value of the center coordinate and the output of the image data of the entire image.

<Variation of scanning method>
Next, a scanning method different from the scanning method described with reference to FIG. 6 (that is, the method of scanning the reflected image) will be described with reference to FIG.

  FIG. 11 is a cross-sectional view showing a configuration in which a photodiode receives external light during scanning. Referring to FIG. 11, part of the outside light is blocked by finger 900. Therefore, the photodiode disposed under the surface region of the liquid crystal panel 140 that is in contact with the finger 900 can hardly receive external light. In addition, although the photodiodes disposed under the surface area where the shadow of the finger 900 is formed can receive a certain amount of external light, the amount of external light received is small compared to the surface area where no shadow is formed.

  Here, by turning off the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the position of the finger 900 with respect to the surface of the liquid crystal panel 140 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), and the liquid crystal panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter also referred to as a shadow image) obtained by blocking external light with the finger 900.

  Further, the display device 102 may be configured to perform scanning by turning on the backlight 179 and then performing scanning again by turning off the backlight 179. Alternatively, the display device 102 may be configured to perform scanning by turning off the backlight 179 and then performing scanning again by turning on the backlight 179.

  In this case, since two scanning methods are used together, two scan data can be obtained. Therefore, it is possible to obtain a highly accurate result as compared with the case of scanning using only one scanning method.

<About display devices>
The operation of the display device 103 is controlled in accordance with a command (for example, a sensing command) from the main body device 101 as in the operation of the display device 102. The display device 103 has the same configuration as the display device 102. Therefore, when the display device 103 receives the same command as the display device 102 from the main body device 101, the display device 103 performs the same operation as the display device 102. For this reason, description of the configuration and operation of the display device 103 will not be repeated.

  Note that the main device 101 can send commands having different commands to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform different operations. Further, the main device 101 may send a command to either the display device 102 or the display device 103. In this case, only one display device performs an operation according to the command. Further, the main device 101 may send a command having the same command to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform the same operation.

  Note that the size of the liquid crystal panel 140 of the display device 102 and the size of the liquid crystal panel 240 of the display device 103 may be the same or different. Further, the resolution of the liquid crystal panel 140 and the resolution of the liquid crystal panel 240 may be the same or different.

  As described above, the liquid crystal panels 140 and 240 of the electronic device 100 according to the present embodiment may not be of the photosensor built-in type. That is, the liquid crystal panels 140 and 240 of the electronic device 100 according to the present embodiment may both be touch panels having a normal tablet function.

<Functional configuration of electronic device 100 according to the present embodiment>
Hereinafter, a functional configuration of electronic device 100 according to the present embodiment will be described with reference to FIGS. 1, 2, and 12. FIG. 12 is a block diagram showing a functional configuration of electronic device 100 according to the present embodiment.

  Electronic device 100 according to the present embodiment includes reception unit 111, first determination unit 112, second determination unit 113, third determination unit 114, fourth determination unit 115, and display control. Part 116. In addition, as shown in FIG. 2, the electronic device 100 includes a first photosensor built-in liquid crystal panel 140 (first display panel) including a RAM 171, a plurality of photosensor circuits 144, and a plurality of pixel circuits 141. And a second photosensor built-in liquid crystal panel 240 (second display panel) including a plurality of photosensor circuits 244 and a plurality of pixel circuits 241.

  The first display panel may be a concept showing the first display device 102 including the first photosensor built-in liquid crystal panel 140, the image processing engine 180, the internal IF 178, and the like. The second display panel may be a concept showing the second display device 103 including the second optical sensor built-in liquid crystal panel 240, the image processing engine 280, the internal IF 278, and the like.

  First, each of the plurality of photosensor circuits 144 of the first display panel 140 receives incident light and generates an electrical signal corresponding to the incident light. The plurality of photosensor circuits 144 as a whole input an electrical signal (image data) corresponding to incident light to the receiving unit 111 via the image processing engine 180 (FIG. 2) or the like. Note that the plurality of optical sensor circuits 144 may read the position of the finger or the stylus pen with the backlight 179 turned off as shown in FIG.

  As described above, the plurality of optical sensor circuits 144 and the image processing engine 180 according to the present embodiment realize an operation unit as a whole. Then, the operation unit accepts a position designation command for designating a position on the first display panel 140 via the first display panel 140. Accordingly, the first display panel 140 can accept a selection command for selecting an object displayed on the first display panel 140 or a move command for moving the object.

  Each of the plurality of pixel circuits 141 of the first display panel 140 emits visible light to the outside based on image data or text data from the display control unit 116, that is, based on an output signal from the CPU 110. More specifically, the plurality of pixel circuits 141 use light from the backlight 179 as a whole based on image data or text data from the display control unit 116 via the image processing engine 180 (FIG. 2) or the like. While displaying objects such as icons and text.

  Thus, the plurality of pixel circuits 141 and the image processing engine 180 according to the present embodiment realize a display unit as a whole. That is, the display unit displays the object on the first display panel 140.

  Although the operation unit and the display unit according to the present embodiment are realized by the first display panel 140 (first display device 102), as described above, the operation unit and the display unit are normal touch panel type displays. It may be realized by a panel. Note that the configuration of the second display panel 240 (second display device 103) is substantially the same as the configuration of the first display panel 140 (first display device 102), and thus description thereof will be repeated here. Absent.

  The reception unit 111, the first determination unit 112, the second determination unit 113, the third determination unit 114, the fourth determination unit 115, and the display control unit 116 are realized by the CPU 110 or the like. It is a function. More specifically, each function of the CPU 110 is a function realized by the CPU 110 executing a control program stored in the RAM 171 or the like to control each hardware shown in FIG.

  First, the receiving unit 111 selects an object selection command or object input to the first display panel 140 based on an electrical signal (image data) input from the plurality of optical sensor circuits 144 via the image processing engine 180. It accepts movement commands. More specifically, the reception unit 111 acquires image data output from the image processing engine 180 of the liquid crystal panel 140 at each sensing time, and receives a selection command, a movement command, and the like based on the image data.

  Similarly, the reception unit 111 receives an object selection command input to the second display panel 240 based on an electrical signal (image data) input from the plurality of photosensor circuits 244 via the image processing engine 280, and the like. Accepts object movement commands. More specifically, the reception unit 111 acquires image data output from the image processing engine 280 of the liquid crystal panel 240 at each sensing time, and receives a selection command, a movement command, and the like based on the image data.

  The accepting unit 111 may store the first image data output from the image processing engine 180 in the RAM 171. That is, the reception unit 111 may constantly update the first image data in the RAM 171 to the latest image data. Similarly, the reception unit 111 may store the second image data output from the image processing engine 280 in the RAM 171. That is, the second reception unit 111 may constantly update the second image data in the RAM 171 to the latest image data.

  The reception unit 111 may have a function realized by the CPU 110, the plurality of photosensor circuits 144 of the first display panel 140, and the plurality of photosensor circuits 244 of the second display panel 240. That is, the accepting unit 111 may be a concept showing a functional block including a part of the function of the CPU 110, the light receiving function of the first display panel 140, and the light receiving function of the second display panel 240.

  In this way, for example, the accepting unit 111 acquires, as the designated position Y, the contact position (such as the center coordinates of the partial image) between the finger 900 or the stylus pen and the first display panel 140 from the first display panel 140. To do. Alternatively, the accepting unit 111 acquires the contact position between the finger 900 or the stylus pen and the second display panel 240 from the second display panel 240 as the designated position Y.

  When the reception unit 111 receives a position designation command from the first display panel 140 or the second display panel 240, the reception unit 111 determines the designated position Y from the first determination unit 112, the second determination unit 113, and the fourth determination. To the unit 115.

  The first determination unit 112 acquires the display area (display range) of the object displayed on the first display panel 140 or the second display panel 240 from the display control unit 116. More specifically, the display area may be indicated by the coordinates of the four corners of the object, may be indicated by the upper left coordinates of the object, the vertical length (number of dots), and the horizontal length, May be indicated by the center coordinate, vertical length, and horizontal length.

  Then, the first determination unit 112 determines whether or not the designated position Y is located within the display area. That is, the first determination unit 112 determines whether an object has been selected. The first determination unit 112 passes the determination result to the second determination unit 113 and the display control unit 116.

  The second determination unit 113 includes first information indicating the first area 140Z determined in advance on the first display panel 140, and the second area 240Z determined in advance on the second display panel 240. 2nd information which shows. More specifically, the first information may include the coordinates of the four corners of the first area 140Z, the upper left coordinates of the first area 140Z, and the vertical length (number of dots). The horizontal length may be included, or the center coordinate, the vertical length, and the horizontal length of the first area 140Z may be included.

  Then, the second determination unit 113 has the designated position Y in the first area 140Z that is predetermined in the first display panel 140 or in the second area 240Z that is predetermined in the second display panel 240. It is judged whether it is located in. The second determination unit 113 passes the determination result to the third determination unit 114, the fourth determination unit 115, and the display control unit 116.

  The third determination unit 114 determines whether or not the object display area extends over the first display panel 140 and the second display panel 240. For example, the third determination unit 114 determines whether or not the upper left coordinate of the object display area is located in the first display panel 140, and the lower left coordinate of the object display area is in the second display panel 240. It is judged whether it is located in. Then, the third determination unit 114 has the upper left coordinates of the object display area located in the first display panel 140 and the lower left coordinates of the object display area located in the second display panel 240. In this case, it is determined that the object is displayed across the first display panel 140 and the second display panel 240.

  The fourth determination unit 115 determines whether or not a position designation command is input to the reception unit 111. In other words, the fourth determination unit 115 determines whether or not the position designation command from the first display panel 140 or the second display panel 240 has been released. That is, the fourth determination unit 115 applies a finger 900 to the first display panel 140 or the second display panel 240 based on the center coordinates or the image data from the first display panel 140 or the second display panel 240. It is determined whether or not a stylus pen or the like is in contact. More specifically, in the fourth determination unit 115 according to the present embodiment, the first determination unit 112 determines that the specified position Y is within the display area of the object, and the second determination unit 113 specifies When it is determined that the position Y is within the first area 140Z, it is determined whether or not the position designation command from the user has been canceled.

  The display control unit 116 outputs an output signal to the first display panel 140 and the second display panel 240 to display an object on at least one of the first display panel 140 and the second display panel 240.

  The display control unit 116 determines that the first determination unit 112 determines that the specified position Y is within the display area of the object, and the second determination unit 113 determines that the specified position Y is within the first area 140Z. If so, the object is displayed again on the second display panel 240. That is, the display control unit 116 moves the object from the first display panel 140 to the second display panel 240.

  When the third determination unit 114 determines that the object is displayed across the first display panel 140 and the second display panel 240, the display control unit 116 according to the present embodiment The object is displayed again on the second display panel 240. That is, in the display control unit 116 according to the present embodiment, the first determination unit 112 determines that the designated position Y is within the display area of the object, and the second judgment unit 113 determines that the designated position Y is the first When it is determined that the area is within the area 140Z and the third determination unit 114 determines that the object is displayed across the first display panel 140 and the second display panel 240, the object is moved.

  However, in the display control unit 116, for example, the first determination unit 112 determines that the designated position Y is within the display area of the object, and the third determination unit 114 determines that the object is the second display panel 140 and the second display panel 140. The object may be moved when it is determined that it is displayed across the display panel 240.

  Conversely, the display control unit 116 determines that the first determination unit 112 determines that the specified position Y is within the display area of the object, and the second determination unit 113 determines that the specified position Y is within the second area 240Z. When it is determined that there is an object, the object is displayed again on the first display panel 140. That is, the display control unit 116 moves the object from the second display panel 240 to the first display panel 140.

  When the third determination unit 114 determines that the object is displayed across the first display panel 140 and the second display panel 240, the display control unit 116 according to the present embodiment Redisplay the object. That is, in the display control unit 116 according to the present embodiment, the first determination unit 112 determines that the designated position Y is within the display area of the object, and the second judgment unit 113 determines that the designated position Y is the second value. When it is determined that the area is within the area 240Z and the third determination unit 114 determines that the object is displayed across the first display panel 140 and the second display panel 240, the object is moved.

  However, in the display control unit 116, for example, the first determination unit 112 determines that the designated position Y is within the display area of the object, and the third determination unit 114 determines that the object is the second display panel 140 and the second display panel 140. The object may be moved when it is determined that it is displayed across the display panel 240.

  Furthermore, the display control unit 116 according to the present embodiment displays the object again when the fourth determination unit 115 determines that the designation of the designated position Y has been cancelled. In other words, the display control unit 116 according to the present embodiment, after an object is selected once, that is, after the user's finger 900 touches either the first display panel 140 or the second display panel 240. When the user's finger 900 moves away from the first area 140Z of the first display panel 140 or the second area 240Z of the second display panel 240, the object is displayed again on the other display panel.

<First operation example>
Hereinafter, with reference to FIG. 1 and FIG. 12, an outline of the operation in the electronic apparatus 100 when an object is moved from the first display panel 140 to the second display panel 240 will be described.

  First, as illustrated in FIG. 1A, the user's finger 900 touches the icon X while the icon X is displayed on the first display panel 140. At this time, the accepting unit 111 accepts a position designation command based on the image data from the first display panel 140. Alternatively, the reception unit 111 receives a position designation command based on the center coordinates of the contact portion between the finger 900 and the first display panel 140 from the first display panel 140.

  The first determination unit 112 acquires the display area of the icon X from the display control unit 116, and determines whether or not the specified position Y is within the display area of the icon X based on the position specifying command. If the first determination unit 112 determines that the designated position Y is within the display area of the icon X, the first determination unit 112 places the icon X in a selected state. More specifically, the display control unit 116 changes the color of the selected icon X to a color different from the other icons based on the determination result by the first determination unit 112, and the icon X is in the selected state. Let the outside know.

  The second determination unit 113 determines whether or not the finger 900 that has selected the icon X is positioned within the first area 140Z defined on the first display panel 140. That is, the second determination unit 113 determines whether or not the designated position Y is located in the first area 140Z. When the designated position Y is not located within the first area 140Z, the third determination unit 114 does not function and the icon X does not move to the second display panel 240.

  Then, as shown in FIG. 1B, the user moves the icon X in the direction of the second display panel 240 (downward) without releasing the finger 900 from the first display panel 140 (dragging). To do). The accepting unit 111 accepts a movement command for the icon X based on a change in a plurality of image data or a change in central coordinates sequentially sent from the first display panel 240.

  The second determination unit 113 determines whether or not the finger 900 that has selected the icon X is located in the first area 140Z (the lower portion of the first display panel 140) defined on the first display panel 140. to decide. That is, the second determination unit 113 determines whether or not the designated position Y is located in the first area 140Z. The third determination unit 114 determines whether or not the icon X is displayed across the first display panel 140 and the second display panel 240 when the designated position Y is located in the first area 140Z. to decide.

  When the icon X is displayed across the first display panel 140 and the second display panel 240, as shown in FIG. 1C, the display control unit 116 displays the icon X on the second display panel. 240 is displayed again. That is, the display control unit 116 moves the icon X from the first display panel 140 to the second display panel 240.

  When the icon X is displayed across the first display panel 140 and the second display panel 240, the fourth determination unit 115 determines whether or not the position designation command is canceled. Also good. Then, when the position designation command is canceled, the display control unit 116 may redisplay the icon X on the second display panel 240.

<Second operation example>
Hereinafter, with reference to FIG. 12 and FIG. 13, an outline of the operation in the electronic apparatus 100 when the object is moved from the second display panel 240 to the first display panel 140 will be described. FIG. 13 is a second schematic diagram showing an outline of the operation of electronic device 100 according to the present embodiment.

  First, as illustrated in FIG. 13A, the user's finger 900 touches the icon X while the icon X is displayed on the second display panel 240. The accepting unit 111 accepts a position designation command based on image data from the second display panel 240 or center coordinates.

  The first determination unit 112 determines whether or not the designated position Y (second position) is within the icon X display area. When the first determination unit 112 determines that the designated position Y is within the display area of the icon X, the first determination unit 112 places the icon X in a selected state. More specifically, the display control unit 116 changes the color of the selected icon X to a color different from the other icons based on the determination result by the first determination unit 112, and the icon X is in the selected state. Let the outside know.

  The second determination unit 113 determines whether or not the finger 900 that has selected the icon X is located within the second area 240Z defined on the second display panel 240. That is, the second determination unit 113 determines whether or not the designated position Y is located in the second area 240Z. When the designated position Y is not located in the second area 240Z, the third determination unit 114 does not function and the icon X does not move to the first display panel 240.

  Then, as shown in FIG. 13B, the user moves the icon X in the direction (upward direction) of the first display panel 140 without dragging the finger 900 from the second display panel 240 (dragging). To do). The accepting unit 111 accepts an instruction to move the icon X based on a change in a plurality of image data or a change in central coordinates sequentially sent from the second display panel 240.

  The second determination unit 113 determines whether or not the finger 900 that has selected the icon X is located in the second area 240Z (upper part of the second display panel 240) defined on the second display panel 240. to decide. That is, the second determination unit 113 determines whether or not the designated position Y is located in the second area 240Z. The third determining unit 114 determines whether or not the icon X is displayed across the first display panel 140 and the second display panel 240 when the designated position Y is located in the second area 240Z. to decide.

  When the icon X is displayed across the first display panel 140 and the second display panel 240, as shown in FIG. 13C, the display control unit 116 displays the icon X on the first display panel. 140 is displayed again. That is, the display control unit 116 moves the icon X from the second display panel 240 to the first display panel 140.

  Note that when the icon X is displayed across the first display panel 140 and the second display panel 240, the fourth determination unit 115 determines whether or not the designated position Y is released. Good. Then, when the designated position Y is canceled, the display control unit 116 may redisplay the icon X on the first display panel 140.

<Third operation example>
Hereinafter, with reference to FIG. 12 and FIG. 14, an outline of operation in the electronic device 100 when the object is not moved beyond the display panel will be described. FIG. 14 is a third schematic diagram illustrating an outline of operation of electronic device 100 according to the present embodiment.

  First, as illustrated in FIG. 14A, the user's finger 900 touches the icon X while the icon X is displayed on the first display panel 140. At this time, the accepting unit 111 accepts a position designation command based on the image data from the first display panel 140 or the center coordinates.

  The first determination unit 112 acquires the display area of the icon X from the display control unit 116, and determines whether or not the specified position Y is within the display area of the icon X based on the position specifying command. If the first determination unit 112 determines that the designated position Y is within the display area of the icon X, the first determination unit 112 places the icon X in a selected state. More specifically, the display control unit 116 changes the color of the selected icon X to a color different from the other icons based on the determination result by the first determination unit 112, and the icon X is in the selected state. Let the outside know.

  The second determination unit 113 determines whether or not the finger 900 that has selected the icon X is positioned within the first area 140Z defined on the first display panel 140. That is, the second determination unit 113 determines whether or not the designated position Y is located in the first area 140Z. When the designated position Y is not located within the first area 140Z, the third determination unit 114 does not function and the icon X does not move to the second display panel 240.

  Then, as shown in FIG. 14B, the user moves the icon X in the direction (downward) of the second display panel 240 without dragging the finger 900 from the first display panel 140 (dragging). To do). The accepting unit 111 accepts a movement command for the icon X based on a change in a plurality of image data or a change in central coordinates sequentially sent from the first display panel 240.

  The second determination unit 113 determines whether or not the finger 900 that has selected the icon X is positioned within the first area 140Z defined on the first display panel 140. That is, the second determination unit 113 determines whether or not the designated position Y is located in the first area 140Z. Here, since the designated position Y is not located in the first area 140Z, the third determination unit 114 does not function and the icon X does not move to the second display panel 240.

  As shown in FIG. 14C, the user moves (drags) the icon X upward without releasing the finger 900 from the first display panel 140. The accepting unit 111 accepts a movement command for the icon X based on a change in a plurality of image data or a change in central coordinates sequentially sent from the first display panel 240. The second determination unit 113 determines whether or not the finger 900 that has selected the icon X is positioned within the first area 140Z defined on the first display panel 140.

  That is, every time the icon X is moved, the second determination unit 113 according to the present embodiment has a designated position Y that is a predetermined area on the side where the icon X is displayed, that is, the first display panel 140 of the first display panel 140. It is determined whether or not it is located within one area 140Z. Again, if the designated position Y is not located in the first area 140Z of the first display panel 140, the third determination unit 114 does not function and the icon X does not move to the second display panel 240.

<Fourth operation example>
Hereinafter, with reference to FIG. 12 and FIG. 15, an outline of operation in the electronic apparatus 100 when moving an object from the first display panel 140 to the second display panel 240 will be described. FIG. 15 is a fourth schematic diagram illustrating an outline of operation of electronic device 100 according to the present embodiment.

  First, as shown in FIG. 15A, an icon X is displayed on the first display panel 140 and the second display panel 240. That is, the icon X is displayed across the first display panel 140 and the second display panel 240.

  As shown in FIG. 15B, the user's finger 900 touches the icon X in this state. At this time, the accepting unit 111 accepts a position designation command based on the image data from the first display panel 140 or the center coordinates.

  The first determination unit 112 acquires the display area of the icon X from the display control unit 116, and determines whether or not the specified position Y is within the display area of the icon X based on the position specifying command. If the first determination unit 112 determines that the designated position Y is within the display area of the icon X, the first determination unit 112 places the icon X in a selected state. More specifically, the display control unit 116 changes the color of the selected icon X to a color different from the other icons based on the determination result by the first determination unit 112, and the icon X is in the selected state. Let the outside know.

  The second determination unit 113 determines whether or not the finger 900 that has selected the icon X is positioned within the first area 140Z defined on the first display panel 140. That is, the second determination unit 113 determines whether or not the designated position Y is located in the first area 140Z. Here, since the designated position Y is located in the first area 140Z, the third determination unit 114 displays whether the icon X is displayed across the first display panel 140 and the second display panel 240. Judge whether or not.

  When the icon X is displayed across the first display panel 140 and the second display panel 240, as shown in FIG. 15C, the display control unit 116 displays the icon X on the second display panel. 240 is displayed again. That is, the display control unit 116 moves the icon X from the first display panel 140 to the second display panel 240.

  When the icon X is displayed across the first display panel 140 and the second display panel 240, the fourth determination unit 115 may determine whether or not the position designation command is canceled. Good. Then, when the position designation command is canceled, the display control unit 116 may redisplay the icon X on the second display panel 240.

<Object display processing according to this embodiment>
Next, with reference to FIG. 1, FIG. 2, FIG. 12, and FIG. 16, an object display process in electronic device 100 according to the present embodiment will be described. FIG. 16 is an image diagram showing a processing procedure of object display processing in electronic device 100 according to the present embodiment. Hereinafter, a case where an object such as the icon X that is displayed in advance on the first display panel 140 is moved to the second display panel 240 will be described.

  First, the plurality of photosensor circuits 144 included in the first display panel 140 always generate electrical signals according to incident light. Each of the optical sensor circuits 144 inputs image data generated via the image processing engine 180 to the CPU 110. That is, the CPU 110 functioning as the reception unit 111 receives electrical signals from the plurality of optical sensor circuits 144 as image data via the image processing engine 180 (step S102). In other words, the CPU 110 always acquires image data from the image processing engine 180.

  The CPU 110 determines whether the image data has changed by comparing the image data stored in the RAM 171 last time with the image data generated this time (step S104). If there is no change in the image data (NO in step S104), CPU 110 repeats the processing from step S102.

  If there is a change in the image data (YES in step S), CPU 110 functioning as first determination unit 112 is displayed on first display panel 140 by the user via finger 900 or the like. It is determined whether or not an object has been selected (step S106). More specifically, the CPU 110 acquires a position designation command for the finger 900 based on the image data. The CPU 110 determines whether or not the designated position Y is located within the object display area. In this way, when CPU 110 determines that the user has not selected an object (NO in step S106), CPU 110 repeats the processing from step S102.

  On the other hand, if it is determined that the user has selected an object (YES in step S106), CPU 110 functioning as second determination unit 113 determines whether designated position Y is located within first area 140Z. It is determined whether or not (step S108). When the designated position Y is not located within the first area 140Z of the first display panel 140 (NO in step S108), the CPU 110 repeats the processing from step S102.

  On the other hand, when the designated position Y is located in the first area 140Z of the first display panel 140 (YES in step S108), the CPU 110 functioning as the third determination unit 114 selects the object. It is determined whether or not the display is performed across the first display panel 140 and the second display panel 240 (step S110). CPU110 repeats the process from step S102, when the object is not displayed over the 1st display panel 140 and the 2nd display panel 240 (when it is NO in step S110).

  On the other hand, CPU 110 functioning as fourth determination unit 115 displays the object across first display panel 140 and second display panel 240 (if YES in step S110), the object It is determined whether or not the selection has been canceled (step S112). That is, the CPU 110 determines whether or not the position designation command is not accepted via the first display panel 140. If the selection of the object is not released (NO in step S112), CPU 110 repeats the processing from step S110.

  On the other hand, when the selection of the object is canceled (YES in step S112), CPU 110 functioning as display control unit 116 redisplays the object at the center position of second display panel 240 (step S114). That is, the CPU 110 moves the object to the second display panel 240.

<Other embodiments>
It goes without saying that the present invention can also be applied to a case where it is achieved by supplying a program to a system or apparatus. Then, a storage medium storing a program represented by software for achieving the present invention is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the program code stored in the storage medium It is possible to enjoy the effects of the present invention also by reading and executing.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card (IC memory card), ROM (mask ROM, flash) EEPROM, etc.) can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code However, it is needless to say that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Further, after the program code read from the storage medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a 1st schematic diagram which shows the operation | movement outline | summary of the electronic device which concerns on this Embodiment. It is a block diagram showing the hardware constitutions of an electronic device. It is the figure which showed the structure of the liquid crystal panel, and the peripheral circuit of the said liquid crystal panel. It is sectional drawing of a liquid crystal panel and a backlight. It is the figure which showed the timing chart at the time of operating an optical sensor circuit. It is sectional drawing which showed the structure which a photodiode receives the light from a backlight in the case of a scan. It is the figure which showed schematic structure of the sensing command. It is the figure which showed the value of the data in each area | region of a sensing command, and the meaning content which the said value shows. It is the figure which showed schematic structure of the response data. It is the figure which showed the image obtained by scanning a finger | toe. It is sectional drawing which showed the structure which a photodiode receives external light in the case of a scan. It is a block diagram which shows the function structure of the electronic device which concerns on this Embodiment. It is a 2nd schematic diagram which shows the operation | movement outline | summary of the electronic device which concerns on this Embodiment. It is a 3rd schematic diagram which shows the operation | movement outline | summary of the electronic device which concerns on this Embodiment. It is the 4th schematic diagram which shows the operation | movement outline | summary of the electronic device which concerns on this Embodiment. It is an image figure which shows the process sequence of the object display process in the electronic device which concerns on this Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 Electronic device, 101 Main body apparatus, 102,103 Display apparatus, 110 CPU, 111 reception part, 112 1st judgment part, 113 2nd judgment part, 114 3rd judgment part, 115 4th judgment part, 116 Display control unit, 130 driver, 131 scanning signal line drive circuit, 132 data signal line drive circuit, 133 photosensor drive circuit, 134 switch, 135 amplifier, 140 first photosensor built-in liquid crystal panel (first display panel), 141,241 pixel circuit, 141b, 141g, 141r sub-pixel circuit, 143 electrode pair, 143a pixel electrode, 143b counter electrode, 144,244 photosensor circuit, 145, 145b, 145g, 145r photodiode, 146 capacitor, 151A active matrix Substrate, 151B Opposing group , 152 Liquid crystal layer, 153b, 153g, 153r Color filter, 157 Data signal line, 161 Polarizing filter, 162 Glass substrate, 163 Shading film, 164 Alignment film, 171 RAM, 173 Memory card reader / writer, 174 Communication device, 175 Microphone, 176 speaker, 177 operation key, 179 backlight, 180 image processing engine, 181 driver control unit, 182 timer, 183 signal processing unit, 240 second optical sensor built-in liquid crystal panel (second display panel), 1731 memory card, DB1, DB2 Data bus.

Claims (7)

First and second display panels;
Display control means for displaying an object on the first display panel;
Accepting means for accepting designation of a first position in the first display panel via the first display panel;
First determination means for determining whether or not the first position is within a display area of the object;
Second determining means for determining whether or not the first position is within a first area preset in the first display panel;
The display control means determines that the first determination means determines that the first position is within the display area, and the second determination means determines that the first position is within the first area. An object display device for redisplaying the object on the second display panel when it is determined that the object is present. The accepting means accepts designation of a second position in the second display panel via the second display panel;
The first determination means determines whether the second position is within the display area of the object;
The second determination means determines whether or not the second position is within a second area preset in the second display panel;
The display control means determines that the first determination means determines that the second position is within the display area, and the second determination means determines that the second position is within the second area. The object display device according to claim 1, wherein when it is determined that the object is present, the object is displayed again on the first display panel. And further comprising third determination means for determining whether or not the object is displayed across the first display panel and the second display panel,
The display control means redisplays the object when the third determination means determines that the object is displayed across the first display panel and the second display panel. The object display device according to claim 1 or 2. When the first determination means determines that the first position is within the display area, and the second determination means determines that the first position is within the first area , Further comprising fourth determination means for determining whether or not the designation of the first position has been canceled,
4. The object according to claim 1, wherein the display control unit redisplays the object when the fourth determination unit determines that the designation of the first position has been canceled. 5. Display device. The first display panel includes:
A plurality of photosensor circuits that generate input signals in response to incident light;
A plurality of pixel circuits that emit light representing the object in response to an output signal;
The accepting means accepts designation of the first position based on the input signals from the plurality of photosensor circuits,
5. The object display device according to claim 1, wherein the display control unit displays the object on the first display panel by outputting the output signal to the plurality of pixel circuits. 6. . An object display method in an object display device having first and second display panels and an arithmetic processing unit,
The arithmetic processing unit accepting designation of a first position in the first display panel via the first display panel;
The arithmetic processing unit determining whether or not the first position is within a display area of the object;
The arithmetic processing unit determining whether or not the first position is within a first area preset in the first display panel;
When the arithmetic processing unit determines that the first position is within the display area and determines that the first position is within the first area, the second display panel Re-displaying the object. An object display program for displaying an object on an object display device having first and second display panels and an arithmetic processing unit,
In the arithmetic processing unit,
Receiving a designation of a first position in the first display panel via the first display panel;
Determining whether the first position is within a display area of the object;
Determining whether the first position is within a first area preset in the first display panel;
When it is determined that the first position is within the display area and the first position is determined to be within the first area, the object is displayed again on the second display panel. An object display program for executing steps.

Images