JP5247139B2 - Display device and method, program, and electronic apparatus - Google Patents

Description

  The present invention relates to a display device and method suitable for application to, for example, a liquid crystal display or an EL (electro-luminescence) display, a program, and an electronic device, and particularly to realize a highly reliable optical touch panel function. It relates to the technology that can.

  Conventionally, when a touch panel function, that is, a function that enables operation by touching a screen of a display device such as a television receiver is realized, a touch panel separate from the display device is superimposed on the screen. The configuration was adopted.

  As a configuration using a separate touch panel, for example, there is one in which a transparent and thin input detection device is pasted on a screen. As a means of detecting contact with the surface of a display, a touch panel using a resistive film type that senses changes in pressure and a capacitive type that senses electrical signals due to static electricity that changes due to contact with the human body is put into practical use. Has been. However, these touch panels can basically detect the position of only one point on the surface, and cannot simultaneously detect a plurality of points. Therefore, in an operation input device using a touch panel, a method of determining an instruction from a user based on the position of one point of contact or a method of determining an instruction based on a change in the position of one point has been common. .

Patent Document 1 accepts a movement input of a designated position in a two-dimensional direction on a display screen, determines the instruction content from the user based on the coordinate change of the designated position from the movement start to the movement end, There is a disclosure of an operation input reception device that performs a process according to the above.
JP 2004-336597 A

  In order to perform operations through the display in an easy-to-understand and efficient manner, the method of directly touching and instructing the display surface is effective, but the conventional touch panel can recognize only one point on the display screen. The operation method is limited, and there is a limit to the processing that can be executed on the screen.

  In order to solve the disadvantages of these conventional touch panels, a touch panel has been devised in which a panel is divided so that a plurality of points can be detected. However, such a touch panel detects each point within a limited range, and it is difficult to detect a plurality of points at free positions on the screen. In addition, a contact position detection device has been devised that can detect the position of a plurality of points by providing a plurality of infrared detection units. However, such an apparatus has a problem that the apparatus becomes large and complicated, such as the need to install a detection unit outside the screen.

  Under such circumstances, it is desired to realize a display device having a touch panel function of a method different from the conventional one and having high reliability.

  In view of such circumstances, an object of the present invention is to realize a display device having an optical touch panel function and having high reliability.

A display device according to an aspect of the present invention is a display device that displays an image and receives light, and includes a first light emitting unit that emits invisible light as the light for receiving light, and a visible light as the display for lighting. A second light emitting means for emitting light; a control means for executing in parallel the light emission control of the first light emitting means and the light emission control of the second light emitting means ; As the light emission control of the first light emitting means, a first period in which invisible light is emitted at the first light emission level and a second period in which invisible light is emitted at the second light emission level are respectively the same time. As a light emission control of the second light emitting means, a constant light emission level is obtained by dynamically emitting light at the timing of the change between the first light emission level and the second light emission level of invisible light. Control to emit visible light with .

  A display method and program according to one aspect of the present invention are a method and program corresponding to the above-described display device according to one aspect of the present invention.

In the display device, method, and program according to one aspect of the present invention, a display device that performs image display and light reception simultaneously or alternately, and that emits invisible light as the illumination for light reception. When, in a display device and a second light emitting means for emitting visible light as illumination for the display, and the emission control of the first light emitting means to emit invisible light in the first light emission level a first period, and a control to repeat a second time period to emit light at respective same time invisible light in the second light emission level, and the emission control of the second light emitting means, said non-visible light Control is performed in parallel to emit visible light at a constant light emission level by dynamically emitting light at the timing of change between the first light emission level and the second light emission level .

An electronic apparatus according to one aspect of the present invention is an electronic apparatus including a display unit that displays and receives an image. The display unit emits invisible light as the illumination for receiving light. If has a second light emitting means for emitting visible light as illumination for said display, said electronic device, and the emission control of the first light emitting means, the non-first illumination level and a first time period to emit visible light, and a control to repeat a second time period to emit light at respective same time invisible light in the second light emission level, a light emission control of the second light emitting means, A controller that executes in parallel the control of emitting visible light at a constant light emission level by dynamically emitting light at the timing of change between the first light emission level and the second light emission level of invisible light. Provide steps .

  According to the present invention, it is possible to realize a display device having an optical touch panel function and an electronic apparatus equipped with the display device. Further, a highly reliable display device and electronic device can be realized.

  Hereinafter, an I / O display will be described as an embodiment of the present invention.

  An I / O display is a display whose display surface has both a display function and an image sensor function (light receiving function). Since the I / O display has such a function, it can be applied to a wide range of fields including a touch panel function.

  FIG. 1 shows a configuration example of an I / O display panel.

  The I / O display panel 11 has a transparent display area (sensor area) 21 at the center thereof. A display horizontal (H) driver 22, a display vertical (V) driver 23, a sensor horizontal (H) receiver 25, and a sensor vertical (V) receiver 24 are arranged on the four end faces of the display area 21, respectively. Yes. The display H driver 22 and the display V driver 23 are supplied with a display signal and a control clock as display data, and drive display pixels arranged in a matrix in the display area 21. A reading clock is supplied to the sensor H receiver 25 and the sensor V receiver 24, and a light reception signal read in synchronization with the clock is output via a light reception signal line.

  2 and FIG. 3 are diagrams showing a configuration example of the pixel circuit arranged in the display area 21. FIG. As shown in FIG. 2, a pixel circuit 51R for red, a pixel circuit 51G for green, and a pixel circuit 51B for blue are arranged for each pixel. The red pixel circuit 51R, the green pixel circuit 51G, and the blue pixel circuit 51B are connected to the constant current sources 53R, 53G, and 53B, respectively, and to the readout circuit 52.

  The readout circuit 52 is a component of the light receiving drive circuit 62 shown in FIG. That is, the “light reception data” shown in FIG. 4 is output from the red pixel circuit 51R, the green pixel circuit 51G, and the blue pixel circuit 51B and provided to the readout circuit 52. Further, the constant current sources 53R, 53G, and 53B can be a component of the light receiving drive circuit 62 of FIG. 4 to be described later and a component of the sensor H receiver 25 of FIG.

  Here, hereinafter, when it is not necessary to individually distinguish the pixel circuits 51R, 51G, and 51B, these are collectively referred to as a pixel circuit 51. The pixel circuit 51 is configured as shown in FIG. 3, for example. That is, in the pixel circuit 51, the gate electrode 51h is arranged in the horizontal direction, the drain electrode 51i is arranged in the vertical direction, the switching element 51a is arranged at the intersection of both electrodes, and the switching element 51a and The pixel electrode 51b is connected. The switching element 51a is controlled to be turned on / off by a signal obtained via the gate electrode 51h, and a display state at the pixel electrode 51b is set by a signal supplied via the drain electrode 51i.

  A sensor 51C (light receiving element 51c) is disposed at a position adjacent to the pixel electrode 51b, and the power supply voltage VDD is supplied. A reset switch 51d and a capacitor 51e are connected to the sensor 51C. After the sensor 51C is reset by the reset switch 51d, a charge corresponding to the amount of light received by the sensor 51C is accumulated in the capacitor 51e. The voltage proportional to the accumulated charge is supplied to the signal output electrode 51j via the buffer amplifier 51f at the timing when the read switch 51g is turned on, and is output to the outside (the read circuit 52 in FIG. 2). The The on / off state of the reset switch 51d is controlled by a signal obtained from the reset electrode 51k. The on / off state of the read switch 51g is controlled by a signal obtained from the read control electrode 51m.

  That is, the following drive operation is performed on the entire screen. After the entire screen is reset and data of the previous imaging frame (accumulated charge of each sensor 51c) is erased, exposure is performed, and imaging data is obtained by reading the entire screen. By repeating such a driving operation, imaging data as a moving image is acquired. The above driving operation is substantially the same as a general CMOS image sensor. The entire screen is reset and read is controlled by various drivers 22 to 25 shown in FIG. As such a control method, for example, the same method as a method of line sequential driving or dot sequential driving used for display can be employed.

  FIG. 4 shows a configuration example of an I / O display system including the I / O display panel 11, that is, a system to which the present invention is applied.

  The I / O display system is configured to include a display drive circuit 61, a light receiving drive circuit 62, and an application program execution unit 63 in addition to the I / O display panel 11. An image processing unit 71 and a frame memory 72 are mounted on the light receiving drive circuit 62.

  Such an I / O display system can be configured as an application of an organic EL display, for example. In the case of an organic EL device, a current proportional to the intensity of light received is generated by applying a reverse bias to the pixel. This is because the imaging function can be performed in a time-sharing manner with the display function.

  The I / O display system can be configured as an application of a liquid crystal panel, for example. This is because the functions of both the display and the imaging can be realized by simultaneously creating the function of the imaging device in the TFT layer that drives the liquid crystal panel.

  Therefore, the I / O display system can realize an optical touch panel function based on display and imaging functions. By realizing such a touch panel function, the following first to third features can be provided.

  The first feature is a feature that enables input with a plurality of fingers.

  The second feature is a feature that the display quality is high, that is, the brightness and the definition are not lowered like a conventional touch panel film attached to the surface.

  The third feature is that a display device that allows gesture input from the surface can be realized.

  By the way, in the I / O display system, as a method of optically realizing the touch panel function, the display light (visible light) is mainly used for light reception (detection of fingers, etc.) when the surrounding brightness is dark. A method to do this is conceivable.

  However, this method causes the following first to third problems.

  The first problem is that the detected image differs significantly depending on the display content and cannot be detected on a black screen.

  The second problem is a method of adjusting the display screen in order to partially solve the first problem. However, when this method is applied, the image quality is deteriorated.

  The third problem is that detection is difficult when the display brightness is lowered in order to suppress glare in a dark place.

  In order to solve these first to third problems, it is preferable to use non-visible light for detection of a finger or the like in addition to visible light for display. However, when controlling invisible light, it is possible to discriminate between external light and reflectors, but solve the following fourth and fifth problems for visible light used simultaneously. There is a need to.

  The fourth problem is that when visible light is controlled at an arbitrary timing, the difference processing of invisible light may be affected.

  The fifth problem is that when the luminance of visible light is greatly changed at an arbitrary timing, the difference processing of invisible light may be affected.

  The invisible light difference process will be described later.

  Hereinafter, a method for solving these problems and realizing an arbitrary display luminance will be described with reference to FIGS.

  FIG. 5 shows a configuration example of a backlight for the I / O display panel 11. The configuration of FIG. 5 is a configuration that takes into account the liquid crystal display type I / O display 11. That is, the configuration example of the backlight is not limited to the example of FIG. 5. For example, when an organic EL system is adopted as the I / O display 11, each pixel may have a light source. Good.

  In the example of FIG. 5, the visible light from the visible light source 82 and the invisible light from the invisible light source 83 are applied to the I / O display panel 11 as a backlight via the light guide plate 81.

  In this case, the I / O display panel 11 performs display once and light reception (imaging) twice within a period of one frame. Note that light reception refers to a read operation of a light reception signal.

  Here, the light receiving operation of the I / O display system of FIG. 4 will be described with reference to FIG.

  The horizontal axis in FIG. 6 is the time axis. The vertical axis indicates the position of a scanning line (horizontal line) for displaying and receiving light. Here, the rewriting of the display signal and the reading of the received light signal are performed by changing the scanning line upward in order from the lowest line on one screen and finally scanning the uppermost line (first line). ing. FIG. 6 shows processing for one frame at an arbitrary frame position. Similar processing is repeatedly executed for subsequent frames.

  Here, one frame period is, for example, 1/60 seconds. As shown in FIG. 6, one frame period is divided into two equal parts, the first half and the second half. The first half is a period during which invisible light from the invisible light source 83 is turned off (hereinafter referred to as an invisible light OFF period). The second half is a period in which invisible light is turned on (hereinafter referred to as invisible light ON period). Light reception is performed in each of the invisible light OFF period and the invisible light ON period.

  Specifically, in the invisible light OFF period, the process RS1 for sequentially resetting the light reception signals of all the lines is performed in the first half, and the process RD1 for sequentially reading the light reception signals of all the lines is performed in the latter half. The received light signals accumulated by the respective sensors 51C for a certain period are read out. That is, in the invisible light OFF period, as the first light reception of the two light receptions, the invisible light by the invisible light source 83 is turned off, and as a result, only the external light is received. become. The light reception data DA obtained as a result of such light reception is temporarily stored in the frame memory 72 of the light reception drive circuit 62.

  Following the invisible light OFF period, in the invisible light ON period, processing RS2 for sequentially resetting light reception signals of all lines is performed in the first half, and processing RD2 for sequentially reading light reception signals of all lines in the second half thereof. And the received light signals accumulated by the respective sensors 51C for a certain period are read out. That is, in the invisible light ON period, the second light reception out of the two times of light reception is light reception based on the emission of invisible light by the invisible light source 83, and the surface of the display area 21 of the I / O display panel 11. Both the reflection of the finger etc. that touched and the external light are received.

  The difference data DC between the light reception data DB obtained as a result of the light reception and the light reception data DA temporarily stored in the frame memory 72 of the light reception drive circuit 62 is sent to the image processing unit 71. The reason for taking the difference is to remove the influence of external light.

  The process until the difference data DC is generated is the “invisible light difference process” mentioned in the fourth and fifth problems.

  That is, in the present embodiment, as shown in FIG. 6, the light emission control of the invisible light source 83 is performed based on the invisible light driving waveform in which the low level period TL and the high level period TH are repeated. That is, the light emission level (intensity) of invisible light is controlled to be low in the low level period TL, and the light emission level of invisible light is controlled to be high in the high level period TH. Here, when 0% is adopted as the low level and 100% or the like is adopted as the high level, the low level period TL is controlled to be the invisible light OFF period, and the high level period TH is the invisible light ON period. It will be controlled to become.

  As described above, in the present embodiment, the invisible light driving waveform is a waveform in which half the time of one period T is the low level period TL and the other half is the high level period TH. The low level period TL and the high level period TH need only be the same time, and the order within one cycle T is not particularly limited. In addition, in the example of FIG. 6, 1/60 seconds that is one frame period is adopted as one period T, but is not particularly limited as shown in FIG. In other words, the switching timing from one of the low-level period TL and the high-level period TH to the other may be arbitrary. For example, the timing can be synchronized with the vertical blanking period. It is also possible to set the timing.

  Thus, in the present embodiment, light emission control of the invisible light source 83 is performed for the purpose of appropriately controlling the light reception.

  In addition to the light emission control of the invisible light source 83, it is preferable to perform the light emission control of the visible light source 82 in parallel for the purpose of appropriately controlling the display brightness. This is because the display brightness can be arbitrarily controlled according to the ambient brightness and the user's desire.

For example, it is possible to adopt a control in which the luminance of visible light (display luminance) is lowered when it is dark so as not to give excessive stimulation to the eyes. In addition, since the surroundings are bright outdoors, it is possible to increase the display brightness and to improve the visibility. In addition, a system called a transflective liquid crystal, that is, a system that has a reflector inside the pixel and that can reduce the backlight power by using external light is used as the I / O display system. In this case, it is possible to employ a control for turning off the visible light source 82 when the external light is strong.

  In any case, the emission control of the visible light source 82 (hereinafter referred to as visible light control) has a completely different purpose from the emission control of the invisible light source 83 (hereinafter referred to as invisible light control). , Is to be done.

  Hereinafter, some specific examples of the visible light control executed in parallel with the invisible light control will be described with reference to FIGS.

  In the following example, an intensity in the range of 100% to 0% of the output of the visible light source 82 is adopted as the intensity (light emission level) of visible light, and the visible light control cycle is 1/60 second or 31.7 seconds. Adopt microseconds. However, it goes without saying that the numbers are not necessarily limited to these.

  Here, a case where a sensor designed not to react to visible light is employed as the sensor 51C incorporated in the I / O display 11 will be considered. As a design method for preventing reaction to visible light, it is necessary to detect the reflection of non-visible light without reacting to visible light. A design method for constructing a filter that transmits visible light can be employed. Alternatively, as a design method for preventing reaction to visible light, a method for designing the spectral sensitivity characteristic of the sensor 51 so as not to react to visible light can be employed.

  When the sensor 51C designed so as not to react to visible light is employed, all examples of visible light control described later with reference to FIGS. 7 to 19 can be employed.

  On the other hand, the sensor 51C can be designed so as not to react to visible light. However, when designed in this way, some of the examples of visible light control to be described later may not be adopted, or there may be points to be noted, and attention is required.

  The visible light control in the example of FIG. 7 uses a constant voltage (analog voltage) for the output (light emission level, here, luminance) of the visible light source 82 when 1/60 second is adopted as the period T of the invisible light control. ).

  In the example of FIG. 7, even when the sensor 51 </ b> C responds to visible light, the visible light source 82 is regarded as external light and is removed by non-visible light differential processing (see data DC in FIG. 6 described above). That is, even when the sensor 51C responds to visible light, the visible light control in the example of FIG. 7 can be realized.

  The visible light control in the example of FIG. 8 is performed at a constant voltage (analog voltage) and the output (luminance) of the visible light source 82 when 1/60 second is adopted as the period T of the non-visible light control. Is a control to change the time.

  When the sensor 51C does not respond to visible light, the visible light control in the example of FIG. 8 can be adopted without any problem.

  On the other hand, when the sensor 51C reacts to visible light, the luminance of the visible light source 82 cannot normally be changed at the speed shown in FIG. This is because the output of the visible light source 82 received by the sensor 51C during the invisible light OFF period is different from the output of the visible light source 82 received by the sensor 51C during the invisible light ON period. That is, in the invisible light difference process, detection is performed to remove the external light by taking the difference between the outputs of the sensor 51C during the two periods. At this time, if the luminance of the visible light source 82 is changed at the speed shown in FIG. 8, a difference occurs in the output by the visible light, and the difference is detected and adversely affected.

  However, if this difference is acceptable, a sensor 51C that responds to visible light can also be employed. An example of such a case is shown in FIG.

  That is, the visible light control in the example of FIG. 9 is also performed at a constant voltage (analog voltage) when 1/60 seconds is adopted as the period T of the invisible light control, and the output of the visible light source 82 ( (Luminance) is time-varying control. In this respect, there is no difference from the visible light control in the example of FIG.

  However, in the visible light control of the example of FIG. 9, the temporal change in the output (luminance) of the visible light source 82 is gentle compared to the visible light control of the example of FIG. 8. Thus, the influence on the sensor 51C can be reduced by gradually changing the output (luminance) of the visible light source 82 into a lamp shape. That is, the amount of change in the visible light source 82 can be reduced between the two pieces of image data acquired by the sensor 51C in the invisible light OFF period and the invisible light ON period. If the change amount of the visible light source 82 between the two image data is 1%, 1.01 ^ 60 = 1.816 per second, and 81% luminance change can be realized.

  In other words, when the sensor 51C responds to visible light and adopts the visible light control of the example of FIG. 7, when it is necessary to change the luminance of the visible light source 82, only at that time. 9 is switched to the visible light control in the example of FIG. 9, and it may be switched again to the visible light control in the example of FIG. 7 when the target luminance (output) is reached.

  That is, it can be understood that FIG. 9 shows an example of a method of changing the luminance of the visible light source 82. However, the method of changing the luminance of the visible light source 82 is not limited to the example shown in FIG. 9, and various methods can be employed.

  For example, in the example of FIG. 10, a method of changing the visible light driving waveform in a step shape is employed in order to instantaneously switch the luminance (output) of the visible light source 82. That is, in the example of FIG. 10, visible light control is executed so that the visible light drive waveform changes stepwise without crossing one cycle T that is a scan unit of the sensor 51C that performs a difference by blinking. . By adopting such a method, the influence on the sensor 51C can be avoided. That is, by adopting such a method, it is possible to execute visible light control so that the sensor 51C operates without any problem even if it reacts to visible light.

  On the other hand, in the example of FIG. 11, a technique is adopted in which the luminance change of visible light is suddenly changed into a lamp shape and the change is completed within one frame. When the sensor 51C does not respond to visible light, this method can be applied without any problem. On the other hand, when the sensor 51C reacts to visible light, the frame at the time of visible light change (in the example of FIG. 11, the frame within the period described as T) cannot be referred to. Here, if the viewpoint is changed, even if there is an effect on the sensor 51C, the effect is only 1 frame (1/60 seconds), so by not referring to this one sensor image, the entire I / O display system It can also be understood that the impact on the environment can be extremely reduced.

  In the above, some embodiments of visible light control using a constant voltage (analog voltage) have been described. Next, some embodiments of visible light control by PWM (pulse width modulation) will be described.

  Visible light control by PWM means the following control. That is, as the visible light driving waveform, a waveform in which a pulse is output every predetermined cycle (hereinafter referred to as PWM cycle) is employed. This pulse period, that is, a period in which the visible light drive waveform is at a high level, is a period in which the visible light source 82 emits light (a period in which the visible light emission level is 100% or the like). The other period is a period in which the visible light source 82 is turned off (a period in which the visible light emission level is 0). Therefore, the luminance control of visible light is realized by varying the time width of the pulse period. That is, visible light control by PWM is an embodiment of control for changing the light emission level (luminance) of visible light.

  The visible light control in the example of FIG. 12 is control performed by PWM when 1/60 seconds is adopted as the period T of the invisible light control. In the example of FIG. 12, the PWM cycle is arbitrary.

  When the sensor 51C responds to visible light, the visible light control in the example of FIG. 12 cannot be employed. The reason is as follows.

  That is, in the example of FIG. 12, the synchronization between the invisible light driving waveform and the visible light driving waveform is not considered, and the PWM cycle is arbitrary. Therefore, the output of the visible light source 82 received by the sensor 51C during the non-visible light OFF period may be different from the output of the visible light source 82 received by the sensor 51C during the non-visible light ON period. For this reason, as shown in FIG. 12, when the number of pulses in the invisible light OFF period is different from that in the invisible light ON period, the influence on the difference processing of the invisible light is increased.

  Further, as described above, the PWM pulse period is a period in which the visible light driving waveform is at a high level, that is, a period in which the visible light source 82 emits light. However, the relationship between the pulse period and the scan time of the sensor 51C of a predetermined pixel is not guaranteed at all. For this reason, when the sensors 51C are individually viewed in units of pixels, the influence on the difference processing of invisible light is increased. For example, in the example of FIG. 12, scanning of the sensor 51C of a predetermined same pixel is performed at time t1 and time t2. However, visible light having completely different intensities is incident on the sensor 51C at time t1 and time t2. As a result, the influence on the difference processing of invisible light is increased.

  On the other hand, when the sensor 51C does not respond to visible light at all, there is no influence on the difference processing of invisible light. Therefore, in such a case, the visible light control in the example of FIG. 12 can be employed.

  The visible light control in the examples of FIGS. 13 and 14 is a control performed by PWM when 1/60 seconds is adopted as the period T of the invisible light control. This is control for synchronizing the drive waveform. As the timing for synchronizing with the rising or falling edge of the invisible light driving waveform, the rising edge of the non-visible light control pulse may be adopted as shown in FIG. 13, or as shown in FIG. The falling edge of the light control pulse may be employed.

  Further, it is sufficient that the PWM frequency is an integral multiple of the examples of FIGS. Specifically, for example, visible light control as in the examples of FIGS. 15 and 16 may be employed.

  In addition, there is a possibility that an influence may appear at the time of switching the illumination depending on the structure of the I / O display panel 11, the sensitivity of the sensor 51C, the illumination used, and the like. In order to reduce this effect, it is possible to respond by changing the phase of PWM. For example, when the control is performed with the PWM waveform (phase) in the example of FIG. 14 and when the reading of the sensor 51C overlaps with the change of visible light, the influence becomes significant, FIG. It is better to change to the PWM waveform (phase) as in the example.

  By adopting such an example of FIGS. 13 to 16, that is, visible light control that synchronizes the invisible light driving waveform and the visible light driving waveform, even when viewed by the individual sensor 51C in pixel units, The light intensity can be the same in the invisible light ON period and the invisible light OFF period. Therefore, the influence of visible light can be eliminated by performing the difference processing of invisible light.

  This means that even when the sensor 51C responds to visible light, visible light control that synchronizes the visible light driving waveform and the visible light driving waveform can be employed without any problem in principle. The reason why there is no problem in principle is that the influence of visible light becomes the output offset of the sensor 51C, and therefore when the offset is different, the sensitivity is different (that is, the linearity of the sensitivity of the sensor 51C is poor). ) Etc., because it is necessary to consider them. This is because in this case, the sensitivity of the sensor 51C that has been struck strongly by visible light and the sensitivity of the sensor 51C that has been struck by weak light appear different. Therefore, for example, since the intensity of the visible light source 82 is known, it is necessary to take measures such as correcting the output of the sensor 51C based on the information.

  Note that even in the case of visible light control using PWM, it may be necessary to change the luminance of the visible light source 82, as in the case of using a constant voltage (analog voltage). The method of changing the luminance of the visible light source 82 at this time is not particularly limited, and various methods can be adopted.

  For example, in the example of FIG. 17, visible light control that synchronizes the invisible light driving waveform and the visible light driving waveform is employed, and therefore, in accordance with the unit of invisible light driving (unit of one cycle T). The method of changing the PWM drive ratio (pulse width) is adopted. By adopting such a method, the influence on the sensor 51C can be avoided. That is, by adopting such a method, visible light control is executed so that the sensor 51C operates without any problem even if it reacts to visible light.

  Also, the invisible light control executed in parallel with the various visible light controls described above is not particularly limited, and it is not particularly necessary to employ 1/60 seconds as in the above-described example. For example, the period T of invisible light control can be much shorter (higher frequency) than 1/60 second. Specifically, for example, as shown in FIGS. 18 and 19, 31.7 μs corresponding to the horizontal synchronization frequency of the I / O display panel 11 can be employed.

  18 and 19, the visible light control employs PWM control that synchronizes the invisible light driving waveform and the visible light driving waveform, and the PWM frequency is the invisible light driving frequency. An integer multiple of (= 1 / T) is employed. At this time, when the display on the I / O display panel 11 or the influence on the sensor 51C is seen, it can be dealt with by changing the phase of the PWM. Here, the method of changing the PWM phase itself is not particularly limited, and for example, the method described above with reference to FIGS. 13 and 14 can be employed as it is. Actually, there are many displays that control the backlight by PWM regardless of the display. By adopting such a display as the I / O display panel 11, the influence on the display is considered to be small. It is done.

  By adopting the I / O panel display system that can execute the visible light control and the invisible light control in parallel as described above, it is possible to cause the reflector that is in contact with the surface of the I / O display panel 11 Regardless of the situation, the shape can be reliably captured. In this case, the received light data received by the image processing unit 71 is data of an image of an object that has already been removed from outside light and touched the surface. Therefore, the image processing unit 71 can execute processing according to various applications.

  For example, if the image processing unit 71 has the same function (single point detection) as a conventional touch panel, after binarization, it can detect the maximum area and determine the presence or absence of finger contact. . After that, the image processing unit 71 can obtain the input coordinates by calculating the center of gravity of the maximum region.

  Furthermore, unlike the existing touch panel, the I / O display system can capture not only the coordinates but also the contact surface as an image, so that it can be used in the following first to third applications, for example.

The first application is an application of gesture recognition using a hand shape, that is, an operation of operation by a gesture. Applications that enable drawing, movement of objects, zoom-in, etc., that is, operations that switch commands on the personal computer (hand tool and zoom, etc.) in a more natural form, This is the first use.

  The second application refers to an application as a drawing panel that realizes even the touch of a brush, that is, an application that detects the contact of a brush by taking advantage of multipoint detection. In other words, some digitizers can be used as an alternative to conventional applications where special pen tilt is realized by electric field detection. By detecting the contact surface of a real writing brush, information can be input in a more realistic sense. The application that can perform the above is the second application.

  The third application refers to an application used as a panel having an acrylic cover or a designed cover on the surface. The third application makes it possible to attach a cover, which is impossible with a pressure-sensitive touch panel, by taking advantage of detection by an optical method. As a result, it is possible to achieve effects such as improvement of product design and easy development of a new product group having design on the cover itself.

  By employing the I / O display system described above, the following first to fourth effects can be achieved.

  The first effect is that the visible light control can be realized in parallel with the non-visible light control for the purpose of receiving light, so that the luminance adjustment of the I / O display panel 11 can be appropriately performed.

  The second effect is that the influence on the sensor 11C and the display at the time of brightness adjustment can be suppressed by adopting the above-described various controls as visible light control.

  The third effect is that the visible light control can be realized in parallel with the non-visible light control for the purpose of receiving light, so that the power consumption can be reduced by adjusting the visible light luminance.

  The fourth effect is that a gesture panel capable of detecting multiple points can be realized as described in the above application.

  By the way, the series of processes described above can be executed by hardware, but can also be executed by software.

  In this case, for example, a personal computer shown in FIG. 20 may be adopted as at least a part of the I / O display of FIG.

  In FIG. 20, a CPU (Central Processing Unit) 201 executes various processes according to a program recorded in a ROM (Read Only Memory) 202 or a program loaded from a storage unit 208 to a RAM (Random Access Memory) 203. To do. The RAM 203 also appropriately stores data necessary for the CPU 201 to execute various processes.

  The CPU 201, the ROM 202, and the RAM 203 are connected to each other via the bus 204. An input / output interface 205 is also connected to the bus 204.

  The input / output interface 205 includes an input unit 206 such as a keyboard and a mouse, an output unit 207 including a display, a storage unit 208 including a hard disk, and a communication unit 209 including a modem and a terminal adapter. It is connected. The communication unit 209 controls communication performed with other devices (not shown) via a network including the Internet.

  A drive 210 is also connected to the input / output interface 205 as necessary, and a removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately installed, and a computer program read from them is loaded. Installed in the storage unit 208 as necessary.

  When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, a general-purpose personal computer is installed from a network or a recording medium.

  As shown in FIG. 20, the recording medium including such a program includes a magnetic disk (including a floppy disk) on which the program is recorded, which is distributed to provide the program to the operator separately from the apparatus main body. ), Optical disk (including compact disk-read only memory (CD-ROM), DVD (digital versatile disk)), magneto-optical disk (including MD (mini-disk)), or semiconductor memory, etc. (package) Medium) 211, and a ROM 202 in which a program is recorded and a hard disk included in the storage unit 208 provided to the operator in a state of being incorporated in the apparatus main body in advance.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the order, but is not necessarily performed in chronological order, either in parallel or individually. The process to be executed is also included.

  Further, in the present specification, the system represents the entire apparatus including a plurality of apparatuses and processing units.

  Furthermore, the display device according to the present invention such as the I / O display described above has a flat panel shape, and various electronic devices such as a digital camera, a notebook personal computer, a mobile phone, and a video camera. It is possible to apply to the display of the electronic device of all fields which display the video signal inputted into the above or generated in the electronic device as an image or video. Examples of electronic devices to which such a display device is applied are shown below.

  For example, the present invention can be applied to a television receiver which is an example of an electronic device. This television receiver includes a video display screen including a front panel, a filter glass, and the like, and is manufactured by using the display device of the present invention for the video display screen.

  For example, the present invention can be applied to a digital camera which is an example of an electronic device. This digital camera includes an imaging lens, a display unit, a control switch, a menu switch, a shutter, and the like, and is manufactured by using the display device of the present invention for the display unit.

  For example, the present invention can be applied to a notebook personal computer. In this notebook personal computer, the main body includes a keyboard that is operated when characters and the like are input, and the main body cover includes a display unit that displays an image. This notebook personal computer is manufactured by using the display device of the present invention for the display portion.

  For example, the present invention can be applied to a mobile terminal device. This portable terminal device has an upper housing and a lower housing. As states of the portable terminal device, there are a state in which the two casings are opened and a state in which the two casings are closed. This portable terminal device includes a connecting portion (here, a hinge portion), a display, a sub-display, a picture light, a camera, and the like in addition to the above-described upper housing and lower housing. It is manufactured by using it for sub-displays.

  For example, the present invention can be applied to a so-called video camera. The video camera includes a main body, a lens for photographing a subject on a side facing forward, a start / stop switch at the time of photographing, a monitor, and the like, and is manufactured by using the display device of the present invention for the monitor 36.

It is a figure which shows the structural example of the I / O display panel to which this invention is applied. FIG. 2 is a diagram illustrating a configuration example of a pixel circuit arranged in the I / O display panel of FIG. 1. FIG. 2 is a diagram illustrating a configuration example of a pixel circuit arranged in the I / O display panel of FIG. 1. It is a functional block diagram which shows the functional structural example of the I / O display to which this invention is applied. 2 shows a configuration example of a backlight for the I / O display panel of FIG. 1. FIG. 5 is a diagram for explaining a light receiving operation of the I / O display of FIG. 4. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a figure explaining an example of the drive method of the visible light source of FIG. It is a block diagram which shows the structural example of the computer which performs the process with which this invention is applied with software.

Explanation of symbols

  11 I / O display panel, 21 display area (sensor area), 22 display H driver, 23 display V driver, 24 sensor V receiver, 25 sensor H receiver, 51 pixel circuit, 51C sensor, 52 readout circuit, 53 Constant current source, 61 display drive circuit, 62 light receiving drive circuit, 63 application program execution unit, 71 image processing unit, 72 frame memory, 81 light guide plate, 82 visible light source, 83 invisible light source, 201 CPU, 202 ROM, 203 RAM, 204 bus, 205 input / output interface, 206 input unit, 207 output unit, 208 storage unit, 209 communication unit, 210 drive, 211 removable media

Claims (4)

In a display device that performs image display and light reception,
First light emitting means for emitting invisible light as the light receiving illumination;
A second light emitting means for emitting visible light as the display illumination;
Control means for executing in parallel the light emission control of the first light emission means and the light emission control of the second light emission means ,
As the light emission control of the first light emitting means, the control means has a first period for emitting invisible light at a first light emission level and a second period for emitting invisible light at a second light emission level. Are repeated at the same time, and as the light emission control of the second light emitting means, light is dynamically emitted at the timing of change between the first light emission level and the second light emission level of invisible light. A display device that performs control to emit visible light at a constant light emission level .   A display device that displays and receives an image,
  First light emitting means for emitting invisible light as the light receiving illumination;
  A second light emitting means for emitting visible light as the display illumination;
  In a display method of a display device comprising:
  As the light emission control of the first light emitting means, a first period in which invisible light is emitted at the first light emission level and a second period in which invisible light is emitted at the second light emission level are the same time. As the light emission control of the second light emitting means and the light emission control of the second light emitting means, a constant light emission level is obtained by dynamically emitting light at the timing of change between the first light emission level and the second light emission level of invisible light. Control to emit visible light in parallel
  A display method including steps.   A display device that displays and receives an image,
  First light emitting means for emitting invisible light as the light receiving illumination;
  A second light emitting means for emitting visible light as the display illumination;
  A computer for controlling a display device comprising:
  As the light emission control of the first light emitting means, a first period in which invisible light is emitted at the first light emission level and a second period in which invisible light is emitted at the second light emission level are the same time. As the light emission control of the second light emitting means and the light emission control of the second light emitting means, a constant light emission level is obtained by dynamically emitting light at the timing of change between the first light emission level and the second light emission level of invisible light. Control to emit visible light in parallel
  A program that executes a step.   In an electronic device comprising display means for performing image display and light reception,
  The display means includes
    First light emitting means for emitting invisible light as the light receiving illumination;
    A second light emitting means for emitting visible light as the display illumination;
  Have
  The electronic device is
    As the light emission control of the first light emitting means, a first period in which invisible light is emitted at the first light emission level and a second period in which invisible light is emitted at the second light emission level are the same time. As the light emission control of the second light emitting means and the light emission control of the second light emitting means, a constant light emission level is obtained by dynamically emitting light at the timing of change between the first light emission level and the second light emission level of invisible light. Means for executing in parallel the control for emitting visible light with
  Electronic equipment comprising.

Images