JP2004240501A - Display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display device capable of detecting position instruction to a display face without using a special instructed object or special display face. <P>SOLUTION: The display device is provided with a transmissive screen, a projecting means for projecting the projected lights of a video to the back face of the transmissive screen, a shielding means for shielding the projected lights from the projecting means in a predetermined timing and a photographing means for photographing the back face of the transmissive screen. Then, an image photographed by the photographing means is acquired in the timing in which the projected lights are shield by the shielding means. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a display device, and more particularly to a display device that detects a position pointed by a pointing object on a display screen.
[0002]
[Prior art]
In an information terminal or a large-screen display device for a conference, etc., the pointing position of the pointing object with respect to the display screen is detected in order to operate the image displayed on the display screen by touching the display screen with a pointing object such as a finger or a pen. Techniques have been proposed in the past.
[0003]
For example, in the invention described in Patent Literature 1, a surface acoustic wave is generated on a display surface, and the pointing position of the pointing object is detected by attenuating the surface acoustic wave due to the contact of the pointing object. Further, in the invention described in Patent Document 2, a transparent resistive film is stuck on the display surface, and the pointing position of the pointing object is detected from a change in resistance due to the contact of the pointing object. In addition, a type in which a transparent electrode wire is arranged on a display surface and a designated position is detected from a change in capacitance caused by contact with a conductive object such as a finger is commercially available.
[0004]
However, the inventions described in Patent Literature 1 and Patent Literature 2 require a special display screen. In general, the display screen frequently breaks due to frequent contact with the pointing object for operation, and therefore, the detection function for detecting the position of the pointing object is easily damaged. Therefore, a display device with a coordinate position detecting function using a rear projection type projector and a camera without using a special display screen for detecting a designated position has been proposed (for example, Patent Document 3, Patent Document 4, Patent Document 4). Reference 5).
[0005]
[Patent Document 1]
JP-A-7-160405
[Patent Document 2]
JP-A-11-353101
[Patent Document 3]
JP-A-5-233148
[Patent Document 4]
JP 2000-148375 A
[Patent Document 5]
JP-A-2002-32193
[Problems to be solved by the invention]
However, in the invention described in Patent Document 3, since a special pen is used, the operation becomes complicated. Further, the inventions described in Patent Literature 4 and Patent Literature 5 have a problem that the pointing position of the pointing object cannot be detected properly due to the influence of the projection light of the projector.
[0011]
The present invention has been made in view of the above points, and has as its object to provide a display device that can detect a position indication on a display surface without using a special pointing object or a special display surface.
[0012]
[Means for Solving the Problems]
Therefore, in order to solve the above-described problem, the present invention provides a transmission screen, a projection unit that projects projection light of an image on a rear surface of the transmission screen, and the projection unit. A light-shielding unit that shields the projection light at a predetermined timing, and a photographing unit that photographs the rear surface of the transmissive screen, and an image photographed by the photographing unit at a timing when the light-shielding unit shields the projection light. It is characterized by obtaining.
[0013]
In such a display device, it is necessary to use a special pointing object or a display screen because the pointing position of the pointing object is detected based on the image captured by the imaging unit when the light shielding unit shields the light projected by the projection unit. Absent.
[0014]
Further, according to the present invention, as described in claim 2, the light-shielding unit intermittently shields the projection light, and a ratio of a time for shielding the projection light to a time for not shielding the projection light is a predetermined value. It is characterized by becoming.
[0015]
In such a display device, since the projection light can be shielded at a timing when the user does not feel uncomfortable, the uncomfortable feeling of the user with respect to the image can be reduced.
[0016]
Further, according to the present invention, as set forth in claim 3, the light shielding means sets a cycle of shielding the projection light from 30 Hz to 120 Hz.
[0017]
With such a display device, it is possible to reduce the user's discomfort with the video and reduce the detection error of the indicated position.
[0018]
According to another aspect of the present invention, a pointing position detecting unit that detects a pointing position of a pointing object with respect to the transmissive screen based on an image captured by the capturing unit, Pointing position detection inhibiting means for inhibiting the detection of the pointing position by means.
[0019]
In such a display device, since the detection of the pointing position of the pointing object by the pointing position detecting means can be prohibited, it is possible to prevent the touch of the pen or the like that writes on the screen with the screen from being recognized as the position pointing. be able to.
[0020]
Further, according to the present invention, as described in claim 5, the pointing position detecting means may be configured such that the pointing means detects an initial image of the rear surface of the screen in advance, and the light shielding means shields the projection light. Preferably, the pointing position is detected based on an image of the rear surface of the screen captured by the image capturing unit.
[0021]
In such a display device, an image that is not information based on the instruction of the pointing object, such as a scratch, dirt, etc. previously attached to the transmissive screen can be excluded, and only information based on the instruction of the pointing object can be extracted. The detection accuracy of the designated position can be improved.
[0022]
Further, according to the present invention, as set forth in claim 6, the designated position detecting means sets, as the designated position, an area in the image where a luminance value is equal to or less than a predetermined value.
[0023]
In such a display device, it is possible to distinguish between writing on the transmissive screen with a pen or the like and pointing at the position by the pointing object, so that the detection accuracy of the pointing position by the pointing object can be improved.
[0024]
Further, according to the present invention, as set forth in claim 7, the pointing position detecting means, based on the plurality of images photographed by the photographing means each time the light shielding means shields the projection light, The pointing position is detected.
[0025]
In such a display device, it is possible to distinguish between writing on the transmissive screen with a pen or the like and pointing at the position by the pointing object, so that the detection accuracy of the pointing position by the pointing object can be improved.
[0026]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an external view of a display device according to an embodiment of the present invention. The display device 10 projects an image generated by a personal computer on a transmission type screen 1. In the display device 10, a position indication operation performed by a mouse in a normal personal computer can be performed by touching the screen 1 with a pointing object such as a finger or a pen.
[0027]
FIG. 2 is a diagram illustrating an internal structure of the display device according to the first embodiment of the present invention. The display device 10 shown in FIG. 2 includes a projector 3, a CCD camera 4, and a personal computer 5 arranged inside a housing 2. The display image generated by the personal computer 5 is projected by the projector 3 via the mirror 6 and projected on the screen 1. The CCD camera 4 captures an image of the screen 1 from behind, and the image is transmitted to the personal computer 5. The personal computer 5 detects the designated position of the designated object from the image transmitted from the CCD camera 4, and executes a predetermined process according to the designated position. Further, a light-shielding plate 7 that can be put in and out so as to block the front part of the light-projecting lens of the projector 3 is arranged, and the solenoid 8 puts the light-shielding plate 7 in and out. The control of the solenoid 8 is performed by the personal computer 5 using the driver 9.
[0028]
FIG. 3 is a diagram for explaining the operation of the light shielding plate. FIG. 3A shows a state where the light shielding plate 7 does not block the projection light of the projector 3, and FIG. 3B shows a state where the light shielding plate 7 blocks the projection light of the projector 3. Is shown. As described above, the display device 1 can intermittently shield the projection light from the projector 3 onto the screen 1 without turning off the power of the light source of the projector 3 by the light shielding plate 7. The operation of the display device 1 when a position support operation is performed on the screen with a pointing object such as a finger or a pen will be described. 4 and 5 are flowcharts for explaining the operation of the display device. It is assumed that an image of a screen of an application having a GUI (Graphical User Interface) component is projected on the screen 1 from the projector 3.
[0029]
In step S1, when the personal computer 5 outputs a signal to the driver 9 to block the projection light of the projector 3, the solenoid 8 sets the light shielding plate 7 to the state shown in FIG. To shield light. In this state, no display image is displayed on the screen 1.
[0030]
Proceeding to step S2 following step S1, when the personal computer 5 instructs the CCD camera 4 to take an image of the screen 1, the CCD camera 4 moves the back side of the screen 1 on which no image is displayed. Shoot. Here, when the pointing object is pointing at the screen 1, a shot image as shown in FIG. 6 is obtained by shooting.
[0031]
FIG. 6 is a diagram illustrating a captured image when the pointing object is pointing on the screen. In the photographed image 101 of FIG. 6, the shadow of the pointing object is identified as black around the pointing object. As described above, when the screen 1 is photographed, the display image from the projector 3 is not projected on the screen 1, so that an image that clearly identifies the area indicating the designated position is photographed.
[0032]
Proceeding to step S3 following step S2, when the personal computer 5 outputs to the driver 9 a signal indicating that the shielding of the projection light of the projector 3 should be released, the solenoid 8 moves the light shielding plate 7 to the position shown in FIG. Then, the shielding of the projection light of the projector 3 is released. As a result, a display image from the projector 3 is displayed on the screen 1.
[0033]
Proceeding to step S4 following step S3, the personal computer 5 acquires a captured image from the CCD camera 4. Proceeding to step S5 following step S4, the personal computer 5 determines whether or not the pointing object points to the screen 1 based on the captured image 101, and specifies the pointing position when pointing. The details of the processing in this step will be described later.
[0034]
If the pointing object cannot be detected in step S5, the process proceeds to step S1, and the processing from step S1 is repeated again. If the pointing object has been detected, the process proceeds to step S7.
[0035]
In step S7, the personal computer 5 searches for a GUI component that controls an application, such as a scroll button or a radio button, within a predetermined range from the designated position, and determines whether the displayed position overlaps the designated position. If there is a GUI component (hereinafter, referred to as an “operation target component”), identification information for specifying the operation target component is held.
[0036]
Proceeding to step S8 following step S7, the personal computer 5 determines whether or not the operation target component exists. If the operation target component does not exist, the process returns to step S1, and if it does, the process proceeds to step S9.
[0037]
In step S9, the personal computer 5 starts a timer. This timer is used to determine whether the instruction by the pointing object is a single click or a double click, as described later. That is, at the time of step S9, since the instruction by the pointing object has been detected once, it is necessary to specify whether the operation by the pointing object is single-click or double-click, and the timer is started for a predetermined time. If the position indication of the pointing object can be detected again within this time, it is determined that a double click has been performed.
[0038]
After step S9, the process proceeds to step S10 in FIG. 5, and the display device 1 repeats the same processing as steps S1 to S5 again. That is, an image of the screen 1 is photographed in a state where the projection light from the projector 3 is shielded, and the pointing position of the pointing object with respect to the screen 1 is detected based on the photographed image 101.
[0039]
Proceeding to step S11 following step S10, the personal computer 5 determines whether or not the current time is within a predetermined time from the start of the timer. If the predetermined time has elapsed, the process proceeds to step S12, and the personal computer 5 generates a single click event for the operation target component. Therefore, the operation target component performs an operation according to the single click event.
[0040]
If it is within the predetermined time, the process proceeds to step S13, and the personal computer 5 determines whether or not the pointing of the pointing object has been detected by the process of step S10. If the pointing of the pointing object has not been detected, the process proceeds to step S14, the flag whose initial value is OFF is turned on, and the processing from step S10 is performed again. If the pointing object has been detected, the process proceeds to step S15, and the personal computer determines whether flag is ON. If the flag is not ON, the process returns to step S10, and if the flag is ON, the process proceeds to step S16.
[0041]
Here, the flag is a flag variable for detecting a double click. The double-click on the screen 1 is to quickly perform an operation of once pointing the screen 1 with the pointing object, separating the pointing object from the screen 1 and pointing again. Therefore, in order to establish a double click, it is necessary to detect once that the pointing object has left the screen 1 before detecting the position indication of the pointing object again. Therefore, when it is detected that the pointing object has left, that is, when the pointing object has not been detected, the flag is set to ON (S14), and it is determined in the subsequent processing that the pointing object has already left the screen once. To be able to do it.
[0042]
In step S16, the personal computer 5 determines whether the current pointing position of the pointing object detected in step S10 is within a predetermined range from the previous pointing position, that is, the pointing position detected in step S5. If it is within the predetermined range, the process proceeds to step S17, and the personal computer 5 generates a double-click event for the operation target component. Therefore, the operation target component performs an operation according to the double-click event.
[0043]
If the current designated position is not within the predetermined range from the previous designated position, the personal computer 5 executes the processing after step S7 assuming that the operation is completely unrelated to the previous designated position.
[0044]
After generating the single click event in step S12 or the double click event in step S17, the personal computer 5 resets the timer and repeats the processing from step S1.
[0045]
Through the above-described processing, the user can perform an operation such as a single click or a double click on the GUI component of the application displayed on the screen 1 by pointing to the position of the pointing object on the screen 1. Other operations such as drag and drop can also be performed on the same principle.
[0046]
Next, details of the process of detecting the designated position of the designated object from the captured image 101 in step S5 will be described. FIG. 7 is a flowchart for explaining the detection processing of the pointing position of the pointing object. First, the computer 5 compares pixels at the same position between the captured image 101 and the initial image, and extracts from the captured image 101 pixels having a difference in luminance value equal to or greater than a predetermined threshold (S31). In FIG. 6, the pixels in the region A correspond to the pixels. Note that, here, the initial image is an image obtained by the CCD camera 4 capturing the screen 1 in advance in a state where no image is projected on the screen 1 and the pointing object does not point at the screen 1. By performing comparison with the initial image, an image that is not information based on the instruction of the pointing object, such as a scratch or a stain attached to the screen 1 in advance, is excluded, and only information based on the instruction of the pointing object can be extracted. The detection accuracy of the pointing position can be improved.
[0047]
The computer 5 determines whether or not the pointing of the pointing object has been detected by determining whether or not the number of pixels in the area A is equal to or smaller than a predetermined value (S32). That is, if the number of pixels in the area A exceeds the predetermined value, it is determined that the pointing of the pointing object has been detected, and if it is less than the predetermined value, it is determined that the pointing of the pointing object has not been detected.
[0048]
When the pointing of the pointing object is detected, the computer 5 calculates the center of gravity of the area A, and sets the position of the center of gravity as the pointing position of the pointing object (S33).
[0049]
As described above, it is possible to detect whether or not the pointing object is pointing, and if so, the pointing position.
[0050]
As described above, according to the display device 10 in the first embodiment, it is necessary to use a special pointing object or a display screen to detect the pointing position of the pointing object based on the image captured by the CCD camera 4. Absent. Further, since the designated position is determined based on an image captured in a state where the projection light from the projector 3 is shielded, the detection accuracy of the designated position can be improved.
[0051]
Next, as a second embodiment, an example in which the light shielding plate 7 and the solenoid 8 in the first embodiment are replaced with a light shielding plate 11 and a motor 12 shown in FIG. 8 will be described. FIG. 8 is a diagram illustrating a light shielding plate according to the second embodiment. The light-shielding plate 11 in FIG. 8 has two wings such that the winged portion and the non-winged portion are alternately spaced at equal intervals. The light-shielding plate 11 intermittently shields the projection light of the projector 3 with two wings by rotating at a constant speed around the rotation axis by driving the motor 12. Since the light-shielding plate 11 rotates at a constant speed and the winged portion and the wingless portion are at equal intervals, the time for shading the projection light from the projector and the time for not shading, ie, the image The time for projecting and the time for not projecting can be switched at substantially equal intervals.
[0052]
An experiment was conducted to determine how long this switching interval (light-shielding period) should be. First, 100 subjects answered when the image displayed on the screen 1 in the process of increasing the rotation speed of the light-shielding plate 11 was free from flicker and other discomfort. FIG. 9 is a diagram illustrating a relationship between the number of rotations of the light-shielding plate and an uncomfortable feeling of an image obtained by an experiment. In the graph 200 of FIG. 9, the horizontal axis represents the number of rotations of the light shielding plate 11 (rotation / second), and the vertical axis represents the number of people who have no discomfort in the image. It can be seen from the graph 200 that the number of persons who did not feel uncomfortable at 15 rotations / second increased rapidly.
[0053]
FIG. 10 is a diagram illustrating a relationship between the rotation speed of the light shielding plate and the detection error rate of the pointing position of the pointing object in a normal use environment. In the graph 300 of FIG. 10, the horizontal axis represents the number of rotations (rotation / second) of the light shielding plate 11, and the vertical axis represents the ratio (detection error rate) (%) at which the designated position could not be detected. It can be seen from FIG. 10 that the detection error rate sharply increases when the number of rotations of the light shielding plate 11 is 60 rotations / second.
[0054]
Therefore, it can be seen from the experimental results of FIGS. 9 and 10 that the desirable rotation speed of the light-shielding plate 11 in applying the present invention is not less than 15 rotations / sec and not more than 60 rotations. That is, in terms of the switching cycle between light projection and light shielding, it can be said that the frequency is preferably 30 Hz or more and 120 Hz or less because the light shielding plate 11 switches between light shielding and light projection twice in one rotation.
[0055]
Next, a third embodiment of the present invention will be described. FIG. 11 is a diagram showing the internal structure of the display device according to the third embodiment of the present invention. 11, the same parts as those of FIG. 2 are denoted by the same reference numerals, and the description thereof will be omitted. In the display device 10 of FIG. 11, a writing surface 21 is arranged in close contact with the surface of the screen 1, and a changeover switch 22 is provided.
[0056]
The writing surface 21 may be formed using a material that is sufficiently transparent to transmit the image projected on the screen 1 to the user side, for example, acrylic. Thereby, the user can see the image projected on the screen 1 from the projector 3 via the writing surface 1. The writing surface 21 is subjected to a process of easily wiping the attached ink with a cloth or the like, for example, a fluororesin process, similarly to a general whiteboard. Therefore, the user can write on the writing surface 21 with a dry marker or the like.
[0057]
However, when writing on the writing surface 21 is permitted, if the process of detecting the pointing position of the pointing object in the embodiment of the first embodiment is applied as it is, the writing on the writing surface 21 may be erroneously recognized as a shadow caused by the pointing object. There is. Therefore, when the changeover switch 22 is OFF, detection of the indicated position is prohibited, that is, the detection process in the computer 5 is stopped. When the changeover switch 22 is turned on to detect the pointing position of the pointing object, there is a possibility that writing has been performed while the changeover switch is off, so the image immediately after the changeover switch 22 was turned on is displayed. If the initial image described in the embodiment of the first embodiment is used, it is possible to perform the detection processing of the designated position ignoring the writing.
[0058]
As described above, according to the display device 10 in the third embodiment, when writing to the writing surface 21 is desired, the detection process of the designated position can be prohibited by the switch 22. It is possible to prevent the computer 5 from erroneously recognizing the writing to 21 as an instruction by the pointing object.
[0059]
Note that the processing for detecting the pointing position of the pointing object in FIG. 7 may be replaced with the processing shown in FIG. 12 or FIG. Hereinafter, these will be described in order. FIG. 12 is a second flowchart for explaining the detection processing of the pointing position of the pointing object. When the processing in FIG. 12 is performed, the computer 5 always holds a recent photographed image (for example, 100 images) photographed during a predetermined time in the past (usually about 3 seconds).
[0060]
First, the computer 5 compares the pixels at the same position between the plurality of held captured images and the captured image 101 newly captured in step S2 in FIG. Then, a pixel whose luminance value decreases with a difference equal to or more than the predetermined threshold value and whose luminance value increases again with a difference equal to or more than the predetermined threshold value is detected (S41).
[0061]
If the change in the luminance value described above also occurs in adjacent pixels within a predetermined range, the computer 5 extracts the area (corresponding to the area A in FIG. 6) (S42, S43). If the region A cannot be extracted, it is assumed that the pointing of the pointing object has not been detected.
[0062]
When the pointing of the pointing object is detected, the computer 5 calculates the position of the center of gravity of the area A, and sets the position of the center of gravity of the area A as the pointing position of the pointing object.
[0063]
This method is more effective than the method in FIG. 4 particularly when writing is performed on the writing surface 21. That is, even when writing is performed on the writing surface 21, the write position can be detected by removing the write information by comparing a plurality of images photographed almost at the same time. This can prevent erroneous recognition that there is.
[0064]
FIG. 13 is a third flowchart illustrating the detection processing of the pointing position of the pointing object. First, the computer 5 compares pixels at the same position between the captured image 101 and the initial image, and determines that the difference between the luminance values is equal to or greater than a predetermined threshold value and the density is higher than or equal to a predetermined value (the luminance value is equal to or greater than the predetermined value) Hereinafter, pixels are extracted from the captured image 101 (S51). In FIG. 6, the pixels in the region A correspond to the pixels. Here, the initial image has the same meaning as that described in FIG.
[0065]
The computer 5 determines whether or not the pointing of the pointing object has been detected by determining whether or not the number of pixels in the area A is equal to or smaller than a predetermined value (S52). That is, if the number of pixels in the area A exceeds the predetermined value, it is determined that the pointing of the pointing object has been detected, and if it is less than the predetermined value, it is determined that the pointing of the pointing object has not been detected.
[0066]
If the pointing of the pointing object is detected, the computer 5 calculates the position of the center of gravity of the area A, and sets the position of the center of gravity as the pointing position of the pointing object (S53).
[0067]
This method is particularly effective when writing is performed on the writing surface 21 as compared with the method in FIG. That is, since the instruction by the pointing object tends to be detected at a higher density than the writing on the writing surface 21, the writing detected in the comparison with the initial image is compared with a predetermined density threshold. Therefore, it is possible to distinguish between the instruction by the pointing object and the writing.
[0068]
Although the preferred embodiments of the present invention have been described in detail, the present invention is not limited to the specific embodiments, and various modifications may be made within the scope of the present invention described in the appended claims.・ Change is possible.
[0069]
【The invention's effect】
As described above, according to the present invention, it is possible to detect a position indication on a display surface without using a special pointing object or a special display surface.
[Brief description of the drawings]
FIG. 1 is an external view of a display device according to an embodiment of the present invention.
FIG. 2 is a diagram showing an internal structure of the display device according to the first embodiment of the present invention.
FIG. 3 is a diagram for explaining the operation of a light shielding plate.
FIG. 4 is a flowchart illustrating the operation of the display device.
FIG. 5 is a flowchart illustrating an operation of the display device.
FIG. 6 is a diagram illustrating a captured image when a pointing object is pointing at a screen.
FIG. 7 is a flowchart illustrating a detection process of a pointing position of a pointing object.
FIG. 8 is a diagram illustrating a light shielding plate according to a second embodiment.
FIG. 9 is a diagram illustrating a relationship between the number of rotations of a light-shielding plate and an uncomfortable feeling of an image obtained by an experiment.
FIG. 10 is a diagram illustrating a relationship between a rotation speed of a light shielding plate and a detection error rate of a pointing position of a pointing object in a normal use environment.
FIG. 11 is a diagram illustrating an internal structure of a display device according to a third embodiment of the present invention.
FIG. 12 is a second flowchart illustrating a process of detecting a pointing position of a pointing object.
FIG. 13 is a third flowchart illustrating the detection processing of the pointing position of the pointing object.
[Explanation of symbols]
Reference Signs List 1 screen 2 housing 3 projector 4 CCD camera 5 personal computer 6 mirror 7, 11 light shielding plate 8 solenoid 9 driver 10 display device 12 motor 21 writing surface 22 changeover switch 101 photographed image

Claims (7)

A transmissive screen,
Projection means for projecting projection light of an image on the back of the transmission screen,
Light shielding means for shielding the projection light by the projection means at a predetermined timing,
Photographing means for photographing the rear surface of the transmission screen,
A display device, wherein an image captured by the image capturing unit is obtained at a timing when the light shielding unit shields the projection light. 2. The display device according to claim 1, wherein the light-shielding unit intermittently shields the projection light so that a ratio of a time during which the projection light is shielded to a time during which light is not shielded has a predetermined value. 3. . The display device according to claim 2, wherein the light shielding unit sets a cycle of shielding the projection light from 30 Hz to 120 Hz. Pointed position detecting means for detecting a pointed position of a pointing object with respect to the transmissive screen based on an image photographed by the photographing means,
The display device according to claim 1, further comprising a designated position detection prohibiting unit that prohibits the designated position detecting unit from detecting the designated position. The pointing position detecting means is based on an initial image in which the photographing means has photographed the rear surface of the screen in advance, and an image in which the photographing means has photographed the rear surface of the screen when the light shielding means has shielded the projection light. The display device according to claim 4, wherein the pointing position is detected. The display device according to claim 4, wherein the designated position detecting unit sets an area in the image where a luminance value is equal to or less than a predetermined value as the designated position. 5. The display according to claim 4, wherein the designated position detecting unit detects the designated position based on a plurality of the images photographed by the photographing unit each time the light blocking unit blocks the projection light. 6. apparatus.

Images