WO2016031038A1

WO2016031038A1 - Video display system and projection-type video display device

Info

Publication number: WO2016031038A1
Application number: PCT/JP2014/072696
Authority: WO
Inventors: 俊彦松▲沢▼; 公舟市川; 信二小野寺; 宣孝堀田
Original assignee: 日立マクセル株式会社
Priority date: 2014-08-29
Filing date: 2014-08-29
Publication date: 2016-03-03

Abstract

A video display system including a projection-type video display device and an information processing device, wherein the projection-type video display device has: a sensor capable of imaging light emitted or reflected by a manipulation object that is in contact with a projection surface; and a first interactive function unit which determines the position of the manipulation object on the basis of an image imaged by the sensor and controls the display of a video in accordance with the position. The information processing device has a second interactive function unit which calculates the position of the manipulation object on the basis of the image imaged by the sensor and received from the projection-type video display device and which is capable of controlling the display of the video in the projection-type video display device in accordance with the position. The projection-type video display device displays a menu window for selecting whether an interactive function relating to the displayed video is to be operated in a first operation mode by the first interactive function unit of the projection-type video display device or in a second operation mode by the second interactive function unit of the information processing device.

Description

Image display system and projection image display device

The present invention relates to a projection-type image display device having an interactive function and a technology related thereto.

In a projection-type image display device that projects an image on a screen or the like, an apparatus having an interactive function has been put into practical use in order to perform a user's presentation comfortably. Thereby, the user can draw a character or a figure on the display image switching operation or the screen using a dedicated pen or finger. At that time, various methods have been proposed to detect the position of a dedicated pen or finger, which is an operation object with respect to the display screen. For example, Patent Document 1 discloses a method of scanning infrared laser light in a plane parallel to the image projection plane and detecting light returning to the operation article. The position of the operation article in the scanning plane is obtained from the time difference between the emission and detection of the laser beam.

JP 2009-2558569 A

In the method of detecting the return light from the operation article by irradiating the laser beam in order to detect the position of the operation article as in Patent Document 1, the obstacle other than the operation article (dedicated pen, finger, etc.) in the detection region If an object is present, the return light reflected by the object is detected, causing malfunction. For example, when a whiteboard is used as a projection surface, unnecessary reflected light (abnormal reflection) is generated if there are protrusions on the whiteboard surface. Therefore, it is necessary to accurately notify the user whether or not there is abnormal reflection, and if so, the location thereof and cope with abnormal reflection.

Furthermore, it is also possible to control the projection video display device from the information processing device side by using the projection video display device as a video display system connected to an information processing device such as a personal computer. However, the interactive function described above is configured to be performed on the projection type video display device side of the main body. For example, calibration data relating to the detection position of the operation article and information relating to the obstacle cannot be handled on the information processing apparatus side, which hinders efficient control.

As described above, the projection type image display device having the conventional interactive function has a problem that the preparation for installation or use is not sufficient.

An object of the present invention is to make it possible to more suitably prepare for installation or use in a projection type video display device and video display system having an interactive function.

The present invention relates to, for example, a video display system including a projection video display device that displays video and an information processing device. The projection video display device projects a display video onto a projection surface, and a projection surface. A sensor capable of imaging light emitted or reflected by a touched operation article, a first interactive function unit that obtains a position of the operation article based on an image captured by the sensor, and controls display of an image according to the position; The information processing device includes a first communication unit capable of transmitting and receiving information, and the information processing device includes a second communication unit capable of transmitting and receiving information to and from the projection video display device, and a second communication unit. The position of the operation article is calculated from the image captured by the sensor received from the projection type video display device or the information generated based on the image captured by the sensor, and the projection type video display device determines the position according to the position. The projection type video display device has a second interactive function unit capable of controlling video display, and the projection type video display device has a first interactive function of the projection type video display device with respect to an interactive function for a display video displayed by the projection type video display device. A menu screen that can select whether to perform the operation in the first operation mode operated by the unit or in the second operation mode operated by the second interactive function unit of the information processing apparatus can be displayed.

The present invention provides, for example, a projection-type image display device that projects and displays an image, a light-emitting unit that emits laser light, and a projection that is used in an image display system that includes a laser light adjustment unit that blocks or reflects the laser light. The projection type image display device includes a projection optical system that projects a display image on a projection surface, and a sensor that can capture reflected light reflected by an operation object that has contacted the projection surface with laser light emitted from a light emitting unit. In order to adjust the installation position of the laser light adjustment unit as an initial adjustment mode, the interactive function unit that obtains the position of the operation article based on the image captured by the sensor and controls the display of the image according to the position is provided. The captured image obtained by capturing the area including the projection surface by the sensor is displayed on the projection surface by the projection optical system.

According to the present invention, it is possible to prepare for installation or use more suitably.

1 is a configuration diagram showing a video display system in Embodiment 1. FIG. The figure explaining the detection method of a user's finger contact. FIG. 2 is a block diagram showing an internal configuration of the projection display apparatus 100. The block diagram which shows the internal structure of the laser sheet generation | occurrence | production part 300. FIG. The figure explaining the example of the light-shielding board installation with respect to abnormal reflection. FIG. 4 is a diagram illustrating an example of a captured image displayed on the screen 1. The figure which shows the example which displayed the confirmation message of the abnormal reflection on the screen. The figure which shows the example of a sensor image in case abnormal reflection exists. The figure which shows the example of the sensor image after installing a light-shielding plate. The figure which shows the basic shape of the light-shielding plate. The figure explaining selection of the light-shielding surface angle of the reflective light-shielding plate. The figure explaining selection of the light-shielding surface angle of the reflective light-shielding plate. The figure explaining selection of the light-shielding surface angle of the reflective light-shielding plate. FIG. 6 is a diagram illustrating an internal configuration of an information processing apparatus 400 according to a second embodiment. The figure which shows the example of the mode selection screen of an interactive function. Explanatory drawing in the case of performing calibration operation automatically. Explanatory drawing in the case of performing calibration operation manually.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a configuration diagram illustrating a video display system according to the first embodiment. The video display system is configured by connecting a video output device 200, a laser sheet generator 300, and an information processing device 400 to a projection video display device (projector) 100. The connection method between the devices may be wired connection or wireless connection.

Projection type image display device (projector) 100 projects and displays an image on screen 1 which is a projection surface. The display screen 20 of the screen 1 includes an operation icon area 22 for displaying operation icons for performing an interactive function, in addition to a video area 21 for displaying an image. The operation icons include, for example, “mouse operation”, “line drawing” and “eraser” as drawing functions, and an icon for calling an operation menu of the main body of the projection display 100. In addition, a function of switching the video display content of the projection type video display device, a function of adjusting other operations (such as audio output) of the projection type video display device, and the like are included. In FIG. 1, as an interactive function, the user performs line drawing 23 and 24 by bringing the light-emitting pen 2 or the finger 3 as the operation article into contact with the screen 1 (video area 21).

The video output device 200 outputs video data to be displayed in the video area 21 to the projection video display device 100. For the video output device 200, for example, various video devices such as a personal computer and a DVD player can be used. Further, as another example of the video output device 200, a mobile terminal such as a tablet computer or a smartphone may be used. The connection between the video output device 200 and the projection video display device 100 may be a wired connection or a wireless communication connection, but is particularly wireless when the video output device 200 is a tablet computer or a mobile terminal such as a smartphone. Communication connection is preferred.

The laser sheet generator 300 irradiates a laser beam in a non-visible light band in a plane near the display surface of the screen 1. The projection display apparatus 100 detects the user's operation (contact position) by detecting the laser light that returns upon hitting the operation article (finger 3) or the light from the light emitting pen 2. Here, the laser light in the non-visible light band does not hinder the viewer from viewing the display image. In addition, since it is assumed that the user (operator) is in the vicinity at the time of operation, infrared laser light is particularly desirable among laser light in the invisible light band.

The information processing apparatus 400 is, for example, a personal computer, a tablet computer, a mobile terminal such as a smartphone, and the like, and inputs and outputs various control signals including an interactive function to and from the projection display apparatus 100. Thereby, the display operation of the projection display apparatus 100 can be controlled on the information processing apparatus 400 side. The information processing apparatus 400 and the video output apparatus 200 may be the same apparatus.

FIG. 2 is a diagram for explaining a detection method of a user's finger contact. (A) is the figure which looked at the screen from the side, (b) is the figure which looked at the screen from the front.

The screen 1 is attached to the wall 9, and the projection type image display device 100 is installed on the top. The projection-type image display device 100 projects an image in an oblique direction from the projection optical system 101 toward the screen 1 and displays the image on the display screen 20. In addition, the projection type image display apparatus 100 is provided with a sensor 150 configured by an infrared camera or the like, and images the screen 1 and detects laser light reflected (scattered) by the user's finger 3.

The laser sheet generator 300 is installed on the upper side of the screen 1 below the projection display apparatus 100 and irradiates infrared laser light in a plane parallel to and close to the display surface of the screen 1. The shape of the laser light irradiation area is a sheet having a thin cross section, and the laser light irradiation area is placed close to the screen 1 so that the gap d is several mm or less. In FIG. 2, the irradiation area is shown in gray, but the irradiation area of the laser light is hereinafter referred to as “laser sheet” 350.

When the user touches the screen 1 with the finger 3, a part of the finger 3 intersects the laser beam, that is, the laser sheet 350, and the laser beam is reflected (scattered). The reflected light is denoted by reference numeral 30. The sensor 150 of the projection display apparatus 100 acquires the captured image including the reflected light 30. The projection display apparatus 100 analyzes the captured image acquired by the sensor 150 to determine the contact position of the finger 3, determines the operation content based on the position of the user's finger, and performs interactive functions such as drawing display. Execute.

FIG. 3 is a block diagram showing the internal configuration of the projection display apparatus 100.
The projection optical system 101 is an optical system that projects an image onto the screen 1 and includes a lens and / or a mirror. The display element 102 is an element that generates an image to be projected, and uses a transmissive liquid crystal panel, a reflective liquid crystal panel, a DMD (Digital Micromirror Device, registered trademark) panel, or the like. The display element driving unit 103 sends a drive signal corresponding to the video signal to the display element 102. The light source 105 generates illumination light for projection, and uses a high pressure mercury lamp, a xenon lamp, an LED light source, a laser light source, or the like. The power source 106 supplies power to the light source 105. The illumination optical system 104 condenses the illumination light generated by the light source 105, makes it more uniform, and irradiates the display element 102. An operation signal input unit 107 is an operation button on the apparatus main body or a light receiving unit of a remote controller, and inputs an operation signal from a user.

The nonvolatile memory 108 stores data for various operations, display icons, and calibration data in the interactive function. The memory 109 stores video data to be projected and control data for the apparatus. The control unit 110 controls the operation of each unit in the apparatus. Particularly, the interactive function is executed by controlling the sensor 150 and the interactive function unit 120.

The sensor 150 is a camera that captures the front surface of the screen 1 and can detect reflected light from the operation article by detecting an infrared light component. In addition, by setting the cut wavelength of the optical filter to the visible light wavelength range (for example, setting it in the middle of the red visible light range), some visible light components other than infrared light (that is, the projected image on the display screen) Can be photographed together with an infrared light component.

The interactive function unit 120 is a part that performs interactive operations such as writing characters and figures in the video area 21 by the user operating the light-emitting pen and fingers. For this purpose, the coordinate calculation unit 121 analyzes the infrared image acquired from the sensor 150, recognizes the light emitted from the light-emitting pen or the scattered light from the laser sheet, and calculates the position (position operated by the user). The application unit 122 synthesizes the video area 21 and the operation icon area 22, an application that performs a drawing process or the like based on a user operation, an application that performs an operation on a video input from the video output apparatus 200, or the like. An application that can be operated with a light-emitting pen or a finger is executed. Here, the imaging range of the sensor 150 and the range of an image projected on the screen 1 (an optical image on the screen 1 in the image area of the display element 102) are unlikely to coincide with each other. Therefore, when the position operated (drawn) by the user is calculated by the coordinate calculation unit 121, it is necessary to convert the coordinates in the shooting range of the sensor 150 and the coordinate position in the image projected on the screen 1. Therefore, the interactive function unit 120 includes a calibration unit 123 that performs the conversion process and a process for creating conversion table data (calibration data) for the conversion process.

The video input unit 131 connects the external video output device 200 and inputs video data. For example, the communication unit 132 is connected to the laser sheet generation unit 300 or the information processing apparatus 400 to input / output various control signals. The power supply unit 133 supplies power and a control signal to the laser sheet generation unit 300.

FIG. 4 is a block diagram showing an internal configuration of the laser sheet generating unit 300.
The power input unit 301 receives power from the projection display apparatus 100 and supplies it to the laser light source 304.

The laser light source 304 generates laser light in the infrared region (for example, near the wavelength of 850 nm). The laser sheet forming unit 305 is an optical system that converts the laser beam generated by the laser light source 304 into a thin sheet (laser sheet 350) and emits it. For example, an existing optical system such as a cylindrical lens may be used. Or you may comprise using the scanning mechanism which scans a laser beam to a wide angle. Although not shown, an adjustment mechanism for adjusting the formation position of the laser sheet 350 in parallel with the screen 1 may be provided.

In this embodiment, an example in which power is supplied from the projection display apparatus 100 to the laser sheet generator 300 will be described. However, the laser sheet generator 300 acquires power without passing through the projection display apparatus 100. You may comprise as follows. When the laser sheet generating unit 300 is configured most simply, the laser light source 304 generates infrared laser light using the power input by the power input unit 301, and the laser sheet is converted into a laser sheet by the laser sheet forming unit 305. Is sufficient, and the control signal input unit 302 and the control unit 303 shown in the figure are not necessary.

However, a control signal input unit 302 that inputs a control signal for controlling laser emission from the projection display apparatus 100 to the laser sheet generation unit 300 and a control unit 303 that controls the laser light source 304 by the input control signal are provided. By installing it, it becomes possible to perform more advanced control. For example, the following controls (1) to (4) can be realized.

(1) For example, a control signal is transmitted from the communication unit 132 of the projection display apparatus 100 to the laser sheet generating unit 300, the laser light source 304 is turned off for a minute time that does not affect the operation, and then turned back on immediately. In conjunction with this, during the time when the laser light source 304 is OFF, an infrared image is imaged by the sensor 150 of the projection display apparatus 100, and an average value, an intermediate value, and the like of the luminance are acquired. A control signal may be transmitted from the communication unit 132 in accordance with the acquired average value, intermediate value, or the like, and the light emission intensity of the laser light source 304 thereafter may be controlled. By repeating this control intermittently, it is possible to realize adjustment of the light emission intensity of the laser light source in consideration of the influence of external light. Thereby, the laser intensity only needs to be increased only when the influence of external light is strong, so that wasteful power consumption can be prevented when the influence of external light is small.

(2) For example, when the coordinate calculation unit 121 of the projection display apparatus 100 does not recognize the light emitted from the light-emitting pen or the scattered light from the laser sheet for a predetermined time (for example, 1 hour) or longer, the communication unit 132 transmits the laser sheet. A control signal may be transmitted to the generator 300 and the laser light source 304 may be turned off. Thereby, it is possible to prevent the laser sheet from being generated unnecessarily when the interactive function is not used for a while. At this time, since the laser light source 304 cannot be returned to the ON state by a finger operation, a message or icon indicating the return method may be displayed in the display image on the screen 1. For example, a display such as “Please press the ○○ button on the remote control to resume finger operation” may be performed. When the return operation signal is input from the operation signal input unit 107 or the like, a control signal may be transmitted from the communication unit 132 to the laser sheet generation unit 300 and the laser light source 304 may be turned on.

(3) As described above, the sensor 150 can capture a part of visible light by setting the cut wavelength of the optical filter in the visible light wavelength region. Imaging can be performed. Therefore, if there is visible light, it is possible to take an image of the arm or the like of the person to be operated without scattering of the laser light. Therefore, in addition to the object for recognizing the light emission of the light emitting pen or the scattered light of the laser sheet as described in (2) above, it is recognized whether there is a moving object in the sensor image, and a control signal is sent from the communication unit 132. It is also possible to switch ON / OFF of the laser light source 304.

That is, when the coordinate calculation unit 121 of the projection display apparatus 100 does not recognize the light emission of the light emitting pen or the scattered light of the laser sheet for a first predetermined time (for example, 30 minutes) or longer, and the projection display apparatus The moving object analysis unit (not shown) included in the sensor 100 analyzes the presence / absence of a moving object in the sensor image, and when the moving object is not recognized for a second predetermined time (for example, 40 minutes) or more, the communication unit 132 controls the laser sheet generation unit 300. A signal may be transmitted and the laser light source 304 may be turned off. Even if there is no light emitted from the light-emitting pen or scattered light from the laser sheet, if the moving object is nearby, there is a high possibility that the operator is in the vicinity, and it is not strange when a finger operation is performed. Therefore, unintended turning off of the laser light source can be prevented by making the second predetermined time longer than the first predetermined time. In addition, the laser light source 304 may be returned to the ON state as in (2) above. However, in the example of (3), the moving body analysis unit may analyze the presence or absence of a moving body in the sensor image without the laser sheet. Since it is possible, when a moving object in the sensor image is recognized again, a control signal may be transmitted from the communication unit 132 to the laser sheet generating unit 300 and the laser light source 304 may be turned on.

(4) A control signal may be transmitted from the communication unit 132 to the laser sheet generation unit 300, and in synchronization with the imaging frame rate of the sensor 150, the laser light source 304 may be blinked at high speed for imaging. For example, the sensor image of the laser sheet ON and the sensor image of the laser sheet OFF can be alternately acquired for each frame. Based on the difference between the sensor image image of the continuous laser sheet ON and the sensor image of the laser sheet OFF, if recognition processing of light emitted from the light emitting pen or scattered light from the laser sheet is performed, the external light in the sensor image of the laser sheet OFF is detected. Since the influence can be excluded, it is possible to reduce an erroneous operation due to external light.

The modified examples (1) to (4) described above may be performed alone or in combination.

Next, FIG. 5 is a diagram for explaining an example of installing a light shielding plate against abnormal reflection. Like FIG. 2, (a) is the figure which looked at the screen from the side, (b) is the figure which looked at the screen from the front.

As shown in FIG. 5, it is assumed that an obstacle 4 protruding to the display side exists at the lower end of the screen 1 and the obstacle 4 intersects the laser sheet 350. For example, when a whiteboard is used as the screen, a writing instrument holder for the whiteboard may become the obstacle 4 in some cases. Since the obstacle 4 intersects with the laser sheet 350, unnecessary reflected light 40 (hereinafter referred to as abnormal reflection) is generated there, and part of it reaches the sensor 150 and is detected. As a result, the projection display apparatus 100 determines that the reflected light is due to an operation by the user's finger 3 and performs an incorrect operation.

Therefore, in order to prevent this malfunction, the light shielding plate 5 is installed so that the reflected light 40 is not generated. The light-shielding plate 5 is installed in front of the obstacle 4 (upward in the figure) as viewed from the laser sheet generating unit 300, but is preferably installed outside the display screen 20 that displays an image. The specific structure of the light shielding plate 5 will be described later. By installing the light shielding plate 5, the laser beam is not irradiated on the obstacle 4, and thus unnecessary abnormal reflection 40 does not occur.

Next, an initial adjustment mode for operating the interactive function by the projection display apparatus 100 will be described. The laser light source 304 of the laser sheet generator 300 is turned on, and the front surface of the screen 1 is photographed by the sensor 150. Then, the photographed image (hereinafter referred to as a sensor image) is displayed on the screen 1 and the adjustment work described below is performed.

FIG. 6A is a diagram illustrating an example of a captured image (sensor image) displayed on the screen 1. When the projection display apparatus 100 and the laser sheet generator 300 are installed in the positional relationship shown in FIG. 2, the sensor image is displayed on the display screen 20 of the screen 1 as shown in FIG. 6A. As the sensor image, an image 20 ′ on the display screen 20 of the screen 1 and an image 300 ′ of the laser sheet generator 300 are displayed. In this embodiment, the image 300 ′ of the laser sheet generator is displayed below the display screen 20 because of the sensor installation position, and the image 20 ′ of the display area is displayed in a trapezoidal shape with the vertical and horizontal directions reversed. The

First, as an initial adjustment, an adjustment reflecting member (not shown) is placed at a predetermined position on the screen, the position and direction of the reflected light are obtained from the sensor image, and the laser sheet 350 is parallel to the screen and has a predetermined position. What is necessary is just to comprise so that it may be notified to a user whether it is formed in distance. For example, as in the example of FIG. 6A, the state in which the reflected light indicated by reference numeral 50 is imaged by the adjustment reflecting member may be displayed as it is to prompt the user to make adjustments. In addition, a display that indicates a method to be adjusted by the user based on the position information of the reflected light 50 acquired from the sensor image may be displayed on the display screen 20. If it is determined by these means that the distance d between the laser sheet 350 and the screen is not within a predetermined range, or if it is determined that they are not in a parallel state, the adjustment mechanism of the laser sheet generating unit 300 is notified to the user. The laser sheet generator 300 may be urged to adjust the installation position and posture, and this may be repeated until it is determined that the distance d between the laser sheet 350 and the screen is in a predetermined range and in a parallel state.

Next, prevention of abnormal reflection (unnecessary light) on the screen will be described. If there is any obstruction on the screen, the laser beam is reflected (scattered) by that, and is detected by the sensor 150 to cause a malfunction. Then, the presence or absence of abnormal reflection is confirmed, and the operation proceeds to the operation of installing a light shielding plate.

FIG. 6B is a diagram showing an example in which an abnormal reflection confirmation message is displayed on the screen. For example, a message 60 “Please check whether there is abnormal reflection” is displayed on the screen 1. As a countermeasure, a message such as “If there is abnormal reflection, install a light shielding plate between the reflector and the light emitting part” is displayed. In accordance with this, the user confirms the presence or absence of abnormal reflection in the sensor image. The “reflector” in the message refers to the obstacle 4, and the “light emitting unit” refers to the laser sheet generating unit 300.

FIG. 6C is a diagram illustrating an example of a sensor image when abnormal reflection exists. In this example, an abnormal reflection 40 is recognized in the upper left part of the sensor image. There is a reflection object (obstacle) 4 that reflects the laser light at the position where the abnormal reflection is recognized, and the user specifies what the reflection object is. Since the sensor image is an infrared light image, the reflector 4 itself is not always visible from the sensor image. In accordance with the message 60, the user installs the light shielding plate 5 between the reflector 4 and the light emitting unit (laser sheet generating unit 300) (see FIG. 5).

FIG. 6D is a diagram illustrating an example of a sensor image after the light shielding plate is installed. In this example, the image of the light shielding plate 5 is denoted by 5 ′ in a state where the light shielding plate 5 is installed in front of the reflector (obstacle) 4. In the sensor image, the reflector is hidden behind the light shielding plate 5 'and cannot be seen. By installing the light shielding plate 5, the laser light does not hit the reflector 4, and the abnormal reflection 40 on the reflector 4 can be eliminated.

As described above, in this embodiment, since the sensor image is displayed on the screen at the time of the initial adjustment, the user can easily find the position of the abnormal reflection from the sensor image. Further, after the light shielding plate is installed, it can be confirmed from the sensor image that the abnormal reflection has been reliably eliminated.

Next, the shape of the light shielding plate 5 will be specifically described.
FIG. 7A is a diagram illustrating a basic shape of the light shielding plate 5. (A) shows an absorption type, and (b) shows a reflection type.
The absorption type (a) absorbs the incident laser beam 31 by the light shielding surface 5a. For example, the light shielding surface 5a is coated with a light absorption film or an antireflection film to reduce the light reflectance. In the case of the absorption type, the shape of the light shielding plate 5 is arbitrary. That is, the angle α between the installation surface 5b on the screen 1 or the wall 9 and the light shielding surface 5a is arbitrary and may be 90 °.

(B) The reflection type has a structure in which the light-shielding plate 5 has a triangular prism shape and is reflected by the light-shielding surface 5a. The material of the light shielding plate 5 is not limited, but the angle α between the installation surface 5b on the screen 1 or the wall 9 and the light shielding surface (reflective surface) 5a needs to be smaller than 90 ° in consideration of the direction of the reflected laser light 32. There is. When the angle α is equal to 90 °, the laser beam 32 reflected by the light shielding surface 5a returns as it is in parallel with the screen 1 or the wall 9 and is likely to reach the sensor 150. If the angle α is larger than 90 °, the laser light 32 reflected by the light shielding surface 5a is irradiated onto the screen 1 (inside the display screen 20), which may be detected by the sensor 150 and cause a malfunction.

Hereinafter, a preferable angle of the light shielding surface (reflection surface) in the reflective light shielding plate will be described.
FIG. 7B is a diagram illustrating selection of the light shielding surface angle of the reflective light shielding plate 5. In this example, the light shielding plate 5 has a triangular prism shape, and the angle α between the installation surface 5b on the screen 1 or the wall 9 and the light shielding surface 5a is smaller than 45 °. (A) is a side view of the entire installation, and (b) is an enlarged view of the light shielding plate 5.

As a premise, projection display devices are often used indoors, and projection display devices having interactive functions are particularly likely to be used in conference rooms and classrooms. In conference rooms and classrooms, there are many vertical surfaces such as walls, tables and back plates of desks that have an angle perpendicular to the floor surface. Considering these, it is desirable that the more preferable value of the angle α is as follows.

When the angle α is smaller than 45 °, the laser light 32 reflected by the light shielding surface 5a travels downward from the horizontal direction. Even if there is a table or desk in the traveling direction and the light is reflected again by the vertical surface 10, it returns to a position below the light shielding plate 5 (reference numeral 33). Therefore, the return light does not travel to the laser sheet generating unit 300 side or irradiate the screen 1, and there is little possibility of erroneous detection by the sensor 150.

FIG. 7C is a diagram illustrating selection of the light shielding surface angle of the reflective light shielding plate 5. In this example, a demerit when the angle α between the installation surface 5b on the screen 1 or the wall 9 and the light shielding surface 5a is 45 ° will be described. (A) is a side view of the entire installation, and (b) is an enlarged view of the light shielding plate 5.

When the angle α is 45 °, the laser beam 32 reflected by the light shielding surface 5a proceeds in the horizontal direction. When a table or desk exists in the traveling direction and is re-reflected by the vertical surface 10, it returns along substantially the same optical path and enters the light shielding plate 5 (reference numeral 33). Further, the light is reflected by the light shielding surface 5a and returned to the laser sheet generator 300 (reference numeral 34), which may be detected by the sensor 150. Therefore, the angle α between the installation surface 5b and the light-shielding surface 5a is most preferably 45 ° and should be avoided.

Although not shown, when the angle α is larger than 45 °, the laser light 32 reflected by the light shielding surface 5a travels upward from the horizontal direction. If a table, desk, or the like exists in the traveling direction and re-reflects on the vertical surface 10, it may return to a position above the light shielding plate 5 and irradiate the screen 1. Therefore, it is not preferable to make the angle α larger than 45 °.

As described above, it is desirable that the angle α is smaller than 45 ° in consideration of a general use environment.

FIG. 7D is a diagram illustrating selection of the light shielding surface angle of the reflective light shielding plate 5. In this example, the installation surface of the light shielding plate 5 is selected from two surfaces. The cross-sectional triangle is a substantially right-angled triangle with internal angles α (<45 °) and β (> 45 °). 5c. Then, the installation surfaces 5b and 5c are switched according to the situation.

(A) is an overall side view, and the reflected light 32 is switched between state 1 and state 2 by switching the installation surface of the light shielding plate. In the state 1, the reflected light 32 travels downward from the horizontal direction, and in the state 2, the reflected light 32 travels upward from the horizontal direction.
(B) is an enlarged view of installation in state 1, and by using the installation surface 5b, the angle with the light shielding surface 5a is α (<45 °) and the reflected light 32 is directed downward.
(C) is an enlarged view of the state 2, and by using the installation surface 5c, the angle with the light shielding surface 5a is β (> 45 °) and the reflected light 32 is directed upward.

Usually, it may be used in the state 1 of (b), but when there is some reflection in the reflected light direction and the return light is detected by the sensor 150, the installation surface is changed from 5b to 5c, Switch to state 2 of (c). This may change the direction of the reflected light and eliminate the return light. As described above, the installation in the two reflecting states can be selectively realized by the single light shielding plate 5, which is practical.

As a modification of the above, the same effect can be obtained by switching the oblique side of the three surfaces of the light shielding plate 5 as the installation surface 5a and switching the other two sides as the two light shielding surfaces (reflection surfaces) 5b and 5c.

The light shielding plate 5 described above may be referred to as a light shielding unit, a reflection unit, or a laser light adjustment unit.

According to the first embodiment, since the sensor image is displayed on the screen at the time of the initial adjustment, the user can easily confirm the presence or absence of abnormal reflection that hinders the interactive function from the sensor image. In addition, since it is possible to confirm from the sensor image that the abnormal reflection has been eliminated after the light shielding plate is installed, the adjustment work is easy and reliable.

Example 2 describes a case where the projection type video display apparatus 100 and the information processing apparatus 400 are connected to realize an interactive function on the information processing apparatus 400 side.

FIG. 8 is a diagram illustrating an internal configuration of the information processing apparatus 400 according to the second embodiment. The information processing apparatus 400 is an example of a personal computer, but may be a mobile terminal such as a tablet computer smartphone.

The video output unit 401 outputs video data to the video input unit 131 of the projection video display device 100. The communication unit 402 inputs and outputs various control signals related to the interactive function with the communication unit 132 of the projection display apparatus 100. The storage 403 stores an interactive function program, video data, and the like. The interactive function program may be acquired in advance from an external device or an external server via the communication unit 402 and stored in the storage 403. The display unit 404 displays various videos and operation screens. The operation input unit 405 is a keyboard or a mouse that receives an operation signal from the user. Also, by operating this, the operation of the projection display apparatus 100 can be operated.

The control unit 406 controls the operation of each unit in the information processing apparatus 400. Further, the interactive operation of the projection display apparatus 100 is controlled using the image signal from the sensor 150 of the projection display apparatus 100 and the interactive function unit 410. The nonvolatile memory 407 stores data for various operations, display icons, and calibration data in the interactive function. These data may be stored in the storage 403.

The memory 408 expands the interactive function program stored in the storage 403, and the control unit 406 cooperates with the memory 408 to realize the interactive function. The interactive function unit 410 calculates a coordinate of the user's operation position based on information acquired from the projection display apparatus 100 via the communication unit 402, and a process for creating calibration data. A calibration unit 413 is provided.

Further, an application unit 412 that is an application program that can be operated using the coordinates calculated by the coordinate calculation unit 411 of the interactive function unit 410 as an operation input can be expanded in the memory 408. The application unit 412 is not only an application such as an electronic whiteboard drawing application that is assumed to be executed together with the projection display apparatus 100 but also an operation input of an operation input unit 405 (such as a keyboard and a mouse) of the information processing apparatus 400. An application for a general personal computer to be operated may be used. In this case, the operation of the application can be realized by replacing the operation coordinates designated by the mouse operation of the operation input unit 405 with the coordinates calculated by the coordinate calculation unit 411. The display content of the application unit 412 operated in this way is output from the video output unit 401 of the information processing device 400 to the projection video display device 100, and the projection video display device 100 displays it.

As described above, in this embodiment, instead of the function of the interactive function unit 120 of the projection display apparatus 100, the information processing apparatus 400 can realize part or all of the interactive function. Note that when the interactive function unit 410 of the information processing apparatus 400 is used, the video displayed by the projection display apparatus 100 may be the video data itself output from the information processing apparatus 400. However, when another video output device (for example, the video output device 200) is connected at the same time, the projection video display device 100 displays the video data output from the information processing device 400 and the video output from the video output device 200. Data may be combined and displayed.

FIG. 9 is a diagram showing an example of an interactive function mode selection screen. An “interactive function mode selection menu” 90 as shown in the figure is displayed on the display screen 20 projected by the projection display apparatus 100. This menu is generated and displayed by the control unit 110 using a menu screen image or the like stored in the non-volatile memory 108 based on a user operation via the operation signal input unit 107 of the projection display apparatus 100. . In this menu, a mode 91 for processing on the projection display apparatus side, a mode 92 for processing on the information processing apparatus side, and a mode 93 not using the interactive function are displayed, and the user selects a desired mode from these. When the user selects the information processing apparatus side processing mode 92, the interactive function is executed using the interactive function unit 410 developed in the memory 408 of the information processing apparatus 400.

When the interactive function unit 410 can realize a function higher than the interactive function unit 120 or a function that the interactive function unit 120 does not have, the information processing apparatus 400 is prepared by enabling the mode switching by the selection menu. Thus, it is possible to select whether to use a more advanced interactive function or to execute the interactive function on the projection display apparatus without preparing the information processing apparatus 400. In the present embodiment, since the projection display apparatus 100 and the information processing apparatus 400 each have an interactive function unit, either of them can execute the interactive function, but the data necessary for preparing for its use, For example, the calibration data is shared with each other to improve efficiency. Hereinafter, the calibration operation of the present embodiment will be described.

FIG. 10A is an explanatory diagram when the calibration operation is automatically executed. The calibration operation refers to data for the calibration unit 123 of the projection display apparatus 100 or the calibration unit 413 of the information processing apparatus 400 to convert the coordinates of the captured image of the sensor 150 into the coordinates of the display screen 20. It is an action to get.

The procedure is as follows. In the following procedure, the operation of the information processing apparatus 400 and the projection video display apparatus 100 is linked with the control signal between the communication unit 402 of the information processing apparatus 400 and the communication unit 132 of the projection video display apparatus 100. This can be realized by inputting and outputting to The projection display apparatus 100 displays a visible light mark 70 (calibration image) at a predetermined position in the display screen 20. The shape, size, and color of the mark may be determined in advance, and the size and shape of the mark may be changed for each position. The calibration video data including the mark 70 may be recorded in the nonvolatile memory 108 of the projection video display device 100, the storage 403 of the information processing device 400, or the like.

The sensor 150 of the projection display apparatus 100 captures a calibration image displayed on the display screen 20, that is, each mark 70. However, since the calibration image is generated with visible light, the sensor 150 can detect not only the laser light (infrared light) from the laser sheet generator 300 but also the visible light component as described above. It is necessary to configure.

The correspondence between the position of each mark 70 in the displayed calibration image and the position of each mark 70 in the image photographed by the sensor 150 is obtained, and calibration data for performing coordinate conversion between the two is obtained.

FIG. 10B is an explanatory diagram when the calibration operation is executed manually. The projection display apparatus 100 displays a visible light mark 80 (calibration image) at a predetermined position in the display screen 20. The mark 80 indicates a position where the user should point using the light-emitting pen 2 or the finger 3. The calibration image is recorded in the nonvolatile memory 108 of the projection display apparatus 100, the storage 403 of the information processing apparatus 400, or the like. While displaying the calibration image, the projection display apparatus 100 uses a message or the like to instruct the user to point the mark 80 at each position one by one using the light-emitting pen 2 or the finger 3. To do.

When the user points to the position where the mark 80 is displayed, the light-emitting pen 2 emits light, or the laser beam is reflected by the finger 3, and this is detected by the sensor 150. Then, a correspondence relationship between the position of each mark 80 in the displayed calibration image and the light detection position (pointing position) detected by the sensor 150 is obtained, and calibration data for performing coordinate conversion between the two is obtained. .

The calibration data acquired in this way is shared by both the projection display apparatus 100 and the information processing apparatus 400. For example, when calibration data acquisition processing is executed by the calibration unit 123 of the projection display apparatus 100, the acquired calibration data is stored in, for example, the non-volatile memory 108 and is naturally interactive on the projection display apparatus 100 side. It can be used to execute a function. Further, the calibration data is transmitted from the projection display apparatus 100 to the information processing apparatus 400 via the communication unit of both at any timing, and stored in the nonvolatile memory 407 or the storage 403. Accordingly, even when the interactive function unit 410 of the information processing apparatus 400 executes an interactive function, the information processing apparatus 400 can take over and use the received calibration data. Of course, conversely, calibration data acquisition processing is executed by the calibration unit 413 of the information processing apparatus 400, and the acquired calibration data is then transmitted to the projection display apparatus 100 via both communication units. The calibration data received by the projection display apparatus 100 can be taken over and used.

In addition, when the aspect ratio of the video input to the projection type video display device 100 is changed, when the output video aspect ratio from the projection type video display device 100 is changed, or in the projection type such as the multi-screen display mode. When the display layout of the video display device 100 is changed, the calibration data may not be used as it is. In this case, a message is displayed in the video of the projection display apparatus 100 so that the calibration data acquisition process is executed again by the calibration unit of the one apparatus selected on the menu screen of FIG. Calibration data acquisition processing may be executed. In that case, the acquired calibration data may be transmitted to the other device, and the other device may take over and use the same as after the initial calibration data acquisition processing described above.

Also, changes in various aspect ratios and display layouts can be geometrically calculated before and after. Therefore, operations that are convenient for the user are as follows. The calibration data acquisition process is performed once in either the calibration unit 123 of the projection video display apparatus 100 or the calibration unit 413 of the information processing apparatus 400, and the acquired calibration data is projected by the above-described method. It is shared by both the video display device 100 and the information processing device 400. When the interactive function is executed on the device side selected by the menu in FIG. 9, when various aspect ratios or display layouts are changed, the various aspect ratios or display layouts are changed. If the calibration unit on the apparatus side selected by the menu of FIG. 9 executes the geometric conversion process on the original calibration data and uses the converted calibration data for the subsequent interactive function process. Good.

In this way, the calibration data acquisition process needs to be performed only once by one apparatus, and even when various aspect ratios or display layouts are changed, or when interactive functions are processed by the menu of FIG. Even if the subject is switched, the interactive function by the system can be used continuously without executing the calibration data acquisition process again, which is very convenient for the user. In addition, when the processing subject of the interactive function is set in the information processing apparatus 400 by the menu in FIG. 9, in order to realize the above-described processing, the control unit 110 of the projection display apparatus 100 has various aspect ratios. When there is a change or a display layout change, it is necessary to transmit control information capable of identifying the change contents to the information processing apparatus 400 via the communication unit 132.

Note that an example of coordinate calculation processing when the processing subject of the interactive function is set in the information processing apparatus 400 using the menu of FIG. 9 will be described. As an example of the coordinate calculation process, at least two methods can be considered as follows.

The first method will be described. First, the information processing apparatus 400 acquires calibration data by the calibration process as described above, and stores it in the storage 403 or the nonvolatile memory 407. Next, the sensor image itself captured by the sensor 150 of the projection display apparatus 100 is transmitted to the communication unit 402 of the information processing apparatus 400 via the communication unit 132. The sensor image received by the communication unit 402 of the information processing apparatus 400 is analyzed by the coordinate calculation unit 411 of the interactive function unit 410 developed in the memory 408, and the light emission of the light emitting pen or the scattered light of the laser sheet is recognized and the light emission is performed. Alternatively, the coordinates (position operated by the user) in the imaging range of the scattered light sensor 150 are calculated.

Further, the coordinate calculation unit 411 uses the calibration data stored in the storage 403 or the non-volatile memory 407 to calculate the coordinates of the user operation position in the imaging range of the sensor 150 in the display screen 20 of the projection display apparatus 100. Convert to coordinates corresponding to the coordinates of. At this time, the calibration data stored in the storage 403 or the nonvolatile memory 407 includes an assumed aspect ratio at the time of calibration, an output aspect ratio of the information processing apparatus 400, a display aspect ratio of the projection display apparatus 100, and a projection. What is necessary is just to use it, after performing geometric correction according to the display layout etc. of the type | mold video display apparatus 100. FIG. Thus, the coordinates calculated by the coordinate calculation unit 411 are used for the operation of the application unit 412.

Next, the second method will be described. First, the information processing apparatus 400 acquires calibration data by the calibration process as described above, and stores it in the storage 403 or the nonvolatile memory 407. In the second method, the sensor image captured by the sensor 150 of the projection display apparatus 100 is sent to the coordinate calculation unit 121 of the interactive function unit 120. The coordinate calculation unit 121 analyzes the sensor image and calculates the coordinates in the imaging range of the sensor 150 for the position of light emitted from the light-emitting pen or the scattered light of the laser sheet (user operation position). The coordinate information in the imaging range of the sensor 150 at the user operation position is transmitted to the communication unit 402 of the information processing apparatus 400 via the communication unit 132.

In the information processing apparatus 400, the coordinate calculation unit 411 uses the calibration data stored in the storage 403 or the nonvolatile memory 407 to obtain the coordinate information in the imaging range of the sensor 150 of the user operation position received by the communication unit 402. Then, the coordinates are converted into coordinates corresponding to the coordinates in the display screen 20 of the projection display apparatus 100. This conversion process is the same as in the first method. Thus, the coordinates calculated by the coordinate calculation unit 411 are used for the operation of the application unit 412.

According to the second coordinate calculation method described above, since it is not necessary to transmit the sensor image itself from the projection display apparatus 100 to the information processing apparatus 400, communication data between the two can be reduced, and an interface with a large bandwidth can be used. There is a merit that it can be realized without it.

In the present embodiment, the example in which the calibration unit is provided in both of the projection display apparatus 100 and the information processing apparatus 400 has been described. However, only the projection display apparatus 100 may be configured to include a calibration unit. In this case, the calibration data creation process is always created by the calibration unit 123 of the projection display apparatus 100, and the created calibration data is transmitted to the information processing apparatus 400 via the communication unit 132 and projected. The type video display device 100 and the information processing device 400 are shared and used.

According to the second embodiment described above, not only the projection display apparatus 100 but also the information processing apparatus 400 can execute a part or all of the interactive function unit, so that both functions are used properly according to the situation. Can build a system that is easy to use. At that time, the calibration data can be shared by both parties, so the efficiency is improved.

Note that the effect of Example 2 is also effective in a system that uses only a light-emitting pen. In this case, the laser sheet generator 300 is not necessary.

1: Screen (projection surface), 2: Light emitting pen, 3: Finger (operation object), 4: Obstacle (reflection object), 5: Light shielding plate, 5a: Light shielding surface (reflection surface), 5b, 5c: Installation surface , 20: display screen, 21: video area, 22: operation icon area, 30: reflected light, 40: extraordinary reflection, 60: message, 70, 80: mark, 90: mode selection menu, 100: projection type video display device , 101: projection optical system, 108: nonvolatile memory, 110: control unit, 120: interactive function unit, 121: coordinate calculation unit, 122: application unit, 123: calibration unit, 150: sensor, 200: video output device , 300: Laser sheet generating unit, 304: Laser light source, 305: Laser sheet forming unit, 350: Laser sheet, 400: Information processing device, 402: Communication unit 403: storage, 410: interactivity unit, 413: calibration unit.

Claims

In a video display system including a projection-type video display device that displays video and an information processing device,
The projection-type image display device
A projection optical system for projecting a display image onto a projection surface;
A sensor capable of imaging light that is emitted or reflected by the operation article in contact with the projection surface;
A first interactive function unit for obtaining a position of the operation article based on an image captured by the sensor and controlling display of an image according to the position;
A first communication unit capable of transmitting and receiving information to and from the information processing apparatus;
Have
The information processing apparatus includes:
A second communication unit capable of transmitting and receiving information to and from the projection display apparatus;
The position of the operation article is calculated from the image captured by the sensor or the information generated based on the image captured by the sensor received from the projection video display device via the second communication unit, and the position is determined according to the position. A second interactive function unit capable of controlling display of video in the projection-type video display device,
Whether the projection-type image display device performs an interactive function for the display image displayed by the projection-type image display device in a first operation mode operated by the first interactive function unit of the projection-type image display device. A video display system capable of displaying a menu screen capable of selecting whether to perform the operation in the second operation mode operated by the second interactive function unit of the information processing apparatus.
The video display system according to claim 1,
The first interactive function unit of the projection-type image display device generates calibration data used for coordinate conversion of the coordinates of the position of the operation article in the image captured by the sensor into the coordinates of the position in the display image. A first calibration unit that performs processing to
The projection-type video display device transmits calibration data generated by the first calibration unit to the information processing device via the first communication unit.
The video display system according to claim 2,
The second interactive function unit of the information processing device generates calibration data used for coordinate conversion of the coordinates of the position of the operation article in the image captured by the sensor into the coordinates of the position in the display image. A second calibration unit for performing
The information processing apparatus transmits calibration data generated by the second calibration unit to the projection display apparatus via the second communication unit.
A projection-type video display device capable of transmitting and receiving information to and from an external information processing device,
A projection optical system for projecting a display image onto a projection surface;
A sensor capable of imaging light that is emitted or reflected by the operation article in contact with the projection surface;
An interactive function unit that obtains a position of the operation article based on an image captured by the sensor and controls display of an image according to the position;
A communication unit capable of transmitting and receiving information to and from the information processing apparatus;
The projection type video display device performs an interactive function for the display video displayed by the projection type video display device by processing of the interactive function unit of the projection type video display device or by processing of the information processing device. A projection type image display device capable of displaying a menu screen capable of selecting either of them.
In the projection type video display device according to claim 4,
The interactive function unit of the projection type video display device generates a calibration data used for coordinate conversion of the coordinates of the position of the operation article in the image captured by the sensor into the coordinates of the position in the display video. A calibration unit to perform,
The projection-type image display apparatus transmits calibration data generated by the calibration unit to the information processing apparatus via the communication unit.
In the projection type video display device according to claim 5,
The projection type video display device, wherein the communication unit of the projection type video display device receives information related to calibration transmitted from the information processing device and uses the information for processing of the interactive function unit.
A projection-type image display device used in an image display system including a projection-type image display device that projects and displays an image, a light-emitting unit that emits laser light, and a laser light adjustment unit that blocks or reflects the laser light. And
The projection-type image display device
A projection optical system for projecting a display image onto a projection surface;
A sensor capable of imaging reflected light reflected by an operation object that has contacted the projection surface with laser light emitted by the light emitting unit;
An interactive function unit that obtains the position of the operation article based on an image captured by the sensor and controls display of an image according to the position;
As an initial adjustment mode, in order to adjust an installation position of the laser light adjustment unit, a captured image obtained by capturing an area including the projection surface by the sensor is displayed on the projection surface by the projection optical system. Projection-type image display device.
In the projection type video display device according to claim 7,
In the initial adjustment mode, an instruction to install a laser light adjustment unit when abnormal reflection is present when a captured image obtained by capturing an area including the projection surface by the sensor is displayed on the projection surface by the projection optical system. A projection-type image display device that displays a message to be displayed together.
A projection display device that is a video display system that projects and displays video on a projection surface;
A light emitting unit that emits laser light;
A reflection unit that reflects the laser light,
The reflection unit is
An installation surface for installation on the projection surface side;
A reflection surface for reflecting the laser beam in a direction different from the incident direction,
An image display system, wherein an angle α formed by the installation surface and the reflecting surface is a triangular prism shape smaller than 45 °.