JP2016184850A - Projector and detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a projector capable of detecting both of an indicator (self-light emitting indicator) which emits infrared light and an indicator (non-light emitting indicator) which does not emit any light.SOLUTION: The projector includes: a projection section for projecting an image on to a projection plane; a first imaging unit and a second imaging unit having an area including the projection plane as an imaging area for picking up an image of a visible light; a third imaging unit which has an area including the projection plane as an imaging area for picking up an image of infrared light; a first detection section that detects the position of a non-light emitting indicator based on the imaging result by the first imaging unit and the second imaging unit; and a second detection section that detects the position of a self-light emitting indicator which has a light emitting part which emits infrared light based on the imaging result by the third imaging unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a projector and a detection method.

  An image based on an image signal output from a computer is projected onto a screen such as a whiteboard by a projector, and the projected image is captured by an imaging device. Recognized interactive systems are known. In such an interactive system, infrared light is transmitted and received by a projector and an electronic pen handled by a user (see, for example, Patent Document 1).

JP 2014-59695 A

However, when the projector is equipped with an infrared camera and the infrared camera receives infrared light from the electronic pen, a human finger that does not emit infrared light cannot be detected.
In order to solve this problem, a light curtain device that forms an infrared light curtain (light curtain) is installed in front of the screen, and the reflected light of the infrared light from a finger or the like is received by the infrared camera. It has been proposed. Thereby, the projector can recognize not only an electronic pen that emits infrared light but also a finger that does not emit infrared light. However, depending on the arrangement of the projector, there is a case where there is no space for installing the light curtain device, and there is a case where a large amount of electric power is consumed to continuously emit infrared light that forms the light curtain.

  The present invention has been made in view of the above points, and detects both an indicator that emits infrared light (self-luminous indicator) and an indicator that does not emit light itself (non-luminous indicator). An object is to provide a projector and a detection method that can be performed.

One aspect of the present invention is made to solve the above-described problem. A projection unit that projects an image on a projection surface, and a first region that captures visible light using the region including the projection surface as an imaging region. The imaging unit and the second imaging unit, the region including the projection plane as an imaging region, a third imaging unit that images infrared light, and the imaging by the first imaging unit and the second imaging unit Based on the result, the position of the self-luminous indicator which has the 1st detection part which detects the position of a non-light-emitting indicator, and the light emission part which emits infrared light based on the image pick-up result by the 3rd image pick-up part A second detection unit for detecting the projector.
With this configuration, the projector detects the position of the non-light-emitting indicator based on the imaging results of the first imaging unit and the second imaging unit that capture visible light, and performs the third imaging that captures infrared light. The position of the self-luminous indicator is detected based on the imaging result of the unit. As a result, the projector can detect both the self-luminous indicator and the non-luminous indicator.

According to one embodiment of the present invention, in the projector, at least one of the first imaging unit and the second imaging unit can switch a function of imaging visible light and a function of imaging infrared light. A configuration that is an imaging unit and that also configures the third imaging unit may be used.
With this configuration, in the projector, the visible light is imaged and the infrared light is imaged by the imaging unit capable of switching between the function of imaging visible light and the function of imaging infrared light. Thereby, in a projector, size reduction can be achieved compared with the case where the imaging part which images visible light, and the imaging part which images infrared light are provided separately.

According to one aspect of the present invention, in the projector, the non-light-emitting indicator or the self-light-emitting instruction is based on at least one of an imaging result by the first imaging unit and an imaging result by the second imaging unit. A configuration that further includes a color detection unit that detects the color of the part designated by the body may be used.
With this configuration, in the projector, based on at least one of the imaging result by the first imaging unit and the imaging result by the second imaging unit, the portion instructed by the non-luminous indicator or the self-luminous indicator Detect color. Thereby, in the projector, it is possible to indicate a color with a non-light-emitting indicator or a self-light-emitting indicator.

According to one aspect of the present invention, in the projector, the non-light-emitting indicator or the self-light-emitting instruction is based on at least one of an imaging result by the first imaging unit and an imaging result by the second imaging unit. The structure further provided with the figure detection part which detects the figure of the part instruct | indicated with the body may be used.
With this configuration, in the projector, based on at least one of the imaging result by the first imaging unit and the imaging result by the second imaging unit, the portion instructed by the non-luminous indicator or the self-luminous indicator Detect shapes. Thereby, in the projector, it is possible to instruct a figure with a non-light-emitting indicator or a self-light-emitting indicator.

One aspect of the present invention is an aspect in which a projector detects an aspect of the self-luminous indicator based on at least one of an imaging result by the first imaging unit and an imaging result by the second imaging unit. A configuration further including a detection unit may be used.
With this configuration, the projector detects the mode of the self-luminous indicator based on at least one of the imaging result by the first imaging unit and the imaging result by the second imaging unit. As a result, the projector can detect the mode of the light emitting part of the self light emitting indicator even if infrared light is not emitted from the light emitting part of the self light emitting indicator.

  One aspect of the present invention is a projector including a thermography camera that captures an image of a heat distribution in an imaging region, wherein the first detection unit includes the first imaging unit, the second imaging unit, and the thermography camera. A configuration may be used in which the position of the non-light emitting indicator is detected based on the imaging result of the above. With this configuration, the projector can improve the detection accuracy of the position of the non-light emitting indicator.

One embodiment of the present invention is made to solve the above-described problem. In the projector, an image is projected onto a projection surface, and the first imaging unit and the second imaging unit include the projection surface. The region is set as an imaging region, visible light is picked up, and the third image pickup unit picks up the region including the projection plane as an image pickup region, picks up infrared light, and the first image pickup unit and the second image pickup unit The position of the non-light-emitting indicator is detected based on the imaging result of, and the position of the self-luminous indicator having a light-emitting unit that emits infrared light is detected based on the imaging result of the third imaging unit. , Detection method.
With this configuration, in the detection method, the projector detects the position of the non-luminous indicator based on the imaging results of the first imaging unit and the second imaging unit that image visible light, and images infrared light. The position of the self-luminous indicator is detected based on the imaging result by the third imaging unit. Thus, in the detection method, the projector can detect both the self-luminous indicator and the non-luminous indicator.

  As described above, according to the projector and the detection method of the present invention, in the projector, the position of the non-light emitting indicator is determined based on the imaging results of the first imaging unit and the second imaging unit that image visible light. And detecting the position of the self-luminous indicator based on the imaging result of the third imaging unit that images infrared light. Accordingly, in the projector and the detection method according to the present invention, the projector can detect both the self-luminous indicator and the non-luminous indicator.

It is a figure which shows the example of a schematic structure of the interactive system which concerns on one Embodiment (1st Embodiment) of this invention. It is a figure which shows the schematic structural example of the interactive system which concerns on one Embodiment (2nd Embodiment) of this invention. It is a figure which shows the schematic structural example of the interactive system which concerns on one Embodiment (3rd Embodiment) of this invention. It is a figure which shows the schematic structural example of the interactive system which concerns on one Embodiment (4th Embodiment) of this invention.

Embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)
FIG. 1 is a diagram illustrating a schematic configuration example of an interactive system 1 according to an embodiment (first embodiment) of the present invention.
The interactive system 1 includes a projector 11, a projection surface 12, and an electronic pen 13. Also shown in FIG. 1 is a human finger 14.
The projection surface 12 is, for example, a screen, but may be a wall or a ceiling as another configuration example.
In the present embodiment, the projection surface 12 is a surface on which an image is projected by the projector 11 and an operation surface (such as an electronic blackboard) on which characters or figures can be drawn with the electronic pen 13 and the finger 14. Surface).

The electronic pen 13 is a pen-like object held by a user, and is used as an indicator that emits infrared light in this embodiment.
The electronic pen 13 has a light emitting unit 21. The light emitting unit 21 emits infrared light.
In addition, the electronic pen 13 includes a light receiving unit (not shown) that receives infrared light, and an operation (pressing operation) in which the tip of the electronic pen 13 is pressed against another object (projection surface 12 in the present embodiment). A pressing switch for detection and a control unit for performing various controls in the electronic pen 13 are provided.
The electronic pen 13 receives a synchronization signal transmitted by infrared light from the projector 11 by the light receiving unit, and transmits infrared light from the light emitting unit 21 in a predetermined light emission pattern (light emission sequence) in synchronization with the synchronization signal. . The light emission pattern may include identification information of the electronic pen 13. The electronic pen 13 changes the emission pattern of the infrared light emitted by the light emitting unit 21 when a pressing operation is detected by the pressing switch. As a result, the electronic pen 13 transmits infrared light in synchronization with the projector 11, and when the electronic pen 13 is held by the user and a pressing operation is performed on the projection surface 12, the pressing operation is performed. Infrared light having a light emission pattern shown is transmitted. By the pressing operation, the user can, for example, draw a character or a figure on the projection surface 12, or designate (click) an arbitrary item from the menu displayed on the projection surface 12. It is.

As an example, the electronic pen 13 transmits infrared light at a periodic timing (timing at a constant time interval) in synchronization with a synchronization signal received from the projector 11, and when the electronic pen 13 is not pressed. Infrared light having a different light emission pattern is transmitted when the pressing operation is performed.
As another configuration example, the electronic pen 13 does not emit infrared light (not transmitted) when not pressed, and emits (transmits) infrared light only when pressed. Good. Also in this configuration, when the electronic pen 13 transmits infrared light, the projector 11 can detect the presence of the electronic pen 13 by the infrared light.
The electronic pen 13 may include an operation unit such as a predetermined button, for example. The electronic pen 13 may transmit infrared light having a light emission pattern different from other times when each button is operated. In this case, the projector 11 can detect that an operation unit such as each button of the electronic pen 13 is operated by the infrared light.

The finger 14 is a user's finger and is used as an indicator that does not emit light in the present embodiment.
The finger 14 can, for example, draw a character or a figure on the projection surface 12 or indicate (click) an arbitrary item from the menu displayed on the projection surface 12.

The projector 11 will be described.
The projector 11 includes a projection unit 31, a first visible light camera 32, a second visible light camera 33, an infrared camera 34, an infrared communication unit 35, a storage unit 36, and a control unit 37. . The control unit 37 includes a first detection unit 51 and a second detection unit 52.

  The projection unit 31 projects an image to be displayed on the projection surface 12. Thereby, the projected image is displayed on the projection surface 12. The display target image may be an arbitrary image, and may include, for example, a menu image that allows the user to specify an item from among a plurality of items.

  Specifically, the projection unit 31 includes a liquid crystal panel, a light source, and a projection lens. The projection unit 31 converts the signal of the image to be displayed into image information representing the gradation of each pixel of the liquid crystal panel for each of RGB (Red, Green, Blue), and a drive voltage corresponding to the image information. Is applied (driven) to each pixel of the liquid crystal panel. By this application, the liquid crystal panel forms light incident on the liquid crystal panel from the light source as light (modulated light) that displays an image corresponding to the image information described above. The projection unit 31 includes a reflector that reflects light emitted from the light source toward the liquid crystal panel, and causes the reflected light to be formed on the liquid crystal panel as light that displays an image corresponding to the image information described above. The projection unit 31 projects light that projects an image formed on the liquid crystal panel through a projection lens.

  Here, the light source is, for example, a discharge-type light source lamp including an ultra-high pressure mercury lamp or a metal halide lamp, but is not limited to a light source lamp, and an LED (Light Emitting Diode) light source or a laser light source may be used. . The liquid crystal panel is, for example, a transmissive liquid crystal panel corresponding to each of RGB, which is the three primary colors of light, in which liquid crystal is sealed between a pair of transparent substrates. The liquid crystal panel is not limited to a transmissive liquid crystal panel, and a reflective liquid crystal panel may be used. Moreover, as a light valve of the projection part 31, the structure provided with DMD (Digital Mirror Device) etc. may be used instead of the structure provided with a liquid crystal panel.

The first visible light camera 32, the second visible light camera 33, and the infrared camera 34 are installed so that the area including the projection surface 12 is an area to be imaged (imaging area).
The first visible light camera 32 has an element that detects visible light and is a camera that captures visible light (visible light image).
The second visible light camera 33 has an element that detects visible light, and is a camera that captures visible light (visible light image).
In the present embodiment, the first visible light camera 32 and the second visible light camera 33 constitute a stereo camera. With the function of the stereo camera, a target object can be imaged simultaneously from a plurality of different directions, and three-dimensional information can be obtained based on the imaging result (image). In the function of the stereo camera, for example, triangulation is used.
As the first visible light camera 32 and the second visible light camera 33, for example, the same camera (camera having the same performance) may be used, or cameras having different performance may be used. Good.
In the present embodiment, two visible light cameras (the first visible light camera 32 and the second visible light camera 33) are used, but three or more visible light cameras may be used.

The infrared camera 34 has an element that detects infrared light, and is a camera that captures infrared light (an infrared light image).
In the present embodiment, one infrared camera (infrared camera 34) is used, but two or more infrared cameras may be used.

Here, in the present embodiment, the images captured by the first visible light camera 32 and the second visible light camera 33 may include an image of the projection surface 12, an image of the electronic pen 13, and an image of the finger 14. .
In the present embodiment, the image captured by the infrared camera 34 may include an infrared light image transmitted from the light emitting unit 21 of the electronic pen 13.

In addition, the coordinates of the image captured by the first visible light camera 32 and the second visible light camera 33 and the coordinates of the image projected by the projection unit 31 and displayed on the projection surface 12 are associated with each other. Yes. This association is set in the storage unit 36 after being calibrated in advance, for example.
Similarly, the coordinates of the image captured by the infrared camera 34 and the coordinates of the image projected by the projection unit 31 and displayed on the projection surface 12 are associated with each other. This association is set in the storage unit 36 after being calibrated in advance, for example.

The infrared communication unit 35 transmits a predetermined synchronization signal with infrared light. In the present embodiment, the electronic pen 13 receives the synchronization signal and synchronizes with the synchronization signal. Thereby, the projector 11 and the electronic pen 13 synchronize and communicate infrared light.
The storage unit 36 stores various types of information. In the present embodiment, the storage unit 36 stores information such as a program for operating the control unit 37 and various parameters.
The control unit 37 performs various controls. In the present embodiment, the control unit 37 has a CPU (Central Processing Unit), and performs various controls by executing a program stored in the storage unit 36.

Here, the projector 11 may include an input unit, for example. The input unit has an operation unit such as a touch panel or a key. 1 or more.
The projector 11 may include an output unit, for example. The output unit includes one or more of a function for displaying information on a screen such as a liquid crystal display, a function for outputting sound information, and a function for outputting information to another device.
The projector 11 may be connected to a terminal device such as a personal computer (PC). An image to be displayed may be input from the terminal device to the projector 11. Various information may be output from the projector 11 to the terminal device.
The projector 11 may be a rear projection (rear projection) type projector.

In the present embodiment, the control unit 37 of the projector 11 has a function (interactive function) for executing processing according to an instruction from an indicator (in this embodiment, the electronic pen 13 and the finger 14).
In the present embodiment, such an interactive function is provided in the control unit 37 of the projector 11. However, as another configuration example, when a terminal device is connected to the projector 11 and communicates with each other, such an interactive function is provided. Some or all of the interactive functions may be provided in the terminal device.

Next, the first detection unit 51 and the second detection unit 52 will be described.
The first detection unit 51 detects three-dimensional information of an object included in these images based on an image captured by the first visible light camera 32 and an image captured by the second visible light camera 33. .
The 1st detection part 51 performs the detection regarding a non-light-emitting type indicator, and the detection regarding a self-light-emitting type indicator.

In the present embodiment, the first detection unit 51 detects the position of the finger 14. Further, the first detection unit 51 detects a temporal change in the position of the finger 14 as an operation of the finger 14.
For example, the first detection unit 51 detects the three-dimensional position of the finger 14 with respect to the projection plane 12. The first detection unit 51 can detect whether or not the finger 14 is in contact with the projection surface 12 based on the three-dimensional position of the finger 14 with respect to the projection surface 12. Further, the first detection unit 51 can detect a change (operation) of the position of the finger 14 in a state where the finger 14 is in contact with the projection surface 12, and the finger 14 is in contact with the projection surface 12. It is possible to detect a change (motion) of the position of the finger 14 in a state where the finger 14 is not.

  Thereby, for example, the first detection unit 51 detects an operation in which the position of the finger 14 changes while the finger 14 is in contact with the projection surface 12, thereby bringing the finger 14 and the projection surface 12 into contact with each other. Characters or figures drawn on the projection surface 12 can be detected by the finger 14. In addition, drawing of a character, a figure, etc. may include deleting a character, a figure, etc. by the function of an eraser. Further, the first detection unit 51 displays, for example, on the projection plane 12 by detecting an operation in which the finger 14 is not in contact with a position of an arbitrary item displayed on the projection plane 12. It is possible to detect that an arbitrary item that has been designated is instructed by the finger 14. For example, the first detection unit 51 can detect a hovering operation in which the position of the finger 14 changes in a state where the finger 14 is not in contact with the projection surface 12 and is separated from the projection surface 12.

As a specific example, the first detection unit 51 can detect that a specific item is instructed by the finger 14 from a menu including a plurality of items displayed on the projection surface 12.
Further, the first detection unit 51 can detect that a specific figure (an example of an item) displayed on the projection plane 12 is instructed by the finger 14. As the specific figure, for example, a QR code (registered trademark) which is a matrix type two-dimensional code may be used.
In addition, when a specific part displayed on the projection plane 12 is instructed by the finger 14, the first detection unit 51 can detect the color of the specific part and accept the designation of the color. In this case, for example, the first detection unit 51 may select a specific part based on one or both of an image captured by the first visible light camera 32 and an image captured by the second visible light camera 33. The color may be detected. In addition, for example, the first detection unit 51 performs switching between a mode for accepting color designation and another mode in accordance with a predetermined operation of the finger 14 (for example, an instruction for an item), and a mode for accepting color designation. Color designation may be accepted at some (only) time.

  Here, the target pointed by the finger 14 may be other than a part of the image displayed on the projection surface 12, for example. The target may be a part of an arbitrary object (here, an object other than the projection surface 12) imaged by the first visible light camera 32 and the second visible light camera 33. As a specific example, when a color is specified, the target is a part of human clothes or a corresponding color part in a palette (multiple color swatches) attached to a human arm or the like. There may be.

In the present embodiment, the first detection unit 51 can also detect the position of the electronic pen 13. The first detection unit 51 can also detect a temporal change in the position of the electronic pen 13 as an operation of the electronic pen 13.
For example, the first detection unit 51 detects the three-dimensional position of the electronic pen 13 with respect to the projection plane 12.
Thereby, in the present embodiment, it is possible to detect the same position and motion (for example, motion such as hovering) as that of the finger 14 for the electronic pen 13.

  However, in the present embodiment, whether or not the finger 14 is in contact with the projection surface 12 (whether or not a pressing operation is performed) is captured by the first visible light camera 32 and the second visible light camera 33. In contrast to the detection (determination) by the first detection unit 51 based on the image to be detected, whether the tip of the electronic pen 13 is in contact with the projection surface 12 (whether the pressing operation is performed) is red. Detection (determination) is performed by the second detection unit 52 based on an image captured by the outer camera 34.

Here, in this embodiment, the detection of the pressing operation of the electronic pen 13 is more infrared than the accuracy of detection based on the images picked up by the first visible light camera 32 and the second visible light camera 33. It is assumed that the accuracy of detection based on the image captured by the camera 34 is better. In the present embodiment, one reason for this is to increase the brightness of the infrared light image included in the image captured by the infrared camera 34 by transmitting infrared light from the electronic pen 13. It is assumed that it is possible. For this reason, in this embodiment, the structure which detects the pressing operation of the electronic pen 13 based on the image imaged with the infrared camera 34 is employ | adopted, and the precision of the detection is improved. In the present embodiment, for example, it is assumed that a finer or more accurate drawing or instruction can be performed by the operation of the electronic pen 13 compared to the operation of the finger 14.
As another configuration example, the first detection unit 51 captures an image of the pressing operation of the electronic pen 13 by the first visible light camera 32 and the second visible light camera 33 similarly to the pressing operation of the finger 14. You may detect based on the image to be performed.

The first detection unit 51 may detect an image of the finger 14 included in images captured by the first visible light camera 32 and the second visible light camera 33 using a pattern matching method. Further, the image of the finger 14 may be followed.
Further, the first detection unit 51 may detect an image of the electronic pen 13 included in images captured by the first visible light camera 32 and the second visible light camera 33 using a pattern matching method. Alternatively, the image of the electronic pen 13 may be followed.
Moreover, the 1st detection part 51 is contained in the image of the electronic pen 13 contained in the image imaged with the 1st visible light camera 32 and the 2nd visible light camera 33 using the pattern matching method, and an image. The image of the finger 14 may be identified (specified).

  Further, the first detection unit 51 detects the color of the image of the object included in the images captured by the first visible light camera 32 and the second visible light camera 33, and relates to the object based on the color. Information may be detected. For example, the first detection unit 51 can detect (determine) that the object is the finger 14 or the electronic pen 13 based on the color of the image of the object. .

The second detection unit 52 detects information on infrared light included in the image based on the image captured by the infrared camera 34.
The 2nd detection part 52 performs the detection regarding a self-light-emitting indicator.
In the present embodiment, the position of infrared light transmitted from the electronic pen 13 (an example of a self-luminous indicator) is regarded as the position of the electronic pen 13. As another configuration example, when the entire image of the electronic pen 13 is detected by pattern matching or the like, a position other than the infrared light transmitted from the electronic pen 13 is regarded as the position of the electronic pen 13. Good. In this case, the position of the electronic pen 13 may be detected together with the position of the electronic pen 13.

  In the present embodiment, the second detection unit 52 detects the position of the electronic pen 13. The second detection unit 52 detects a temporal change in the position of the electronic pen 13 as an operation of the electronic pen 13. In the present embodiment, the second detector 52 receives infrared light transmitted from the electronic pen 13 at periodic timing, and detects the position or operation of the electronic pen 13 based on the infrared light. In addition, it is detected whether or not the electronic pen 13 is pressed based on the infrared light emission pattern.

  The second detection unit 52 can detect (determine) whether or not the tip of the electronic pen 13 is in contact with the projection surface 12 (whether or not a pressing operation is performed). In the present embodiment, the second detection unit 52 has a predetermined light emission pattern (light emission pattern corresponding to the pressing operation) transmitted from the light emitting unit 21 of the electronic pen 13 to a plurality of images sequentially captured by the infrared camera 34. If the image of infrared light is included, the electronic pen 13 is detected as being pressed on the projection surface 12. The second detection unit 52 can detect a change (operation) of the position of the electronic pen 13 in a state where the tip of the electronic pen 13 is in contact with the projection surface 12.

  Accordingly, the second detection unit 52 detects, for example, an operation in which the position of the electronic pen 13 changes in a state where the tip of the electronic pen 13 is in contact with the projection surface 12, so that the electronic pen 13, the projection surface 12, and the like are detected. A character or a figure drawn on the projection plane 12 by the electronic pen 13 can be detected. In addition, drawing of a character, a figure, etc. may include deleting a character, a figure, etc. by the function of an eraser. In addition, the second detection unit 52 detects, for example, an operation that changes from a state in which the tip of the electronic pen 13 is not in contact with a position of an arbitrary item displayed on the projection surface 12 to thereby change the projection surface. It is possible to detect that any item displayed on the screen 12 is instructed by the electronic pen 13.

As a specific example, the second detection unit 52 can detect that a specific item is instructed by the electronic pen 13 from a menu including a plurality of items displayed on the projection surface 12.
In addition, the second detection unit 52 can detect that a specific figure (an example of an item) displayed on the projection surface 12 is instructed by the electronic pen 13. As the specific figure, for example, a QR code (registered trademark) which is a matrix type two-dimensional code may be used.

  Here, the target to be instructed by the electronic pen 13 may be other than a part of the image displayed on the projection surface 12, for example. The target may be a part of an arbitrary object (here, an object other than the projection surface 12) imaged by the first visible light camera 32 and the second visible light camera 33. As a specific example, when a color is designated, the target may be a part of human clothes or a corresponding color part in a pallet attached to a human arm or the like. In the present embodiment, the electronic pen 13 transmits infrared light corresponding to the pressing operation from the light emitting unit 21 even when the tip of the electronic pen 13 contacts an object other than the projection surface 12. Thereby, the 2nd detection part 52 can detect that the tip of the electronic pen 13 contacted arbitrary objects, and was pressed.

  The second detection unit 52 includes, for example, an operation unit such as a predetermined button on the electronic pen 13, and infrared light having a predetermined light emission pattern is transmitted from the electronic pen 13 according to the operation of the operation unit. In this case, it may be detected that a predetermined operation has been performed in response to detecting that the infrared light has been received. As the content of the predetermined operation, for example, switching between a mode for accepting a color designation and another mode may be used.

In this embodiment, the 1st detection part 51 and the 2nd detection part 52 may detect the information regarding the electronic pen 13 or the finger | toe 14 based on the information which combined each detection result. For example, for each object detected by the first detection unit 51, the second detection unit 52 detects (determines) whether infrared light is transmitted from the object, and based on this result, each object is detected. Can be detected (determined) whether it is an object that emits light (self-luminous object) or an object that does not emit light (non-luminous object).
In the present embodiment, after the electronic pen 13 is detected by the first detection unit 51, the first detection unit 51 detects the electronic pen 13 even if infrared light from the electronic pen 13 is not detected by the second detection unit 52. It is possible to detect the position and movement of the pen 13 (for example, movement such as hovering). In this case, the periodic timing (timing at a constant time interval) for transmitting infrared light from the electronic pen 13 may be a long-period timing. In this case, as another configuration example, when the electronic pen 13 is not pressed, a configuration in which the electronic pen 13 does not transmit infrared light may be used.

The control unit 37 executes processing corresponding to the detection result based on the detection result of one or both of the first detection unit 51 and the second detection unit 52.
For example, the control unit 37 superimposes a character or a graphic drawn on the projection surface 12 by the electronic pen 13 or the finger 14 on the display target image, and thereby the character or the graphic is displayed on the display target image. Can be projected onto the projection plane 12 by the projection unit 31. In addition, the control unit 37 can select an arbitrary item (item of the item) from the menu (menu image) displayed on the projection surface 12 by being superimposed on the image to be displayed by the electronic pen 13 or the finger 14. When an (image) is instructed, processing corresponding to the item can be executed. In addition, for example, when a specific graphic is instructed by the electronic pen 13 or the finger 14, the control unit 37 can execute processing corresponding to the graphic. Further, for example, when a specific color is instructed (designated) by the electronic pen 13 or the finger 14, the control unit 37 can execute processing corresponding to the color. As a process corresponding to the color, for example, a process using the color for drawing a character or a graphic may be used. Note that the control unit 37 may accept a color designation according to the type of the indicator (non-light-emitting type or self-light-emitting type). Further, the control unit 37 may accept designation of the color according to the color of the indicator.

As described above, in the projector 11 according to the present embodiment, both the indicator that emits infrared light (in this embodiment, the electronic pen 13) and the indicator that does not emit light itself (in this embodiment, the finger 14). Can be detected.
For example, in the configuration using only the visible light camera (in the present embodiment, the first visible light camera 32 and the second visible light camera 33) without using the infrared camera 34, the infrared light is detected. Therefore, the electronic pen 13 capable of drawing with high accuracy cannot be used. On the other hand, in the projector 11 according to the present embodiment, the electronic pen 13 can be used as the indicator by using the infrared camera 34.

  Further, for example, in order to install a light curtain device that irradiates infrared light so that the finger 14 can be used as an indicator, the equipment (light curtain device) is required. Installation space is required, and the power consumed by the equipment is required. On the other hand, in the projector 11 according to the present embodiment, a light curtain device is installed by using a visible light camera (in the present embodiment, the first visible light camera 32 and the second visible light camera 33). Even if it does not do, it is possible to use the finger 14 as an indicator. For this reason, in this embodiment, it is possible to eliminate the limitation of the installation space etc. which arises when installing the light curtain device, and it is possible to save the power required for the light curtain device. However, in the interactive system 1 according to the present embodiment, a light curtain device may be installed and used.

  In the projector 11 according to the present embodiment, the aspect of the electronic pen 13 is based on an image captured by the visible light camera (in the present embodiment, the first visible light camera 32 and the second visible light camera 33). By detecting a change in the position of the electronic pen 13 by detecting (for example, a shape, a color, or a pattern), it is possible to detect an operation such as hovering of the electronic pen 13. In this case, the projector 11 can detect the operation without transmitting infrared rays from the electronic pen 13 during an operation such as hovering of the electronic pen 13. For this reason, by reducing the frequency of infrared transmission from the electronic pen 13 in such an operation or by eliminating the transmission, the power consumption of the electronic pen 13 is reduced and the battery life of the electronic pen 13 is reduced. It can be extended significantly.

  As described above, in the projector 11 according to the present embodiment, combined detection by the visible light stereo camera (in the present embodiment, the first visible light camera 32 and the second visible light camera 33) and the infrared camera 34 is performed. By performing the above, it is possible to make both the operation with the finger 14 and the operation with the electronic pen 13 compatible, and power saving can be achieved.

As an example of the configuration, in the projector 11, a projection unit 31 that projects an image on a projection surface, and a first imaging unit that captures visible light using an area including the projection surface 12 as an imaging region (in the present embodiment, a first imaging unit). Of the visible light camera 32) and the second imaging unit (in this embodiment, the second visible light camera 33), and a third imaging unit that images infrared light using the region including the projection surface 12 as an imaging region. (In the present embodiment, the position of the non-luminous indicator (in this embodiment, the finger 14) is detected based on the imaging results of the infrared camera 34) and the first imaging unit and the second imaging unit. Based on the imaging results of the first detection unit 51 and the third imaging unit, the position of a self-luminous indicator (in this embodiment, the electronic pen 13) having a light emitting unit 21 that emits infrared light is detected. 2 detection unit 52.
As one configuration example, in the detection method (in this embodiment, the detection method performed in the projector 11), the projector 11 projects an image on the projection surface 12, and the first imaging unit and the second imaging unit project the image. The area including the surface 12 is defined as an imaging area, and visible light is imaged. The third imaging unit defines the area including the projection surface 12 as an imaging area, images infrared light, and includes the first imaging unit and the second imaging unit. Based on the imaging result by the imaging unit, the position of the non-luminous indicator is detected, and based on the imaging result by the third imaging unit, the position of the self-luminous indicator having a light emitting unit that emits infrared light is detected. To do.

As an example of the configuration, the projector 11 is instructed by a non-luminous indicator or a self-luminous indicator based on at least one of an imaging result by the first imaging unit and an imaging result by the second imaging unit. You may further provide the color detection part which detects the color of a part. The color detection unit may be provided in the control unit 37, for example.
As an example of the configuration, the projector 11 is instructed by a non-luminous indicator or a self-luminous indicator based on at least one of an imaging result by the first imaging unit and an imaging result by the second imaging unit. You may further provide the figure detection part which detects the figure of a part. The graphic detection unit may be provided in the control unit 37, for example.
As an example of the configuration, the projector 11 further includes an aspect detection unit that detects an aspect of the self-luminous indicator based on at least one of the imaging result by the first imaging unit and the imaging result by the second imaging unit. You may prepare. The aspect detection unit may be provided in the control unit 37, for example.

(Second Embodiment)
FIG. 2 is a diagram illustrating a schematic configuration example of the interactive system 101 according to an embodiment (second embodiment) of the present invention.
The interactive system 101 includes a projector 111, a projection surface 12, and an electronic pen 13. The electronic pen 13 has a light emitting unit 21. FIG. 2 also shows a human finger 14.
Here, the projection surface 12, the electronic pen 13 having the light emitting unit 21, and the human finger 14 are the same as those shown in FIG.

The projector 111 will be described.
The projector 111 includes a projection unit 131, a visible light camera 132, a visible light / infrared camera 133, an infrared communication unit 134, a storage unit 135, and a control unit 136. The control unit 136 includes a first detection unit 151 and a second detection unit 152.
Here, in the present embodiment, the outline of the projection unit 131, the infrared communication unit 134, the storage unit 135, and the control unit 136 (points for performing various controls) are respectively the projection unit 31 and the red unit illustrated in FIG. This is the same as the outline of the external communication unit 35, the storage unit 36, and the control unit 37 (points where various controls are performed).

The visible light camera 132 and the visible light / infrared camera 133 are installed so that the area including the projection plane 12 is an area to be imaged (imaging area).
The visible light camera 132 has a function similar to that of the first visible light camera 32 shown in FIG.
The visible light / infrared camera 133 is a camera capable of switching between a visible light camera function and an infrared camera function. In this embodiment, the visible light / infrared camera 133 includes a visible light camera and a predetermined filter (for example, a filter that blocks visible light and allows infrared light to pass). When it is removed, it has a function of a visible light camera, and when the filter is applied to the visible light camera, it has a function of an infrared camera. In the present embodiment, the control unit 136 performs control to switch the state of the filter of the visible light / infrared camera 133.

  Here, in a state where the visible light / infrared camera 133 has a function of a visible light camera, the visible light / infrared camera 133 plays the same role as the second visible light camera 33 shown in FIG. In this state, the first detection unit 151 is based on the image captured by the visible light camera 132 and the image captured by the visible light / infrared camera 133 (the function of the visible light camera). The same detection as that of the first detection unit 51 is performed.

  In the state where the visible light / infrared camera 133 has the function of an infrared camera, the visible light / infrared camera 133 plays the same role as the infrared camera 34 shown in FIG. In this state, the second detection unit 152 performs the same detection as the second detection unit 52 shown in FIG. 1 based on the image captured by the visible light / infrared camera 133 (infrared camera function). .

As described above, the projector 111 according to the present embodiment can obtain the same effects as those of the projector 11 shown in FIG.
Further, since the projector 111 according to the present embodiment uses the visible light / infrared camera 133, for example, the number of cameras is reduced compared to the case where the visible light camera and the infrared camera are separately provided, and the projector It is possible to reduce the size.

  As an example of the configuration, in the projector 111, at least one of the first imaging unit (imaging unit that captures visible light) and the second imaging unit (imaging unit that captures visible light) (one in the present embodiment) Only) is an imaging unit (in this embodiment, visible light / infrared camera 133) that can switch between a function for imaging visible light and a function for imaging infrared light, and a third imaging unit (red). An imaging unit that images external light is also configured.

(Third embodiment)
FIG. 3 is a diagram illustrating a schematic configuration example of an interactive system 201 according to an embodiment (third embodiment) of the present invention.
The interactive system 201 includes a projector 211, a projection surface 12, and an electronic pen 13. The electronic pen 13 has a light emitting unit 21. FIG. 3 also shows a human finger 14.
Here, the projection surface 12, the electronic pen 13 having the light emitting unit 21, and the human finger 14 are the same as those shown in FIG.

The projector 211 will be described.
The projector 211 includes a projection unit 231, a first visible light / infrared camera 232, a second visible light / infrared camera 233, an infrared communication unit 234, a storage unit 235, and a control unit 236. . The control unit 236 includes a first detection unit 251 and a second detection unit 252.
Here, in the present embodiment, the outline of the projection unit 231, the infrared communication unit 234, the storage unit 235, and the control unit 236 (points where various controls are performed) are the projection unit 31 and red shown in FIG. This is the same as the outline of the external communication unit 35, the storage unit 36, and the control unit 37 (points where various controls are performed).

The first visible light / infrared camera 232 and the second visible light / infrared camera 233 are installed so that the area including the projection plane 12 is an area to be imaged (imaging area).
The first visible light / infrared camera 232 is a camera capable of switching between a visible light camera function and an infrared camera function. In the present embodiment, the control unit 236 performs control to switch the state of the first visible light / infrared camera 232 (a state having a visible light camera function and a state having an infrared camera function).
Similarly, the second visible light / infrared camera 233 is a camera capable of switching between a visible light camera function and an infrared camera function. In the present embodiment, the control unit 236 performs control to switch the state of the second visible light / infrared camera 233 (a state having a function of a visible light camera and a state having a function of an infrared camera).
The configuration for switching the state of each visible light / infrared camera 232, 233 is the same as the configuration for switching the state of the visible light / infrared camera 133 shown in FIG.

  Here, in the state where both of the two visible light / infrared cameras 232 and 232 have the function of the visible light camera, the visible light / infrared cameras 232 and 233 have the two visible lights shown in FIG. It plays the same role as the optical camera (the first visible light camera 32 and the second visible light camera 33). In this state, the first detection unit 251 detects an image captured by the visible / infrared camera 232 (function of visible light camera) and an image captured by the visible / infrared camera 233 (function of visible light camera). Based on the above, the same detection as that of the first detection unit 51 shown in FIG. 1 is performed.

  In the state where one of the two visible light / infrared cameras 232 and 232 has the function of an infrared camera, the visible light / infrared camera (one of the visible light / infrared cameras 232 and 232 is displayed). ) Plays the same role as the infrared camera 34 shown in FIG. In this state, the second detection unit 252 performs detection similar to the second detection unit 52 shown in FIG. 1 based on the image captured by the visible light / infrared camera (function of the infrared camera). .

  In the state where both of the two visible light / infrared cameras 232 and 232 have the function of the infrared camera, the two visible light / infrared cameras 232 and 232 (functions of the infrared camera) can be used as a stereo camera. And three-dimensional information about infrared light is obtained. In this state, for example, the second detection unit 252 captures an image captured by the visible light / infrared camera 232 (infrared camera function) and an image captured by the visible light / infrared camera 233 (infrared camera function). Based on the obtained image, three-dimensional information related to infrared light transmitted from the electronic pen 13 may be detected. The second detection unit 252 may use this detection information when detecting information about the electronic pen 13.

As described above, the projector 211 according to the present embodiment can obtain the same effects as those of the projector 11 shown in FIG.
Further, since the projector 211 according to the present embodiment uses the visible light / infrared cameras 132 and 133, for example, the number of cameras is reduced as compared with the case where the visible light camera and the infrared camera are provided separately. It is possible to reduce the size of the projector.
In the projector 211 according to the present embodiment, it is possible to detect three-dimensional information about infrared light, and it is possible to improve the detection accuracy regarding the electronic pen 13.

  As a configuration example, in the projector 211, both the first imaging unit (imaging unit that captures visible light) and the second imaging unit (imaging unit that captures visible light) have a function of capturing visible light and red. An imaging unit (in this embodiment, visible light / infrared cameras 232 and 233) that can switch the function of imaging external light, and a third imaging unit (imaging unit that captures infrared light) are also configured. To do.

(Fourth embodiment)
FIG. 4 is a diagram illustrating a schematic configuration example of an interactive system 301 according to an embodiment (fourth embodiment) of the present invention.
The interactive system 301 includes a projector 311, a projection surface 12, and an electronic pen 13. The electronic pen 13 has a light emitting unit 21. FIG. 4 also shows a human finger 14.
Here, the projection surface 12, the electronic pen 13 having the light emitting unit 21, and the human finger 14 are the same as those shown in FIG.

The projector 311 will be described.
The projector 311 includes a projection unit 331, a first visible light camera 332, a second visible light camera 333, an infrared camera 334, a thermography camera 335, an infrared communication unit 336, a storage unit 337, A control unit 338 is provided. The control unit 338 includes a first detection unit 351 and a second detection unit 352.
Here, in the present embodiment, the projection unit 331, the first visible light camera 332, the second visible light camera 333, the infrared camera 334, the infrared communication unit 336, the storage unit 337, and the control The outline of the unit 338 (points for performing various controls) are respectively the projection unit 31, the first visible light camera 32, the second visible light camera 33, the infrared camera 34, and the like shown in FIG. This is the same as the outline of the infrared communication unit 35, the storage unit 36, and the control unit 37 (points where various controls are performed).

The first visible light camera 332, the second visible light camera 333, the infrared camera 334, and the thermography camera 335 are installed so that the area including the projection plane 12 is an area to be imaged (imaging area). Yes.
The thermography camera 335 detects infrared light (infrared rays) emitted from the human body (in this embodiment, the finger 14) based on the body temperature of the human body and captures an image of the infrared light (heat distribution image). Camera. The thermography camera 335 can be said to be a kind of infrared camera, but is provided separately from the infrared camera 334 in this embodiment.

  As with the first detection unit 51 shown in FIG. 1, the first detection unit 351 is based on the image captured by the first visible light camera 332 and the image captured by the second visible light camera 333. It has a function of detecting three-dimensional information of an object included in these images. Furthermore, in the present embodiment, the first detection unit 351 can use an image captured by the thermography camera 335 when performing detection related to a human or the like (in this embodiment, the finger 14). The first detection unit 351 uses, for example, an image captured by the thermographic camera 335 together with an image captured by the first visible light camera 332 and an image captured by the second visible light camera 333, so that a human It is possible to increase the accuracy of detection related to the above.

As a specific example, the first detection unit 351 detects an image of a person or the like based on an image captured by the thermography camera 335, thereby enabling a visible light camera (in the present embodiment, the first visible light camera 332 and the first visible light camera 332). Detecting (determining) whether the object detected based on the image captured by the second visible light camera 333) is a human or the like or an object having no body temperature (in this embodiment, the electronic pen 13). Can do. In this case, for example, by reducing the frequency of infrared transmission from the electronic pen 13 or eliminating the transmission, the power consumption of the electronic pen 13 is reduced and the battery life of the electronic pen 13 is greatly extended. It is possible.
In addition, when the first detection unit 351 detects the image of the finger 14 included in the images captured by the first visible light camera 332 and the second visible light camera 333 using a pattern matching method, By using the image captured by the thermography camera 335, the accuracy of pattern matching can be increased.

As described above, the projector 311 according to the present embodiment can obtain the same effects as those of the projector 11 shown in FIG.
Further, in the projector 311 according to the present embodiment, it is possible to improve the accuracy of detection related to a human or the like by using an image captured by the thermography camera 335.

  Here, in this embodiment, the case where the function of the thermography camera 335 is provided in the same configuration as the interactive system 1 shown in FIG. 1 is shown, but as another configuration example, the interactive system 101 shown in FIG. A thermographic camera function may be provided in a similar configuration, or a thermographic camera function may be provided in a configuration similar to the interactive system 201 illustrated in FIG.

  As an example of the configuration, the projector 311 includes a thermography camera 335 that captures an image of the heat distribution in the imaging region, and the first detection unit 351 includes a first imaging unit (in this embodiment, the first visible light camera). 332), based on the imaging results of the second imaging unit (second visible light camera 333 in this embodiment) and the thermography camera 335, the position of the non-luminous indicator (finger 14 in this embodiment) is determined. To detect.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

Note that a program for realizing the function of an arbitrary component in the apparatus described above (for example, the projector 11, 111, 211, 311) is recorded (stored) in a computer-readable recording medium (storage medium). The program may be loaded into a computer system and executed. The “computer system” here includes an operating system (OS) or hardware such as peripheral devices. The “computer-readable recording medium” means a portable medium such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory) and a CD (Compact Disk) -ROM, and a hard disk built in the computer system. Refers to the device. Furthermore, “computer-readable recording medium” means a volatile memory (RAM: Random Access) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. As in the case of (Memory), a program that holds a program for a certain period of time is also included.
In addition, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the functions described above. Further, the program may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 1, 101, 201, 301 ... Interactive system 11, 111, 211, 311 ... Projector, 12 ... Projection surface, 13 ... Electronic pen, 14 ... Finger, 21 ... Light emission part, 31, 131, 231, 331 ... Projection part 32, 33, 132, 332, 333 ... Visible light camera, 34, 334 ... Infrared camera, 35, 134, 234, 336 ... Infrared communication unit, 36, 135, 235, 337 ... Storage unit, 37, 136 236, 338: control unit 51, 151, 251 and 351 ... first detection unit, 52, 152, 252, 352 ... second detection unit, 133, 232, 233 ... visible light / infrared camera, 335 ... thermography camera

Claims (7)

A projection unit that projects an image on a projection surface;
A region including the projection plane as an imaging region, a first imaging unit and a second imaging unit that capture visible light;
A region including the projection plane as an imaging region, a third imaging unit that images infrared light;
A first detection unit for detecting a position of a non-light emitting indicator based on an imaging result by the first imaging unit and the second imaging unit;
A second detection unit that detects a position of a self-luminous indicator having a light-emitting unit that emits infrared light based on an imaging result of the third imaging unit;
A projector comprising: At least one of the first imaging unit and the second imaging unit is an imaging unit capable of switching a function of imaging visible light and a function of imaging infrared light, and the third imaging The part also constitutes,
The projector according to claim 1. Based on at least one of an imaging result by the first imaging unit and an imaging result by the second imaging unit, a color of a portion indicated by the non-light emitting indicator or the self-luminous indicator is detected. A color detecting unit
The projector according to claim 1 or 2. Based on at least one of an imaging result by the first imaging unit and an imaging result by the second imaging unit, a figure of a portion indicated by the non-light emitting indicator or the self-luminous indicator is detected. A graphic detection unit
The projector according to any one of claims 1 to 3. A mode detection unit that detects a mode of the self-luminous indicator based on at least one of an imaging result by the first imaging unit and an imaging result by the second imaging unit;
The projector according to any one of claims 1 to 4. It has a thermographic camera that captures an image of the heat distribution in the imaging area,
The first detection unit detects the position of the non-light emitting indicator based on the imaging results of the first imaging unit, the second imaging unit, and the thermography camera.
The projector according to any one of claims 1 to 5. In the projector,
Project an image on the projection surface,
The first imaging unit and the second imaging unit use the region including the projection plane as an imaging region, and capture visible light,
A third imaging unit sets an area including the projection plane as an imaging area, images infrared light,
Based on the imaging results of the first imaging unit and the second imaging unit, the position of the non-luminous indicator is detected,
Detecting a position of a self-luminous indicator having a light emitting unit that emits infrared light based on an imaging result by the third imaging unit;
Detection method.

Images