JP2018160265A - Position detection system, and control method of position detection system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a position detection system that can accurately detect positions of indication by indicators of different types even with a simple configuration by using photographic image data.SOLUTION: A projection system 1 includes: a light emission device 60 that emits detection light to a detection area for detecting positions of indication by indicators 70 and 80; and a projector 10 that detects the position of indication by the indicators 70 and 80 in the detection area. The projector 10 includes: an imaging unit 51 that photographs the detection area; and a position detection unit 50 that detects, from the photographic image data of the imaging unit 51, at least one of an image of light emitted from the indicator 70 and an image of detection light reflected on the indicator 80, and individually detects the positions of the indication by the indicators 70 and the indicators 80 on the basis of light emission timing of the indicator 70 and the light emission device 60.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a position detection system and a control method for the position detection system.

  A technique for detecting the position of the indicator within the detection region by detecting the change range of the light amount distribution obtained from each of the plurality of sensor means is known (see, for example, Patent Document 1).

Japanese Patent No. 4757144

When optically detecting the indication position of the indicator, light does not necessarily reach uniformly from all locations in the detection region, and this affects the detection accuracy. With respect to this point, in the configuration of Patent Document 1, the accuracy is improved by obtaining the light amount distribution. However, for example, such a method cannot be applied when the detection position is detected by photographing the detection region. . Further, when the indication positions of a plurality of different types of indicators are distinguished and detected, a plurality of sensors must be provided, which increases the size of the apparatus.
The present invention has been made in view of the above circumstances, and a position detection system that can accurately detect the pointing position of different types of pointers using captured image data even with a simple configuration, and An object of the present invention is to provide a control method for a position detection system.

In order to achieve the above object, a position detection system of the present invention detects a light emitting device that emits detection light to a detection region that detects an indication position of an indicator, and detects an indication position of the indicator in the detection region. A position detection system comprising a position detection device, wherein the position detection device is a light emitted from a first indicator as the indicator from an imaging unit that images the detection region and captured image data of the imaging unit. And at least one of the image of the detection light reflected by the second indicator as the indicator, and based on the light emission timing of the first indicator and the light emitting device, the first A position detection unit that distinguishes and detects the pointing positions of the pointer and the second pointer.
According to the configuration of the present invention, at least one of an image of light emitted from the first indicator as the indicator and an image of detection light reflected by the second indicator as the indicator is detected from the captured image data. . Further, the indication positions of the first indicator and the second indicator are distinguished and detected based on the light emission timings of the first indicator and the light emitting device. For this reason, even if it is a simple structure, the indication position of the indicator from which a kind differs can be detected accurately using picked-up image data.

Further, the present invention is characterized in that, in the above configuration, the position detecting unit detects the position of a bright spot reflected in the photographed image data after the light emitting device is turned off as an indication position of the first indicator. To do.
According to the configuration of the present invention, it is possible to easily distinguish between the first indicator and the second indicator, and it is possible to accurately detect the indication positions of different types of indicators.

Further, according to the present invention, in the configuration described above, the position detection unit is configured to detect light that a plurality of first indicators blink according to identification information assigned to each first indicator based on the captured image data. The determination is performed by distinguishing and detecting the indication positions of the plurality of first indicators.
According to the configuration of the present invention, even if there are a plurality of first indicators, it is possible to distinguish and detect the indication position of each first indicator.

According to the present invention, in the configuration described above, the position detection device transmits the detection light to a first transmission unit that transmits a synchronization signal that notifies the lighting timing of the first indicator, and the light emitting device. And a second transmitter that transmits a signal notifying the timing of emission to the light emitting device.
According to this configuration, the position detection device can photograph the detection region with the photographing unit in synchronization with the lighting timing of the first indicator and the light emitting device. For this reason, the indication position of the 1st indicator and the 2nd indicator can be detected.

The control method of the position detection system of the present invention includes a light emitting device that emits detection light to a detection region that detects the indication position of the indicator, and a position detection device that detects the indication position of the indicator in the detection region. A method for controlling a position detection system comprising: a step of photographing the detection area by a photographing unit; an image of light emitted by a first indicator as the indicator from the photographed image data of the photographing unit; At least one of the detection light image reflected by the second indicator serving as the indicator is detected, and the first indicator and the first indicator are detected based on the light emission timings of the first indicator and the light emitting device. And a step of distinguishing and detecting a pointing position with respect to the two pointers.
According to the configuration of the present invention, at least one of an image of light emitted from the first indicator as the indicator and an image of detection light reflected by the second indicator as the indicator is detected from the captured image data. . Further, the indication positions of the first indicator and the second indicator are distinguished and detected based on the light emission timings of the first indicator and the light emitting device. For this reason, even if it is a simple structure, the indication position of the indicator from which a kind differs can be detected accurately using picked-up image data.

  According to the present invention, it is possible to accurately detect the pointing positions of different types of pointers using captured image data even with a simple configuration.

It is a figure which shows schematic structure of a projection system. It is a functional block diagram of a projection system. It is a flowchart which shows operation | movement of a projector. (A) shows a state of detecting an indication position of a pen-type indicator, and (B) is a diagram showing a state of detecting an indication position of a finger as an indicator. It is a figure which shows an example of an auto calibration image. It is a figure which shows an example of the picked-up image data image | photographed with the imaging part. It is a figure which shows an example of a calibration data management table. It is a figure which shows an example of a manual calibration image. It is a flowchart which shows the detail of a background image creation process. It is a figure which shows an example of a mask image. It is a figure for demonstrating the preparation method of a mask image. It is a figure which shows an example of 1st background image data. It is a figure which shows an example of 2nd background image data. It is a flowchart which shows the detail of an instruction | indication position detection process. (A) shows density threshold values set in accordance with the mask image, and (B) is a diagram showing density threshold values at points A to D shown in (A). It is a figure which shows an example of light source noise data. It is a figure which shows the light emission timing of a projector, an indicator, and a light-emitting device.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a configuration of a projection system 1 according to an embodiment to which the present invention is applied. The projection system 1 includes a projector 10 (position detection device) installed above a screen SC (projection surface) and a light emitting device (light source unit) 60 installed on the screen SC.

  The projector 10 is installed directly above or obliquely above the screen SC and projects an image toward the screen SC obliquely below. Further, the screen SC exemplified in the present embodiment is a flat plate or a curtain fixed to a wall surface or standing on a floor surface. The present invention is not limited to this example, and the wall surface can be used as the screen SC. In this case, the projector 10 and the light emitting device 60 may be attached to the upper part of the wall surface used as the screen SC.

The projector 10 is connected to an external image supply device such as a PC (personal computer), a video reproduction device, a DVD reproduction device, etc., and an image is displayed on the screen SC based on an analog image signal or digital image data supplied from the image supply device. Project. The projector 10 may be configured to read out image data stored in a built-in storage unit 110 (FIG. 2) or an externally connected storage medium and display an image on the screen SC based on the image data.
The light emitting device 60 includes a light source unit 61 (FIG. 2) that is a solid light source, and diffuses (emits) light emitted from the light source unit 61 (infrared light in the present embodiment) along the screen SC. . The emission range of the light emitting device 60 is shown in FIG. The light emitting device 60 is installed above the upper end of the screen SC and emits light downward in the range of the angle θ, and this light forms a layer of light along the screen SC. In this embodiment, the angle θ reaches approximately 180 degrees, and a light layer is formed on almost the entire screen SC. The surface of the screen SC and the light layer are preferably close to each other. In this embodiment, the distance between the surface of the screen SC and the light layer is approximately in the range of 10 mm to 1 mm.

The projection system 1 detects the indicated position by the projector 10 when an instruction operation is performed on the screen SC.
A pen-type indicator 70 can be used as the indicator used for the instruction operation. Since the distal end portion 71 of the indicator 70 incorporates an operation switch 75 (FIG. 2) that operates when pressed, the operation switch 75 is turned on when the distal end portion 71 is pressed against the wall or the screen SC. become. The indicator 70 is operated so that the user holds the rod-shaped shaft portion 72 in his / her hand and brings the tip portion 71 into contact with the screen SC, and an operation of pressing the tip portion 71 against the screen SC is also performed. The distal end portion 71 includes a transmission / reception unit 74 (FIG. 2) that emits infrared light. The projector 10 detects the position of the distal end portion 71 as the indication position based on the infrared light emitted from the indicator 70.

Further, when a position indicating operation is performed with the indicator 80 that is a user's finger, the user brings the finger into contact with the screen SC. In this case, the position where the indicator 80 contacts the screen SC is detected. That is, when the tip (for example, fingertip) of the indicator 80 contacts the screen SC, the light layer formed by the light emitting device 60 is blocked. At this time, the light emitted from the light emitting device 60 strikes and reflects the indicator 80, and part of the reflected light travels from the indicator 80 toward the projector 10. Since the projector 10 has a function of detecting light from the screen SC side, that is, light from below by the position detection unit 50 described later, the reflected light of the indicator 80 can be detected. The projector 10 detects an instruction operation on the screen SC by the indicator 80 by detecting the reflected light reflected by the indicator 80. Further, the projector 10 detects the indicated position indicated by the indicator 80.
Since the light layer emitted from the light emitting device 60 is close to the screen SC, the position where the light is reflected by the indicator 80 can be regarded as the tip closest to the screen SC or the indication position. For this reason, the indicated position can be specified based on the reflected light of the indicator 80.

The projection system 1 functions as an interactive whiteboard system, detects an instruction operation performed by the user using the indicators 70 and 80, and reflects the indicated position on the projection image. Specifically, the projection system 1 performs a process of drawing a graphic or placing a character or a symbol at a designated position, a process of drawing a graphic along the locus of the designated position, a drawn figure, or a placed character or symbol. The process etc. which erases are performed. In addition, graphics drawn on the screen SC, arranged characters or symbols can be stored as image data, and can be output to an external device.
Further, it may operate as a pointing device by detecting the designated position, and output the coordinates of the designated position in the image projection area where the projector 10 projects an image on the screen SC. Further, a GUI (Graphical User Interface) operation may be performed using these coordinates.

FIG. 2 is a functional block diagram of each part constituting the projection system 1.
The projector 10 includes an I / F (interface) unit 11 and an image I / F (interface) unit 12 as interfaces connected to external devices. The I / F unit 11 and the image I / F unit 12 may include a connector for wired connection, and may include an interface circuit corresponding to the connector. The I / F unit 11 and the image I / F unit 12 may include a wireless communication interface. Examples of the connector and interface circuit for wired connection include those based on wired LAN, IEEE 1394, USB, and the like. Examples of the wireless communication interface include those that comply with a wireless LAN, Bluetooth (registered trademark), or the like. The image I / F unit 12 may be an image data interface such as an HDMI (registered trademark) interface. The image I / F unit 12 may include an interface through which audio data is input.

The I / F unit 11 is an interface that transmits and receives various data to and from an external device such as a PC. The I / F unit 11 inputs and outputs control data related to image projection, setting data for setting the operation of the projector 10, coordinate data of the designated position detected by the projector 10, and the like. The control unit 30 described later has a function of transmitting / receiving data to / from an external device via the I / F unit 11.
The image I / F unit 12 is an interface through which digital image data is input. The projector 10 of the present embodiment projects an image based on digital image data input via the image I / F unit 12. The projector 10 may have a function of projecting an image based on the analog image signal. In this case, the image I / F unit 12 converts the analog image signal into digital image data and an analog image interface. And an A / D conversion circuit.

The projector 10 includes a projection unit 20 that forms an optical image. The projection unit 20 includes a light source unit 21, a light modulation device 22, and a projection optical system 23. The light source unit 21 includes a light source including a xenon lamp, an ultra high pressure mercury lamp, an LED (Light Emitting Diode), a laser light source, or the like. The light source unit 21 may include a reflector and an auxiliary reflector that guide light emitted from the light source to the light modulation device 22. Further, a lens group (not shown) for enhancing the optical characteristics of the projection light, a polarizing plate, or a dimming element that reduces the amount of light emitted from the light source on the path to the light modulation device 22 may be provided. Good.
The light modulation device 22 includes, for example, three transmissive liquid crystal panels corresponding to the three primary colors of RGB, and generates image light by modulating light transmitted through the liquid crystal panel. The light from the light source unit 21 is separated into three color lights of RGB, and each color light enters each corresponding liquid crystal panel. The color light modulated by passing through each liquid crystal panel is combined by a combining optical system such as a cross dichroic prism and emitted to the projection optical system 23.

  The projection optical system 23 includes a lens group that guides the image light modulated by the light modulation device 22 toward the screen SC and forms an image on the screen SC. Further, the projection optical system 23 may include a zoom mechanism for enlarging / reducing the projected image on the screen SC and adjusting the focus, and a focus adjusting mechanism for adjusting the focus. When the projector 10 is a short focus type, the projection optical system 23 may include a concave mirror that reflects image light toward the screen SC.

  The projection unit 20 is connected to a light source drive unit 45 that turns on the light source unit 21 according to the control of the control unit 30 and a light modulation device drive unit 46 that operates the light modulation device 22 according to the control of the control unit 30. The light source drive unit 45 may have a function of switching on / off the light source unit 21 and adjusting the light amount of the light source unit 21.

  The projector 10 includes an image processing system that processes an image projected by the projection unit 20. The image processing system includes a control unit 30 that controls the projector 10, a storage unit 110, an operation detection unit 17, an image processing unit 40, a light source driving unit 45, and a light modulation device driving unit 46. Further, a frame memory 44 is connected to the image processing unit 40, and an attitude sensor 47, an emission device driving unit (second transmission unit) 48, and a position detection unit 50 are connected to the control unit 30. These units may be included in the image processing system.

  The control unit 30 controls each unit of the projector 10 by executing a predetermined control program 111. The storage unit 110 stores the control program 111 executed by the control unit 30 and the data processed by the control unit 30 in a nonvolatile manner. The storage unit 110 stores setting screen data 115 of a screen for setting the operation of the projector 10 and setting data 116 indicating contents set using the setting screen data 115. In addition to this, the storage unit 110 stores an auto calibration image 121 and a manual calibration image 122. Further, the storage unit 110 stores auto calibration data 123, manual calibration data 124, initial correction data 125, and manual correction data 126. Details of these data will be described later.

The image processing unit 40 processes the image data input via the image I / F unit 12 under the control of the control unit 30, and outputs an image signal to the light modulation device driving unit 46. The processing executed by the image processing unit 40 includes 3D (stereoscopic) image and 2D (planar) image discrimination processing, resolution conversion processing, frame rate conversion processing, distortion correction processing, digital zoom processing, color tone correction processing, luminance correction processing, and the like. is there. The image processing unit 40 executes processing designated by the control unit 30 and performs processing using parameters input from the control unit 30 as necessary. Of course, it is also possible to execute a combination of a plurality of processes.
The image processing unit 40 is connected to the frame memory 44. The image processing unit 40 expands the image data input from the image input I / F 12 in the frame memory 44, and executes the various processes described above on the expanded image data. The image processing unit 40 reads out the processed image data from the frame memory 44, generates R, G, B image signals corresponding to the image data, and outputs them to the light modulation device driving unit 46.
The light modulation device driving unit 46 is connected to the liquid crystal panel of the light modulation device 22. The light modulator driving unit 46 drives the liquid crystal panel based on the image signal input from the image processing unit 40 and draws an image on each liquid crystal panel.

The operation detection unit 17 is connected to a remote control light receiving unit 18 and an operation panel 19 that function as an input device, and detects an operation via the remote control light reception unit 18 and the operation panel 19.
The remote control light receiving unit 18 receives, by the remote control light receiving unit 18, an infrared signal transmitted in response to a button operation by a remote control (not shown) used by the user of the projector 10. The remote control light receiving unit 18 decodes the infrared signal received from the remote control, generates operation data indicating the operation content of the remote control, and outputs the operation data to the control unit 30.
The operation panel 19 is provided in the exterior housing of the projector 10 and has various switches and indicator lamps. The operation detection unit 17 appropriately turns on and off the indicator lamp of the operation panel 19 according to the operation state and setting state of the projector 10 according to the control of the control unit 30. When the switch of the operation panel 19 is operated, operation data corresponding to the operated switch is output from the operation detection unit 17 to the control unit 30.

  The emission device driving unit 48 is connected to the light emission device 60 via the connection unit 49. The connection portion 49 is a connector having a plurality of pins, for example, and the light emitting device 60 is connected to the connection portion 49 via a cable 60a. The emission device driving unit 48 generates a pulse signal according to the control of the control unit 30 and outputs the pulse signal to the light emission device 60 via the connection unit 49. Further, the emission device driving unit 48 supplies power to the light emission device 60 through the connection unit 49.

  As shown in FIG. 1, the light emitting device 60 is configured by housing a light source unit 61 and optical components in a substantially box-shaped case. The light emitting device 60 of the present embodiment includes a solid light source (not shown) that emits infrared light in the light source unit 61. Infrared light emitted from the solid-state light source is diffused by the collimating lens and the Powell lens to form a surface along the screen SC. The light source unit 61 may include a plurality of solid light sources, and a light layer may be formed so as to cover the image projection range of the screen SC by diffusing light emitted from the plurality of solid light sources. The light emitting device 60 may include an adjustment mechanism that adjusts the distance and angle between the light layer emitted from the light source unit 61 and the screen SC.

  The light emitting device 60 turns on the light source unit 61 by the pulse signal and the power source supplied from the emitting device driving unit 48. The emission device driving unit 48 controls the timing at which the light source unit 61 is turned on and off. The control unit 30 controls the emission device driving unit 48 to turn on the light source unit 61 in synchronization with the timing when the imaging unit 51 described later performs imaging.

The position detection unit 50 detects an operation on the screen SC by the indicators 70 and 80. The position detection unit 50 includes an imaging unit (imaging unit) 51, a transmission unit (first transmission unit) 52, an imaging control unit 53, a captured image data processing unit 56, a frame memory 58, a pointer detection unit 54, and a coordinate calculation unit 55. It comprises each part of.
The imaging unit 51 captures the screen SC and its peripheral part (detection area) as an imaging range in order to detect the indication positions of the indicators 70 and 80. The imaging unit 51 includes an imaging optical system, an imaging element, an interface circuit, and the like, and captures the projection direction of the projection optical system 23. The imaging optical system of the imaging unit 51 is arranged so as to face substantially the same direction as the projection optical system 23, and has an angle of view that covers a range in which the projection optical system 23 projects an image on the screen SC. Examples of the imaging device include a CCD and a CMOS that receive light in the infrared region and the visible light region. The imaging unit 51 may include a filter that blocks a part of the light incident on the imaging element. For example, when receiving infrared light, a filter that mainly transmits infrared light is provided in front of the imaging element. May be arranged. Further, the interface circuit of the imaging unit 51 reads and outputs the detection value of the imaging element.

  The imaging control unit 53 causes the imaging unit 51 to perform imaging and generates captured image data. When the imaging device performs imaging with visible light, an image projected on the screen SC is captured. For example, an auto-calibration image to be described later is taken with visible light. In addition, the imaging control unit 53 can cause the imaging unit 51 to capture infrared light. In this case, the captured image includes infrared light (infrared signal) emitted from the indicator 70 or reflection reflected by the indicator 80. Light is reflected.

  The captured image data processing unit 56 develops the captured image data captured by the imaging unit 51 and acquired from the imaging control unit 53 in the frame memory 58. The captured image data processing unit 56 stores a mask image to be described later, and performs mask processing by superimposing the mask image on the captured image data developed in the frame memory 58. The captured image data processing unit 56 outputs the captured image data after the mask processing to the indicator detection unit 54.

The indicator detection unit 54 detects the indication positions of the indicators 70 and 80 using density threshold values that differ depending on the position of the screen SC based on the captured image data captured by the imaging control unit 53. The density threshold value is set to a different value depending on the distance from the imaging unit 51 to each position on the screen SC. More specifically, the density threshold value is set so that the value increases as the distance from the imaging unit 51 decreases. Details of the density threshold and the process of detecting the designated positions of the indicators 70 and 80 from the captured image data using the density threshold will be described later with reference to FIG.
The indicator detection unit 54 reflects the image of the infrared light emitted by the indicator 70 and the reflection of the indicator 80 from the captured image data when the imaging control unit 53 causes the imaging unit 51 to perform infrared imaging. At least one of the reflected light image is detected. Further, the indicator detection unit 54 may determine whether the detected image is an image of light emitted from the indicator 70 or an image of reflected light from the indicator 80.
The coordinate calculation unit 55 calculates the coordinates of the pointing positions of the pointers 70 and 80 in the captured image data based on the position of the image detected by the pointer detection unit 54 and outputs the calculated coordinates to the control unit 30. The coordinate calculation unit 55 may also calculate the coordinates of the indication positions of the indicators 70 and 80 in the projection image projected by the projection unit 20 and output them to the control unit 30. Further, the coordinate calculation unit 55 is configured to display the coordinates of the pointing positions of the indicators 70 and 80 in the image data drawn by the image processing unit 40 in the frame memory 44 and the indicators 70 and 80 in the input image data of the image I / F unit 12. The coordinates of the designated position may be calculated.

  The transmission unit 52 transmits an infrared signal to the indicator 70 according to the control of the indicator detection unit 54. The transmission unit 52 has a light source such as an infrared LED, and turns on and off the light source according to the control of the indicator detection unit 54.

Moreover, the indicator 70 is provided with the control part 73, the transmission / reception part 74, the operation switch 75, and the power supply part 76, and these each part is accommodated in the axial part 72 (FIG. 1). The control unit 73 is connected to the transmission / reception unit 74 and the operation switch 75 and detects the on / off state of the operation switch 75. The transmission / reception unit 74 includes a light source such as an infrared LED and a light receiving element that receives infrared light. To 73.
The power supply unit 76 includes a dry battery or a secondary battery as a power supply, and supplies power to the control unit 73, the transmission / reception unit 74, and the operation switch 75.
The indicator 70 may include a power switch that turns on / off the power supply from the power supply unit 76.

Here, a method of specifying the indicator 70 from the captured image data of the imaging unit 51 by mutual communication between the position detection unit 50 and the indicator 70 will be described.
When detecting the position pointing operation by the pointer 70, the control unit 30 controls the pointer detection unit 54 to transmit a synchronization signal from the transmission unit 52. That is, the indicator detection unit 54 turns on the light source of the transmission unit 52 at a predetermined period in accordance with the control of the control unit 30. The infrared light periodically emitted from the transmission unit 52 functions as a synchronization signal that synchronizes the position detection unit 50 and the indicator 70.
On the other hand, after the supply of power from the power supply unit 76 is started and a predetermined initialization operation is performed, the control unit 73 receives infrared light emitted from the transmission unit 52 of the projector 10 by the transmission / reception unit 74. When the transmission / reception unit 74 receives infrared light periodically generated by the transmission unit 52, the control unit 73 lights the light source of the transmission / reception unit 74 in a preset lighting pattern in synchronization with the timing of the infrared light. (Light emission). This lighting pattern represents data specific to the indicator 70 by turning on and off the light source corresponding to ON and OFF of the data. The control unit 73 turns on and off the light source in accordance with the set pattern turn-on time and turn-off time. The control unit 73 repeatedly executes the above pattern while power is supplied from the power supply unit 76.
That is, the position detection unit 50 periodically transmits an infrared signal for synchronization to the indicator 70, and the indicator 70 synchronizes with the infrared signal transmitted by the position detection unit 50 to set the infrared signal set in advance. Send.

The shooting control unit 53 of the position detection unit 50 performs control to match the shooting timing of the imaging unit 51 with the timing when the indicator 70 is turned on. This shooting timing is determined based on the timing when the indicator detection unit 54 turns on the transmission unit 52. The indicator detection unit 54 can specify a pattern in which the indicator 70 is turned on depending on whether or not the image of the light of the indicator 70 is captured in the captured image data of the imaging unit 51.
The pattern in which the indicator 70 is lit may include a pattern unique to each individual indicator 70, or a pattern common to a plurality of indicators 70 and a pattern unique to each individual. In this case, the indicator detection unit 54 can distinguish each image as an image of a different indicator 70 when the captured image data includes images of infrared light emitted from the plurality of indicators 70.

  In addition, the control unit 30 controls the emission device driving unit 48 to synchronize the lighting timing of the light source unit 61 with the imaging timing of the imaging unit 51. When the light source unit 61 turns on the pulse in synchronization with the imaging timing of the imaging unit 51, the reflected light of the indicator 80 appears in the captured image of the imaging unit 51 when the indicator 80 points on the screen SC. When the light source unit 61 is turned on in a pattern that can be distinguished from the lighting timing of the indicator 70, the indicator detection unit 54 determines whether the image shown in the captured image data is the indicator 70 or the indicator 80. it can. For example, consider a case in which the indicator 70 is turned on in synchronization with all the shooting timings of the imaging unit 51 and the light source unit 61 is turned on in a pattern of “1010101010” (1 indicates lighting and 0 indicates light extinction). In this case, it can be determined that the image taken when the light source unit 61 is not lit is due to the indicator 70.

  Further, the control unit 73 included in the indicator 70 switches the pattern for lighting the transmission / reception unit 74 according to the operation state of the operation switch 75. For this reason, the indicator detection unit 54 can determine the operating state of the indicator 70, that is, whether or not the distal end portion 71 is pressed against the screen SC, based on a plurality of captured image data.

The attitude sensor 47 is configured by an acceleration sensor, a gyro sensor, or the like, and outputs a detection value to the control unit 30. The attitude sensor 47 is fixed to the main body of the projector 10 so that the installation direction of the projector 10 can be identified.
As shown in FIG. 1, the projector 10 is installed in a suspended state in which it is suspended from a wall surface or a ceiling surface, in an installation state in which projection is performed from below the screen SC, or in a horizontal plane such as the top surface of a desk. Can be used in situations. Depending on the installation state of the projector 10, the light emitting device 60 may not be suitable for use. For example, when the projection is performed on the screen SC from below, the user's body may block the emitted light of the light emitting device 60, which is inappropriate. The attitude sensor 47 is provided in the main body of the projector 10 so that a plurality of installation states assumed as the installation state of the projector 10 can be identified. The attitude sensor 47 is configured using, for example, a biaxial gyro sensor, a monoaxial gyro sensor, an acceleration sensor, or the like. The control unit 30 can automatically determine the installation state of the projector 10 based on the output value of the attitude sensor 47. When the control unit 30 determines that the installation state is inappropriate for the use of the light emitting device 60, for example, the emission device driving unit 48 stops outputting the power supply voltage and the pulse signal.

The control unit 30 reads and executes the control program 111 stored in the storage unit 110 to thereby execute functions of the projection control unit 31, the detection control unit 32, the emission control unit 33, and the calibration control unit 39 (mask image generation unit). To control each part of the projector 10.
The projection control unit 31 acquires the content of the operation performed by the user based on the operation data input from the operation detection unit 17. The projection control unit 31 controls the image processing unit 40, the light source driving unit 45, and the light modulation device driving unit 46 according to the operation performed by the user, and projects an image on the screen SC. The projection control unit 31 controls the image processing unit 40 to determine the above-described 3D (stereoscopic) image and 2D (planar) image, resolution conversion processing, frame rate conversion processing, distortion correction processing, digital zoom processing, and tone correction. Processing, brightness correction processing, and the like are executed. Further, the projection control unit 31 controls the light source driving unit 45 in accordance with the processing of the image processing unit 40 and controls the light amount of the light source unit 21.

  The detection control unit 32 controls the position detection unit 50 to detect the operation positions of the indicators 70 and 80 and acquires the coordinates of the operation positions. Further, the detection control unit 32 obtains data for identifying whether it is the operation position of the indicator 70 or the operation position of the indicator 80 and data indicating the operation state of the operation switch 75 together with the coordinates of the operation position. . The detection control unit 32 executes a preset process based on the acquired coordinates and data. For example, the image processing unit 40 performs a process of drawing a figure based on the acquired coordinates and superimposing the drawn figure on an input image input to the image I / F unit 12 and projecting it. The detection control unit 32 may output the acquired coordinates to an external device such as a PC connected to the I / F unit 11. In this case, the detection control unit 32 may convert the acquired coordinates into a data format recognized as an input of the coordinate input device in the operating system of the external device connected to the I / F unit 11 and output the converted data. Good. For example, when a PC operating on a Windows (registered trademark) operating system is connected to the I / F unit 11, data processed as input data of HID (Human Interface Device) in the operating system is output. Further, the detection control unit 32 outputs the data indicating the operation position of the indicator 70 or the operation position of the indicator 80 and the data indicating the operation state of the operation switch 75 together with the coordinate data. Also good.

  Further, the detection control unit 32 controls position detection using the indicator 80. Specifically, the detection control unit 32 determines whether or not the light emitting device 60 can be used based on whether or not the light emitting device 60 is connected. When the light emitting device 60 cannot be used, the detection control unit 32 performs setting so that the light emitting device 60 cannot be used. Here, the detection control unit 32 may notify that the light emitting device 60 cannot be used.

  The emission control unit 33 controls the emission device driving unit 48 to execute or stop the output of the power source and the pulse signal to the light emission device 60 connected to the connection unit 49. The emission control unit 33 controls the detection control unit 32 to stop the power supply of the emission device driving unit 48 and the output of the pulse signal when the light emission device 60 cannot be used or is not used. When the light emitting device 60 is used, the emission control unit 33 outputs the power supply and pulse signal of the emission device driving unit 48.

  The calibration control unit 39 detects the indication positions of the indicator 70 and the indicator 80 and executes calibration for conversion into coordinates in the input image of the image I / F unit 12.

  The processing procedure of the control unit 30, particularly the processing procedure of the calibration control unit 39, will be described with reference to the flowchart shown in FIG.

  The calibration is executed as one of initial settings when the projector 10 is used for the first time. For example, the calibration is a process of associating a position in an image drawn in the frame memory 44 and projected by the projection unit 20 with a position on the captured image data captured by the imaging unit 51. The indication positions of the indicators 70 and 80 detected by the position detection unit 50 from the photographed image data are positions in the photographed image data, and are indicated by coordinates in a coordinate system set for the photographed image, for example. The user gives instructions using the indicators 70 and 80 in consideration of the projection image projected on the screen SC. Therefore, the projector 10 needs to specify the designated position for the projection image on the screen SC. By calibration, the coordinates of the position detected in the captured image data can be converted into the coordinates on the projection image data. Data for performing this association is referred to as calibration data. The calibration data is data that associates the coordinates on the photographed image data output from the photographing control unit 53 with the coordinates on the projection image. Specifically, it may be a table in which the coordinates on the captured image data and the coordinates on the projected image are associated one-to-one, or a function that converts the coordinates on the captured image data into the coordinates on the projected image. May be.

The calibration control unit 39 executes calibration according to the type of the indicator. That is, two calibrations are executed: calibration relating to detection of the indicated position of the indicator 70 and calibration relating to detection of the indicated position of the indicator 80.
4A and 4B are explanatory diagrams showing how the pointing positions of the pointers 70 and 80 are detected. FIG. 4A shows how the pointing position of the pointer 70 is detected. FIG. 4B shows the pointing position of the pointer 80. A state of detecting is shown.
In FIG. 4A, the imaging direction in which the imaging unit 51 captures the screen SC is indicated by symbol PA. When detecting the position of the indicator 70, the transmission / reception unit 74 emits infrared light from the light emission position 70 a at the tip of the indicator 70. The light emission position 70a is very close to the contact point 70b where the indicator 70 contacts the screen SC. For this reason, when detecting an image of light emitted from the indicator 70 from the captured image data captured from the imaging direction PA, the position of this image can be regarded as the position of the contact point 70b.

On the other hand, when the indication position of the indicator 80 is detected as shown in FIG. 4B, the reflected light reflected by the indicator 80 is detected from the detection light L emitted from the light emitting device 60. That is, the reflected light image of the detection light L is detected from the captured image data captured from the capturing direction PA. The emission direction of the detection light L is substantially parallel to the screen SC, and the detection light L is separated from the screen SC by a predetermined distance (hereinafter referred to as distance G1). Although the distance G1 varies depending on the mounting position of the light emitting device 60 with respect to the screen SC, it is difficult to make the distance G1 zero due to the structure. For this reason, in the captured image data captured from the capturing direction PA, an image of the reflected light reflected at the reflection position 80c separated from the screen SC by the distance G1 is captured at the tip of the indicator 80.
As shown in FIG. 4B, the reflection position 80c is separated in a direction oblique to the imaging direction PA. For this reason, the position of the image of the reflected light reflected in the captured image data is the same position as the image in the case where the indicator 70 indicates a more distant position in the capturing direction PA. That is, the reflected light when the indicator 80 contacts the screen SC at the contact point 80b and the light when the indicator 70 contacts the screen SC at the contact point 70b are the same position in the captured image data of the imaging unit 51. It is reflected in. For this reason, the contact point 80b pointed to by the indicator 80 is detected as the contact point 70b away from the imaging unit 51 in the shooting direction PA, and a shift of the distance G2 occurs.

  The shift in the distance G2 is caused by the imaging unit 51 taking an image obliquely from a position away from the screen SC. For example, the positional relationship between the shooting direction PA and the indicators 70 and 80 shown in FIGS. 4A and 4B occurs not only in the vertical direction but also in the horizontal direction. In the present embodiment, as shown in FIG. 1, since one imaging unit 51 provided in the main body of the projector 10 located above the screen SC shoots the screen SC in an overhead view, both in the vertical and horizontal directions. A shift of the distance G2 occurs.

Therefore, when detecting the indication position of the indicator 80, the projector 10 corrects the detected position after detecting the indication position in the same manner as when detecting the indication position of the indicator 70.
Specifically, the calibration control unit 39 generates calibration data by performing calibration relating to the detection of the indication position of the indicator 70. By using this calibration data, for example, as shown in FIG. 4A, when the light emission position 70a is close to the contact point 70b with the screen SC, the indicated position can be detected with high accuracy.

Further, the projector 10 uses correction data for correcting the coordinates obtained from the calibration data when detecting the indication position of the indicator 80. Specifically, the correction data is initial correction data 125 and manual correction data 126.
The correction data can be data that defines the distance G1 in FIG. In this case, it is possible to use a table format or map data in which data indicating the size of the distance G1 is associated with each coordinate on the captured image data or each coordinate on the projection image. Moreover, it can be set as the table format which matches the data which show the magnitude | size of the distance G1 about the representative point preset in the coordinate on picked-up image data, or the coordinate on a projection image. When it is necessary to determine the size of the distance G1 of the coordinates deviating from the representative point, a method of applying the distance G1 of the representative point close to the coordinates to be corrected, or the coordinates of the correction target from the distance G1 of the representative points by interpolation calculation A method for obtaining the distance G1 can be used.
Further, for example, the correction data may be data that shifts coordinates detected on the captured image data or coordinates on the projection image obtained based on the calibration data. Specifically, it may be data for determining the shift amount of coordinates or a function for correcting coordinates. Further, the correction data may be data that realizes a different shift amount for each coordinate on the captured image data or each coordinate on the projection image. In this case, it is good also as a table which matched the shift amount of the coordinate with the coordinate of correction object. In this table, the shift amount may be associated with a representative point selected from the coordinates on the captured image data or the coordinates on the projection image. When correcting coordinates other than the representative point, use the method of applying the shift amount of the representative point close to the correction target coordinate, or the method of obtaining the shift amount of the correction target coordinate from the shift amount of the representative point by interpolation calculation. it can.

The calibration control unit 39 can execute auto calibration and manual calibration as calibration related to the indication position of the indicator 70.
Auto-calibration is a process of projecting an image for auto-calibration onto the screen SC, photographing it with the imaging unit 51, and generating calibration data using the photographed image data. The auto-calibration is a process that can be automatically executed by the projector 10 and does not require the user to operate the indicators 70 and 80. The auto calibration is not limited to the case where the user gives an instruction to execute using the remote controller or the operation panel 19, but can also be executed at a timing controlled by the control unit 30. For example, it may be performed at the start of operation such as immediately after the projector 10 is turned on, or may be performed during normal operation described later. The auto calibration image 121 projected by auto calibration is stored in the storage unit 110 in advance.

FIG. 5 shows an example of the auto calibration image 121. In the auto calibration image 121, a plurality of marks (symbols) are arranged at predetermined intervals. The mark in the calibration image is a figure or symbol that can be detected from the captured image data, and its shape and size are not particularly limited.
FIG. 6 shows an example of photographed image data obtained by photographing the auto calibration image 121 projected on the screen SC by the imaging unit 51. The captured image data of the imaging unit 51 is a distorted image because it is captured from obliquely above the screen SC when the projector 10 is suspended and installed as shown in FIG. FIG. 5 illustrates a rectangular auto-calibration image 121 in which marks are arranged at equal intervals. However, the photographed image data in FIG. 6 shows a distorted image, and the interval between the marks arranged in the image is as follows. , Depending on the position of the mark.

The calibration control unit 39 operates the image processing unit 40 and the projection unit 20 on the basis of the auto calibration image 121 stored in the storage unit 110 by the function of the projection control unit 31, and displays the auto calibration image 121. Project onto the screen SC. The calibration control unit 39 controls the position detection unit 50 to cause the imaging unit 51 to perform imaging and acquire captured image data. The captured image data is temporarily stored in a memory (not shown) from the imaging control unit 53 and output to the control unit 30. The calibration control unit 39 detects the mark from the captured image data, and acquires the position of the center of gravity of each mark as the coordinate value of the mark. The calibration control unit 39 associates the mark detected from the captured image data with the projected image drawn in the frame memory 44, that is, the mark of the auto calibration image 121.
The calibration control unit 39 creates auto-calibration data 123 in a table format or a function format by associating the coordinate value of the mark in the captured image with the coordinate value of the mark in the projection image. The coordinate value in the projected image of the mark of the auto calibration image 121 is stored in advance in the storage unit 110 together with the auto calibration image 121 or included in the auto calibration image 121. The calibration control unit 39 updates the auto calibration data 123 when the auto calibration data 123 is already stored.

  FIG. 7 shows an example of a calibration data management table in which calibration data is registered. The calibration data management table table shown in FIG. 7 is a table in which the identification numbers of the marks arranged on the auto calibration image 121 and the center coordinates of each mark on the auto calibration image 121 are recorded in association with each other. is there. The calibration data management table is stored in the storage unit 110 in association with the auto calibration image 121. In the calibration management table, the center coordinates of each mark on the frame memory 44 are recorded in association with the identification number of each mark. As the position coordinates of each mark, instead of the center coordinates, coordinate values (maximum value and minimum value in the X coordinate and Y coordinate) of the range in which each mark is located may be recorded. Based on the coordinates recorded in the calibration data management table and the coordinates detected by the position detection unit 50 from the captured image data, the coordinates on the captured image data and the coordinates on the frame memory 44 are associated with each other, and auto calibration is performed. Action data 123 is generated.

  The calibration control unit 39 executes one calibration and creates or updates one auto calibration data 123. Alternatively, the calibration control unit 39 switches the auto-calibration image 121 and associates a plurality of shootings with coordinates. Then, the calibration control unit 39 may create the auto calibration data 123 with higher accuracy by integrating the association results obtained by the plurality of auto calibrations.

  In the present embodiment, the storage unit 110 stores a plurality of auto calibration images 121. The calibration control unit 39 selects one auto calibration image 121 from the auto calibration images 121 stored in the storage unit 110. By preparing a plurality of auto-calibration images 121, it is possible to perform auto-calibration a plurality of times and increase the detection accuracy of the indicated coordinates by the indicators 70 and 80. For example, a mark having different mark positions is used in the first auto-calibration and the second auto-calibration. In order to increase the detection accuracy of the designated coordinates by the indicators 70 and 80, the mark needs to have a certain size (area). For this reason, the detection accuracy of the designated coordinates by the indicators 70 and 80 can be increased by dividing the auto calibration into two times and using the ones having different mark positions on the auto calibration image 121. The calibration control unit 39 may use a plurality of auto calibration images 121 in one auto calibration. Although FIG. 5 shows an example using X-shaped marks, the shape and size of the marks, the number of marks included in the auto-calibration image 121, the arrangement of the marks, and the like are not limited.

Manual calibration is a process of projecting an image for manual calibration on the screen SC, detecting an operation of the indicator 70 corresponding to the projected image, and generating manual calibration data.
FIG. 8 shows an example of the manual calibration image 122. The manual calibration image 122 includes a mark indicating the indicated position in order to allow the user to make an instruction with the indicator 70. In the manual calibration image 122 of FIG. 8, a plurality of instruction marks (◯) are arranged, and the user indicates the position of the mark with the indicator 70.

The manual calibration image 122 includes a plurality of marks, and these marks are projected onto the screen SC one by one. Therefore, the manual calibration image 122 is specifically composed of a combination of a plurality of images with different numbers of marks.
Each time a mark is displayed on the screen SC, the user designates the newly displayed mark with the indicator 70. The calibration control unit 39 detects the indicated position every time the user performs an operation. Then, the calibration control unit 39 associates the designated position detected in the captured image with the projected image drawn in the frame memory 44, that is, the mark of the manual calibration image 122. The calibration control unit 39 creates the manual calibration data 124 by associating the coordinate value of the indicated position detected from the captured image data with the coordinate value of the mark on the projection image.
The manual calibration data 124 can be in the same data format as the auto calibration data 123, but can be correction data for correcting the auto calibration data 123. The auto calibration data 123 is data for converting coordinates on the captured image into coordinates on the projected image. On the other hand, the manual calibration data 124 is data for further correcting the coordinates after conversion using the auto calibration data 123.

  The calibration control unit 39 can execute auto calibration or manual calibration when performing calibration related to detection of the pointing position of the pointer 70. When the storage unit 110 stores auto-calibration data 123 generated in the past, auto-calibration and manual calibration can be selected and executed. Here, when auto calibration is executed, the calibration control unit 39 updates the auto calibration data 123 in the storage unit 110. When manual calibration is executed, manual calibration data 124 is generated or updated. Further, when the auto calibration data 123 is not stored in the storage unit 110, it is necessary to execute auto calibration. This is because the manual calibration data 124 cannot be used when the auto-calibration data 123 is not stored.

The calibration control unit 39 can execute the calibration related to the detection of the indication position of the indicator 80 in the same manner as the manual calibration of the indicator 70. In this case, the calibration control unit 39 generates manual correction data 126. The manual correction data 126 is used when detecting the indication position of the indicator 80.
As described with reference to FIG. 4B, the manual correction data 126 is data for correcting the coordinates detected as the indication position of the indicator 70 to the coordinates of the indication position of the indicator 80. When manual calibration is not performed regarding the detection of the pointing position of the pointer 80, the calibration control unit 39 selects the initial correction data 125. The initial correction data 125 is correction data when the distance G1 in FIG. 4B is set to an initial value, and is stored in the storage unit 110 in advance. When the light emitting device 60 is installed, the distance G1 between the screen SC and the detection light L is adjusted to be, for example, 10 mm to 1 mm, and actually changes in the plane of the screen SC. The initial correction data 125 is correction data when the initial value of the distance G1 is assumed to be 5 mm, for example. If the initial correction data 125 is used, the indicated position of the indicator 80 can be detected without performing manual calibration. . By using the manual correction data 126 created by the manual calibration, it is possible to detect the indication position of the indicator 80 with higher accuracy by performing the correction reflecting the difference in the plane of the distance G1.
That is, in the position detection of the position detection unit 50, the detection control unit 32 obtains the coordinates of the specified position using the auto calibration data 123 or the manual calibration data 124 when detecting the indication position of the indicator 70. . When detecting the indication position of the indicator 80, the detection control unit 32 performs correction using the initial correction data 125 or the manual correction data 126 in a process of obtaining coordinates using the auto calibration data 123 or the manual calibration data 124. . In other words, the initial correction data 125 and the manual correction data 126 are difference data for obtaining the indication position of the indicator 80 from the indication position of the indicator 70.

  In the flowchart of FIG. 3, the user selects whether to execute auto calibration or manual calibration (step S1). The calibration control unit 39 detects the operation of the remote controller or the operation panel 19 (step S2). If auto calibration is selected, the process proceeds to step S2. If manual calibration is selected, the process proceeds to step S7. Transition. As described above, when the auto-calibration data 123 is not stored in the storage unit 110, a menu screen in which only auto-calibration can be selected may be projected in step S1.

In step S <b> 2, the calibration control unit 39 selects the auto calibration image 121. The storage unit 110 stores a plurality of auto calibration images 121, and the calibration control unit 39 selects one auto calibration image 121 from the auto calibration images 121 stored in the storage unit 110. To do.
Subsequently, the calibration control unit 39 projects the selected auto calibration image 121 on the screen SC by the projection unit 20 (step S3). With the auto-calibration image 121 projected on the screen SC, the user adjusts the display size and the display position so that the auto-calibration image 121 fits in the display area of the screen SC by operating the remote control or the operation panel 19. May be.
The calibration control unit 39 controls the position detection unit 50 to cause the imaging unit 51 to perform imaging (step S4), acquires captured image data of the imaging unit 51, and performs auto calibration based on the acquired captured image data. Action data 123 is created (step S5).

The calibration control unit 39 performs background image creation processing (step S6), and proceeds to step S15.
FIG. 9 is a flowchart showing details of the background image creation process.
The calibration control unit 39 creates a mask image (step S111). The mask image is an image for cutting out image data of a portion corresponding to the display area of the screen SC from the captured image data captured by the imaging unit 51. FIG. 10 shows an example of the mask image.

A method for creating a mask image will be described with reference to FIG. FIG. 11 shows an example of photographed image data. In FIG. 11, the lower left of the captured image data is the origin, the vertical direction is the X axis direction, and the horizontal direction is the Y axis direction.
The calibration control unit 39 determines a range in which the screen SC is captured from captured image data obtained by capturing a projection image on the screen SC.

In the projector 10, the projection image has already been adjusted by the user in step S3 so that the projection image fits in the projection area of the screen SC. In addition, the calibration control unit 39 acquires data indicating which mark row is the outermost mark row in the vertical and horizontal directions among the marks included in the auto-calibration image 121. This data is stored in the storage unit 110 in association with the auto-calibration image 121, for example.
In the example shown in FIG. 11, the mark row located on the outermost side on the left side of the auto-calibration image 121 is the mark row T. The calibration control unit 39 acquires the center coordinates of each mark included in the mark row T from the calibration data management table illustrated in FIG. The calibration control unit 39 determines a left end of the screen SC by adding a predetermined value as a margin to the acquired center coordinates of each mark. Since the mark row T is the outermost mark row on the left side of the auto calibration image 121, a predetermined value is subtracted from the Y coordinate value of each mark to obtain the coordinate value at the left end of the screen SC. For example, in the case of the mark T3 (X3, Y3) in the mark row T shown in FIG. 11, the coordinate T3 ′ (X3, Y3-α) obtained by subtracting the predetermined value α from the Y coordinate value Y3 is the left end of the screen SC at T3. Become. The calibration control unit 39 obtains the coordinate value of the end in each of the top, bottom, left and right directions of the screen SC. It should be noted that for the region where no mark exists, the coordinate value of the end portion may be obtained by complement processing. The calibration control unit 39 saves the obtained coordinate values in the respective directions of up, down, left, and right in the storage unit 110. Next, the calibration control unit 39 creates a mask image using the obtained data of the range of the screen SC. The calibration control unit 39 generates a mask image set so that the pixel value becomes 0 in an area outside the range of the screen SC. The calibration control unit 39 sends the generated mask image to the captured image data processing unit 56 of the position detection unit 50.

  Next, the calibration control unit 39 turns off the light source unit 61 of the light emitting device 60 and causes the imaging unit 51 to capture the imaging range (step S112). At the time of shooting by the imaging unit 51, for example, the projection unit 20 projects a message image that calls attention so as not to perform an operation with the indicators 70 and 80 so that the indicators 70 and 80 and the user are not shot. To. The captured image data captured in step S112 and captured with the light source 61 of the light emitting device 60 turned off is referred to as first background image data. FIG. 12 shows an example of the first background image data. In the first background image data shown in FIG. 12, the screen SC and reflected light that is unintentionally reflected on the screen SC are recorded. The calibration control unit 39 stores the first background image data in the storage unit 110.

Next, the calibration control unit 39 turns on the light source unit 61 of the light emitting device 60 and causes the imaging unit 51 to capture an imaging range (step S113). Also at the time of photographing, for example, the message image is projected onto the projection unit 20 so that the indicators 70 and 80 and the user are not photographed. The photographed image data photographed in step S113 and photographed with the light source unit 61 of the light emitting device 60 turned on is referred to as second background image data. FIG. 13 shows an example of the second background image data. The second background image data shown in FIG. 13 includes the screen SC and a pen tray that reflects the light from the light source unit 61 in addition to the reflected light that the detection light of the light emitting device 60 reflects on the screen SC. The calibration control unit 39 stores the second background image data in the storage unit 110.
When the calibration control unit 39 captures the first background image data and the second background image data, it subtracts the first background image data from the second background image data to generate difference image data (step S114). Hereinafter, this difference image data is referred to as light source noise data. The calibration control unit 39 stores the generated light source noise data in the storage unit 110. Note that the shooting of the first background image data and the second background image data may be performed during a normal operation described later.

On the other hand, when manual calibration is selected, the calibration control unit 39 proceeds to step S7.
In step S7, the background image creation process in step S6 described above is executed. In step S111 in FIG. 9, the auto-calibration image 121 is projected on the screen SC to create a mask image. The same applies to step S7.

  After creating the mask image, the calibration control unit 39 selects the manual calibration image 122 (step S8), and projects the selected manual calibration image 122 onto the screen SC by the projection unit 20 (step S9). In a state where the manual calibration image 122 is projected on the screen SC, the user may adjust the display size and the display position so that the manual calibration image 122 fits in the display area of the screen SC.

  Here, an operation using the indicator 70 is performed by the user (step S10). As shown in FIG. 8, a predetermined mark is arranged in the manual calibration image 122. When the manual calibration image 122 is displayed on the screen SC, the user uses the indicator 70 to indicate the marks projected on the screen SC one by one. The transmission unit 52 of the projector 10 periodically transmits an infrared signal for synchronization. The indicator 70 turns on the infrared light in synchronization with the infrared signal. The calibration control unit 39 causes the imaging unit 51 to capture the imaging range in synchronization with the light emission timing of the indicator 70. As a result, photographed image data (hereinafter referred to as first position detection image data) in which the indicator 70 points to the mark is photographed. The calibration control unit 39 performs a designated position detection process for acquiring photographed image data and detecting the designated position of the pointer 70 (step S11).

  Details of step S11 will be described with reference to the flowchart shown in FIG. The first background image data is an image of the screen SC and its surroundings taken with the light source 61 of the light emitting device 60 turned off so that the indicator 70 and the user are not reflected. In the first background image data, external light entering from a window, illumination light such as a fluorescent lamp, disturbance due to reflected light reflected by the screen SC, and the like are recorded. These are called background noise.

  The captured image data processing unit 56 acquires first position detection image data captured by the imaging unit 51. The captured image data processing unit 56 develops the acquired first position detection image data in the frame memory 58. The captured image data processing unit 56 performs mask processing by further superimposing a mask image on the frame memory 58 in which the first position detection image data is expanded (step S121). In the following, the first position detection image data in the area corresponding to the screen SC that has been cut out by masking will be referred to as cut-out image data. The captured image data processing unit 56 passes the cut-out image data cut out by the mask process to the indicator detection unit 54. The indicator detection unit 54 subtracts the first background image data from the cut-out image data acquired from the captured image data processing unit 56 to generate difference image data from which background noise has been removed (step S122). Next, the indicator detection unit 54 binarizes the difference image data using the density threshold value (step S123). That is, the indicator detection unit 54 compares the pixel value of the difference image data with the density threshold value, and detects a pixel whose pixel value is equal to or higher than the density threshold value. FIGS. 15A and 15B show examples of density threshold values. 15A shows density threshold values set in accordance with the mask image shown in FIG. 10, and FIG. 15B shows density threshold values at points A to D shown in FIG. Is shown.

  In the present embodiment, the density threshold value is changed according to the distance from the imaging unit 51. In the captured image data captured by the imaging unit 51, the brightness of the captured image decreases as the distance from the imaging unit 51 to the subject increases, and the brightness tends to increase as the distance from the imaging unit 51 to the subject decreases. For this reason, in the position detection unit 50, the density threshold value in the range where the distance from the imaging unit 51 is close is large, and the density threshold value in the range where the distance from the imaging unit 51 is long is set small. Since the density threshold value is set with an inclination in this way, it becomes easy to distinguish the infrared light of the indicator 70 from other images in the captured image. In the example shown in FIG. 15B, the density threshold is set to be the largest at the point D where the distance from the imaging unit 51 is the closest, and at the point A where the distance from the imaging unit 51 is the farthest. Is set to the smallest.

When the indicator detection unit 54 detects pixels detected by binarization and whose pixel value is equal to or higher than the density threshold value, the indicator detection unit 54 divides the detected pixels into blocks for each block, and the area of the blocks is within a predetermined range The barycentric coordinates of the block to be obtained are obtained (step S124). The barycentric coordinates are the positional center coordinates of the block. The calculation of the barycentric coordinates is performed, for example, by adding the horizontal axis X component and the vertical axis Y component of the coordinates assigned to the block and taking the average. The position detection unit 50 passes the calculated barycentric coordinates to the calibration control unit 39. The calibration control unit 39 determines the barycentric coordinates acquired from the indicator detection unit 54 as the instruction coordinates on the captured image data.
In the example illustrated in FIG. 2, the case where the position detection unit 50 includes the frame memory 58 has been described as an example. When the position detection unit 50 does not include the frame memory 58, the captured image data processing unit 56 instructs only the first position detection image data in the screen SC area out of the first position detection image data. Mask processing may be performed by outputting to the body detection unit 54. Specifically, the captured image data processing unit 56 calculates the coordinate value of the area corresponding to the screen SC on the captured image data from the mask image acquired from the calibration control unit 39. Then, when the captured image data processing unit 56 acquires the first position detection image data from the imaging control unit 53, only the first position detection image data of the area corresponding to the screen SC is indicated on the basis of the calculated coordinate value. It outputs to the detection part 54.
Returning to FIG. 3, the calibration control unit 39 stores the designated coordinates on the captured image data and the coordinates of the corresponding mark on the auto-calibration image 121 in the storage unit 110 in association with each other (step S12).

  The calibration control unit 39 determines whether or not the indicated position has been detected for all the marks in the manual calibration image 122 (step S13), and if there is an unprocessed mark, the process returns to step S9. When the indication positions of all the marks have been detected, the calibration control unit 39 creates the manual calibration data 124 based on the coordinates of the indication positions and the mark positions temporarily stored in step S12. (Step S14). The manual calibration data 124 created here is stored in the storage unit 110. Thereafter, the calibration control unit 39 proceeds to step S15.

In step S <b> 15, the calibration control unit 39 causes the projection unit 20 to project a user interface for selecting whether or not to perform manual calibration related to the detection of the pointing position of the pointer 80, and the user's selection input is performed.
The calibration control unit 39 detects the operation of the remote controller or the operation panel 19 and determines whether or not to execute manual calibration (step S16).

When manual calibration is not executed (step S16; No), the calibration control unit 39 selects the initial correction data 125 (step S17) and shifts to normal operation (step S18).
In the normal operation, an image is projected on the screen SC based on an input image input to the image I / F unit 12, an instruction position indicated by the indicators 70 and 80 is specified, and processing according to the instruction content is performed. It is an operation to perform.

When performing manual calibration regarding the operation of the indicator 80 (step S16; Yes), the calibration control unit 39 selects the manual calibration image 122 (step S19).
Subsequently, the calibration control unit 39 projects the selected manual calibration image 122 on the screen SC by the projection unit 20 (step S20). Here, an operation using the indicator 80 is performed by the user (step S21), and the calibration control unit 39 executes an indication position detection process for detecting the indication position of the indicator 80 (step S22). The designated position detection process in step S22 is the same process as the designated position detection process in step S11 described above, and is executed as shown in FIG.

  In the indicated position detection process in step S22, the position detection unit 50 (indicator detection unit 54) uses the second position detection image data, the mask image, the first background image data, and the second background image data, The designated coordinate pointed by the finger 80a is detected. Note that the second background image data is an image of the screen SC taken around the light source 61 of the light emitting device 60 and its surroundings. For this reason, in addition to the background noise recorded in the first background image data, the light emitted from the light source 61 of the light emitting device 60 and the reflected light thereof are recorded in the second background image data. Noise caused by the light emitted from the light source unit 61 is referred to as light source noise. In step S <b> 8, the calibration control unit 39 generates light source noise data by subtracting the first background image data from the second background image data, and stores the light source noise data in the storage unit 110. FIG. 16 shows an example of light source noise data.

  The position detection unit 50 subtracts the first background image data and the light source noise data from the second position detection image data to generate difference image data from which the background noise and the light source noise are removed. The processing after generating the difference image data is the same as the processing shown in FIG. The position detection unit 50 passes the calculated barycentric coordinates to the calibration control unit 39. The calibration control unit 39 determines the center-of-gravity coordinates acquired from the position detection unit 50 as designated coordinates on the captured image data. Then, the calibration control unit 39 associates the designated coordinates on the captured image data with the coordinate values of the mark on the manual calibration image 122 of the corresponding mark and stores them in the storage unit 110 (step S23).

  The calibration control unit 39 determines whether or not the indicated position has been detected for all the marks in the manual calibration image 122 (step S24), and if there is an unprocessed mark, returns to step S20. When the indication positions of all the marks have been detected, the calibration control unit 39 creates the manual correction data 126 based on the indication position coordinates and the mark positions stored in step S23 (step S25). . The manual correction data 126 created here is stored in the storage unit 110. Thereafter, the calibration control unit 39 proceeds to step S18 and starts a normal operation.

Note that the calibration control unit 39 may generate manual calibration data 124 including data similar to the auto calibration data 123 by manual calibration of the indicator 70. In this case, the calibration control unit 39 generates manual calibration data 124 similar to the auto calibration data 123 by the processing of steps S8 to S14 in FIG. The auto calibration data 123 and the manual calibration data 124 may be the same data. In this case, the auto calibration data 123 generated in the past is overwritten by the data generated in step S14.
In this configuration, if the calibration control unit 39 executes either auto calibration or manual calibration, the coordinates of the pointing position of the pointer 70 can be obtained. Therefore, in the operation of FIG. 3, it is possible to select manual calibration in step S1 in a state where the auto calibration data 123 is not stored.

  Next, the light emission timings of the projector 10 (transmission unit 52), the indicator 70, and the light emitting device 60 (light source unit 61) during normal operation will be described with reference to the sequence diagram shown in FIG.

  In this sequence, the projector 10 becomes a master device, and notifies the indicator 70 and the light emitting device 60 of the light emission timing. The light source unit 61 of the light emitting device 60 and the indicator 70 emit light according to the timing notified from the projector 10. In addition, this sequence includes four phases from the first phase to the fourth phase, and the first phase to the fourth phase are repeated in order. When the process reaches the fourth phase, the process returns to the first phase, and the first to fourth phases are repeated again. Accordingly, the projector 10 can notify the indicator 70 of the light emission timing, and can perform shooting in synchronization with the light emission timing of the indicator 70 and the light emitting device 60.

  The first phase is a synchronization phase. In the first phase, the light source of the transmission unit 52 of the projector 10 is turned on, and an infrared signal for synchronization is transmitted to the indicator 70. The indicator 70 can recognize the start timing of the first phase by receiving the synchronization infrared signal. In addition, since each time of the 1st-4th phase is set to the same time, the indicator 70 also recognizes the start timing of the 2nd-4th phase by recognizing the start timing of the 1st phase. Can be recognized.

  The second phase is a position detection phase in which the light source unit 61 of the light emitting device 60 and the indicator 70 are turned on. The projector 10 captures an imaging range by the imaging unit 51 in accordance with the lighting timing of the light source unit 61 and the indicator 70. Thereby, when the indicator 80 points on the screen SC, the reflected light of the indicator 80 appears in the captured image data of the imaging unit 51. In addition, lighting light of the indicator 70 is reflected in the captured image data of the imaging unit 51.

The third phase is a phase provided for distinguishing the indication positions of the indicator 70 and the indicator 80 based on the light emission timings of the indicator 70 and the light emitting device 60. The third phase is an indicator determination phase in which the indicator 70 is lit with a unique lighting pattern set in the indicator 70, and the light source unit 61 is not lit. For this reason, the lighting light of the indicator 70 is reflected in the captured image data of the imaging unit 51. Among the bright spots photographed in the second phase, the position of the bright spot reflected in the photographed image data photographed in the third phase when the light emitting device 60 is turned off can be detected as the designated position of the indicator 70. That is, among the indicated coordinates detected from the captured image data captured in the second and fourth phases, the indicated coordinates close to the indicated coordinates detected from the captured image data captured in the third phase are the indicated coordinates of the indicator 70. It can be determined that
The indicator 70 is set with an ID for identifying the indicator 70. In the third phase, the indicator 70 is turned on according to the set ID. Even when a plurality of indicators 70 are used, an ID is set for each indicator 70, and each indicator 70 is lit according to the set ID. For example, it is assumed that “1000” is assigned as the ID of the indicator 70, and the setting is such that “1” turns on the indicator 70 and “0” turns off the indicator 70. In this case, the indicator 70 is turned on in the first third phase, and is turned off in the second to fourth third phases thereafter. Therefore, even when a pointing operation is performed using a plurality of pointers 70, the projector 10 can detect the pointing position of each pointer 70.

  The fourth phase is a position detection phase as in the second phase. The light source unit 61 and the indicator 70 of the light emitting device 60 are turned on, and imaging is performed by the imaging unit 51. Note that, for example, in the first phase, the second phase, and the fourth phase, the projector 10 may capture the imaging range by the imaging unit 51 and update the first and second background image data. By updating the first and second background image data even during normal operation, it is possible to improve the detection accuracy of the indicated coordinates by the indicators 70 and 80. For example, in the first phase, the projector 10 captures an imaging range by the imaging unit 51 after transmitting a synchronization infrared signal. The captured image data captured in the first phase is image data in which the indicator 70 and the light source unit 61 of the light emitting device 60 are not lit, and thus can be used as first background image data. In at least one of the second phase and the fourth phase, the projector 10 captures an image by the imaging unit 51 while the light source unit 61 of the light emitting device 60 is lit after the indicator 70 is turned on. Shoot the range. The captured image data is image data in which the indicator 70 is turned off and the light source unit 61 of the light emitting device 60 is turned on. For this reason, it can be used as the second background image data.

  As described above, the projection system 1 according to the present embodiment includes the light emitting device 60 that emits detection light to the detection area for detecting the indication positions of the indicators 70 and 80 and the indicators 70 and 80 in the detection area. And a projector 10 for detecting a designated position. The projector 10 includes an imaging unit 51 and a position detection unit 50. The imaging unit 51 images the detection area. The position detection unit 50 detects at least one of an image of light emitted from the indicator 70 and an image of detection light reflected by the indicator 80 from the captured image data. The position detection unit 50 distinguishes and detects the indication positions of the indicator 70 and the indicator 80 based on the light emission timings of the indicator 70 and the light emitting device 60. For this reason, even if it is a simple structure, the indication position of the indicator from which a kind differs can be detected accurately using picked-up image data.

  Further, the position detection unit 50 detects the position of the bright spot reflected in the captured image data captured after the light emitting device 60 is turned off as the indication position of the indicator 70. Therefore, the indicator 70 and the indicator 80 can be easily distinguished from each other, and the indication positions of the different types of indicators 70 and 80 can be detected with high accuracy.

  Further, the position detection unit 50 determines the light emitted by the plurality of indicators 70 according to the identification information assigned to each indicator 70 based on the captured image data of the imaging unit 51, and the plurality of indicators 70. The designated position is distinguished and detected. Therefore, even if there are a plurality of pointers 70, each pointer 70 can be distinguished and the pointing position of each pointer 70 can be detected.

  In addition, the projector 10 transmits to the light emitting device 60 a signal for notifying the light emitting device 60 of the timing for emitting the detection light to the transmission unit 52 that transmits a synchronization infrared signal that notifies the lighting timing of the indicator 70. And an emission device driving section 48 for transmitting. For this reason, the projector 10 can photograph the detection region by the imaging unit 51 in synchronization with the lighting timing of the indicator 70 and the light emitting device 60. Therefore, it is possible to detect the pointing positions of the indicator 70 and the indicator 80.

Note that the above-described embodiments and modifications are merely examples of specific modes to which the present invention is applied, and the present invention is not limited thereto, and the present invention can be applied as different modes. For example, the indicator is not limited to the pen-type indicator 70 or the indicator 80 that is a user's finger, and a laser pointer, an indicator bar, or the like may be used, and the shape and size thereof are not limited.
Moreover, in the said embodiment and modification, although the light-projection apparatus 60 was comprised separately from the main body of the projector 10, and illustrated the structure connected with the cable 60a, this invention is not limited to this. For example, the light emitting device 60 may be integrally attached to the main body of the projector 10 or may be configured to be built in the main body of the projector 10. Further, the light emitting device 60 may be supplied with power from the outside and connected to the emitting device driving unit 48 by a wireless communication line.

Moreover, in the said embodiment, although the position detection part 50 image | photographed the screen SC by the imaging part 51 and pinpointed the position of the indicator 70, this invention is not limited to this. For example, the imaging unit 51 is not limited to the one provided in the main body of the projector 10 and photographing the projection direction of the projection optical system 23. The imaging unit 51 may be arranged separately from the main body of the projector 10, or the imaging unit 51 may perform imaging from the side or front of the screen SC. Further, a plurality of imaging units 51 may be arranged, and the detection control unit 32 may detect the positions of the indicators 70 and 80 based on the captured image data of the plurality of imaging units 51.
In the above embodiment, the configuration in which the projector 10 transmits the synchronization signal to the indicator 70 using the infrared signal emitted from the transmission unit 52 to the indicator 70 has been described. However, the synchronization signal is an infrared signal. It is not limited to. For example, a synchronization signal may be transmitted by radio wave communication or ultrasonic wireless communication. This configuration can be realized by providing the projector 10 with a transmission unit that transmits signals by radio wave communication or ultrasonic radio communication, and providing the indicator 70 with a similar reception unit.

In the above embodiment, the light modulation device 22 that modulates the light emitted from the light source has been described by taking as an example a configuration using three transmissive liquid crystal panels corresponding to RGB colors. Is not limited to this. For example, a configuration using three reflective liquid crystal panels may be used, or a method in which one liquid crystal panel and a color wheel are combined may be used. Alternatively, a system using three digital mirror devices (DMD), a DMD system combining one digital mirror device and a color wheel, or the like may be used. When only one liquid crystal panel or DMD is used as the light modulation device, a member corresponding to a composite optical system such as a cross dichroic prism is unnecessary. In addition to the liquid crystal panel and DMD, any light modulation device capable of modulating light emitted from the light source can be employed without any problem.
In the above embodiment, a mode has been described in which the user performs an instruction operation with the indicators 70 and 80 on the screen SC (projection surface, display surface) on which the front projection type projector 10 projects (displays) an image. The user may perform an instruction operation on a display screen (display surface) on which an image is displayed on a display device (display unit) other than the projector 10. Also in this case, the light emitting device 60 and the imaging unit 51 may be configured integrally with the display device, or may be configured separately from the display device. Display devices other than the projector 10 include a rear projection type projector, a liquid crystal display, an organic EL (Electro Luminescence) display, a plasma display, a CRT (cathode ray tube) display, and a SED (Surface-conduction Electron-emitter Display). Etc. can be used.
Moreover, each function part of the projection system 1 shown in FIG. 2 shows a functional structure, and a specific mounting form is not particularly limited. That is, it is not always necessary to mount hardware corresponding to each function unit individually, and it is of course possible to adopt a configuration in which the functions of a plurality of function units are realized by one processor executing a program. In addition, in the above embodiment, a part of the function realized by software may be realized by hardware, or a part of the function realized by hardware may be realized by software. In addition, specific detailed configurations of other parts of the projection system 1 can be arbitrarily changed without departing from the spirit of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Projection system (position detection system), 10 ... Projector (position detection apparatus), 20 ... Projection part, 21 ... Light source part, 22 ... Light modulation apparatus, 23 ... Projection optical system, 30 ... Control part, 31 ... Projection control 32, detection control unit, 33 ... emission control unit, 39 ... calibration control unit, 40 ... image processing unit, 48 ... emission device drive unit (second transmission unit), 50 ... position detection unit (position detection unit) , 51 ... imaging unit (imaging unit), 52 ... transmission unit (first transmission unit), 54 ... indicator detection unit, 55 ... coordinate calculation unit, 60 ... light emitting device, 70 ... indicator (indicator), 80 ... Indicator (indicator) 110 ... Storage unit 121 ... Auto calibration image 122 ... Manual calibration image 123 ... Auto calibration data 124 ... Manual calibration Over data, 126 ... manual correction data, SC ... screen (detection area).

Claims (5)

A position detection system comprising: a light emitting device that emits detection light to a detection region that detects an indication position of an indicator; and a position detection device that detects an indication position of the indicator in the detection region,
The position detection device includes: an imaging unit that captures the detection region; an image of light emitted by a first indicator as the indicator from a captured image data of the imaging unit; and a second indicator as the indicator And detecting at least one of the detection light image reflected at the position of the first indicator and the second indicator based on a light emission timing of the first indicator and the light emitting device. A position detection unit that detects and distinguishes;
A position detection system comprising:   2. The position detection system according to claim 1, wherein the position detection unit detects a position of a bright spot reflected in captured image data after the light emitting device is turned off as an indication position of the first indicator.   The position detection unit determines, based on the captured image data, a light in which a plurality of first indicators blink according to identification information assigned to each first indicator, and the plurality of first indicators The position detection system according to claim 1, wherein the detection is performed by distinguishing the designated position of the body.   The position detection device transmits a signal for notifying the timing of emitting the detection light to the first transmission unit that transmits a synchronization signal for notifying the lighting timing of the first indicator, and the light emitting device. The position detection system according to claim 1, further comprising a second transmission unit that transmits to the emission device. A control method for a position detection system, comprising: a light emitting device that emits detection light to a detection region that detects an indication position of an indicator; and a position detection device that detects an indication position of the indicator in the detection region. ,
Photographing the detection area by a photographing unit;
Detecting at least one of an image of light emitted by the first indicator as the indicator and an image of the detection light reflected by the second indicator as the indicator from the captured image data of the imaging unit; Distinguishing and detecting the indication positions of the first indicator and the second indicator based on the light emission timing of the first indicator and the light emitting device;
A method for controlling a position detection system comprising: