KR20190041984A - Input system for a computer incorporating a virtual touch screen - Google Patents

Abstract

The present invention relates to a computer input system inputting information into a computer by including: a display device displaying an image of the computer; an input unit emitting infrared light of a predetermined wavelength through manipulation by a user or by making contact with or approaching a display area in which the display device displays an image; and a camera configured to photograph the display area and transmit information about the photographed image to the computer. The camera device includes a photographing element comprising a plurality of pixels and a multiband pass filter transmitting only light of a predetermined wavelength range in a wavelength range and visible light area of the infrared light emitted from the input device. Since the emission of the infrared light is synchronized with the photographing of the camera device, the infrared light is emitted or not emitted alternately by frame during the photographing of the camera device. The computer extracts information of an image corresponding to the infrared light from the input unit among images taken through the camera device and calculates position information on the coordinate system of the image transmitted from the display device to process the information as information inputted into the computer through the display device.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a computer input system in which a virtual touch screen is implemented.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a computer input system in which a virtual touch screen is implemented. More particularly, the present invention relates to a computer input system capable of controlling a computer or inputting various information, ≪ / RTI >

In a training site or a business scene, a large number of people frequently transmit learning contents or business contents from a computer to an image, and a display device such as a large-sized monitor or projector connected to the computer is used to provide such an image.

In the case of using a display device with a touch panel, a user controls the provided image while touching the display, or displays a character or a graphic on the display screen to input information to the computer and display the input information again on the display screen However, when the image provided by the computer is provided by being projected on a screen by a projector, or when a display device having no screen input function such as a normal LCD monitor is used, the user can proceed or stop the image with a remote controller or the like It is impossible to control the computer in the displayed screen or the monitor or to display the figure or letter on the screen or the monitor so as to be inputted into the computer.

As a technique for solving such an inconvenience, there is disclosed an invention relating to a computer input device that compares a laser-pointed image with a computer image using a camera to recognize a position and flicker, and a patent (Patent Document 1) of Patent No. 820573 936666 (Document 2) discloses an invention related to a 'projection screen image touch device using an infrared screen'.

In the invention of Document 1, a point light source is irradiated to a screen or a monitor with a laser pointer, a monitor is photographed with a camera, an image of the point light source from the laser pointer is recognized and received as an input to a computer, The camera corrects the coordinate system of the image captured by the camera according to the coordinate system of the image displayed by the computer.

However, in the invention of Document 1, since an image taken by a camera includes an image corresponding to an indoor illumination light or natural light in addition to a coordinate point correction image and a point light source of a laser pointer, the point light source of the laser pointer and the coordinate system Since significant errors must occur in the extraction of the correction image, the illumination can be lowered and the natural light can be used only under blocked or reduced conditions.

The invention of Document 2 is a configuration in which an infrared ray screen is formed by an infrared ray array and an infrared camera directed to an infrared screen is used to detect a distortion of an infrared ray on an infrared screen by a user's finger or the like to detect a point where a user's finger is located It is equipped.

However, in the invention of Document 2, since various apparatuses are used so that the coordinate system of the plane formed by the infrared screen coincides with the coordinate system of the image projected by the computer, there is a problem that the installation of the system is very complicated.

Further, when an external light source including an infrared ray is incident on the screen, an error is recognized which is recognized as a distortion in the infrared screen taken by the camera. Therefore, like the invention of Document 1, there is a problem that it can be used only under natural light of a certain intensity or less.

Document 1: Patent No. 820573 Document 2: Patent No. 936666

The present invention seeks to realize a virtual touch screen as a computer input system. Specifically, it is intended to provide a computer input system capable of interacting with a computer by controlling an operation of bringing a hand or a tool into contact with or approaching an area where a computer-provided image is displayed, or by inputting various information and displaying it on a display.

In particular, in consideration of the problems of the related art described above, the present invention intends to implement a virtual touch screen type computer input system which is relatively simple and can be used in any environment by a simple operation.

It is another object of the present invention to provide a computer input system that does not cause errors without being affected by such illumination or natural light even in an environment with illumination or natural light.

The above-mentioned object of the present invention can be achieved by a display device for displaying an image of a computer, input means for bringing a display device into contact with or approaching a display area for displaying an image or emitting infrared light of a predetermined wavelength by a user's operation, And a camera device configured to photograph a display area of the display device and transmit information of the photographed image to the computer.

In the computer input system of the present invention,

The camera device includes a multi-band pass filter for transmitting only the light of a predetermined wavelength band of the visible light region and the wavelength band of the infrared light emitted by the input device, and an image pickup device composed of a plurality of pixels,

The input means is arranged such that the light emission of the infrared light is synchronized with the photographing of the camera apparatus, the light emission of the infrared ray and the non-light emission are alternately performed in frame units of photographing during the photographing of the camera apparatus,

The computer extracts information of an image corresponding to infrared light from the input means among the images taken by the camera device, calculates position information in the coordinate system of the image to be transmitted to the display device, and processes the information as information input to the computer through the display device However,

G and B values of each pixel constituting an image pickup element are calculated from the information of the image taken by the camera device, and the R, G and B values in the frame at the time of infrared light emission of the input means and the values of R, G, and B values in the frame of the frame, extracts pixels whose result of the difference operation is not 0, and specifies the extracted pixels as pixels to which infrared light from the input means is incident, Is specified by the information of the image corresponding to the infrared light from the input means.

The operation of the computer input system of the present invention having such a configuration will be described.

An image is transmitted to a display device through a computer and a user controls the computer or inputs information such as a character or a graphic using input means, And emits light.

When the input means emits infrared light in the display area, the infrared light is taken by the camera device together with the image of the entire display area.

The entire image to be incident on the camera device is incident on the imaging element of the camera through visible light of a predetermined wavelength band and infrared light of a wavelength range of the infrared light of the input means while passing through the dual bandpass filter.

In an image taken by an image pickup device, in a case where infrared light is emitted while the input means is located in the display region and when the infrared light is not emitted from the image pickup element, the ratio of R, G, But the R, G, and B values or the ratios of these values in the pixel on which the infrared light from the input means is incident are different from the values or ratios when the infrared light from the input means is not incident.

In the computer input system of the present invention, since the infrared light emitted by the infrared light is synchronized with the photographing by the camera device, the infrared light and the non-light emitted during the photographing of the camera device are alternately arranged in frame units of photographing, The ratio of the R, G, and B values of the pixel to the pixel of which the ratio is different can be specified as the pixel on which the infrared light from the input means is incident.

Specifically, when the ratio of the R, G, and B values of the respective pixels constituting the image pickup element is calculated and the R, G, and B values of the contrasting pixels in adjacent frames are calculated differentially, Since the R, G, and B values are not changed in the pixel where the light is not incident, the result of the difference operation becomes 0, and the result of the difference operation becomes non-zero in the pixel in which the infrared light from the input means is incident.

Therefore, the pixel on which the infrared light from the input means is incident can be discriminated, and as a result of the discrimination, the pixel can be identified and treated as the input information from the input means.

As described above, in the computer input system according to the present invention, the infrared light of the input device is tuned to the photographing of the camera device, and the information of the image of the frame adjacent to the frames constituting the image captured by the camera device is compared with the contrast , It is possible to clearly distinguish between a pixel to which infrared light is input and a pixel to which no infrared light is input.

Particularly, according to this configuration, in addition to the input means of the present invention, even in an environment in which infrared light from a pointer or other device that emits infrared light passing through a multi-band pass filter is used, The visible light source can be removed so that the infrared light emission of the input means can be clearly discriminated as the input to the computer without being influenced by the input of the infrared light from this other device.

FIG. 1 is a schematic view showing a schematic configuration of an image system according to an embodiment of the present invention, wherein each element constituting an image system is conceptually shown.
2 is a view showing an embodiment in which a laser curtain is provided as an input device on a screen.
3 is a graph showing the light intensity according to the optical spectrum of each pixel in the CMOS.
4 is a graph showing the result of performing a difference operation between the RGB value of a frame that emits infrared rays and the frame that does not emit light.
5 is an image projected on a screen for coordinate system matching.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

First, with reference to FIG. 1, elements constituting a computer input system according to an embodiment of the present invention will be described.

The computer input system of the present embodiment includes a screen 20 as a display device and a projector 10 for projecting an image on the screen. The projector 10 is configured to be used by a user to grasp the screen and to emit infrared light of 850 nm wavelength at its tip A camera device 40 configured to capture a reflection image reflected from the screen 20, and a computer 50 to which the camera device and the projector are connected.

The projector 10 has a typical configuration for projecting a screen of a program executed in the computer 50 or an image such as various presentation contents onto the screen 20.

The screen 20 is a normal screen that can be installed on a wall surface of a room such as a lecture room or a conference room where the computer input system of the present embodiment is implemented and can be seen as an image is projected as needed. It is sufficient if the image is projected and reflected so that the projected image can be seen by people. For example, a wall surface of a building made of a white flat surface can also be used as the screen of the present invention, Is defined as an element having a surface that reflects the projected image of the projector.

The display device of the present invention is not limited to such a screen 20 and the projector 10 but may be configured to display an image of the computer 50 connected to the computer 50 such as an LCD monitor.

The camera device 40 is arranged to photograph the screen 20. The camera device 40 includes a lens mechanism for focusing incident light onto an image pickup element as in a typical camera, an image pickup element for converting light that is incident and focused through a lens mechanism into an electric signal, And a processor for processing and storing the image data as image data.

However, in the camera device 10 of this embodiment, a band pass filter for passing only light having a wavelength of a specific band is disposed between the lens mechanism and the image pickup element, and this band pass filter is composed of 850 nm infrared light And a multi band pass filter formed so as to pass only the light in the visible light region of the 550 nm band.

In the present embodiment, the pen 30 may be a computer 50, such as an operation of switching or stopping an image projected on a screen by a user who is presenting or lecturing using an image projected on the screen, And to display characters and figures on the screen.

A switch for providing power to the infrared light emitting element is provided at the tip end of the pen 30. When the tip of the pen contacts the screen 20, (on) so that the infrared light emitting element emits infrared light at a point where the pen 30 contacts the screen 20. [

An input device having a configuration similar to that of the pen 30 is shown in Fig. 3 of Document 1. The pen-shaped pointer of Document 1 is equipped with a switch for allowing the user to operate and emit a laser, and the pen of this embodiment may also be provided with a switch operated by the user. For example, when the pen of this embodiment is equipped with a switch that can be operated by a user, the user can manipulate the pen to approach a specific point on the screen and press the switch to emit infrared light.

On the other hand, in the present embodiment, the pen 30 that emits infrared light is used as the input means. Alternatively, a means that does not emit infrared light, such as a user's finger or a normal pen or pointer, may be used. Such an example is shown in Fig.

Fig. 2 shows an alternative embodiment in which a laser module 31 is provided as input means, which creates a laser curtain 32 parallel to the plane of the screen, close to the screen 20. Fig.

The laser module 31 is installed on the upper side of the screen 20 and scans a line beam of infrared light parallel to the screen 20 to form a laser curtain 32. The screen 20 displays an image 41 projected by the projector 40 and a laser curtain 32 is formed in front of the image.

The user's finger 33 forms one of the input means because the infrared light of the laser curtain 32 is not incident on the camera device 10 due to its linearity but allows the user to bring the finger close to the screen 20 When the laser curtain 32 is brought into contact with the finger, infrared light that forms a laser curtain hits the finger and is incident on the camera device 10 due to reflection and interference.

Infrared light from the tip of the finger 33, which is in contact with the laser curtain 32, is incident on the camera device 10, as in the case of using the pen 30 that emits infrared light.

2 and the infrared light generated by the laser module 31 in the alternative embodiment of FIG. 2, the wavelength of the infrared region through which the multi-band pass filter of the camera device 40 passes, That is, it emits infrared rays of 850 nm.

However, the infrared light emitted by the input device according to the present invention is not limited to this, but may be configured to emit infrared light belonging to an infrared wavelength range through which the multi-band pass filter passes.

On the other hand, the infrared ray transmitting module is provided in the camera device 30, and the processor of the camera device transmits a synchronizing signal to the imaging device through the infrared ray transmitting module, and the pen 30 is equipped with the infrared ray receiving module, And synchronizes the emission of the infrared light according to the synchronization signal.

The infrared light of the pen 30 is emitted alternately in a manner that the infrared light is emitted in one frame and the infrared light is not emitted in the next frame with respect to the frames constituting one period of the photographing of the camera apparatus 40. [ .

For example, if the camera device 40 shoots at 60 frames per second, the pen 30 is configured to emit alternately in accordance with the shooting of one frame per two frames at the time of the light emission.

That is, when the pen 30 comes into contact with the screen 20 to emit infrared light, the pen 30 is synchronized with the photographing of the camera device 40 during the entire time of the light emission and emits light in one frame, The infrared light from the pen 30 is incident on one frame of the image captured by the camera device 40 and the infrared light is not incident on the next frame.

In the case where the input device is constituted by the laser module 31 shown in Fig. 2, the laser module 31 is configured so that light emission and non-light emission are alternately performed in accordance with the synchronization signal from the camera device 40. [

The computer 50 is configured as a typical computer and executes a program or the like that shows learning content or business presentation content and sends an image of such content to the projector 10 or an execution screen of the computer on the screen 20, As shown in FIG.

The camera device 40 is connected to the computer 50 so that information on the image of the screen 20 captured by the camera device 40 is input and processed.

Next, with reference to FIG. 3 and FIG. 4, the image capturing in the camera device 40 and the processing in the computer 50 in the computer input system of the present embodiment will be described.

In a state in which the pen 30 does not emit light, the image photographed by the camera device 40 is photographed by external light that is irradiated on the screen area and reflected, and the image projected onto the screen 20 by the projector 10. Since the visible light and the infrared component are both included in the light source of natural light or illumination irradiated on the screen 20 and only the visible light component is included in the image from the projector 10, .

In this mixed image, only the component corresponding to the 550 nm band and the component corresponding to the 850 nm band of the multi-band pass filter are photographed by the image pickup element of the camera device 40 made of CMOS through the multi-band pass filter.

The image by the external light source and the image from the projector 10 and the infrared light (wavelength: 850 nm) from the pen 20 overlap each other at the time of photographing the frame in which the pen 20 emits light Is obtained.

In this mixed image, the external light source corresponding to the 550 nm band of the multi-band pass filter and the image of the projector 10, the external light source corresponding to the 850 nm band of the multi-band pass band filter, and the infrared light component of the pen 30, Passes through the band pass filter and is photographed by the image pickup element of the camera device 40. [

The R, G, and B pixels of the image sensor are arranged in a specific manner and the light-to-electricity conversion efficiency of each of the R, G, and B pixels is determined by the light of the wavelength corresponding to each pixel B, 450nm, G: 550nm, R: 650nm), but R, G, and B pixels exhibit similar photoelectric conversion efficiency in the infrared region. In FIG. 3, reference symbols A and B indicate the transmission band and transmittance of the multi-band pass filter.

In the image processing unit of the image pickup device, the R, G, and B pixels of a certain region are calculated, and the R, G, and B values of the pixels are set and output. do.

Depending on the characteristics of the CMOS and the characteristics of the multi-band pass filter, the image captured by the camera device 40 can be divided into a component of external light and a component of infrared light from the pen 30 as an input device.

The external light means light from all the light sources except for the infrared light from the pen 30, and includes light from natural light or illumination light or an image projected by the projector 10. [ The external light is separated into a visible light component in the 550 nm band and an infrared light component in the 850 nm band while being transmitted through the multi-band pass filter of the camera device 40, and the respective components are again represented by R, G, and B values. .

These separated components are shown below. In the following, 'EL' indicates external light, 'VF' indicates a visible band filter of 550 nm band passed through the multi band pass filter, 'IRF' indicates a band pass filter passing through the multi band pass filter And displays infrared light (IR band filter) of 850 nm band.

R_EL_VF: The R component that appears when external light passes through the 550 nm of the dual filter and enters the CMOS

R_EL_IRF: An infrared component of external light passes through 850 nm,

G_EL_VF: G component that appears when ambient light passes through 550nm of dual filter and enters CMOS

G_EL_IRF: The G component that appears when the infrared component of external light passes through 850nm and enters the CMOS

B_EL_VF: B component that appears when ambient light passes through 550nm of dual filter and enters CMOS

EL_IRF: The component that acts on the B pixel that appears when the infrared component of external light passes through 850nm and enters the CMOS.

Infrared light from the pen 30, which is an input device, does not contain a visible light component, so it is composed of the following three components.

R_IR_IRF: The R component that appears when the IR light source enters the CMOS through 850 nm.

G_IR_IRF: The G component that appears when the IR light source enters the CMOS through 850 nm.

B_IR_IRF: The B component that appears when the IR light source enters the CMOS through 850nm.

1. The ratio of the light intensity in the RGB pixels of the imaging element at the time of non-emission of the pen 30

When the image photographed in the non-light emitting state of the pen 30 passes through the multi-band pass filter and enters the CMOS, only the 550 nm band and the 850 nm band components pass through the CMOS due to the action of the multi-band pass filter.

In this case, the R and B components of the visible light region are removed by the action of the 550 nm band filter, and R_EL_IRF, G_EL_IRF, and B_EL_IRF have the same value because the RGB photoelectric conversion efficiencies of the CMOS are similar in the 850 nm band. However, The value of G_EL_VF exists due to the action of the filter, and the magnitude (n) of the value depends on the transmittance of the filter or the like.

For example, if the transmittance of the multiband pass filter is adjusted such that G_EL_VF is four times the value of G_EL_IRF, the relative value of the G component of the external light becomes approximately 5 when the other component is set to 1, and the ratio of the light intensity Is measured as follows.

R: G: B = (R_EL_IRF): (G_EL_IRF + G_EL_VF): (B_EL_IRF)

= 1: (1 + G_EL_VF): 1

= 1: (1 + n): 1

= 1: 5: 1

Where n is the relative size of G_EL_VF when G_EL_IRF is 1.

2. The ratio of the R, G, and B values in the pixel of the imaging element at the time of light emission of the pen 30

On the other hand, in the light emission mixed image captured by the camera device 40 when the pen 30 emits infrared light, in the remaining pixels except the pixels in which the light source of the pen 30 is incident in the CMOS, The pixels corresponding to the light source of the pen 30 are composed of the sum of the external light source and the infrared light from the pen 30. [

Since the infrared ray IR of the pen 30 does not have a visible light component, the light intensities of the infrared rays incident on the R, G, and B pixels of the CMOS have the same value (a), and thus they can be expressed as follows.

R: G: B = R_EL_IRF + R_IR_IRF: G_EL_IRF + G_EL_VF + G_IR_IRF: (B_EL_IRF + B_IR_IRF)

= (1 + a): (1 + n + a): (1 + a)

In this case, the value of a is relatively larger than 1, and the value of n is equal to the value of non-emission, so n = 4 is applied

(1 + a): (1 + a) = (1 + a) + (4 +

If the RGB value is divided by (1 + a)

R: G: B = 1 + a / 1 + a + 1 / a + 1 + a / 4 / (1 + a): 1

Here, the light intensity (a) of the infrared light incident on the R, G, and B pixels of the CMOS is always equal to or greater than 0, so 1 / (1 + a) <4.

Assuming that the light intensity of the infrared light component G_EL_IRF of the external light and the infrared light G_IR_IRF of the pen 30 are the same, that is, G_IR_IRF is 1, 4 / (1 + a) = 2, 30 emit infrared light, the infrared light from the pen of the CMOS is incident on the pixel

R: G: B = 1: 3: 1.

The ratio of the light intensities in the RGB pixels is determined according to the intensity of the infrared light of the pen 30 and varies in a range larger than 1 and smaller than 5.

That is, when the pen 30 emits light, the ratio of the R, G, and B values in the pixel where the infrared light from the pen 30 does not enter is

R: G: B = 1: 5: 1,

The ratio of the R, G, and B values in the pixel on which the infrared light from the pen 30 is incident is

R: G: B = 1: less than 5: 1, so that a portion where infrared light from the pen 30 is not incident and a portion where infrared light is incident are distinguished at a ratio of light intensity of RGB pixels in CMOS.

As a result, when the values of R: G: B of adjacent pixels in the CMOS are continuously analyzed, as shown in Fig. 4, the light intensities at the pixels of R and B in the region other than the light source of the pen 30 There is little change in the value, but a change in the ratio of the value of the light intensity in the G pixel occurs sharply.

4, the ratio of the R: G: B value for each pixel to the output image of the CMOS with respect to the area of the screen 20 on which the infrared ray of the pen 30 emits infrared light is relatively large in the light source region of infrared light, Is lowered.

Therefore, it can be determined that the pixel in the section where the value in the G pixel relatively decreases-increases is the light source area.

As another method of determining the area of the infrared light of the pen 30, the camera device 40 carries out a difference operation with respect to pixels of the image photographed by the pen 30 in the infrared light emitting state and the non-light emitting state, Can be determined.

For the non-light source area, the R, G, and B values of the pixels in the frame of the image at the time of light emission and the frame of the image at the time of the non-light emission immediately adjacent thereto are the same, The other part is removed.

That is, when the difference calculation is performed by defining the RGB values of the light source area when F (R: G: B) is emitted and the RGB values of the light source area when D (R: G: B)

(B_EL_IRF): (R_EL_IRF): (G_EL_IRF + G_EL_VF): (B_EL_IRF): (R_EL_IRF) )

(R: G: B) = 0: 0: 0 is finally obtained because the components due to the external light at the time of light emission and at the time of non-light emission are the same.

The difference calculation in the light source area is expressed as RGB value of the light source area when F_L (R: G: B) is emitted and D_L (R: G: B) And performs a difference operation.

(R_EL_IRF + R_IR_IRF): (G_EL_IRF + G_EL_VF + G_IR_IRF): (B_EL_IRF + B_IR_IRF)

D_L (R: G: B) = (R_EL_IRF) :( G_EL_IRF + G_EL_VF): (B_EL_IRF)

Here, the RGB components of the visible light are the same at the time of light emission and at the time of non-light emission

G_EL_IRF + G_EL_IRF: G_EL_IRF + G_IR_IRF: (B_EL_IRF + B_IR_IRF) - (R_EL_IRF): (G_EL_IRF + G_EL_VF): (B_EL_IRF) )

= R_IR_IRF: G_IR_IRF: B_IR_IRF

In this way, only the light source area portion remains at a constant ratio of R: G: B, and the light source area can be extracted.

According to this method, in addition to the pen 30 which is synchronized with the camera device 40 on the screen 20 in which the light emission and the non-light emission alternate, a device for emitting infrared light of the same 850 nm band, for example, Even when the light from an infrared microwave or an infrared remote controller using light is irradiated on the screen, since these devices do not alternately emit / not emit infrared light, the infrared light of these devices is transmitted to the multi Even if the band-pass filter is transmitted, the result of the difference calculation of RGB in the area corresponding to these light sources is 0: 0: 0 by the above-described difference calculation, and it is judged that it is not infrared light of the pen 30 like external light .

As described above, the position of the light source of the infrared light from the pen 30 is discriminated from the image taken by the camera device 40, and the position and the movement locus of the light source discriminated in the continuous photographing can be discriminated.

The thus-determined information is determined by a control operation of the computer 30 by the pen 30 or an operation of drawing or writing a figure on the screen image provided by the computer 50 by the pen 30, May be controlled or characters or graphics may be input and displayed on the screen 20 through the projector 10 again.

On the other hand, since the image captured by the camera device 40 is distorted, the position of the light source of the infrared light from the pen 30 extracted from the distorted image is directly converted into the coordinate system of the image projected onto the screen by the computer 50 The position of an accurate light source can not be determined.

Therefore, in the computer input system of the present embodiment, the computer 50 is capable of displaying the image of the screen 20 captured by the camera device 40 in a coordinate system of the image projected on the screen, And the coordinate system of the image of the screen 20 to be photographed by the camera device 40 is matched.

5 shows an image of a reference point for matching of a coordinate system projected onto the screen 20 by the computer 50 via the projector 10. In FIG.

5, the frame line is not actually an image projected through the projector 10. This is a virtual representation of the circumference of the image projected through the projector 50 by the computer 50. The projector 10 has a plurality of green reference points To the projector. Here, the reference point is green, which has a wavelength band of 550 nm through which the multi-band pass filter of the camera apparatus 40 can pass.

These circles are arranged in a specific position with respect to the image projected by the projector 10 and the camera 50 captures the image and the computer 50 determines the positions of the circles in the captured image, The coordinate system of the photographed image can be set in accordance with the coordinate system of the image to be projected through the projector 10. [

An image for setting the coordinate system is input to the camera device 40. An image for a green point serving as a reference point for setting a coordinate system in the multiband pass filter is passed through a filter in a visible light region and is incident on an imaging element, .

In addition to the green reference point image, the image captured by the camera device includes natural light or illumination light incident on the building and reflected from the screen. However, in the multi-band pass filter of the camera device 40, most of the light components of natural light and illumination light are not transmitted Only the light component passing through the multi-band pass filter passes through the image relating to the green reference point and the natural light or the illumination light so that the light component of the image corresponding to the reference point is emphasized in the light incident on the image pickup device, The image of the reference point can be clearly identified in the captured image.

The configuration of the computer input system of the present embodiment described above and the overall operation according to each individual operation will be described.

The computer input system of the present embodiment is configured such that the computer 50 uses the coordinate system of the image projected on the screen 30 through the projector 10 and the coordinate system of the image captured by the camera apparatus 40, A pattern image including information of a reference coordinate system, that is, an image including an image of a reference point shown in Fig. 5, is projected onto the screen 20 through the projector 10 in order to correct distortion caused by photographing in the projector 40 .

The image of this image is applied to the CMOS image sensor through the multi-band pass filter in the camera device 40, and the CMOS transmits the photographed image to the computer 50. The computer 50 compares the distorted image with the image of the reference pattern projected through the projector 10, analyzes the distortion information of the image acquired by the camera device to generate matching data with the reference pattern, The position distorted by the camera device 40 is converted into the position of the reference coordinate system by using the matching data when the touching screen 20 emits light.

When the pen 30 touches the screen 20 when the user draws a picture using the pen 30 or performs a mouse operation, the user uses the alternate light emission during the contact time in accordance with the infrared optical synchronization signal sent by the camera device 40 .

The camera device 40 continuously photographs the screen area including the light emission / non-light emission of the continuous pen 30, performs a difference operation between the light emission frame and the non-light emission frame, and calculates the ratio of the RGB value of each pixel And extracts the light source area. The extracted light source area is converted into a reference coordinate system using matching data obtained in the matching process, and the computer 50 draws a line or performs a mouse operation according to the movement of the light source area.

In the computer input system of the present embodiment described above, the light emission of the input device and the photographing of the camera device are synchronized to perform a difference operation between the light emitted by the input device and the non-lighted image during the time the input device contacts the screen, It is possible not only to detect the position of the correct input device by analyzing the RGB ratio but also to eliminate interference with devices using infrared light of the same wavelength band as the input device as well as noise such as natural light and illumination.

10: Projector 20: Sprin
30: pen 40: camera device
50: Computer

Claims (3)

A display device for displaying an image of a computer, input means for causing the display device to come in contact with or close to a display area for displaying an image, or to emit infrared light of a predetermined wavelength by a user's operation, A computer input system for inputting information to a computer, the camera input device comprising:
The camera device includes a multi-band pass filter for transmitting only the light of a predetermined wavelength band of the visible light region and the wavelength band of the infrared light emitted by the input device, and an image pickup device composed of a plurality of pixels,
The input means is arranged such that the light emission of the infrared light is synchronized with the photographing of the camera apparatus, the light emission of the infrared ray and the non-light emission are alternately performed in frame units of photographing during the photographing of the camera apparatus,
The computer extracts information of an image corresponding to infrared light from the input means among the images taken by the camera device, calculates position information in the coordinate system of the image to be transmitted to the display device, and processes the information as information input to the computer through the display device However,
G and B values of each pixel constituting the image pickup device from the information of the image photographed by the camera device, and calculates R, G and B values of the respective pixels in the frame at the time of infrared light emission of the input means, G, and B values of the respective pixels in the adjacent non-light emitting frame are calculated to extract pixels whose result of the difference operation is not 0, and the extracted pixels are output to the pixels to which the infrared light from the input means is incident And specifies the information of the pixel specified and specified as the information of the image corresponding to the infrared light from the input means. The method according to claim 1,
Wherein the input device is configured to be grasped by a user of the information processing apparatus and to include infrared light emitting means to contact the screen or emit infrared light by user's manipulation. The method according to claim 1,
The input device includes infrared light irradiating means for irradiating the infrared light in the vicinity of the display area of the display device and in parallel with the plane of the screen so that a part of the body of the computer user or a part of the article carried by the user approaches the display area And is configured to reflect infrared light from the infrared light irradiating means.

Images