WO2018203595A1

WO2018203595A1 - Projector capable of touch interaction

Info

Publication number: WO2018203595A1
Application number: PCT/KR2018/002237
Authority: WO
Inventors: 임성현
Original assignee: 임성현
Priority date: 2017-05-04
Filing date: 2018-02-23
Publication date: 2018-11-08

Abstract

Provided is a projector, which can be carried by a user and is capable of touch interaction without any other operation when moving. The provided projector comprises: a panel for converting light from a light source into an image; a pattern beam providing unit for outputting a pattern beam; a single lens for projecting, onto a screen, the pattern beam and the image from the panel, and receiving a signal generated by a touch interaction on the screen; a sensor for recognizing, on the basis of the signal generated by a touch interaction, location coordinates of a touch object currently touched on the screen; and a light splitting element for sending the light from the light source to the panel, passing the image through the panel so as to send the same to the lens, reflecting the pattern beam so as to send the same to the lens, and sending the signal generated by a touch interaction, received from the lens, to the sensor.

Description

Projector with touch interaction

The present invention relates to a projector that is capable of touch interaction, and more particularly, to a projector that enables touch interaction on a screen projected from a small beam projector.

Recently, as portable devices such as smartphones and tablet PCs are widely used, products linked with portable devices have gained great popularity among consumers.

In particular, portable projectors capable of displaying large screens such as smartphones are gaining popularity.

However, the portable projector can only view the screen largely, and when a user tries to control and manipulate the contents of the screen, the user inputs a command by directly touching the touch screen or pressing a key of the portable device (eg, a smartphone). In this case, when the user touches the screen of the portable device, the position of the portable device which is placed at a specific position in a focused state may also be shifted. In this case, the image projected to the outside may be shaken.

To solve this problem, a technology for making a direct touch of the projector screen has been developed, an interactive projector is being released, and consumers are interested in related products.

However, currently available touch-interacting projectors are large in size because they have a lens for projecting a screen and a lens for detecting an interaction. In addition, it is inconvenient to perform a calibration for interaction every time the projector is moved. That is, conventionally, in order to recognize a touch in the beam projector, a lens for recognizing the touch must be separately provided in addition to the lens for projecting the screen, and the touch area must be calibrated according to the illuminated screen area.

The present invention has been proposed to solve the above-mentioned conventional problems, and an object thereof is to provide a projector that can be carried by a user and enables touch interaction without any other operation (correction process) every time the user moves.

In order to achieve the above object, a projector capable of touch interaction according to a preferred embodiment of the present invention includes a panel for converting light from a light source into an image; A pattern beam providing unit configured to output a pattern beam; A single lens for projecting the pattern beam and the image from the panel onto a screen and receiving a signal by touch interaction on the screen; A sensor for recognizing position coordinates of the touch object currently touched on the screen based on the signal generated by the touch interaction; And transmitting light from the light source to the panel, transmitting an image from the panel to the lens, reflecting the pattern beam to the lens, and receiving a signal by the touch interaction received from the lens. It includes; an optical separation element for sending to the sensor.

The panel may be configured as a reflective panel, the light source may be installed opposite the first surface of the optical separation element, the panel may be installed opposite the second surface of the optical separation element, The lens may be installed to face the third surface of the light splitter, and the pattern beam providing unit and the sensor may be installed to face the fourth face of the light splitter.

And an interaction signal processor configured to determine which touch object is generated based on the position coordinates from the sensor and convert the touch event into a corresponding touch event.

It may further include a filter for removing the sunlight or the light of the light source does not affect the sensor.

The pattern beam may be formed of infrared light.

The signal by the touch interaction may be infrared light whose pattern is distorted or changed in shape at a corresponding touch point as the user touches a touch object with a body part in the image projected on the screen.

The signal generated by the touch interaction may be infrared light having a changed amount of light at a corresponding touch point compared to another point as the user touches a touch object with a body part in the image projected on the screen.

On the other hand, a projector capable of touch interaction according to another preferred embodiment of the present invention, a panel for converting light from the light source into an image; A pattern beam providing unit configured to output a pattern beam; A single lens for projecting the pattern beam and the image from the panel onto a screen and receiving a signal by touch interaction on the screen; A sensor for recognizing position coordinates of the touch object currently touched on the screen based on the signal generated by the touch interaction; And an optical separation element for transmitting an image from the panel to the lens, reflecting the pattern beam to the lens, and sending a signal of the touch interaction received from the lens to the sensor. do.

The panel may be formed of a transmissive panel, the light source and the panel may be installed to face the first surface of the optical separation element, the light source may be installed at the rear of the panel, and the lens is the optical separation element The pattern beam providing unit and the sensor may be installed to face the third surface of the optical splitter.

On the other hand, a projector capable of touch interaction according to another preferred embodiment of the present invention, a panel for converting light from the light source into an image; A single lens for projecting an image from the panel onto a screen and receiving a signal by touch interaction on the screen; A sensor for recognizing position coordinates of the touch object currently touched on the screen based on the signal generated by the touch interaction; And an optical separation element for transmitting light from the light source to the panel, transmitting an image from the panel to the lens, and transmitting a signal of the touch interaction received from the lens to the sensor. do.

The panel may be configured as a reflective panel, the light source may be installed opposite the first surface of the optical separation element, the panel may be installed opposite the second surface of the optical separation element, The lens may be installed to face the third surface of the optical separation element, and the sensor may be installed to face the fourth surface of the optical separation element.

The signal by the touch interaction may be specific light emitted by the touch device at the touch point as the user touches a touch object with the touch device in the image projected on the screen.

The specific light may be infrared light.

On the other hand, a projector capable of touch interaction according to another preferred embodiment of the present invention, a panel for converting light from the light source into an image; A single lens for projecting an image from the panel onto a screen and receiving a signal by touch interaction on the screen; A sensor for recognizing position coordinates of the touch object currently touched on the screen based on the signal generated by the touch interaction; And an optical separation element configured to transmit an image from the panel to the lens and to transmit a signal due to the touch interaction received from the lens to the sensor.

The panel may be formed of a transmissive panel, the light source and the panel may be installed to face the first surface of the optical separation element, the light source may be installed at the rear of the panel, and the lens is the optical separation element It may be installed to face the second surface of the sensor, the sensor may be installed to face the third surface of the optical separation element.

According to the present invention having such a configuration, it is possible to enable screen projection and touch recognition through a single lens, and to operate the touch area according to the screen area without additional calibration.

For various projection surfaces (non-planar object screens such as rectangles, circles, and cylinders, and ultra-wide-angle projection planes), projectors that can be touched on any surface using a dedicated pen can be used to control the contents or OS that are displayed on portable and portable small projectors. Can be implemented.

In the related art, the screen may be operated only on a flat surface, but according to the present invention, it may be used while changing the interaction environment without the complicated correction process that has to be performed every time the environment changes.

In addition, interactions can be applied to contents that are created without changing a space configuration, such as an object screen, and can be utilized as various services and content technologies.

Recently, as the head-up display (HUD) of the automotive electronics market or aftermarket is activated, the utilization of the present invention is expected to increase further.

In addition, the pattern beam and the recognition sensor have been developed as a sensor with integrated coordinate planes, so that 3D recognition and texture mapping of natural input images can be performed at the same time, thus serving as a 3D scanner for smartphones. It can be used.

1 is a view for explaining the overall operation concept of a projector capable of touch interaction according to an embodiment of the present invention.

2 is a diagram illustrating an example of an internal configuration of the projector illustrated in FIG. 1.

3 to 5 are diagrams for explaining the reason why the calibration of the configuration of Figure 2 is not necessary.

FIG. 6 is a diagram illustrating a recognition area of a sensor illustrated in FIG. 2.

FIG. 7 is a flowchart for describing a touch interaction operation using the projector illustrated in FIG. 2.

8 is a diagram illustrating another example of an internal configuration of the projector illustrated in FIG. 1.

FIG. 9 is a flowchart for describing a touch interaction operation using the projector illustrated in FIG. 8.

10 is a view for explaining the overall operation concept of the projector capable of touch interaction according to another embodiment of the present invention.

FIG. 11 is a diagram illustrating an example of an internal configuration of the projector illustrated in FIG. 10.

FIG. 12 is a diagram illustrating a recognition area of a sensor illustrated in FIG. 11.

FIG. 13 is a flowchart for describing a touch interaction operation using the projector illustrated in FIG. 11.

14 is a diagram illustrating another example of the internal configuration of the projector shown in FIG. 10.

FIG. 15 is a flowchart for describing a touch interaction operation using the projector illustrated in FIG. 14.

16 is a block diagram showing a configuration of an infrared ray-based copyboard system according to an embodiment of the present invention.

FIG. 17 is a block diagram illustrating an exemplary embodiment of the configuration of the infrared signal processing apparatus illustrated in FIG. 16.

18 is a flowchart illustrating an embodiment of a method of operating an infrared signal processing apparatus according to the present invention.

19 is a block diagram illustrating an embodiment of a configuration of a terminal device illustrated in FIG. 16.

20 is a flowchart illustrating an embodiment of a method of operating a terminal device according to the present invention.

21 is a flowchart illustrating a parameter automatic setting method for infrared signal processing according to an embodiment of the present invention.

22 is a flowchart illustrating an embodiment of a method of setting an infrared brightness threshold of an infrared sensor.

23 and 24 are diagrams for describing embodiments of a method of changing an infrared brightness threshold according to a radius of an infrared region.

FIG. 25 is a block diagram illustrating another exemplary embodiment of the configuration of the terminal device illustrated in FIG. 16.

26 is a flowchart illustrating a parameter setting method for infrared signal processing according to another embodiment of the present invention.

27 is a flowchart illustrating another embodiment of a method of setting an infrared brightness threshold of an infrared sensor.

28 and 29 are diagrams for describing an example of a method of setting an infrared brightness threshold according to a brightness item designated by a user.

30 is a flowchart illustrating a filtering method for infrared signal processing according to an embodiment of the present invention.

31 is a flowchart illustrating an embodiment of a method of determining a frame motion event by analyzing an infrared signal.

32 to 36 are diagrams for describing embodiments of a method of detecting an infrared coordinate of an electronic pen by filtering an infrared signal.

37 is a flowchart illustrating a view angle setting method according to an embodiment of the present invention.

38 to 44 are diagrams for describing embodiments of a method of operating an angle of view guide in a terminal device.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description.

However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. In the following description of the present invention, the same reference numerals are used for the same elements in the drawings and redundant descriptions of the same elements will be omitted.

The projector 100 capable of touch interaction according to the exemplary embodiment of the present invention is positioned to be spaced apart from the screen 200 by a predetermined distance.

The projector 100 is small enough to be portable and mobile, and may be referred to as a pico projector, for example.

Of course, since the projector 100 can be utilized in various digital signages, it is applicable not only to a mobile projector but also to a large and difficult environment for calibration (correction) for interaction such as a large object screen.

The projector 100 may project a predetermined image (eg, including one or more touch objects (selection menu)) and a pattern beam (eg, an infrared pattern beam) to the screen 200 through the

lenses

33 and 43. have. Here, the pattern beam will spread evenly so as to cover the entire front surface of the screen 200.

In addition, when a touch interaction occurs on the screen 200, the projector 100 may receive a signal corresponding thereto.

In addition, the projector 100 may process a touch event corresponding to the generated touch interaction.

Here, the touch interaction is generated by the user's touch. For example, if a user touches a touch object with a body part (eg, a finger) on an image projected on the screen 200 (eg, including one or more touch objects (selection menu)), the touch point Pattern distortion or shape change will occur. That is, an infrared pattern beam (for example, a structured light beam) is formed on the front surface of the screen 200. When a touch object is touched by a finger, a pattern distortion or shape change is performed at the touch point by the finger. Will cause. As such, when the touch object is touched by a body part, pattern distortion or shape change may occur at a corresponding touch point. Accordingly, the infrared light whose pattern is distorted or changed in shape at the corresponding touch point as compared to other points may be referred to as an example of a signal by touch interaction described in the claims of the present invention. Using the pattern distortion or shape change described above will also contribute to the enlargement of the distance between the projector 100 and the screen 200.

On the other hand, in addition to using the above-described pattern distortion or shape change, a light amount change can also be used. For example, if a user touches a touch object with a body part (eg, a finger) on an image projected on the screen 200 (eg, including one or more touch objects (selection menu)), the touch point Infrared light amount changes occur. That is, the infrared pattern beam of a certain amount of light is uniformly distributed on the front surface of the screen 200. When one touch object is touched with a finger, the amount of light of the infrared pattern beam at the corresponding touch point is determined by the infrared pattern beam at another position. It will be different from the amount of light. In other words, since diffused reflection or the like occurs in the finger, the receiving side will receive an infrared pattern beam having a smaller amount of light than other positions. As such, when the touch object is touched by a body part, a change in the amount of infrared light may occur at a corresponding touch point. Therefore, infrared light having a changed amount of light at the corresponding touch point compared to other points may be referred to as an example of a signal by touch interaction described in the claims of the present invention.

The screen 200 may be a projection screen, and the screen 200 may be formed of a flat surface as shown in FIG. 1.

However, the screen 200 may be made of an uneven surface. For example, even if a predetermined image is projected toward an area where the boundaries of two flat surfaces touch each other, the projected images will be divided into two surfaces and appear to be naturally connected to each other.

2 is a diagram illustrating an example of an internal configuration of the projector illustrated in FIG. 1. The projector of FIG. 2 may be referred to as a digital light processing (DLP) projector, and may directly touch the screen 200 with a body part (eg, a finger).

The projector 100 illustrated in FIG. 2 includes a light source 30, a panel 11, an infrared pattern beam providing unit 32, a lens 33, a filter 34, a sensor 35, and an interaction signal processor 36. And an optical separation element 37.

The light source 30 emits predetermined light. For example, the light source 30 may include a laser diode, an LCD, an LED, and the like.

Light emitted from the light source 30 may be referred to as light that becomes an image to be displayed on the screen 200.

The panel 31 may convert light (light) generated by the light source 30 into an image that can be viewed by a person. In other words, an image based on the light from the light source 30 is formed on the panel 31. That is, the panel 31 may be formed with an image to be displayed on the screen 200.

Here, the panel 31 may be composed of a reflective panel. Accordingly, the light source 30 is provided on the side spaced apart from the panel 31 by a predetermined distance.

The infrared pattern beam providing unit 32 outputs an infrared pattern beam projected on the front surface of the screen 200. The infrared pattern beam will spread evenly to cover the entirety of the screen 200. The infrared pattern beam may refer to a beam of a predetermined pattern formed of infrared light. The infrared pattern beam may be an example of the pattern beam described in the claims of the present invention.

The lens 33 magnifies an image from the panel 31 and sends it to the screen 200.

The lens 33 sends the infrared pattern beam from the infrared pattern beam providing unit 32 to spread evenly over the front surface of the screen 200.

In addition, the lens 33 may receive an infrared signal (light) due to a touch interaction on the screen 200. For example, when a user touches a touch object with a body part, the lens 33 may receive infrared light having a pattern distortion or a shape change at a corresponding touch point. As another example, when the user touches the touch object with a body part, the lens 33 may receive infrared light having a changed amount of light at the touch point. Of course, lens 33 will also receive infrared light at points other than touch points.

The lens 33 is installed at the front of the projector 100.

The lens 33 may be installed to be exposed from the projector 100 as shown in FIG. 1.

The filter 34 removes the sunlight or light of the light source 30 so as not to affect the sensor 35. In other words, the filter 34 reflects only the infrared signal (infrared signal with light quantity change and infrared signal without light quantity change) incident on the screen 200 and reflected through the lens 33 and the optical separation element 37. Other signals are removed.

The filter 34 is preferably installed at the front end of the sensor 35.

In FIG. 2, the light emitted from the light source 30 is not all reflected by the light splitting element 37 to be incident on the panel 31, but some of the light may pass through the light splitting element 37 as it is. For example, assuming that the light emitted from the light source 30 is a laser, the laser includes an infrared region, so that light corresponding to the infrared region passes through the optical separation element 37 as it is and enters the sensor 35. Can be. In this case, the sensor 35 will perform abnormal sensing. Accordingly, the filter 34 is installed at the front end of the sensor 35 to remove light (that is, light of the light source 30) that does not need to be applied to the sensor 35.

In addition, sunlight may flow into the projector 100 from the outside of the projector 100. In this case, when the sunlight introduced into the projector 100 is incident to the sensor 35, the sensor 35 may perform abnormal sensing. Accordingly, the filter 34 is installed at the front of the sensor 35 to remove light (that is, sunlight) that can be applied to the sensor 35.

The sensor 35 recognizes the position coordinates of the touch object currently touched based on the received infrared signal (that is, including a signal by touch interaction). That is, the sensor 35 may receive an infrared signal when the user touches any one of the touch objects on the screen 200, and recognize the position coordinates of the touch object based on the received infrared signal. A detailed description of the position coordinate recognition operation of the sensor 35 will be described later.

The interaction signal processor 36 may determine to which touch object a touch event occurs based on the position coordinates from the sensor 35.

Meanwhile, the interaction signal processor 36 may convert the touch event into a corresponding touch event when determining which touch object has occurred.

The interaction signal processor 36 may process the touch event.

The light separation element 37 reflects the light from the light source 30 toward the panel 31, and transmits the image from the panel 31 to the lens 33. The optical separation element 37 reflects the infrared pattern beam from the infrared pattern beam providing unit 32 and sends it to the lens 33. In addition, the optical separation element 37 reflects the infrared signal (light) incident through the lens 33 toward the sensor 35 side.

More specifically, the inclined surface 37a is formed inside the optical separation element 37. The oblique surface 37a may reflect the light from the light source 30 toward the panel 31 side, and transmit the image from the panel 31 as it is. The inclined surface 37a may reflect the infrared pattern beam from the infrared pattern beam providing unit 32 toward the lens 33 side. In addition, the inclined surface 37a may reflect the infrared signal (which may be light) incident through the lens 33 toward the sensor 35.

In FIG. 2, the optical separation element 37 may be configured of a hexahedron. The light source 30 may be provided to face the first surface (eg, the left surface) of the light splitter 37. The panel 31 may be provided to face the second side (eg, the rear side) of the optical separation element 37. The lens 33 may be provided to face the third surface (eg, the front surface) of the light separation element 37. The infrared pattern beam providing unit 32, the filter 34, and the sensor 35 may be provided to face the fourth surface (eg, the right side) of the optical separation element 37.

In FIG. 2, the interaction signal processor 36 is installed inside the projector 100, but may be installed inside a portable device (eg, a smartphone) or a PC on which an OS is mounted. If the interaction signal processor 36 is installed in the portable apparatus or the PC, a separate transmission module may be included in the sensor 35 to transmit the position coordinate value to the portable apparatus or the PC.

Conventional touch interaction projectors are provided with a projection lens for projecting a predetermined image onto the screen as shown in Figure 3, and the interaction lens for receiving a signal (light) according to the touch interaction on the screen, respectively.

Accordingly, in the conventional touch-interactable projector, a difference occurs in the recognition area due to the distance between the two lenses as shown in FIG. 3. That is, the interaction range and the projection range are different from each other.

Thus, as shown in FIG. 4, the range of simultaneous recognition is increased by adjusting the installation angles of the projection lens and the interaction lens. In this process, since the optical axes of the projection lens and the interaction lens are different from each other, a calibration is performed to eliminate the error. The error between the two lenses varies with the distance and angle between the screen and the projector, so a calibration was required whenever the projector installation position was changed.

However, in the present invention, as shown in Fig. 5, since the projection and the interaction are processed by one lens 33, the optical axes of the projection lens and the interaction lens are coincident with each other. Therefore, since the optical axes are the same, there is no error and there is no error, so no calibration is necessary.

In other words, according to the present invention, the projection range and the interaction range become the same as in Fig. 5, and the recognizable range (i.e., the recognition area) is enlarged.

As described above, since a separate calibration is not required by the configuration of FIG. 2, the recognition area (range) of the sensor 35 will be fixed as shown in FIG. 6.

In the case of using the pattern distortion or the shape change, without the touch, the sensor 35 will recognize the infrared signal (ie the infrared pattern beam) without the pattern distortion or the shape change at all positions in the recognition area. However, if there is a touch, pattern distortion or shape change will occur at the corresponding touch point in the recognition area. Therefore, when the touch point is present in the recognition area, the sensor 35 can easily calculate the X and Y coordinate values of the touch point based on the recognition area.

On the other hand, in the case of using the light amount, if there is no touch, the sensor 35 will recognize that the infrared signal (that is, the infrared pattern beam) at all positions in the recognition area is the same light amount. However, if there is a touch, only the amount of light of the infrared signal at the corresponding touch point in the recognition area will be different. Therefore, when the touch point is present in the recognition area, the sensor 35 can easily calculate the X and Y coordinate values of the touch point based on the recognition area.

For example, the above-described X, Y coordinate values may range from 0 to 4095. That is, the position coordinate values of X and Y in the sensor 35 may be calculated as a value between 0 and 4095 and input to the interaction signal processor 36. If necessary, the sensor 35 may further calculate an X, Y coordinate value of the touch point by further employing an algorithm for identifying the fingertip point. Algorithms for identifying the fingertips will be fully understood by those skilled in the art.

The coordinate values required by the projector 100 will vary depending on the resolution of the projector 100. For example, if the resolution of the projector 100 is 1920 × 1080, the coordinate values required by the projector 100 may be 0 to 1980 horizontally and 0 to 1080 vertically.

Accordingly, the interaction signal processor 36 converts the coordinate data of the sensor 35 into real world coordinate data (that is, coordinate data used by the projector) using a warping algorithm or a geometric transformation. can do. In addition, the interaction signal processor 36 may map the converted coordinate data into a corresponding touch event. The interaction signal processor 36 may process the touch event.

This time, the touch interaction operation using the projector shown in FIG. 2 will be described with reference to the flowchart of FIG. 7.

First, the power of the projector 100 is turned on (S100).

Accordingly, the light source 30 is turned on, and the infrared pattern beam providing unit 32 is turned on (S102, S104).

The light source 30 emits light (for example, laser light) to the light splitting element 37, and the light emitted from the light source 30 is reflected on the inclined surface 37a of the light splitting element 37 so that the panel ( It enters into 31) (S106). As a result, a predetermined image is formed on the panel 31. That is, since light emitted from the light source 30 may be referred to as light to be displayed on the screen 200, a predetermined image may be formed on the panel 31 by the light from the light source 30. In addition, the infrared pattern beam providing unit 32 outputs the infrared pattern beam to the optical separation element 37.

Subsequently, the image of the panel 31 passes through the optical separation element 37 to be directed to the lens 33, and the infrared pattern beam is reflected from the oblique surface 37a to be directed to the lens 33 (S108).

The image transmitted to the lens 33 is projected onto the screen 200 through the lens 33, and the infrared pattern beam is spread evenly on the front surface of the screen 200 through the lens 33 (S110).

Accordingly, the screen 200 not only distributes the infrared pattern beam but also displays one or more touch objects (ie, a selection menu). Here, there is no limitation of the touch object. If the smartphone screen is projected on the screen 200 through the projector 100, all menus (calling, KakaoTalk, message, Internet, etc.) in the smartphone may be touch objects. If the PC screen is projected on the screen 200, all menus in the PC may be touch objects.

Thereafter, the user views an image (eg, including one or more touch objects (selection menus)) displayed on the screen 200 and touches a touch object with a body part (eg, a finger).

Accordingly, when the position of the touch interaction is detected based on the pattern distortion or the shape change, the pattern distortion or the shape change will occur at the corresponding touch point. That is, an infrared pattern beam (for example, a structured light beam) is formed on the front surface of the screen 200. When a touch object is touched by a finger, a pattern distortion or shape change is performed at the touch point by the finger. Will cause.

In contrast, when the location of the touch interaction is detected based on the change in the amount of infrared light, the change in the amount of infrared light will occur at the corresponding touch point. That is, the infrared pattern beam of a certain amount of light is uniformly distributed on the front surface of the screen 200. When one touch object is touched with a finger, the amount of light of the infrared pattern beam at the corresponding touch position is determined by the infrared pattern beam at the other position. It will be different from the amount of light.

As such, when a user touches a touch object with a body part (for example, a finger) in an image projected on the screen 200, a touch interaction occurs at a corresponding touch point (“Yes” in S112).

Thereafter, the infrared signal at the point where the touch interaction occurs and the infrared signal at other points without the touch interaction are incident on the screen separation element 37 through the lens 33. Infrared signals (lights) incident on the optical separation element 37 are reflected by the oblique plane 37a and passed through the filter 34 to the sensor 35 (S114).

The infrared signal (light) at the point where the touch interaction occurs and the infrared signal (light) at other points without the touch interaction enter both the recognition area of the sensor 35. Accordingly, the sensor 35 coordinates (ie, X, Y coordinates) of the corresponding touch point at which the pattern distortion or shape change has occurred, or coordinate values (ie, X, Y coordinates) of the corresponding touch point having a difference in the amount of light. Value) can be easily recognized (calculated or converted) (S116).

Then, the sensor 35 transmits the recognized position coordinate value to the interaction signal processor 36 (S118).

The interaction signal processor 36 converts the position coordinate value of the sensor 35 into real world coordinate data, and maps the converted coordinate data into a corresponding touch event (S120).

Then, the interaction signal processor 36 processes the corresponding touch event (S122).

As described above, screen projection and touch recognition are possible through one lens.

In addition, each time it moves, touch interaction is possible without going through another operation (correction process). For example, even if the angle and direction of the screen projected after the movement are different from those before the movement, or the projected shape such as the curved TV is changed, the touch interaction can be performed without calibration.

8 is a diagram illustrating another example of an internal configuration of the projector illustrated in FIG. 1. The projector of FIG. 8 may be referred to as an LCD projector or an LED projector, and may directly touch the screen 200 with a body part (eg, a finger).

The projector 100 illustrated in FIG. 8 includes a light source 40, a panel 41, an infrared pattern beam providing unit 42, a lens 43, a filter 44, a sensor 45, and an interaction signal processor 46. , And an optical separation element 47.

The light source 40 emits predetermined light. For example, the light source 40 may include a laser diode, an LCD, an LED, and the like.

Light emitted from the light source 40 may be referred to as light that becomes an image to be displayed on the screen 200.

The panel 41 may convert light (light) generated by the light source 40 into an image that can be viewed by a person. In other words, the panel 41 has an image based on the light from the light source 40. That is, it can be said that the panel 41 has an image to be displayed on the screen 200.

Here, the panel 41 may be composed of a transmissive panel. Thus, the light source 40 is installed behind the panel 41.

The infrared pattern beam providing unit 42 outputs an infrared pattern beam of a predetermined amount of light projected on the front surface of the screen 200. The infrared pattern beam will spread evenly to cover the entirety of the screen 200. The infrared pattern beam may refer to a beam of a predetermined pattern formed of infrared light. The infrared pattern beam may be an example of the pattern beam described in the claims of the present invention.

The lens 43 magnifies an image from the panel 41 and sends it to the screen 200.

The lens 43 sends the infrared pattern beam from the infrared pattern beam providing unit 42 to spread evenly over the front surface of the screen 200.

In addition, the lens 43 may receive an infrared signal (light) due to a touch interaction on the screen 200. For example, when the user touches the touch object with a body part, the lens 43 may receive infrared light whose pattern is distorted or changed in shape at the touch point. As another example, when the user touches the touch object with a part of the body, the lens 43 may receive infrared light having a changed amount of light at the touch point. Of course, lens 43 will also receive infrared light at points other than touch points.

The lens 43 is installed at the front of the projector 100.

The lens 43 may be installed to be exposed from the projector 100 as shown in FIG. 1.

Filter 44 removes sunlight or light from light source 40 so as not to affect sensor 45. In other words, the filter 44 reflects only the infrared signal (infrared signal with light quantity change and infrared signal without light quantity change) incident on the screen 200 and reflected through the lens 43 and the light splitter 47. Other signals are removed.

The filter 44 is preferably installed at the front end of the sensor 45.

Detailed description of the purpose of employing the filter 44 is sufficient to explain the filter 34 in FIG. 2 described above.

The sensor 45 recognizes the position coordinates of the touch object currently touched based on the received infrared signal (that is, including a signal by touch interaction). That is, the sensor 45 may receive an infrared signal when the user touches any touch object on the screen 200, and recognize the position coordinates of the touch object based on the received infrared signal. A more detailed description of the position coordinate recognition operation of the sensor 45 may be replaced with the contents of the sensor 35 described above with reference to FIG. 6.

The interaction signal processor 46 may determine to which touch object a touch event occurs based on the position coordinates from the sensor 45.

On the other hand, the interaction signal processor 46 may determine which touch object the touch event has occurred to convert it to the corresponding touch event.

The interaction signal processor 46 may process the touch event.

In other words, the interaction signal processor 46 converts the coordinate data of the sensor 45 into real world coordinate data (that is, coordinate data used by the projector) using a warping algorithm or a geometric transformation. can do. In addition, the interaction signal processor 46 may map the converted coordinate data into a corresponding touch event. The interaction signal processor 46 may process the touch event.

The optical separation element 47 transmits the image from the panel 41 to the lens 43. The optical separation element 47 reflects the infrared pattern beam from the infrared pattern beam providing unit 42 and sends it to the lens 43. In addition, the optical separation element 47 reflects the infrared signal (light) incident through the lens 43 toward the sensor 45.

More specifically, the inclined surface 47a is formed inside the optical separation element 47. The oblique surface 47a can transmit the image from the panel 41 as it is. The inclined surface 47a may reflect the infrared pattern beam from the infrared pattern beam provider 42 toward the lens 43. The oblique surface 47a may reflect the infrared signal (which may be light) incident through the lens 43 toward the sensor 45.

In FIG. 8, the optical separation element 47 may be configured of a hexahedron. The light source 40 and the panel 41 may be provided to face the first surface (eg, the rear surface) of the light splitter 47. The lens 43 may be provided to face the second surface (eg, the front) of the light splitter 47. The infrared pattern beam providing unit 42, the filter 44, and the sensor 45 may be provided to face the fourth surface (eg, the right side) of the light splitter 47.

In FIG. 8, the interaction signal processor 46 is installed inside the projector 100, but may be installed inside a portable device (eg, a smartphone) or a PC on which an OS is mounted. If the interaction signal processor 46 is installed in the portable apparatus or the PC, a separate transmission module may be included in the sensor 45 to transmit the position coordinate value to the portable apparatus or the PC.

In addition, calibration is unnecessary even with the configuration of FIG. 8 as described above. The reason why no calibration is required will be fully understood with reference to the above-described contents of FIGS. 3 to 5.

Next, the touch interaction operation using the projector shown in FIG. 8 will be described with reference to the flowchart of FIG. 9.

First, the power of the projector 100 is turned on (S200).

As a result, the light source 40 is turned on, and the infrared pattern beam providing part 42 is turned on (S202 and S204).

A light beam (for example, laser light) of the light source 40 is directly incident to the rear surface of the panel 41, and a predetermined image is formed on the panel 21 (S206 and S208). That is, since the light emitted from the light source 40 may be referred to as light to be an image to be displayed on the screen 200, a predetermined image will be formed on the panel 41 by the light from the light source 40. In addition, the infrared pattern beam providing unit 42 outputs the infrared pattern beam to the optical separation element 47.

Subsequently, the image of the panel 41 passes through the optical separation element 47 and is directed to the lens 43, and the infrared pattern beam is reflected from the inclined surface 47a and directed to the lens 43 (S210).

The image transmitted to the lens 43 is projected onto the screen 200 through the lens 43, and the infrared pattern beam is spread evenly on the front surface of the screen 200 through the lens 43 (S212).

As such, when a user touches a touch object with a body part (for example, a finger) in the image projected on the screen 200, a touch interaction occurs at a corresponding touch point (“Yes” in S214).

Thereafter, the infrared signal at the point where the touch interaction occurs and the infrared signal at other points without the touch interaction are incident on the screen separation device 47 through the lens 43. Infrared signals (lights) incident on the light splitter 47 are reflected by the oblique plane 47a and passed through the filter 44 to the sensor 45 (S216).

Both the infrared signal (light) at the point where the touch interaction occurred and the infrared signal (light) at other points without the touch interaction enter the recognition area of the sensor 45. Accordingly, the sensor 45 may determine a coordinate value (ie, X and Y coordinate values) of the corresponding touch point where the pattern distortion or shape change has occurred, or a coordinate value (ie, X and Y coordinates) of the corresponding touch point with a difference in light quantity. Value) can be easily recognized (calculated or converted) (S218).

Then, the sensor 45 transmits the recognized position coordinate value to the interaction signal processor 46 (S220).

The interaction signal processor 46 converts the position coordinate value of the sensor 45 into real world coordinate data, and maps the converted coordinate data into a corresponding touch event (S222).

Then, the interaction signal processor 46 processes the touch event (S224).

The projector 100 capable of touch interaction according to another embodiment of the present invention is positioned to be spaced apart from the screen 200 by a predetermined distance.

The projector 100 may project a predetermined image (eg, including one or more touch objects (selection menu)) to the screen 200 through the lenses 12 and 22.

Here, the touch interaction is a touch device when the user touches any touch object with a dedicated touch device such as an infrared pen on an image projected on the screen 200 (eg, including one or more touch objects (selection menu)). May mean to emit infrared light with respect to the touch point. Therefore, the infrared light emitted by the touch device at the touch point can be referred to as an example of the signal by the touch interaction described in the claims of the present invention.

The screen 200 may be a projection screen, and the screen 200 may be formed of a flat surface as shown in FIG. 10.

FIG. 11 is a diagram illustrating an example of an internal configuration of the projector illustrated in FIG. 10. The projector of FIG. 11 may be referred to as a digital light processing (DLP) projector and uses a dedicated touch device such as an infrared pen. Here, the touch device may output infrared light.

The projector 100 illustrated in FIG. 11 includes a light source 10, a panel 11, a lens 12, a filter 13, a sensor 14, an interaction signal processor 15, and an optical separation element 16. Include.

The light source 10 emits predetermined light. For example, the light source 10 may include a laser diode, an LCD, an LED, and the like.

Light emitted from the light source 10 may be referred to as light that becomes an image to be displayed on the screen 200.

The panel 11 may convert light (light) generated by the light source 10 into an image that can be viewed by a person. In other words, the panel 11 bears an image based on the light from the light source 10. That is, the panel 11 may be formed with an image to be displayed on the screen 200.

Here, the panel 11 may be composed of a reflective panel. Accordingly, the light source 10 is provided on the side spaced apart from the panel 11 by a predetermined distance.

The lens 12 magnifies an image from the panel 11 and sends it to the screen 200.

The lens 12 may receive an infrared signal (light) by touch interaction on the screen 200. That is, the lens 12 may receive infrared light emitted by the touch device at the touch point.

The lens 12 is preferably installed at the front of the projector 100 and is positioned to face the screen 200.

The lens 12 may be installed to be exposed from the projector 100 as shown in FIG. 10.

The filter 13 removes the sunlight or light of the light source 10 so as not to affect the sensor 14. In other words, the filter 13 passes only the infrared signal reflected by the screen 200 and incident through the lens 12 and the optical separation element 16 and removes other signals.

The filter 13 is preferably installed at the front end of the sensor 14.

In FIG. 11, the light emitted from the light source 10 inside the projector 100 is not reflected by the light separating element 16 and is incident on the panel 11, but some of the light is separated from the light separating element 16. Permeable. For example, assuming that the light emitted from the light source 10 is a laser, the laser includes an infrared region, so that light corresponding to the infrared region passes through the optical separation element 16 as it is and enters the sensor 14. Can be. In this case, the sensor 14 will perform abnormal sensing. Thus, the filter 13 is installed at the front end of the sensor 14 to remove light (that is, light of the light source 10) that does not need to be applied to the sensor 14.

In addition, sunlight may flow into the projector 100 from the outside of the projector 100. In this case, when the sunlight introduced into the projector 100 is incident on the sensor 14, the sensor 14 may perform abnormal sensing. Accordingly, the filter 13 is installed in front of the sensor 14 to remove light (ie, sunlight) that may be applied to the sensor 14.

The sensor 14 recognizes the position coordinates of the touch object currently touched based on the received infrared signal (ie, a signal by touch interaction). That is, the sensor 14 receives an infrared signal (ie, generated by a touch device) when the user touches any touch object on the screen 200, and based on the received infrared signal, position coordinates of the touch object. Can be recognized. A detailed description of the position coordinate recognition operation of the sensor 14 will be described later.

The interaction signal processor 15 may determine to which touch object a touch event occurs based on the position coordinates from the sensor 14.

Meanwhile, the interaction signal processor 15 may convert a touch event into a corresponding touch event when determining which touch object has occurred.

The interaction signal processor 15 may process the corresponding touch event.

The light separation element 16 reflects the light from the light source 10 toward the panel 11 side, transmits the image from the panel 11 to the lens 12. In addition, the optical separation element 16 reflects the infrared signal (light) incident through the lens 12 to the sensor 14 side.

More specifically, the inclined surface 16a is formed inside the optical separation element 16. The inclined surface 16a may reflect light from the light source 10 toward the panel 11 side, and transmit the image from the panel 11 as it is. In addition, the inclined surface 16a may reflect the infrared signal (which may be light) incident through the lens 12 toward the sensor 14.

In FIG. 11, the optical separation element 16 may be configured of a hexahedron. The light source 10 may be provided to face the first surface (eg, the left surface) of the light separation element 16. The panel 11 may be installed to face the second side (eg, the rear side) of the light separation element 16. The lens 12 may be provided to face the third surface (eg, the front surface) of the light separation element 16. The filter 13 and the sensor 14 may be provided to face the fourth side (eg, the right side) of the optical separation element 16.

In FIG. 11, the interaction signal processor 15 is installed inside the projector 100, but may be installed inside a portable device (eg, a smartphone) or a PC on which an OS is mounted. If the interaction signal processor 15 is installed in the portable apparatus or the PC, a separate transmission module may be included in the sensor 14 to transmit the position coordinate value to the portable apparatus or the PC.

The reason why no calibration is required by the configuration of FIG. 11 is the same as the content described above with reference to FIGS. 3 to 5. Those skilled in the art will be able to fully understand based on the description of Figures 3 to 5, so a description thereof will be omitted.

As described above, since a separate calibration is not required by the configuration of FIG. 11, the recognition area (range) of the sensor 14 will be fixed as shown in FIG. 12. The sensor 14 can easily calculate the X and Y coordinate values based on the recognition area when the infrared light enters the recognition area. For example, the range of X, Y coordinate values may be 0 to 4095. That is, the position coordinate values of X and Y in the sensor 14 may be calculated as a value between 0 and 4095 and input to the interaction signal processor 15. If necessary, the sensor 35 may further calculate an X, Y coordinate value of the touch point by further employing an algorithm for identifying the end point of the touch device. Algorithms that can grasp the end point of the touch device will be fully understood by those skilled in the art through known techniques.

Meanwhile, the coordinate values required by the projector 100 will vary depending on the resolution of the projector 100. For example, if the resolution of the projector 100 is 1920 × 1080, the coordinate values required by the projector 100 may be 0 to 1980 horizontally and 0 to 1080 vertically.

Accordingly, the interaction signal processor 15 converts the coordinate data of the sensor 14 into real world coordinate data (that is, coordinate data used by the projector) using a warping algorithm or a geometric transformation. I can convert it. In addition, the interaction signal processor 15 may map the converted coordinate data into a corresponding touch event. The interaction signal processor 15 may process the corresponding touch event.

This time, touch interaction operations using the projector shown in FIG. 11 will be described with reference to the flowchart in FIG.

First, the power of the projector 100 is turned on (S10).

Accordingly, the light source 10 is turned on to operate. That is, as the power of the projector 100 is turned on, the light source 10 emits light (for example, laser light) to the light splitter 16 (S12).

The light emitted from the light source 10 is reflected by the oblique surface 16a of the light separation element 16 and is incident on the panel 11 (S14).

Accordingly, a predetermined image is formed on the panel 11. That is, since light emitted from the light source 10 may be referred to as light to be displayed on the screen 200, a predetermined image may be formed on the panel 11 by light from the light source 10. Since the image also emits light, the image of the panel 11 passes through the optical separation element 11 and is directed to the lens 12 (S16).

The image transmitted to the lens 12 is projected onto the screen 200 through the lens 12 (S18). Accordingly, one or more touch objects (ie, selection menus) are displayed on the screen 200. Here, there is no limitation of the touch object. If the smartphone screen is projected on the screen 200 through the projector 100, all menus (calling, KakaoTalk, message, Internet, etc.) in the smartphone may be touch objects. If the PC screen is projected on the screen 200, all menus in the PC may be touch objects.

Thereafter, the user views an image (eg, including one or more touch objects (selection menu)) displayed on the screen 200 and touches one touch object with a dedicated touch device such as an infrared pen. Accordingly, the touch device outputs an infrared signal (light) at the corresponding point (that is, the position where the touch object is touched).

As such, when a user touches a touch object with a dedicated touch device in an image projected on the screen 200, a touch interaction occurs in which the touch device emits an infrared signal (light) at a corresponding touch point (S20). "Yes").

Accordingly, the infrared signal (light) at the touch point is reflected by the screen 200 and is incident on the optical separation element 16 through the lens 12. The infrared signal (light) incident on the optical separation element 16 is reflected by the oblique plane 16a and passed through the filter 13 to the sensor 14 (S22).

As the infrared signal (light) enters the recognition area, the sensor 14 recognizes (calculates or converts) coordinate values (that is, X and Y coordinate values) of the corresponding position (touch point) (S24).

Then, the sensor 14 transmits the recognized position coordinate value to the interaction signal processor 15 (S26).

The interaction signal processor 15 converts the position coordinate value of the sensor 14 into real world coordinate data, and maps the converted coordinate data into a corresponding touch event (S28).

Then, the interaction signal processor 15 processes the corresponding touch event (S30).

14 is a diagram illustrating another example of the internal configuration of the projector shown in FIG. 10. The projector of FIG. 14 may be referred to as an LCD projector or an LED projector, and uses a dedicated touch device such as an infrared pen. Here, the touch device may output infrared light.

The projector 100 illustrated in FIG. 14 includes a light source 20, a panel 21, a lens 22, a filter 23, a sensor 24, an interaction signal processor 25, and an optical separation element 26. Include.

The light source 20 emits predetermined light. For example, the light source 20 may include a laser diode, an LCD, an LED, and the like.

The light emitted from the light source 20 may be referred to as light that becomes an image to be displayed on the screen 200.

The panel 21 may convert light (light) generated by the light source 20 into an image that can be viewed by a person. In other words, the panel 21 bears an image based on the light from the light source 20. That is, it can be said that the image to be displayed on the screen 200 is formed on the panel 21.

Here, the panel 21 may be composed of a transmissive panel. Accordingly, the light source 20 is installed behind the panel 21.

The lens 22 magnifies an image from the panel 21 and sends it to the screen 200.

The lens 22 may receive an infrared signal (light) by touch interaction on the screen 200. That is, the lens 22 may receive infrared light emitted by the touch device at the touch point.

The lens 22 is preferably installed at the front of the projector 100 and is positioned to face the screen 200.

The lens 22 may be installed to be exposed from the projector 100 as shown in FIG. 10.

The filter 23 removes the sunlight or light of the light source 20 so as not to affect the sensor 24. In other words, the filter 23 passes only the infrared signal reflected by the screen 200 and incident through the lens 22 and the optical separation element 26 and removes other signals.

The filter 23 is preferably installed in front of the sensor 24.

A detailed description of the purpose of employing the filter 23 is sufficient to describe the filter 13 in FIG. 11 described above.

The sensor 24 recognizes the position coordinates of the touch object currently touched based on the received infrared signal (light) (that is, a signal by touch interaction). That is, the sensor 24 receives an infrared signal (ie, generated by the touch device) when the user touches any touch object on the screen 200 and based on the received infrared signal, position coordinates of the touch object. Can be recognized. A more detailed description of the position coordinate recognition operation of the sensor 24 will be replaced with the above description based on FIG. 12.

The interaction signal processor 25 may determine to which touch object a touch event occurs based on the position coordinates from the sensor 24.

Meanwhile, the interaction signal processor 25 may convert a touch event into a corresponding touch event when determining which touch object has occurred.

The interaction signal processor 25 may process the touch event.

In other words, the interaction signal processor 25 converts the coordinate data of the sensor 24 into real world coordinate data (that is, coordinate data used in the projector) using a warping algorithm or a geometric transformation. I can convert it. In addition, the interaction signal processor 25 may map the converted coordinate data into a corresponding touch event. The interaction signal processor 25 may process the touch event.

The optical separation element 26 transmits the image from the panel 21 to the lens 22. In addition, the optical separation element 26 reflects the infrared signal (light) incident through the lens 22 toward the sensor 24.

More specifically, the slanted surface 26a is formed inside the light separation element 26. The oblique surface 26a may transmit the image from the panel 21 as it is. In addition, the inclined surface 26a may reflect the infrared signal (which may be light) incident through the lens 22 toward the sensor 24.

In FIG. 14, the optical separation element 26 may be configured of a hexahedron. The light source 20 and the panel 21 may be installed to face the first surface (eg, the rear surface) of the light separation element 26. The lens 22 may be installed to face the second surface (eg, the front surface) of the light separation element 26. The filter 23 and the sensor 24 may be provided to face the third side (eg, the right side) of the optical separation element 26.

In FIG. 14 described above, the interaction signal processing unit 25 is installed inside the projector 100. The interaction signal processing unit 25 may be installed inside a portable device (for example, a smartphone) or a PC equipped with an OS. If the interaction signal processor 15 is installed in the portable apparatus or the PC, a separate transmission module may be included in the sensor 14 to transmit the position coordinate value to the portable apparatus or the PC.

In addition, the configuration of FIG. 14 as described above also eliminates the need for calibration. The reason why no calibration is required will be fully understood with reference to the above-described contents of FIGS. 3 to 5.

This time, touch interaction operations using the projector shown in FIG. 14 will be described with reference to the flowchart of FIG. 15.

First, the power of the projector 100 is turned on (S40).

Accordingly, the light source 20 is turned on to operate. That is, as the power of the projector 100 is turned on, a light beam (eg, laser light) of the light source 20 is directly incident on the rear surface of the panel 21 (S42 and S44).

Accordingly, a predetermined image is formed on the panel 21 (S46).

Since the image of the panel 21 also emits light, the image of the panel 21 passes through the optical separation element 21 and is directed to the lens 22 (S48).

The image transmitted to the lens 22 is projected onto the screen 200 through the lens 22 (S50). Accordingly, one or more touch objects (ie, selection menus) are displayed on the screen 200. Here, there is no limitation of the touch object. If the smartphone screen is projected on the screen 200 through the projector 100, all menus (calling, KakaoTalk, message, Internet, etc.) in the smartphone may be touch objects. If the PC screen is projected on the screen 200, all menus in the PC may be touch objects.

As such, when a user touches a touch object with a dedicated touch device in an image projected on the screen 200, a touch interaction occurs in which the touch device emits an infrared signal (light) at the touch point (S52). "Yes").

Accordingly, the infrared signal (light) at the touch point is reflected by the screen 200 and is incident on the optical separation element 26 through the lens 22. The infrared signal (light) incident on the optical separation element 26 is reflected by the oblique plane 26a and passed through the filter 23 to the sensor 24 (S54).

As the infrared signal (light) enters the recognition area, the sensor 24 recognizes (calculates or converts) coordinate values (that is, X and Y coordinate values) of the corresponding position (touch point) (S56).

Then, the sensor 24 transmits the recognized position coordinate value to the interaction signal processing unit 25 (S58).

The interaction signal processor 25 converts the position coordinate value of the sensor 24 into real world coordinate data, and maps the converted coordinate data into a corresponding touch event (S60).

Then, the interaction signal processing unit 25 processes the touch event (S62).

The projector that is capable of touch interaction as described above is applicable to the infrared-based copyboard system described below.

16 is a block diagram illustrating a configuration of an infrared based electronic blackboard system according to an embodiment of the present invention. The illustrated electronic blackboard system includes an electronic pen 300, an infrared signal processing apparatus 400, and a terminal device 500. And a display device 600.

Referring to FIG. 16, when the user moves the electronic pen 300 on the screen of the display apparatus 600 to write, the infrared signal processing apparatus 400 recognizes infrared rays emitted from the electronic pen 300. By sending the corresponding coordinate information to the terminal device 500.

The terminal device 500 processes the coordinate information received from the infrared signal processing device 400 and transmits the coordinate information to the display device 600 so that the writing content is visualized through the display device 600.

More specifically, the electronic pen 300 may have an infrared light source (not shown) for emitting infrared rays at one end thereof to emit infrared rays.

The infrared signal processing apparatus 400 may include an infrared sensor (or an infrared camera module) for recognizing infrared rays. The infrared signal processing apparatus 400 may analyze an infrared signal output from the infrared sensor, perform noise filtering and alignment, and may include an electronic pen ( The infrared signal emitted from 300 may be screened to obtain coordinate information corresponding thereto.

According to an embodiment of the present invention, the infrared signal processing apparatus 400 automatically sets the parameters for processing the infrared signal as described above according to the surrounding environment, and the infrared based electronic blackboard system according to the embodiment of the present invention. Wherever it is installed, it can be operated without errors adaptively to the mobile environment.

The terminal device 500 is equipped with electronic blackboard operating software for processing and visualizing coordinate information transmitted from the infrared signal processing apparatus 400. The electronic blackboard operating software corrects the coordinate information or provides an angle of view setting function. And a function of designating, recognizing, and processing a region of interest (ROI).

The terminal device 500 may be implemented as a mobile phone, a smart phone, a tablet PC, a notebook computer, a personal computer (PC) in a desktop form, and the like, but the present invention is not limited thereto. Installed in the display apparatus 600, the configuration and functions of the terminal apparatus 300 according to an embodiment of the present invention may be integrated into the display apparatus 600 and implemented.

In addition, the copyboard operating software may be installed and operated in the form of a cloud service or an application service provider (ASP) in a server.

The display device 600 may be implemented as a TV, a monitor, a projector, or the like, but the present invention is not limited thereto, and the display device 600 may include a screen such as a smartphone, a tablet PC, or the like, in which a writing using the electronic pen 300 may be performed. It may be a mobile device provided.

According to another embodiment of the present invention, the terminal device 500 transmits the coordinate information of the electronic pen 300 to a relay communication server (not shown), and is also a display device which is various service clients capable of communicating with the relay communication server. (Not shown) allows the writing to be synchronized.

FIG. 17 is a block diagram illustrating an embodiment of the configuration of the infrared signal processing apparatus shown in FIG. 16. The infrared signal processing apparatus 400 illustrated includes an infrared receiver 410, a signal processor 420, and parameter settings. The unit 430 may include a short range communication module 440, a control unit 450, and a storage unit 460. Meanwhile, since the components illustrated in FIG. 17 are not essential, the infrared signal processing apparatus 400 having more components or fewer components may be implemented.

Referring to FIG. 17, the infrared receiver 410 may receive and recognize infrared rays emitted from the electronic pen 300 in the ROI, and output an infrared signal corresponding thereto. For that purpose, the infrared receiver 410 may be implemented as an infrared camera module including one or more infrared sensors.

The signal processor 420 sequentially analyzes, filters, and sorts the infrared signals output from the infrared receiver 410, and performs a function of selecting coordinates of the infrared signals emitted from the electronic pen 300 to obtain coordinate information. can do.

On the other hand, the parameter setting unit 430 may set the optimal parameter for the infrared signal processing according to the surrounding environment and transmit it to the infrared receiver 410 or the signal processor 420.

The short range communication module 440 may transmit coordinate information obtained from the signal processor 420 to the terminal device 500 or receive a command message from the terminal device 500 using a short range communication method such as Bluetooth.

The controller 450 may control the overall operation of the infrared signal processing apparatus 400 as described above, and may perform a function such as initial setting of the infrared signal processing apparatus 400.

The storage unit 460 may store reference values for signal processing, coordinate information detected by the signal processor 420, and parameters set by the parameter setting unit 430.

18 is a flowchart illustrating an embodiment of an operating method of the infrared signal processing apparatus according to the present invention, and the infrared signal processing apparatus 400 according to the exemplary embodiment of the present invention shown in FIG. A description will be given in conjunction with the block diagram showing the configuration of the "

Referring to FIG. 18, the controller 450 of the infrared signal processing apparatus 400 checks a current power state value and an operating state value, and an interrupt generated when the switch is pressed for 3 seconds during the operation of the infrared signal processing apparatus 400. If there is an interrupt, the power mode state value is changed (step S300).

If the interrupt is not generated, the short-range communication module 440 may receive a command message transmitted from the outside, for example, the terminal device 500 (step S310), and perform an operation corresponding to the received command message. .

In addition, the parameter setting unit 430 sets a parameter including a brightness threshold of the infrared light recognized by the infrared sensor of the infrared receiver 410 (step S320).

In operation S320, the parameter setting unit 430 operates a mobile environment recognition engine to apply a noise filter processing algorithm that is robust to a mobile environment in which external signals such as sunlight and remote control reflections interfere. The optimal parameter values required can be set automatically.

For example, since the radius of the infrared signal may change according to the ambient brightness, the parameter setting unit 430 may determine the radius of the infrared ray recognized by the infrared receiver 410 and the reference value stored in the storage unit 460. By comparing the parameters, the parameters can be set automatically according to the comparison result.

The infrared receiver 410 recognizes the infrared rays of the front area including the infrared rays emitted from the electronic pen 300 and outputs an infrared signal (step S330), and the signal processor 420 is an infrared ray output from the infrared receiver 410. The signal is analyzed to detect attributes of an infrared signal including an area, an average brightness value avgBright, a maximum brightness maxBright, and the like (S340).

For example, the signal processor 420 may analyze the brightness of the surroundings by using three attributes of the area, the average brightness value avgBright, and the maximum brightness value maxBright. The information about the ambient brightness analyzed in the parameter setting unit 430 may be used to set the optimal parameter.

Here, the area may represent an intensity value of infrared rays among the attributes of infrared rays, the average brightness value avgBright may represent an average value of infrared brightness, and a brightness maximum value maxBright may represent a maximum value of infrared brightness. These can be used later to filter or align infrared signals.

In addition, in step S340, the signal processor 420 analyzes the infrared signal converted by the infrared sensor of the infrared receiver 410 to the digital infrared light emitted from the electronic pen 300, and analyzes the infrared signal. The current operation state of may be determined as any one frame motion event among three state values (MotionDown, MotionUp, MotionMove).

For example, if the infrared signal value is not recognized in the previous frame and the infrared signal recognized in the current frame is "down", if the infrared signal value recognized in the previous frame is "down" or "move", it is recognized in the current frame. If the infrared signal is "move" and the infrared signal value recognized in the previous frame is "down" or "move" and the infrared signal value is not recognized in the current frame, it may be determined as "up".

Thereafter, the signal processor 420 filters the infrared signal analyzed in operation S340 to separate the infrared coordinates recognized by the infrared receiver 410 into coordinates within a reference signal range and coordinates outside the reference signal range (S350). .

In step S350, the signal processing unit 420 filters out infrared rays or noise by DSLR flash, sunlight, etc. to select a signal by infrared rays emitted from the electronic pen 300 from among the analyzed infrared coordinates. In addition, the infrared signal analyzed together with the infrared signal may be excluded from the ROI by determining whether the IR signal is within the ROI.

To this end, the signal processor 420 filters the infrared coordinates if the infrared signal value corresponding to each infrared coordinate is equal to or greater than or equal to a preset value according to the infrared rays emitted from the electronic pen 300, and then sorts the infrared coordinates. Can be excluded.

Thereafter, the signal processor 420 sorts the infrared coordinates filtered in step S350 to select an infrared signal corresponding to the infrared light emitted from the electronic pen 300 as a reference signal (step S360).

For example, in step S360, the signal processor 420 tracks the infrared signal recognized in the previous frame by using the unique ID of the infrared signal, so that the signal value recognized in the previous frames is changed from "up" to "down". If changed, the infrared coordinates are sorted according to the sum of the area, average brightness, and maximumbright values of the infrared signals acquired in the current frame, and then stored in the storage unit 460. The infrared signal closest to the reference value can be selected as the reference signal.

On the other hand, when the signal value recognized in the previous frame is "move" or "down", the signal processor 420 selects the infrared signal having the same ID as the ID of the infrared signal selected as the reference signal in the previous frame as the reference signal. Can be.

The short range communication module 440 transmits the coordinate information of the reference signal selected above to the terminal device 500 (operation S370).

Some of the steps described above with reference to FIG. 18 may be omitted, and the order of performing the steps illustrated in FIG. 18 may also be changed as necessary.

FIG. 19 is a block diagram illustrating an embodiment of the configuration of the terminal device illustrated in FIG. 16. The terminal device 500 illustrated includes a short range communication module 510, a coordinate converter 520, and a display unit 530. The controller may include a parameter setting unit 540, a control unit 550, a storage unit 560, and an angle of view setting unit 570. Since the components illustrated in FIG. 19 are not essential, the terminal device 500 may be implemented to have more components or fewer components.

Referring to FIG. 19, the short range communication module 510 communicates with the infrared signal processing apparatus 400 or the display apparatus 600 using a short range communication scheme such as Bluetooth, for example, from the infrared signal processing apparatus 400. The coordinate information of the reference signal may be received or the command message may be transmitted to the infrared signal processing apparatus 400.

The coordinate converter 520 may convert the infrared coordinates received from the infrared signal processing apparatus 400 into two-dimensional planarized coordinates in accordance with the resolution of the terminal apparatus 500 or the display apparatus 600.

The display unit 530 may represent the infrared coordinates converted by the coordinate converter 520 to perform application program functions such as drawing, sharing, and content control as an application layer.

The parameter setting unit 540 performs a function for setting an optimal parameter required for infrared signal processing according to the surrounding environment.

According to an embodiment of the present invention, the parameter setting unit 540 dynamically sets a bright threshold value according to a value designated by a user (for example, an ambient brightness value), thereby executing the short range communication module 510. Through the infrared signal processing apparatus 400 may be transmitted.

On the other hand, the angle of view setting unit 570 is a guide so that the direction toward the infrared sensor or infrared camera module provided in the infrared signal processing device 400, more specifically, the infrared signal processing device 400 and its recognition area can be set. In this case, even when the user wants to install the infrared signal processing apparatus 400 by moving to various places and locations, the distance from the display apparatus 600 and the pan tilt position can be correctly corrected. have.

The controller 550 controls the overall operation of the terminal device 500 as described above including an initial setting and the like, and the storage unit 560 is responsible for storing various information or data described above. .

20 is a flowchart illustrating an embodiment of a method of operating a terminal device according to the present invention. The operation method shown in FIG. 19 is a configuration of the terminal device 500 according to an embodiment of the present invention shown in FIG. It will be described in conjunction with the block diagram shown.

Referring to FIG. 20, the controller 550 of the terminal device 500 first performs a setup process of turning on power and an initial setting of configuring a short range communication channel with the infrared signal processing apparatus 400 (S500).

The angle of view setting unit 570 guides the angle of view setting so as to set the direction and the recognition area that the infrared signal processing apparatus 400 faces (S510).

For example, after the angle of view setting guide function is executed, when the electronic pen 300 contacts the screen of the display device 600 to emit infrared rays, the angle of view setting unit 570 may operate through the short range communication module 510. The distance and pan tilt to move the infrared signal processing apparatus 400 may be displayed by determining the position and direction of the infrared signal processing apparatus 400 according to the property information of the infrared coordinates received from the infrared signal processing apparatus 400. have.

More specifically, the angle of view setting unit 570 may display a large circle through the display unit 530 when the distance between the infrared signal processing device 400 and the display device 600 is closer than a reference value. 400 to move closer to the display device 600, and if the distance between the infrared signal processing device 400 and the display device 600 is farther than the reference value, a small circle is displayed through the display unit 530 to infrared The signal processing apparatus 400 may be moved to move away from the display apparatus 600.

In addition, the angle of view setting unit 570 may display the lower left arrow through the display unit 530 when the direction toward which the infrared signal processing apparatus 400 is directed is the upper right side of the display apparatus 600. The pan tilt adjustment guide to guide the adjustment of the direction of the lower left direction 200 may be performed together.

On the other hand, the control unit 550 stores the infrared signal of the moving environment recognized by the infrared signal processing apparatus 400 in the storage unit 560 to use as a reference for determining whether the infrared coordinates are noise (step S520). ).

Thereafter, the parameter setting unit 540 sets an optimal parameter required for operating the moving environment recognition engine of the infrared signal processing apparatus 400 according to a user-specified value (step S530).

For example, the parameter setting unit 540 may set the brightness threshold value of the infrared signal recognized by the infrared camera included in the infrared signal processing apparatus 400 within a range of 0 to 255, when the room is dark, b normally, and bright. Can be set to c (a <b <c).

In addition, the parameter setting unit 540 sets, in addition to the brightness threshold of the infrared camera included in the infrared signal processing apparatus 400, attribute information of a corresponding signal value when recognizing an infrared signal value. Can be set to one of two or more formats, ranging from simplified to large properties, and can set the number of infrared signals that an infrared camera can recognize.

Thereafter, the controller 550 performs a calibration to designate the ROI (operation S540).

The ROI refers to an area that the infrared signal processing apparatus 400 intends to recognize in the entire area (eg, 4095 x 4095) that the infrared signal processing apparatus 400 can recognize. Infrared coordinates generated within the range of the rectangle (LeftTop, RightTop, LeftBottom, RightBottom) designated as the moving environment coordinates within the recognizable area are designated by the infrared signal processing apparatus 400 as a rectangular range. It can be managed by specifying left and right.

The controller 550 drives the middleware to perform a de-warping and smoothing algorithm (step S550).

The de-warping algorithm is a matrix algorithm that converts the distorted two-dimensional square data obtained through calibration into two-dimensional square data flattened according to the resolution of the terminal device 500, and the smoothing. ) Algorithm is to use the average value of the previous coordinates and the current coordinates to correct the shaking of the recognized infrared signal.

Some of the steps described above with reference to FIG. 20 may be omitted, and the order of performing the steps illustrated in FIG. 20 may also be changed as necessary.

Meanwhile, an electronic blackboard operating application for performing the steps illustrated in FIG. 20 may be installed in the terminal device 500.

FIG. 21 is a flowchart illustrating a method for automatically setting parameters for infrared signal processing according to an embodiment of the present invention, wherein the infrared signal processing apparatus 400 performs the parameter setting step (step S320) shown in FIG. 18. An example of the method is shown.

Referring to FIG. 21, the infrared signal processing apparatus 400 analyzes an infrared signal output from an infrared sensor (step S600) and detects size information among attributes of the infrared signal obtained according to the analysis result (step S610).

An infrared sensor (or an infrared camera including one or more infrared sensors) may recognize infrared rays having a predetermined brightness threshold of infrared rays and output an infrared signal corresponding thereto.

For example, the infrared sensor provided in the infrared signal processing apparatus 400 recognizes infrared rays that are equal to or greater than an infrared brightness threshold among the infrared rays emitted from the electronic pen 300, and outputs corresponding infrared signals, and analyzes the infrared signals. The attributes of the infrared signal including an area, an average brightness value avgBright, a maximum brightness maxBright, and the like may be detected.

The attribute information detected according to the result of analyzing the infrared signal may be changed according to the surrounding environment such as ambient brightness, and thus may be obtained from information about the surrounding environment based on the attribute information.

The infrared signal processing apparatus 400 compares the detected size information of the infrared signal with a reference value (step S620), and changes the infrared brightness threshold of the infrared sensor based on the comparison result (step S630).

According to one embodiment of the invention, the size information (for example, the radius of the infrared region detected by the infrared sensor) for the infrared signal is changed according to the ambient brightness, using the infrared signal processing apparatus 400 Can adjust the infrared brightness threshold of the infrared sensor adaptively to the ambient brightness.

For example, the signal processor 420 of the infrared signal processing apparatus 400 receives and analyzes an infrared signal from an infrared receiver 410 including at least one infrared sensor to form infrared rays emitted from the electronic pen 300. Infrared region can be detected.

The infrared brightness threshold is initially set to a default value calculated according to the property of the infrared light emitted from the electronic pen 300 (for example, intensity or brightness), and thus the infrared sensor emits from the electronic pen 300. Among the infrared rays, the infrared signal stored in the storage unit 460 or more may be recognized to output an infrared signal.

The signal processor 420 may detect an infrared region formed by the infrared rays emitted from the electronic pen 300 based on a result of analyzing the infrared signal output from the infrared sensor, and calculate the radius of the detected infrared region.

In this case, the radius of the infrared region of the electronic pen 300 may vary according to the ambient brightness, and the darker the ambient brightness, the smaller the radius of the infrared region detected by the signal processor 420.

The parameter setting unit 430 may adaptively change the brightness threshold of the infrared light recognized by the infrared sensor, based on a correlation between the radius of the infrared region detected by the signal processor 420 and the ambient brightness.

Hereinafter, an embodiment of a method of setting an infrared light intensity threshold of an infrared sensor by using a radius of an infrared ray area detected by the electronic pen 300 detected by the signal processor 420 will be described with reference to FIG. 22. do.

Referring to FIG. 22, the parameter setting unit 430 of the infrared signal processing apparatus 400 checks whether the radius of the infrared region detected by the signal processing unit 420 matches a preset reference value (S700).

For example, if the radius of the infrared region detected by the signal processor 420 approaches the reference value of the radius stored in the storage unit 460 within an error range, the parameter setting unit 430 determines that the two values match. In this case, the preset IR brightness threshold is not changed.

On the other hand, if the radius of the infrared region detected by the signal processor 420 does not match the reference value, it is checked whether the radius of the infrared region detected by the signal processor 420 is smaller than the reference value (step S710).

As a result of the check, if the radius of the infrared region detected by the signal processor 420 is smaller than the reference value, the parameter setting unit 430 decreases the infrared brightness threshold (step S720).

Referring to FIG. 23, infrared rays emitted from the electronic pen 300 located on the ROI 800 displayed through the screen of the display apparatus 600 are recognized by an infrared sensor, and output an infrared signal output from the infrared sensor. In this case, the infrared region 810 by the electronic pen 300 may be detected in a form similar to a circle.

On the other hand, the darker the ambient brightness is, the smaller the radius of the infrared region 810 recognized by the infrared sensor is, and as shown in FIG. 23, the radius of the infrared region 810 is larger than the radius of the infrared region 820 on which the reference is made. Can be small.

In this case, the parameter setting unit 430 decreases the infrared brightness threshold of the infrared sensor, and as the infrared brightness threshold decreases, the radius of the infrared region 810 recognized by the infrared sensor increases to approach the reference value. Can be.

If the radius of the infrared region detected by the signal processor 420 is greater than the reference value, the parameter setting unit 430 increases the infrared brightness threshold (step S730).

Referring to FIG. 24, the brighter the ambient brightness is, the larger the radius of the infrared region 810 recognized by the infrared sensor is, so that the radius of the infrared region 810 may be larger than the radius of the infrared region 820 on which it is a reference.

In this case, the parameter setting unit 430 increases the infrared brightness threshold of the infrared sensor, and as the infrared brightness threshold increases, the radius of the infrared region 810 recognized by the infrared sensor decreases to approach the reference value. Can be.

Until the radius of the infrared region detected by the signal processor 420 coincides with the reference value within the error range, steps S700 to S730 as described with reference to FIG. 22 are repeated, and thus the magnitude of the infrared signal output from the infrared sensor. Can be maintained within a preset range adaptively to the surrounding environment.

FIG. 25 is a block diagram illustrating another exemplary embodiment of the configuration of the terminal device illustrated in FIG. 16. The terminal device 500 illustrated in FIG. 25 is viewed in comparison with the terminal device 500 illustrated in FIG. 19. Only when the user input unit 580 is further provided, the remaining components are the same. Therefore, the same components as those described in FIG. 19 will be omitted. Since the components illustrated in FIG. 25 are not essential, the terminal device 500 may be implemented to have more or fewer components.

Referring to FIG. 25, the user input unit 580 generates input data for the user to control the operation of the terminal device 500. The user input unit 380 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like.

According to an embodiment of the present invention, the user input unit 580 may serve to receive a user selection necessary for the operation of the parameter setting unit 540 or the angle of view setting unit 570 as described above.

Meanwhile, the operation method of the terminal device of FIG. 25 described above will proceed according to the flowchart of FIG. 20 described above.

FIG. 26 is a flowchart illustrating a parameter setting method for infrared signal processing according to another exemplary embodiment of the present invention. In the method in which the terminal device 500 performs the parameter setting step shown in FIG. 20 (step S530). It shows an example.

Referring to FIG. 26, the user input unit 580 of the terminal device 500 selects one of a plurality of brightness items representing ambient brightness from the user (operation S900), and the parameter setting unit 540 is the user input unit ( In operation S910, an infrared brightness threshold corresponding to the brightness item selected by the user is set.

Here, the infrared sensor (or an infrared camera including one or more infrared sensors) may recognize an infrared ray having a predetermined brightness threshold of infrared rays and output an infrared signal corresponding thereto.

For example, the infrared sensor provided in the infrared signal processing apparatus 400 recognizes infrared rays that are equal to or greater than an infrared brightness threshold value among the infrared rays emitted from the electronic pen 300, and outputs an infrared signal corresponding to the infrared light threshold value. After the user checks the brightness of the surroundings, the user may dynamically set the brightness according to the designated brightness item.

To this end, the user may select a brightness item closest to the current ambient brightness among a plurality of brightness items representing different ambient brightness displayed on the display unit 530 of the terminal device 500.

In this case, the storage unit 560 provided with the memory may store in advance infrared brightness thresholds corresponding to each of the plurality of user-selectable brightness items, and when the user selects the brightness item through the user input unit 580. The parameter setting unit 540 may search the memory and read an infrared brightness threshold corresponding to the selected brightness item.

For example, the plurality of brightness items include “dark”, “normal”, and “bright” based on the ambient brightness, and the brighter the ambient brightness according to the selected brightness item is, the higher the infrared brightness threshold is to be set. Can be.

More specifically, the infrared brightness threshold recognized by the infrared sensor (or infrared camera) may be set to a value between '0' and '255', and the smallest value when the brightness item selected by the user is "dark". (T1), the middle value T2 in the case of "normal", and the highest value T3 in the case of "bright".

Thereafter, the short-range communication module 510, which is a communication unit of the terminal device 500, transmits the infrared brightness threshold set in step S910 to the infrared signal processing apparatus 400 equipped with the infrared sensor (S920).

Hereinafter, an embodiment of a method for setting the parameter by the infrared signal processing apparatus 400 by using the infrared brightness threshold value of the infrared sensor set in the terminal device 500 will be described with reference to FIG. 27.

Referring to FIG. 27, the parameter setting unit 430 of the infrared signal processing apparatus 400 sets one or more parameters for infrared signal processing to a preset initial value (step S1000).

The parameters include an infrared brightness threshold recognized by the infrared sensor, in addition to the format of the infrared signal attribute value recognized by the infrared sensor or the number of infrared signals recognized by the infrared sensor. (tracking object number) and the like.

For example, the infrared signal attribute value format may be selected from one of three formats ranging from a simplified attribute to a large number of attributes, and a value “0x09” corresponding to the simplified attribute format is initially set. Can be.

On the other hand, the number of infrared signal (tracking object number) represents the number of infrared signals that can be recognized by the infrared sensor to the maximum, "2" may be set as an initial value.

In addition, the infrared brightness threshold may be set to “200” as an initial value.

The short-range communication module 440 of the infrared signal processing device 400 receives a parameter from the terminal device 500 (step S1010), and the parameter setting unit 430 presets the parameter received from the terminal device 500. It is checked whether or not the parameter is changed by comparing with the same parameter (or set as the initial value in step S1000) (step S1020).

For example, the user may set one of the "default setting", "user-based dynamic setting", and "perimeter-based auto setting" on the screen 900 of the terminal device 500 shown in FIG. Can be selected as a method to set.

When the user selects the "default setting" button 910, the parameters set as initial values in step S1000 may not be changed.

On the other hand, when the user selects the "user-based dynamic setting" button 920, as shown in Figure 29, the user is the brightness of any one of "dark", "normal" and "bright" based on the current

ambient brightness Items

940, 950, and 960 can be specified.

When the user selects the "bright" button 960 on the screen 900, the highest brightness T3 (eg, "3") is set in advance according to the parameter setting method described with reference to FIG. 26. 255 ", and the set infrared brightness threshold T3 may be transmitted to the infrared signal processing apparatus 500 using a short range wireless communication method such as Bluetooth.

In this case, the infrared signal processing apparatus 400 may confirm that the infrared brightness threshold of the infrared sensor is increased from "200" which is an initial value to "255".

Thereafter, when the parameter is changed, the parameter setting unit 430 of the infrared signal processing apparatus 400 resets the parameter to the value transmitted from the terminal device 500 (step S1030).

On the other hand, when the "main conversion mirror based automatic setting" button 930 is selected on the screen 900 of the terminal device 500 shown in Figure 28, the infrared signal processing apparatus 400 automatically recognizes the surrounding environment and Appropriate optimal parameter values can be set.

On the other hand, the above has been described an embodiment of the present invention by taking the "user-based dynamic setting" method and the "main conversion mirror-based automatic setting" method as an example, the present invention is not limited to this, the user is to change the ambient brightness Even if it is specified, the " main conversion mirror-based automatic setting " method is periodically performed, so that the copyboard system according to the embodiment of the present invention can set the optimal parameters required for operation whenever the surrounding environment changes.

30 is a flowchart illustrating a filtering method for processing an infrared signal according to an embodiment of the present invention. In the infrared signal processing apparatus 200, the infrared signal analyzing, filtering, and sorting steps S340 to FIG. 18 are performed. An example of how to perform step S360) is shown.

Referring to FIG. 30, the signal processor 420 of the infrared signal processing apparatus 400 analyzes attributes of one or more infrared signals recognized by the infrared sensor (step S1100).

For example, the infrared sensor provided in the infrared signal processing apparatus 400 recognizes infrared rays that are equal to or greater than an infrared brightness threshold among the infrared rays emitted from the electronic pen 300, and outputs corresponding infrared signals, and analyzes the infrared signals. The attributes of the infrared signal are detected including an area, an average brightness value avgBright, a maximum brightness maxBright, and the like.

As described above, the area of the infrared signal represents the intensity value of the infrared signal, the average brightness value avgBright represents the average value of the infrared brightness, and the brightness maximum value maxBright is the maximum value of the infrared brightness. Can be represented.

Meanwhile, the infrared sensor may recognize and output a plurality of infrared signals having different coordinates, and in step S1100, an attribute may be set for each of the plurality of infrared signals (or a plurality of infrared coordinates corresponding thereto). Can be analyzed.

The signal processor 420 checks whether the corresponding infrared signal is within the reference signal property range by comparing the analyzed infrared signal property with the property of the reference signal (step S1110), and according to the result of the check, out of the reference signal property range. The infrared signal having the property is excluded from the alignment target (S1120).

For that purpose, an attribute of the reference signal is defined based on an attribute of infrared rays emitted from the electronic pen 300, and the minimum value of the attribute is centered on the attribute value of the reference signal together with the attribute value of the defined reference signal. The maximum value may be previously stored in the storage unit 460 as a reference signal attribute range.

When the attribute value of the infrared signal analyzed in step S1100 is smaller than the minimum value or larger than the maximum value stored in the storage unit 460, the signal processor 420 determines that the infrared signal is out of the reference signal range, and then determines the Can be excluded.

For example, among the plurality of infrared coordinates recognized by the infrared sensor, any one of the area, the average brightness value avgBright and the maximum brightness maxBright, which are properties of the infrared signal, is preset The infrared coordinates deviating from may be excluded from the alignment object that may be selected as the electronic pen coordinates.

In addition, the signal processor 420 may exclude the infrared signal located outside the preset ROI from the alignment target by using the coordinate information of the infrared signal recognized by the infrared sensor.

Meanwhile, the signal processor 420 detects an infrared signal having the same unique identification information among the infrared signals recognized in the previous frame by using the unique identification information of the infrared signals included in the alignment target (S1130).

For example, the infrared signals output from the infrared sensor may have IDs that are unique identification information, and the infrared signals may be distinguished from each other by the IDs.

The signal processor 420 may track a specific infrared signal among frames sequentially obtained over time using the ID of the infrared signal.

Accordingly, in step S1130, whether an infrared signal having the same ID as the infrared signal recognized in the current frame exists in the previous frame, coordinates in the previous frame with respect to the infrared signal recognized in the current frame, and the like may be checked. .

Thereafter, the signal processor 420 may select an infrared signal having an attribute closest to the preset reference signal or an infrared signal having the same unique identification information as the infrared signal detected in the previous frame among the infrared signals included in the alignment target. Is selected as the infrared signal, the coordinate information of the selected infrared signal is transmitted to the terminal device 500 through the short-range communication module 440 (step S1140).

For example, during, or before or after the analysis of step S1100, the signal processor 420 determines a motion event for the current frame using infrared signals recognized in the previous frame and the current frame. An infrared signal may be selected in step S1140 based on the determined frame motion event.

More specifically, when the infrared signal is not recognized in the previous frame, the motion event of the current frame according to the infrared signal recognized in the current frame has a first value, and when the motion event of the previous frame has the first value, The motion event of the current frame according to the infrared signal recognized in the frame may have a second value.

Meanwhile, when the motion event of the previous frame has the second value, the motion event of the current frame according to the infrared signal recognized in the current frame has the second value, and the motion event of the previous frame is the first value or the second value. If the value has a value, the motion event of the current frame according to the infrared signal not recognized in the current frame may have a third value.

When the motion event of the current frame has the first value, the signal processor 420 may output an infrared signal having an attribute value closest to the reference signal attribute value stored in the storage unit 460 among the infrared signals included in the alignment target. You can choose.

Meanwhile, when the motion event of the current frame has the second value, the signal processor 420 may select an infrared signal having the same unique identification information as the infrared signal detected in the previous frame.

FIG. 31 is a flowchart illustrating an example of a method of determining a frame motion event by analyzing an infrared signal. The frame motion event may be determined as one of 'down', 'move', and 'up'.

Referring to FIG. 31, it is determined whether an infrared signal is recognized in a current frame (S1200).

For example, when one or more normal infrared coordinates having attribute values within a preset reference signal attribute range exist in the ROI, it is determined that the infrared signal is recognized, and the normal infrared coordinates are within the ROI. If none exists, it may be determined that the infrared signal is not recognized.

As shown in FIG. 32, an entire recognition area 1000 recognizable by an infrared sensor (or an infrared camera having one or more infrared sensors) exists, and a display screen, a projector screen, or the like is present in the entire recognition area 1000. The screen area 1010 of the same display device 600 may exist.

In addition, the writing content may be recognized by the electronic blackboard system only when the coordinates of the infrared rays emitted from the electronic pen 300 are within the ROI 1020, and the ROI 1020 is the display apparatus 600. It may be set smaller than the screen area 1010.

Accordingly, of the plurality of infrared coordinates P1 to P5 recognized by the infrared sensor, the infrared coordinates P1 and P2 outside the region of interest 1020 are excluded as non-normal coordinates, and within the region of interest 1020. Only infrared coordinates P3, P4, and P5 may be determined to be normal coordinates.

When the infrared signal is not recognized in the current frame, if the motion event of the previous frame is not 'down' or 'move' (step S1210), the motion event of the current frame has an initial value (step S1220).

Meanwhile, when the infrared signal is not recognized in the current frame, if the motion event of the previous frame is 'down' or 'move' (step S1210), the motion event of the current frame is determined as 'up' (step S1230).

When the infrared signal is recognized in the current frame, if the motion event of the previous frame is the initial value or 'up' (step S1240), the motion event of the current frame is determined to be 'down' (step S1250).

In addition, when the infrared signal is recognized in the current frame, if the motion event of the previous frame was not 'up', that is, 'down' or 'move' (step S1240), the motion event of the current frame is determined as 'move'. (Step S1260).

For example, when the motion event of the previous frame shown in FIG. 33 is 'up' and the motion event of the current frame shown in FIG. 34 is 'down', infrared coordinates by the electronic pen do not exist in the previous frame. The signal processor 420 detects the coordinate P3 of the infrared signal having the attribute value closest to the attribute value of the reference signal among the plurality of infrared signals recognized by the infrared sensor, and the corresponding coordinate information is short-range communication module 440. ) To be transmitted to the terminal device 500.

Meanwhile, when the motion event of the previous frame shown in FIG. 35 is 'down' and the motion event of the current frame shown in FIG. 36 is 'move', the infrared coordinates of the electronic pen are moved between the previous / current frames. The signal processor 420 detects an infrared signal having the same ID as the infrared coordinate P3 of the previous frame among the plurality of infrared signals recognized by the infrared sensor, and the corresponding coordinate information is transmitted through the short range communication module 440. May be sent to the device 500.

According to another embodiment of the present invention, by setting the angle of view according to the properties and coordinates of the infrared signal output from the infrared sensor during the mobile installation of the infrared-based copyboard system, even if the infrared-based copyboard system is installed at any position It can be operated without error adaptively to mobile environment.

FIG. 37 is a flowchart illustrating a view angle setting method according to an embodiment of the present invention, and illustrates an example of a method in which the terminal device 500 performs the view angle setting step (step S510) of FIG. 20.

Referring to FIG. 37, the terminal device 500 acquires attribute information and coordinate information of an infrared signal recognized by an infrared sensor (S1300).

When the guide for setting the angle of view is executed in the terminal device 500, the user places the electronic pen 300 at a point on the screen of the display device 600 to emit infrared rays, and the infrared sensor (or one or more infrared sensors). An infrared camera including a) may recognize the infrared rays emitted from the electronic pen 300 and output an infrared signal corresponding thereto.

Meanwhile, the infrared signal processing apparatus 400 analyzes the infrared signal output from the infrared sensor and analyzes the properties of the infrared signal including the area, the average brightness value (avgBright), the maximum brightness value (maxBright), and the infrared rays. Coordinates may be detected, and attribute information and coordinate information of the detected infrared signal may be transmitted to the terminal device 500.

For example, the area value among the attributes of the infrared signal may indicate the intensity of the infrared signal, and may vary according to the distance between the screen of the display apparatus 600 and the infrared signal processing apparatus 400.

More specifically, the closer the distance between the screen of the display apparatus 600 and the infrared signal processing apparatus 400 is, the larger the area value among the attributes of the infrared signal may be.

The angle of view setting unit 570 of the terminal device 500 generates a first image having a size corresponding to the attribute information of the infrared signal acquired in step S1300 (step S1310).

For example, the first image may be processed to be displayed as a circle having a larger radius as the distance between the screen of the display apparatus 600 and the infrared signal processing apparatus 400 determined based on the attribute information of the infrared signal is closer. have.

The angle of view setting unit 570 determines whether the corresponding coordinate information is normal based on the coordinate information of the infrared signal obtained in step S1300 (step S1320).

Here, in order to execute the angle of view guide, the electronic pen 300 may be induced to touch a specific position (eg, a center point) on the screen.

In this case, when the coordinates of the infrared signal obtained in step S1300 do not coincide with the coordinates of a specific position (for example, the center point) on the screen within a preset error range, it may be determined that the coordinate information is not normal. have.

On the contrary, when the coordinates of the infrared signal obtained in step S1300 coincide with the coordinates of a specific position (for example, the center point) on the screen within a preset error range, the coordinate information may be determined to be normal.

When the coordinate information is not normal as a result of the determination, the angle of view setting unit 570 generates a second image for adjusting the direction of the infrared signal processing apparatus 400 (step S1330).

For example, the second image may be processed to be displayed as an arrow indicating a direction for compensating for a difference between the coordinates of the infrared signal acquired in step S1300 and the coordinates of a specific position (for example, a center point).

Thereafter, the display unit 530 displays at least one of the first and second images generated by the angle of view setting unit 570 under the control of the controller 550 (step S1340).

Hereinafter, embodiments of a method of executing an angle of view guide in the terminal device 500 will be described in detail with reference to FIGS. 38 to 44.

Referring to FIG. 38, after the user installs the infrared signal processing apparatus 400 to face the display apparatus 600, the user turns on the power and executes the copyboard operation application installed in the terminal apparatus 500 to execute the infrared signal processing apparatus ( After enabling communication with 400, the angle of view guide menu can be selected. Here, the method of installing the infrared signal processing device 400 toward the display device 600 will be described in detail. The display device 600 may be a screen or a TV, and the infrared sensor of the infrared signal processing device 400 may be positioned approximately 1.5 times the horizontal length of the screen or the screen of the TV. For example, it is preferable to position the infrared sensor of the infrared signal processing apparatus 400 in a diagonal direction at a distance of approximately 1.5 times the horizontal length of the screen or the screen of the TV. This is because it is not recognized when the infrared sensor of the infrared signal processing apparatus 400 is covered. That is, when the user is right-handed, the infrared signal processing apparatus 400 may be positioned to face the display apparatus 600 in a diagonal direction to the left of the display apparatus 600. When the user is left-handed, the infrared signal processing apparatus 400 may be positioned to face the display apparatus 600 in a diagonal direction to the right of the display apparatus 600. Of course, the infrared sensor of the infrared signal processing apparatus 400 may be positioned in front of the screen at a distance of approximately 1.5 times the width of the screen of the screen or the TV. In this case, the user may stand in front of the screen so as to be positioned at the center of the screen of the screen or the TV so that the infrared sensor of the infrared signal processing apparatus 400 looks at the user's eyes.

According to the guidance displayed on the screen 1200 shown in FIG. 38, when the user presses the “START” button 1210 and touches the electronic pen 300 at a center point on the screen of the display device 600, FIG. 37 is shown. View angle setting steps as described with reference to may be performed in the terminal device 500.

Referring to FIG. 39, when the distance between the screen of the display apparatus 600 and the infrared camera included in the infrared signal processing apparatus 400 is close to each other, a first circle indicating an attribute of the current infrared signal on the screen 1200 ( 1220 may be larger than the second circle 1225.

In this case, when the user moves the infrared signal processing apparatus 400 in a direction opposite to the screen of the display apparatus 600 according to the guidance displayed on the screen 1200 illustrated in FIG. 39, the first display displayed on the screen 1200 is performed. The size of the circle 1220 may be gradually reduced.

When the user moves the infrared signal processing apparatus 400 away from the screen of the display apparatus 600 until the first circle 1220 coincides with the second circle 1225, an alarm indicating that the distance adjustment is completed is voice or It may be provided through the screen 1200.

Referring to FIG. 40, when the distance between the screen of the display apparatus 600 and the infrared camera provided in the infrared signal processing apparatus 400 is far, a first circle indicating the attribute of the current infrared signal on the screen 1200 ( 1220 may be smaller than the second circle 1225.

In this case, when the user moves the infrared signal processing apparatus 400 toward the screen of the display apparatus 600 according to the guidance displayed on the screen 1200 illustrated in FIG. 40, the first circle displayed on the screen 1200 is displayed. The size of 1220 may increase gradually.

When the user moves the infrared signal processing apparatus 400 adjacent to the screen of the display apparatus 600 until the first circle 1220 coincides with the second circle 1225, an alarm indicating that the distance adjustment is completed is a voice. Or it may be provided through the screen 1200.

Referring to FIG. 41, the user touches the electronic pen 300 at the center point C on the screen of the display apparatus 600, but the infrared camera provided in the infrared signal processing apparatus 400 may be an infrared sensor. Toward the lower right of the screen, it can be recognized that the infrared coordinate P of the electronic pen 300 is located on the upper left of the region of interest 1020. Here, when the user touches a point that becomes the center on the screen of the display apparatus 600 using the electronic pen 300, the center point C appears on the screen as shown in FIG. 41. At this time, since the infrared camera faces the lower right direction of the screen, the infrared coordinate P displays the infrared coordinates currently viewed by the center of the infrared camera.

In this case, as illustrated in FIG. 42, an arrow 1230 which guides the tilt direction of the infrared camera provided in the infrared signal processing apparatus 400 toward the upper left is adjusted to the screen 1200 of the terminal device 500. ) May be displayed.

According to the guidance displayed on the screen 1200 illustrated in FIG. 42, when the user pan tilts the direction toward which the infrared signal processing apparatus 400 is directed to the upper left, the size of the arrow 1230 displayed on the screen 1200 is increased. It may decrease gradually.

When the user pan tilts the direction of the infrared signal processing apparatus 400 until the arrow 1230 displayed on the screen 1200 disappears, an alarm indicating that the direction adjustment is completed may be provided through the voice or the screen 1200. Can be.

The above-described pan tilt adjustment will be described in detail.

As shown in FIG. 43, when the infrared coordinate P is above or below the center point C in the screen of the display apparatus 600, the user moves the head of the infrared sensor of the infrared signal processing apparatus 400 up and down. Rotate it to face the center of the screen. In this case, the infrared coordinate P may move in the direction of the center point C in the center of the screen, and the size of the arrow may gradually decrease.

On the other hand, as shown in Figure 44, if the infrared coordinates (P) to the left or right of the center point C in the screen of the display device 600, the user rotates the infrared signal processing device 400 to the left / right to the center of the screen Adjust to look at In this case, the infrared coordinate P may move in the direction of the center point C in the center of the screen, and the size of the arrow may gradually decrease.

In the above description, the first and second images for inducing the angle of view setting are displayed through the display unit 530 of the terminal device 500. However, the present invention is not limited thereto. The first image or the second image may be received and displayed on the screen.

As described above, the best embodiment has been disclosed in the drawings and the specification. Although specific terms have been used herein, they are used only for the purpose of describing the present invention and are not used to limit the scope of the present invention as defined in the meaning or claims. Therefore, those skilled in the art will understand that various modifications and equivalent other embodiments are possible from this. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Claims

A panel for converting light from a light source into an image;

A pattern beam providing unit configured to output a pattern beam;

A single lens for projecting the pattern beam and the image from the panel onto a screen and receiving a signal by touch interaction on the screen;

A sensor for recognizing position coordinates of the touch object currently touched on the screen based on the signal generated by the touch interaction; And

Sends light from the light source to the panel, transmits the image from the panel to the lens, reflects the pattern beam to the lens, and receives a signal from the touch interaction received at the lens. And an optical separation element sent to the sensor.
The method according to claim 1,

The panel consists of a reflective panel,

The light source is provided opposite to the first surface of the optical separation element,

The panel is provided to face the second surface of the optical separation element,

The lens is provided to face the third surface of the optical separation element,

And the pattern beam providing unit and the sensor are disposed to face the fourth surface of the optical separation element.
The method according to claim 1,

And an interaction signal processor configured to determine which touch object has occurred on the basis of the position coordinates from the sensor and convert the touch event into a corresponding touch event.
The method according to claim 1,

And a filter for removing sunlight or light from the light source so as not to affect the sensor.
The method according to claim 1,

And the pattern beam is formed of infrared light.
The method according to claim 5,

The signal by the touch interaction is,

And an infrared light having a pattern distortion or a shape change at a corresponding touch point as a user touches a touch object with a body part in an image projected on the screen.
The method according to claim 5,

The signal by the touch interaction is,

And an infrared light having a changed amount of light at a corresponding touch point as a user touches a touch object with a body part in the image projected on the screen.
A panel for converting light from a light source into an image;

A pattern beam providing unit configured to output a pattern beam;

A single lens for projecting the pattern beam and the image from the panel onto a screen and receiving a signal by touch interaction on the screen;

A sensor for recognizing position coordinates of the touch object currently touched on the screen based on the signal generated by the touch interaction; And

And an optical separation element for transmitting an image from the panel to the lens, reflecting the pattern beam to the lens, and transmitting a signal of the touch interaction received from the lens to the sensor. Projector capable of touch interaction, characterized in that.
The method according to claim 8,

The panel consists of a transmissive panel,

The light source and the panel are installed opposite to the first surface of the optical separation element, the light source is installed behind the panel,

The lens is provided to face the second surface of the optical separation element,

And the pattern beam providing unit and the sensor are provided to face the third surface of the optical separation element.
The method according to claim 8,

And an interaction signal processor configured to determine which touch object has occurred on the basis of the position coordinates from the sensor and convert the touch event into a corresponding touch event.
The method according to claim 8,

And a filter for removing sunlight or light from the light source so as not to affect the sensor.
A panel for converting light from a light source into an image;

A single lens for projecting an image from the panel onto a screen and receiving a signal by touch interaction on the screen;

A sensor for recognizing position coordinates of the touch object currently touched on the screen based on the signal generated by the touch interaction; And

And an optical separation device configured to transmit light from the light source to the panel, transmit an image from the panel to the lens, and transmit a signal of the touch interaction received from the lens to the sensor. Projector capable of touch interaction, characterized in that.
The method according to claim 12,

The panel consists of a reflective panel,

The light source is provided opposite to the first surface of the optical separation element,

The panel is provided to face the second surface of the optical separation element,

The lens is provided to face the third surface of the optical separation element,

And the sensor is installed to face the fourth surface of the optical separation element.
The method according to claim 12,

And an interaction signal processor configured to determine which touch object has occurred on the basis of the position coordinates from the sensor and convert the touch event into a corresponding touch event.
The method according to claim 12,

And a filter for removing sunlight or light from the light source so as not to affect the sensor.
The method according to claim 12,

The signal by the touch interaction is,

And a specific light emitted by the touch device at a touch point as the user touches any touch object with the touch device in the image projected on the screen.
A panel for converting light from a light source into an image;

A single lens for projecting an image from the panel onto a screen and receiving a signal by touch interaction on the screen;

A sensor for recognizing position coordinates of the touch object currently touched on the screen based on the signal generated by the touch interaction; And

And an optical separation element configured to transmit an image from the panel to the lens, and to transmit a signal of the touch interaction received from the lens to the sensor.
The method according to claim 17,

The panel consists of a transmissive panel,

The light source and the panel are installed opposite to the first surface of the optical separation element, the light source is installed behind the panel,

The lens is provided to face the second surface of the optical separation element,

And the sensor is installed to face the third surface of the optical separation element.
The method according to claim 17,

And an interaction signal processor configured to determine which touch object has occurred on the basis of the position coordinates from the sensor and convert the touch event into a corresponding touch event.
The method according to claim 17,

And a filter for removing sunlight or light from the light source so as not to affect the sensor.