KR101102953B1 - Large scale interactive display system based multi touch and method of the same - Google Patents

Abstract

The present invention relates to a large interactive display system and a method based on a multi-touch, and an interactive surface that can be interacted with by installing an infrared LED array bar (IR-LEDs array bar) on the ceiling or floor of the front of the screen. ), A system that allows a user to interact with content freely. According to the present invention, when the user is far from the display, it acts like a normal billboard but when the user enters the interactive area, the interactive screen is changed to allow the user to interact. Through this, an individual or a plurality of people can enjoy various contents.
The multi-touch large interactive display system according to the present invention comprises a screen for displaying an image; An infrared LED array bar generating infrared light by generating infrared rays in front of the screen; A projector for projecting an image onto the screen; An infrared camera for photographing infrared light reflected when a human body or an object comes into contact with the interactive film; A server configured to extract a coordinate value of an interaction position of the human body or an object through image processing using the information obtained from the infrared camera, and generate a packet generating an interaction at a corresponding point; And a client PC analyzing the packet received from the server and executing the content according to the packet on the screen through the project.

Description

LARGE SCALE INTERACTIVE DISPLAY SYSTEM BASED MULTI TOUCH AND METHOD OF THE SAME}

The present invention relates to a multi-touch based large interactive display system and method thereof. More specifically, the present invention relates to a large interactive display system and a method thereof. The present invention relates to a multi-touch large interactive display system and a method for providing a user with a function of interacting with content.

Recently, interactive displays provided by various ubiquitous computing technologies have provided various interaction methods to users. In other words, in a ubiquitous computing environment called invisible computing, in addition to the existing explicit interaction using a keyboard or a mouse, implicit interactions through user's actions grasped by naturally occurring user's actions or sensors ( Implicit interaction method is required.

Just as humans communicate through various senses such as sight, hearing, and touch, it is more important to synthesize and analyze multimodality from multiple devices in the human-computer interaction of the ubiquitous computing environment. For example, the overall reliability can be improved by analyzing the user's hand movements, body movements, facial expressions, gaze direction, etc. rather than grasping the user's intention using only a hand touch or a simple input device.

In the prior art, only a simple touch-type display is provided or various sensor devices are used to grasp a user's intention. There is a Gossip Wall using a distance sensor, and there is a shared interactive ambient display system that uses a variety of cameras to determine user location information.

Specifically, there is a Gossip Wall developed by the Fraunhofer Institute in Germany. When the distance between the user and the display is far away from the ambient area, general information is displayed, and when the user approaches the display more and the display recognizes the user, the appropriate content is displayed for the user. In other words, when the user approaches the interaction area that is very close to the display, the mobile terminal can obtain more detailed information and connect with the outside.

As such, the Gossip Wall obtains information from the display using a mobile terminal adopting a distance sensor method, and determines the location of the user through the mobile terminal. However, the Gossip Wall has a problem such that the user can keep track of the user only when the user always carries the mobile terminal, and various display devices are not provided.

The shared interactive ambient display system at the University of Toronto, meanwhile, is designed to allow for different interactions in four stages, depending on the proximity between the user and the display. However, this system has a big problem that the user has to wear several types of recognition devices.

In addition, the interaction method adopting a recognition method using a device device according to the prior art has a problem that can not provide a natural interaction to the user. In addition, the sensor recognition method has a limitation in that multiple users must be equipped with the device and thus cannot participate together.

Document 1 D.M. Russell, N. Streitz, T. Winograd, "Building Disappearing Compters", Communications of the ACM, Vol. 48, No. 3, pp. 42-48, 2005. D. Vogel, R. Balakrishnan, "Interactive Public Ambient Displas: Transitioning from Implicit to Explicit Plublic to Personal, Interaction with Multiple Users", Proceedings of UIST 2004, pp. 137-146, 2004. [3] E. Mynatt, J. Rowan, S. Craighill, "Digital Family Portraits: Supporting Peace of Mind For Extended Family Members", Proceedings of HCI 2001, ACM Press, pp. 333-340, 2001. [Reference 4] Yoon, H.S. Ryu, S.J. Park, S.J. Yoo, S.M. Choi, "An Ambient Display for the Elderly", HC International 2007, LNCS 4555, Springer-Verlag, pp. 1043-1049, 2007. Document 5 S, M. Choi, Y.G. Kim, et al., "Non-photorealistic 3D Facial Animation on the PDA Based on Facial Expression Recognition", Smart Graphics 2004, LNCS 3031, pp. 11-20, 2004.

The technical problem to be solved by the present invention is to provide an interactive zone on the front of the screen so that the user can freely interact with the content (large interactive based on multi-touch) interactive display system and method thereof.

In addition, another technical problem to be achieved by the present invention is a large multi-touch-based large size that provides the ability to interact with the content when the user enters the interactive area of the screen only when the user is away from the screen and the user enters the interactive area of the screen An interactive display system and method thereof are provided.

In addition, another technical problem to be achieved by the present invention is to install an infrared LED array bar (IR-LEDs Array Bar) on the ceiling or floor of the front of the screen to create a multi-touch interactive surface (interactive) that the user can interact with The present invention provides a large interactive display system and a method thereof.

In addition, another technical problem to be achieved by the present invention is to provide a multi-touch to the user in the ubiquitous ambient environment, a multi-touch based large interactive display that can be interacted with a simple touch operation of the user without the help of the device device for recognition A system and method are provided.

The problem of the present invention is not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The present invention for solving the above technical problem is a "large interactive display system based on multi-touch,
A screen for displaying an image;
An infrared LED array bar generating infrared light by generating infrared rays in front of the screen;
A projector for projecting an image onto the screen;
An infrared camera for photographing infrared light reflected when a human body or an object comes into contact with the interactive film;
A server configured to extract a coordinate value of an interaction position of the human body or an object through image processing using the information obtained from the infrared camera, and generate a packet generating an interaction at a corresponding point; And
A client PC that analyzes the packet received from the server and executes the content according to the packet on the screen through a project;
Including,
The infrared LED array bar is configured by arranging a plurality of infrared LEDs (IR-LEDs) at regular intervals, and a partition wall is formed on both sides of the infrared LEDs, and is installed on the top or ceiling of the screen, and the projector and the The infrared camera is a multi-touch-based large interactive display system, characterized in that the infrared LED array bar is installed on the top or the ceiling separated by a predetermined distance.

And the present invention, "In the large interactive display system based on multi-touch,
A screen for displaying an image;
An infrared LED array bar generating infrared light by generating infrared rays in front of the screen;
A projector for projecting an image onto the screen;
An infrared camera for photographing infrared light reflected when a human body or an object comes into contact with the interactive film;
A server configured to extract a coordinate value of an interaction position of the human body or an object through image processing using the information obtained from the infrared camera, and generate a packet generating an interaction at a corresponding point; And
A client PC that analyzes the packet received from the server and executes the content according to the packet on the screen through a project;
Including,
The infrared LED array bar is configured by arranging a plurality of infrared LEDs (IR-LEDs) at regular intervals, and a partition wall is formed on both sides of the infrared LEDs, and is installed at the bottom or bottom of the front of the screen, and the projector and the An infrared camera is installed at the bottom or bottom of the rear side of the screen,
The screen is a multi-touch-based large interactive display system characterized in that the rear screen (Rear Screen) on a glass window or an acrylic plate.

In addition, the present invention "In the multi-interactive large interactive display method,
(a) generating infrared film by generating infrared rays in front of the screen;
(b) photographing infrared light reflected by a human body or an object in contact with the interactive film with an infrared camera;
(c) extracting a coordinate value of an interaction position of the human body or an object through image processing using the information obtained from the infrared camera, and generating a packet generating an interaction at a corresponding point; And
(d) analyzing the packet and executing content according to the packet on the screen through a project;
Including,
The method of extracting the coordinate value of the interaction position in the step (c),
(c1) a binarization step of separating the information obtained from the infrared camera into an area where the LED light is reflected and an area that is not through image binarization;
(c2) a blob to bind the reflected area;
(c3) labeling the numbered areas bounded by the blobs;
(c4) a homography step of correcting signal distortion using a homography matrix after the labeling;
(c5) a window API coordinate correction step of correcting distortion by inputting coordinates of four corners of an image on a screen viewed by the infrared camera and converting an image in a quadrangle of four points into a quadrangle of four points; And
(c6) extracting coordinate values by identifying interaction positions of the human body or an object;
It is another object solving means to provide a multi-touch-based large interactive display method characterized in that it comprises a.

delete

delete

delete

delete

delete

According to the present invention, when the user is far from the screen, only the display function may be provided, and when the user enters the interactive area of the screen, the user may be provided with a function to interact with the content.

In addition, it can be provided as an interactive ambient display environment where individual or large crowds can enjoy various contents.

In addition, various interaction information is recognized through a preprocessing process, and as the size of the screen increases, a plurality of users can use it together.

In addition, it has scalability that can be operated from a small space to a large space without any restrictions.

In addition, by touching the screen, the user can directly feel and experience various contents.

The effects of the present invention are not limited to those mentioned above, and other effects that are not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a schematic view showing a multi-touch based large size interactive display system according to an exemplary embodiment of the present invention.
2 is a diagram illustrating an interactive surface through which a user can interact with content.
FIG. 3 is a schematic diagram illustrating the principle of improving the linearity of light using an IR-LEDs array bar. FIG.
4 and 5 are product photographs and installation images of the IR-LEDs array bar.
6 is a view schematically showing the principle that the infrared light is reflected to the hand and extracted through the infrared camera
7 is a flowchart schematically illustrating a multi-touch based large interactive display method according to an exemplary embodiment of the present invention.
8 is an exemplary view showing a homography algorithm process for image distortion correction
9 is a schematic diagram of a multi-touch based large size interactive display system according to another exemplary embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In order to clearly explain the present invention in the drawings, parts not related to the description are omitted, and similar parts are denoted by similar reference numerals throughout the specification.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

Multi touch based large Interactive  Embodiment of the display system

1 is a block diagram schematically illustrating a multi-touch based large size interactive display system according to an exemplary embodiment of the present invention, and FIG. 2 is a diagram illustrating an interactive surface 160 that a user may interact with content. .

As shown in FIG. 1, the multi-touch-based large interactive display system of the present invention is a screen or wall display device (hereinafter, referred to as a 'screen': 100), an infrared LED array bar (IR-LEDs Array Bar: 110). ), A projector 120, an infrared camera 130 equipped with an IR-Filter 131, a server 140, and a client PC 150.

Here, the screen 100 forms an image so that the image of the projector 120 can be projected. The infrared LED array bar 110 is configured by arranging a plurality of infrared LEDs (IR-LEDs) 111 at predetermined intervals, and an infrared ray generator having barrier ribs formed on both sides of the infrared LEDs, the ceiling of the front of the screen 100. It is installed on the (10), and generates an interactive surface (160) that the user can interact with the content by generating infrared rays in front of the screen (100). The projector 120 functions to project an image onto the screen 100, and the infrared camera 130 is an IR-Filter in front of a lens of a general camera (for example, a USB CCD camera). 131, the camera is mounted, and when the user's hand touches the interactive layer 160, the infrared light reflected from the user's hand (back of the hand) is photographed and transmitted to the server 140. The server 140 uses the image signal transmitted from the infrared camera 130 to calculate the coordinate (x, y) value of the user's hand region through image processing to extract the interaction position of the user, and then perform the interaction at the corresponding point. The generated packet is generated and transmitted to the client PC 150. The client PC 150 analyzes the packet transmitted from the server 140 and executes the content according to the packet through the projector 120.

The present invention provides various interaction techniques by generating the interactive film 160 on the front of the screen 100 using the IR-LEDs array bar 110. Hereinafter, the areas touched by the human hand on the interactive layer 160 will be referred to as an interactive zone, and an area in which the touches are not touched will be defined as an ambient zone.

1 and 2, the projector 120 and the infrared camera 130 find and locate an optimal place for user interaction. In this case, when the camera 130 obtains the screen itself without using a band pass filter, the camera 130 receives all images projected by the beam projector, for example, a band pass filter (eg, 850 nm). By using a filter to block below), the image from the projector 120 may be blocked to receive infrared light.

As a result, when the user reaches the screen 100 to interact with the content, the infrared rays descending from the IR-LEDs Array Bar 110 located on the screen 100 are released. You will not be able to pass through the back of your hand). In this case, the infrared camera 130 photographs the infrared light reflected from the hand (back of the hand) and transmits the captured image signal to the server 140. The server 140 calculates coordinate (x, y) values of the hand region through image processing of the signal transmitted from the infrared camera 130.

Here, the method of calculating the coordinate (x, y) value of the hand region is as follows.

This function operates by calculating the coordinates that recognize the highest point of the coordinates after correcting the distortion using a homography matrix through the blob / labeling only in the region where the LED is reflected through the image binarization and the region that is not. do. Here, the blob binds the areas where the LED is reflected when the image is binarized. That is, since the human hand does not reflect light to one pixel (point) but reflects light to a plurality of pixels (planes), it serves to bind the areas where the LED is continuously reflected. The labeling is an operation of numbering the regions bounded by the blobs as they are. By assigning a number for each region, it is possible to calculate the coordinate values from when the region is created to disappear. At this time, since the image is two-dimensional rather than three-dimensional, only the coordinate information value of (x, y) is calculated by calculating only the (x, y) coordinate.

The overall configuration of the system is composed of the server 140 which finds the interaction position of the user through image processing with the information obtained from the infrared camera 130 and the client PC 150 which proceeds with the contents. The infrared camera 130 connects to the server 140. Then, the projector 120 is connected to the client PC 150 to analyze the packet received from the server 140 to execute content.

infrared ray LED  Array Bars IR - LEDs Array Bar )

3 is a schematic view showing the principle of improving the linearity of light using an infrared LED array bar (IR-LEDs Array Bar), Figures 4 and 5 is a product picture and installation of the infrared LED array bar (IR-LEDs Array Bar). It is a photograph of the appearance.

The infrared LED array bar 110 installs a plurality of infrared LEDs (IR-LEDs) at regular intervals (for example, 3 cm intervals) as shown in the photo of FIG. 4. At this time, the reason why the infrared LEDs (IR-LEDs) 111 are arranged at regular intervals is to make the amount of infrared light uniform for all regions. In addition, the reason for setting the spacing of the infrared LEDs (IR-LEDs) to 3 cm is a distance of 3 cm, which was found to be adequate as a result of testing a distance that can be supplied with sufficient amount of infrared light while considering the amount of heat generated by each LED. However, it is not necessarily limited to this value and may vary depending on the LED device or other environment.

The infrared LEDs (IR-LEDs) 111 generally emit light at a 45 degree angle. Accordingly, the IR-LEDs array bar 110 of the present invention vertically blocks both sides of the IR LEDs 111 to a wall to prevent the light of the LEDs from radiating at 45 °. Only the light is configured to go out to improve the straightness of the light (see Figure 3).

The IR-LEDs Array Bar 110 is located at an upper portion (ceiling or sidewall top) of the front of the screen 100. It attaches to the front surface at intervals (for example, about 3 cm) to capture the user's interaction, and generates the interaction film 160 constituting the user's interaction area. This means that the user's hand touches the interaction layer 160 when the user reaches out, thereby entering the interaction area where the user can freely interact.

Interaction  Zone extraction method

6 is a view schematically showing the principle that the infrared light is reflected by the hand through the infrared camera and extracted.

Infrared light of the infrared LED array bar 110 is spread in a straight line by the principle of 'straightness of light'. If an object is blocked in front of it, the property of straightness is lost, causing a shadow to appear on the opposite side of the object. The present invention is constructed as follows using this principle.

First, when a user touches with his hand to interact with the screen 100, the user touches the interaction layer 160 formed on the front surface of the screen 100. As described above, the interaction layer 160 is an interaction region formed by infrared light generated by the infrared LED array bar 110.

As shown in FIG. 6, when the hand touches the interaction layer 160, light of the infrared LEDs (IR-LEDs) is blocked at the bottom of the hand to generate shadows, and the infrared LEDs (IR-LEDs) are placed on the upper hands. ) Light is reflected from the back of the hand. In this case, the infrared light reflected from the back of the hand is photographed and collected by the infrared camera 130 photographing the interaction region. The infrared camera 130 transmits the captured image signal to the server 140, and the server 140 detects the user's hand region through the image processing the signal transmitted from the infrared camera 130, Extract the coordinates (x, y) of the hand.

As described above, the infrared LED array bar 110 places a plurality of infrared LEDs (IR-LEDs) at regular intervals (for example, 3 cm intervals) so that the regions of light output from the infrared LEDs 111 overlap each other. Provides a wide area infrared light film.

Multi touch based large Interactive  Display method

7 is a flowchart schematically illustrating a multi-touch based large interactive display method according to an exemplary embodiment of the present invention.

Referring to FIGS. 1, 2, and 6, the infrared film array bar 110 installed at the upper front side of the screen 100 generates infrared rays vertically in front of the screen 100 to form an interactive film 160. Create Thereafter, the infrared film 130 captures infrared light reflected by the human body or an object in contact with the interactive layer 160 and hits the human body or the object. The image signal captured by the infrared camera 130 is transmitted to the server 140.

Next, referring to FIG. 7, the server 140 separates the information obtained from the infrared camera 130 into an area where the LED light is reflected and an area that is not through the image binarization (steps S10 to S20). .

Then, the reflected area is bundled through the blob (step S30). That is, since a human hand or an object reflects light to a plurality of pixels (planes) instead of reflecting light to one pixel (dot), it binds an area where the LED is continuously reflected.

Next, the areas bound through the blob are numbered by labeling (step S40). By assigning a number for each region, it is possible to calculate the coordinate values from when the region is created to disappear.

Then, the signal distortion is corrected using the homography matrix after the labeling (step S50). The homography matrix will be described in detail later with reference to FIG. 8.

Then, the coordinates are corrected and tracked at the Windows API level (steps S60 to S70). In this case, the method of correcting coordinates and tracking method at the Windows API level is as follows. Through the cvWarpPerspective and cvGetPerspectiveTransform provided by the OpenCV library, input coordinates of four corners of the image on the screen viewed by the infrared camera 130 to convert an image in a quad of four points into a quad of four points. Correct the distortion. The tracking transmits coordinate values obtained by locating a person's hand or an object to the network and generates an interaction at a corresponding point.

Next, the server 140 generates the post-tracking packet and transmits the packet to the client PC 150 (step S80).

Finally, the client PC 150 analyzes the packet received from the server 140 and executes a packet command to execute the content according to the packet to the screen 100 through the projector 120 (step). S90 to S120).

As described above, the present invention may be classified into an image processing module for recognizing an image acquired through a camera and a network communication connection module for transmitting information processed by the image processing module.

Here, the image processing module designates an area of the actual projection screen seen in the acquired image after acquiring an image by using the infrared camera 130. In this case, a homography matrix is used to correct distortion of the generated image. In addition to the constant illuminance, the incoming image is binarized to remove noise, and then the position of the hand (finger) or object is determined by blob / labeling.

However, the distortion of the actual screen is corrected by the homography matrix, but the extracted coordinates are only coordinates in the camera view area, not coordinates applied to the actual window coordinates. To do this, coordinates are calculated at the API level to match the camera view with the actual view coordinates. Specifically, the screen 100 viewed by the projector 120 and the screen of the screen 100 viewed by the infrared camera 130 are different in shape and size. In order to correct the distortion of the image, a homography matrix is used to solve the problem.

Next, the size of the resolution viewed by the infrared camera 130 is 640 × 480, but the size of the screen displayed on the actual monitor (1024 × 768) or the projector 120 is different and coordinate correction is performed using a homography matrix. Is also incorrect. To do this, correcting the coordinates at the WindowsAPI level ensures proper tracking. Here, the homography is calculated through FIG. 8. 8 schematically illustrates a process of a homography algorithm for correcting distortion in an image.

First, the shape of the image of the screen of the viewing area viewed by the actual user and the screen of the viewing area viewed by the infrared camera 130 is different. The viewing area the user looks at fits into a rectangular area for convenience (FIG. 8 a). The viewing area viewed by the infrared camera 130 generates trapezoidal image distortion due to spatial constraints (b of FIG. 8). The application of the coordinates of the finger (or object) of the distorted image to the coordinates without correction cannot be returned to the coordinates of the desired position. In order to solve this problem, a homography matrix is applied.

The homography matrix may represent a point on the plane with respect to one plane and a corresponding point of the other plane with a 3x3 matrix. Here the 3x3 matrix represents the use transformation relationship between the two planes, which is called homography. Multiplying a homogeneous matrix by a 9x1 vector by an nx9 matrix that combines points on the plane with corresponding points on the other plane results in zero. In this process, a 9 × 1 vector in which the homography matrix is arranged in a dictionary is obtained through SVD (Singular Value Decomposition).

As such, the distortion of the image is corrected by using a homography matrix. The corrected coordinates are coordinates in the viewing area of the infrared camera 130 but are not coordinates applied to the actual window coordinates. If the entire view area of the infrared camera 130 is 640 × 480, and the resolution of the window is 1024 × 768, the positional error may be much higher. In order to reduce this position, the mouse range can be calculated and controlled at the API level. Since the actual coordinates of the mouse are in the range of 0 to 65535, regardless of the resolution, multiply the current resolution (width, height) and multiply the coordinates resulting from the homography calculation. (C of FIG. 8).

Here, the network communication module continuously transmits the values input by the users to the content in real time in order to interact with the content. The server / client model may be 1: 1, 1: N, but if the environment for executing the content is provided, the infrared camera 130 and the projector 130 may be increased by one. The actions that the user interacts with the content are to analyze the packet that the content has received from the server and to proceed the content by the number of packets.

Multi touch based large Interactive  Another embodiment of a display system

9 is a schematic diagram of a multi-touch based large size interactive display system according to another exemplary embodiment of the present invention.

First, in the case of the large interactive display system described with reference to FIG. 1, the infrared LED array bar 111 is installed on the front top or ceiling of the screen 100, and the upper part is separated from the infrared LED array bar 110 by a predetermined distance. Alternatively, the projector 120 and the infrared camera 130 are installed on the ceiling. However, in this system, since the infrared camera 130 which detects when the user touches the interactive layer 160 by hand is located behind the user's back, the system detects infrared light that is reflected from the user's hand. Shooting).

In order to solve this problem, the present invention proposes a multi-touch large interactive display system as shown in FIG. 9.

In the large interactive display system of FIG. 9, the infrared LED array bar 110 is installed at the bottom or the bottom of the front of the screen 100 to form the interactive film 160 on the front of the screen 100, and the screen 100. The projector 120 and the infrared camera 130 are installed at the bottom or bottom of the rear side. At this time, the screen 100 is configured by installing a rear screen (Rear Screen) on a glass window or an acrylic plate. Accordingly, when the user touches the interactive layer 160 by hand (or with another object), the infrared camera 130 detects (shoots) infrared light that is reflected from the hand and is reflected (server 140 in FIG. 1). Reference), it is possible to extract the coordinate value of the hand region. At this time, the method of extracting the coordinate value of the hand region in the server is the same as described in Figures 1 and 7 will be omitted here.

The server 140 extracts an interaction location of a user, generates a packet generating an interaction at a corresponding point, and transmits the packet to the client PC (see 150 of FIG. 1), and the server 140 at the client PC 150. Analyzes the packet received from the and executes the content according to the packet through the projector 120.

The multi-touch-based large interactive display system and method thereof according to the present invention configured as described above are provided with an interactive film (IR) that can be interacted with by installing an infrared LED array bar (IR-LEDs array bar) on the ceiling or floor of the front of the screen. The technical problem of the present invention can be solved by generating a surface to allow a user to freely interact with the content.

Preferred embodiments of the present invention described above are disclosed to solve the technical problem, and those skilled in the art to which the present invention pertains (man skilled in the art) various modifications, changes, additions, etc. within the spirit and scope of the present invention. It will be possible to, and such modifications, changes, etc. will be considered to be within the scope of the following claims.

The multi-touch-based large interactive display system and method thereof according to the present invention can be provided as an interactive ambient display environment in which individual or large crowds can enjoy various contents, and can be parallelized through interworking with multiple cameras and multi computers. It can be scaled by computing.

10: ceiling
100: screen or wall display device
110: IR-LEDs Array Bar
111: IR-LEDs 120: projector
130: Infrared Camera
131: IR-Filter
140: server 150: client PC
160: interactive zone or interactive surface

Claims (8)

delete In the large interactive display system based on multi-touch,
A screen for displaying an image;
An infrared LED array bar generating infrared light by generating infrared rays in front of the screen;
A projector for projecting an image onto the screen;
An infrared camera for photographing infrared light reflected when a human body or an object comes into contact with the interactive film;
A server configured to extract a coordinate value of an interaction position of the human body or an object through image processing using the information obtained from the infrared camera, and generate a packet generating an interaction at a corresponding point; And
A client PC that analyzes the packet received from the server and executes the content according to the packet on the screen through a project;
Including,
The infrared LED array bar is configured by arranging a plurality of infrared LEDs (IR-LEDs) at regular intervals, and a partition wall is formed on both sides of the infrared LEDs, and is installed on the top or ceiling of the screen, and the projector and the The infrared camera is a multi-touch large interactive display system, characterized in that installed on the top or ceiling separated by a predetermined distance to the infrared LED array bar.
In the large interactive display system based on multi-touch,
A screen for displaying an image;
An infrared LED array bar generating infrared light by generating infrared rays in front of the screen;
A projector for projecting an image onto the screen;
An infrared camera for photographing infrared light reflected when a human body or an object comes into contact with the interactive film;
A server configured to extract a coordinate value of an interaction position of the human body or an object through image processing using the information obtained from the infrared camera, and generate a packet for generating an interaction at a corresponding point; And
A client PC that analyzes the packet received from the server and executes the content according to the packet on the screen through a project;
Including,
The infrared LED array bar is configured by arranging a plurality of infrared LEDs (IR-LEDs) at regular intervals, and a partition wall is formed on both sides of the infrared LEDs, and is installed at the bottom or bottom of the front of the screen, and the projector and the An infrared camera is installed at the bottom or bottom of the rear side of the screen,
The screen is a large touch-based large interactive display system, characterized in that consisting of a rear screen (Rear Screen) on a glass window or an acrylic plate.
delete The method according to claim 2 or 3,
The infrared camera is one of a camera including a CCD camera, a USB CCD camera, and a web camera, and the infrared touch filter (IR-Filter) is installed in front of the lens of the camera.
In the large interactive display method based on multi-touch,
(a) generating infrared film by generating infrared rays in front of the screen;
(b) photographing infrared light reflected by a human body or an object in contact with the interactive film with an infrared camera;
(c) extracting a coordinate value of an interaction position of the human body or an object through image processing using the information obtained from the infrared camera, and generating a packet generating an interaction at a corresponding point; And
(d) analyzing the packet and executing content according to the packet on the screen through a project;
Including,
The method of extracting the coordinate value of the interaction position in the step (c),
(c1) a binarization step of separating the information obtained from the infrared camera into an area where the LED light is reflected and an area that is not through image binarization;
(c2) a blob to bind the reflected area;
(c3) labeling the numbered areas bounded by the blobs;
(c4) a homography step of correcting signal distortion using a homography matrix after the labeling;
(c5) a window API coordinate correction step of correcting distortion by inputting coordinates of four corners of an image on a screen viewed by the infrared camera and converting an image in a quadrangle of four points into a quadrangle of four points; And
(c6) extracting coordinate values by identifying interaction positions of the human body or an object;
Multi-touch based large interactive display method comprising a.
delete The method according to claim 6,
The method of generating the packet in step (c),
and (c7) generating a packet to generate an interaction at a corresponding point by transmitting a coordinate value of an interaction position of the human body or an object to a network through a content.

Images