KR101280840B1 - Method of identifying the laser point position on screen of presentation system - Google Patents

Abstract

PURPOSE: A laser point position coordinate recognizing method on a screen of a presentation system is provided to recognize the laser point coordinate quickly and exactly on a screen image photographed by a camera, thereby exactly indicating the line, shape, or character that most match to the intention of a presenter in the position where the presenter indicated in the presentation image. CONSTITUTION: A presentation system converts the size of an image which is obtained from a camera into the fixed size (S101). If the image is colored, the system converts the image into a black and white image (S102). The system calculates a conversion matrix through the homographic calculation, and prospectively transforms the monitor screen area of the image into the monitor screen by using the conversion matrix (S103,S104). The system removes the noise from the binary image of the converted black and white image (S105,S106). The system changes the calculated central point coordinate into a monitor coordinate by using the labeling of the binary image (S107-S111). [Reference numerals] (AA) Start; (BB) Return; (S101) Convert a size of images; (S102) Convert images into a black and white image; (S103) Homographic calculation; (S104) Perspective transform; (S105) Image binarization; (S106) Noise removal with morphology; (S107) Labeling; (S108) Size calculation of each label; (S109) Select the largest label; (S110) Central point calculation; (S111) Conversion into a monitor coordinate

Description

TECHNICAL FIELD [0001] The present invention relates to a method of recognizing a laser point position on a screen of a presentation system,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of recognizing coordinates of a position indicated by a laser pointer in order to display a character or a picture on a screen in a presentation system, And automatically recognizing the position coordinates of the laser point pointed by the laser pointer.

When a presenter makes a presentation using a user terminal such as a beam projector and a notebook, the user directs a laser beam to the screen by using a laser pointer. At this time, underlining, triangle, square, It may be displayed in a circle or the like, or the additional description may be simply displayed in characters.

However, when the laser pointer is turned on in the presentation, the current position indicated by the presenter is referred to as a point (referred to as "laser point"). However, since the laser pointer does not continue to be displayed on the presentation image, Has a disadvantage in that it has temporal characteristics that disappear from the screen.

In order to solve this disadvantage, a method of sequentially inputting shapes such as an underline, a triangle, a rectangle, and a circle to be input by a laser pointer in generating a data for a presentation image is taken in advance, .

Accordingly, in the related art, by designing a shape instructed without interruption such as an underline, a triangle, a rectangle, and a circle, which are images instructed by a laser pointer, together with a presentation image, The technology has been patented by the present applicant and registered with Registration No. 10-122543.

[Related Technical Literature]

1. A presentation system using a laser pointer mouse (Patent Application No. 10-2000-0016712)

2. An input apparatus using a laser pointer and a presentation providing system using the same (Patent Application No. 10-2005-0109878)

3. Application of Laser Pointer Image Extraction to Presentation Image (Patent Registration No. 10-1222543)

Patent No. 10-1222543 pre-registered by the applicant of the present invention extracts and stores the trajectory of the laser pointer, compares the trajectory of the laser pointer with a preset figure, applies the most similar figure image to the corresponding position of the presentation image, So that the graphics are continuously displayed on the presentation image. Therefore, in order to display letters, pictures, etc. on an image on a screen as disclosed in Japanese Patent No. 10-1222543, it is required to obtain a position coordinate of the laser point. That is, when a laser pointer is shot using a laser pointer while a presentation is being performed, a coordinate value for the position of the laser point with respect to the entire screen must be set so that an image such as an ellipse or an underline at that position is set to a laser point You will be able to draw in that position.

It is an object of the present invention to provide a method and apparatus for recognizing position coordinates of a laser point on a screen of a presentation system capable of quickly and accurately recognizing coordinates of a position indicated by a laser pointer Method.

According to an aspect of the present invention, there is provided a method of displaying a presentation image on a screen, the method comprising: displaying a presentation image provided from a user terminal on a screen, the laser pointer scanning a laser point on the screen, A first step of setting four corners of a screen full screen in an outline point setting mode; A second step of converting the size of the image obtained from the camera to a predetermined size in the laser point recognition mode; A third step of converting the image obtained from the camera into a monochrome image if the image is color; A fourth step of calculating a transformation matrix H through homography calculation; A fifth step of performing a perspective transformation of an area corresponding to the monitor screen among the images acquired using the conversion matrix in the same manner as the monitor screen; A sixth step of converting the perspective transformed monochrome image into a binarized image using a set threshold value; A seventh step of removing noise from the binarized image; An eighth step of assigning a label to the connected region of the one portion in the binarized image from which the noise is removed, and adding the number of 1s of each label to calculate the size; A ninth step of selecting a label having the largest size among labels within a specific size, and calculating a center point of the selected label; And a tenth step of converting the calculated center point coordinates into monitor coordinates.

The method includes the steps of changing a camera mode to an outlier point setting mode, displaying a white dialog on a screen by scanning a monitor screen in white by a beam projector to find an outline point of the screen, And converting the captured dialogue image into black and white, and binarizing it, and then setting the upper left corner, the lower left corner, the upper right corner, and the lower right coordinate of the one-person region to set four corner coordinates of the entire screen.

으로 변환행렬(H)을 산출하고, 상기 제6단계는 수학식

에 따라 해당 위치의 이미지 값이 임계치(T) 미만이면 0으로, 임계치(T) 이상이면 1로 이진화하는 것이다.The fourth step uses the camera pinhole model and the projective transformation to obtain the following equation

(H), and the sixth step is a step of calculating a transformation matrix

0 " if the image value of the corresponding position is less than the threshold value T, and " 1 "

로 산출하며, 상기 제10단계는 모니터 화면의 가로크기(W), 세로크기(H), 이진 영상의 가로크기(B_W), 세로크기(B_H)라고 할 때, 모니터 좌표(M_x,M_y)는 수학식

으로 환산한다.
In the seventh step, the noise is removed using the erosion and expansion in the morphology. In the erosion operation, the 3 * 3 size filter is used less than 3 times, and the expansion operation is performed 3 times or less ( _Cx ) of the x-coordinate of the selected label and the center of the y-coordinate ( _Cy )

When called, the step 10 is a horizontal dimension (W) of the monitor screen, the vertical size (H), width of the binary image size (B _W), the vertical size (B _H) is calculated, the monitor coordinates (M _x, M _y )

.

A method of automatically recognizing coordinates of a position indicated by a laser pointer on a screen of a presentation system according to the present invention is a method of quickly and accurately recognizing the coordinates of a laser point on a screen image captured by a camera, It is possible to accurately display a straight line, a figure, a character, and the like that best match the presenter's intention in one place.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a schematic diagram showing the overall configuration of a typical presentation system to which the present invention may be applied;
Fig. 2 is a block diagram showing the configuration of the camera shown in Fig. 1,
3 is a block diagram showing a configuration of a user terminal in FIG. 1,
4 is a flowchart illustrating an automatic coordinate calculation procedure for recognizing a position of a laser pointer in a presentation system according to the present invention.
5 is a flowchart illustrating a procedure for calculating coordinates in real time in the presentation system according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a detailed description of preferred embodiments of the present invention will be given with reference to the accompanying drawings. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In the present specification, when any one element 'transmits' data or signals to another element, the element can transmit the data or signal directly to the other element, and through at least one other element Data or signal can be transmitted to another component.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic diagram showing the overall configuration of a typical presentation system to which the present invention may be applied; FIG. 1, a presentation system according to the present invention includes a beam projector 100, a screen 200, a laser pointer 300, a camera 400, a user terminal 500, and an IP network 600 . Here, the IP network 600 is a communication network capable of a large-capacity, long-distance voice and data service, and can be, for example, the Internet. Also, the IP network may be a next generation wired or wireless network for providing high-speed multimedia service based on ALL IP (Internet Protocol).

The camera 400 may be a camera or an infrared camera that recognizes the light of the laser pointer 300 according to the type of the laser pointer 300. The camera 400 may recognize the laser point pointed to the screen 200 most brightly Brightness, contrast, white balance, and the like, so that the beam projector 100 captures the screen 200 to be scanned. In this case, since the area of the screen obtained according to the position of the camera 400 is changed, the position of the laser point is different from the position of the screen. Therefore, in order to solve this problem, And transforms them using homography and perspective transform so that each corner point becomes a corner point of the user terminal screen.

Fig. 2 is a diagram showing a configuration of the camera 400 in Fig. 2, the camera 400 includes a CMOS module 410, an MCU (Micro Control Unit) 420, a laser imaging module 430, a first storage unit 440, a first transmission / reception unit 450, And an I / O interface 460. The laser photographing module 430 includes a laser photographing unit 431 and a laser position value extracting unit 432. [

In this specification, a module may mean a functional and structural combination of hardware for carrying out the technical idea of the present invention and software for driving the hardware. For example, the module may mean a logical unit of a predetermined code and a hardware resource for executing the predetermined code, and it does not necessarily mean a physically connected code or a kind of hardware. Can be easily deduced to the average expert in the field of < / RTI >

The MCU 420 connects the data session with the user terminal 500 under the control of the first transmission / reception terminal 450 or the first I / O interface 460, The image capturing command for the screen 200 and each position value set by the user terminal 500, thereby driving the CMOS module 410 to enter the photographing standby mode. At this time, the MCU 420 stores the received position values in the first storage unit 440. That is, when the MCU 420 connects the data session through the IP network 600 with the user terminal 500, the MCU 420 controls the first transmission / reception terminal 450 and transmits the data session with the user terminal 500 via the direct data cable And controls the first I / O interface 460 when connecting.

The laser from the laser pointer 300 carried by the user is irradiated by the user terminal 500 onto the screen 200 with one position value set on the screen 200 and through the CMOS module 410 The sensed MCU 420 wakes-up the laser imaging module 430. Accordingly, the laser photographing unit 431 of the laser photographing module 430 photographs the laser image during the continuous irradiation time of the laser output from the laser pointer 300, and then stores it in the first storage unit 440. At the same time, the laser position value extracting means 432 of the laser photographing module 430 extracts the position value of the laser continuously irradiated to the laser image photographed by the laser photographing means 431 at a predetermined time interval, (440). That is, when the user makes a presentation, the laser photographing module 430 irradiates a laser to the screen 200 by using the laser pointer 300. At this time, the laser pointer 300 is used to underline, If you specify a triangle, rectangle, or circle, you can recognize the shape of one such underline, triangle, square, and circle as one continuous laser irradiation.

Here, the laser photographing means 431 photographs the laser image during the continuous irradiation time output from the laser pointer on the screen 200 at the time of photographing the laser image, The range is preferably set to 1/100 to 1/2 second.

The first storage unit 440 and the second storage unit 550, which will be described later, are non-volatile memories (NVMs), which do not delete the data stored in the flash memory Memory, a magnetic random access memory (MRAM), a phase-change random access memory (PRAM), and a ferroelectric RAM (FRAM).

The MCU 420 may transmit the extracted positional value to the user terminal 500 according to a predetermined time interval between the laser image and the laser image for the continuous irradiation time taken through the data session connected to the user terminal 500 1 transmission / reception terminal 450 and the first I / O interface 460.

FIG. 3 is a diagram showing a configuration of the user terminal 500 in FIG. 3, the user terminal 500 includes a second transmission / reception terminal 510, a second I / O interface 520, a central control unit 530, a presentation image implementation module 540, a second storage unit 550, an input / output unit 560, and a laser pointer writing module 570. The central control unit 530 includes a camera / beam projector control module 531. The presentation image implementation module 540 includes a screen position value setting unit 541, a laser image extracting unit 542, 543). Meanwhile, the user terminal 500 stores a presentation program such as a power point in the second storage unit 550, and the central control unit 530 loads the program in the system memory according to a user's request through the input / output unit 560 do.

The camera / beam projector control module 531 of the central control unit 530 controls the second transmission / reception terminal 510 or the second I / O interface 520 so that the beam projector 100 and the camera 400 connect data sessions. . Then, the camera / beam projector control module 531 controls the beam projector 100 to implement the presentation image through the connected data session, and then wakes up the presentation image implementing module 540.

The screen position value setting means 541 of the presentation image implementing module 540 sets the screen position value of the presentation image output from the beam projector 100 in the horizontal direction X and the vertical direction Y) is set and stored in the second storage unit 550. Here, the position value can be set as a pixel value or a coordinate value, and can be set to various numbers according to the resolution of the presentation image implemented by the beam projector 100.

In the embodiment of the present invention, the screen position value setting means 541 is configured to recognize the position coordinates indicated by the laser pointer 300 in the image acquired by the camera 400 in cooperation with the camera / beam projector control module 531 The operation mode of the camera is controlled by the outline point setting mode and the laser point recognition mode. The outline point setting mode is to adjust the camera exposure, brightness, gamma and sharpness in the setting program so that the screen can be seen on the screen. exposure, brightness, gamma, and sharpness in the setting program so that only the laser point can be acquired without viewing the screen on the screen.

The laser image extracting unit 542 of the presentation image implementing module 540 compares the received laser image with a plurality of images stored in the second storage unit 550 to extract an image having the closest shape.

The image converting means 543 of the presentation image implementing module 540 divides the shape of the nearest image extracted by the laser image extracting means 542 at predetermined time intervals and then extracts the divided shape image Matching the position values extracted according to the predetermined time intervals of the laser images to generate presentation-matching type laser images, and stores the generated laser images in the second storage unit 550.

The camera / beam projector control module 531 synchronizes the presentation-matching type laser image generated by the presentation image implementation module 540 with the presentation image being output to the beam projector 100 according to the flow of time, So as to control the beam projector 100 as much as possible.

The laser pointer writing module 570 uses an elliptic fitting and a linear fitting to determine a trajectory of the laser pointer 300 on the screen 200 using the laser pointer 300, Or a straight line, and displays various colors on the screen screen. That is, when the laser pointer 300 is operated and exposed to the screen 200, the camera 400 acquires it and stores it in the coordinate system of the monitor, and continuously stores the laser pointer 300 when the laser pointer 300 is continuously operated. If the number of stored frames (A) exceeds the number of stored frames (A) when the laser pointer 300 is not on the screen and the number of stored coordinates is greater than or equal to a predetermined number B, an ellipse fitting is performed to search for a long axis length, a short axis length, If it is horizontal and the short axis length / long axis length is less than or equal to the specific value C, it is determined as a straight line. When ellipse is judged, the ellipse is calculated as the calculated value on the screen. Then, the straight line is fitted to the coordinates stored in the straight line, and the starting point and the ending point of the drawn line are calculated using the left and right coordinates of the stored coordinates, , And repeats this process by looping until the end of the program.

In order to recognize the position coordinates on the screen indicated by the laser pointer in this presentation system, in the present invention, first, the coordinates of the four corners of the entire screen are searched for and thresholds are set, Point coordinates are recognized.

4 is a flowchart illustrating an automatic coordinate calculation procedure for recognizing a position of a laser pointer in a presentation system according to the present invention.

The automatic coordinate calculation procedure for recognizing the coordinates of the laser point indicated by the laser pointer 300 according to the present invention includes a camera mode changing step S1 and a white dialogue- (S2), acquiring a camera image (S4), and searching for upper left, lower left, upper right, and lower right coordinates (S4).

First, when the screen scanned by the beam projector 100 on the screen 200 is acquired by the camera 400, the coordinates of the monitor screen of the user terminal 500 are not matched. To correct this, the homography and perspective transform ). Since four points are required for the homography and perspective transform, conventionally, in order to set the area corresponding to the monitor screen in the image obtained by the camera, coordinates to be converted are obtained by clicking four points of the corner with the mouse, It is not easy to obtain by clicking on the image. Therefore, the present invention uses an automatic coordinate calculation algorithm through image processing. The automatic coordinate calculation algorithm according to the present invention changes the camera mode to the outline point setting mode and then displays a white screen on the monitor in the setting program. If the screen is displayed on the screen, it is acquired by the camera, It will find the corner point of the area.

4, in the camera mode change step S1, the camera mode is changed to the outline point setting mode, and then the exposure, brightness, and the like of the camera 400 are displayed on the screen 200, , Gamma, and sharpness settings. In the embodiment of the present invention, there are two camera modes. In the outline point setting mode, the exposure program, the brightness, the gamma, and the sharpness of the camera are adjusted in the setting program, The laser point recognition mode can be used to adjust the camera exposure, brightness, gamma, and sharpness in the setting program so that the screen can not be seen on the screen, It is a mode that can acquire only.

Then, in step S 2 of displaying a white dialogue, the beam projector 100 scans the monitor screen in white to find the end point of the screen and displays a white dialog on the screen.

In the camera image acquiring step S3, the dialog image scanned on the screen 200 is captured. In step S4, the dialog image is changed to black and white, and then the binarized image is binarized. Find the upper left, lower left, upper right, and lower right coordinates. That is, the monochrome image f is converted into the binarized image B as shown in the following Equation 1 using the set threshold value T to obtain the four corner coordinates of the dialog screen.

5 is a flowchart illustrating a procedure for calculating coordinates in real time in the presentation system according to the present invention.

The procedure for calculating coordinates in real time according to the present invention is performed in a state in which the camera mode is the laser point recognition mode. The step of converting the size of the image obtained from the camera 400 to a predetermined size (S102) of converting the image obtained from the camera into a monochrome image (S102); calculating a transformation matrix (H) through homography calculation (S103); A step S104 of perspectively transforming an area corresponding to the monitor screen in the same manner as the monitor screen, a step S105 of converting the perspective-converted monochrome image into a binarized image using a preset threshold value, A labeling step S107, a step S108 of calculating the size of each level, a step S109 of selecting the largest label, And a step (S110) of calculating, consists of a step (S111) for converting the monitor coordinates.

Referring to FIG. 5, the image size conversion step (S101) is to reduce the image size of the image acquired by the camera to a predetermined size for real-time processing. That is, since the size of the image varies depending on the resolution of the camera, the image size is limited so as to enable real-time processing even if a high-resolution camera is used.

The image monochrome conversion step S102 converts the obtained image into monochrome if the camera 400 is a color camera, and omits this step if it is a monochrome camera.

Homography calculation step S103 calculates a transformation matrix H to perform a perspective transformation. To do this, the camera model uses a camera pinhole model as shown in Equation (2).

The transformation between the planes in the camera model uses Projective Transformation.

If h is a projective transformation, there exists a 3x3 matrix H in which an inverse matrix exists and can be expressed by the following equation (3).

In the perspective transformation step (S104), an area corresponding to the monitor screen among the images obtained using the H calculated by the homography is transformed into the same as the monitor screen. In other words, generally, when a photograph of a building is acquired by using a camera, Perspective Distortion exists. Here, 4 points are selected and H is obtained so that these four points are squares. do.

In the image binarization step S105, the monochrome image f is converted into the binarized image B by using the above-described equation (1) using the set threshold value T. In other words, the laser point comes in brightly, so it can be found by interpreting the one part in the binarized image.

On the other hand, in the noise removal step using the morphology (S106), noise is removed by using erosion and expansion in the morphology since the one part in the binarized image can be generated by noise even in the non-laser point. The erosion operation can cause the objects of small chunks to disappear according to the 3 * 3 size filter and the use frequency (less than 3 times), and the expansion operation can be performed by using the 3 * 3 size filter and the use frequency Can be made to disappear.

In the labeling step, the numbers (labels) are differently assigned to the connected regions of one portion in the binary image, and the sizes of 1s of the labels are summed up to calculate sizes (S107, S108).

And searching the one that corresponds to the largest size of label of the labels within a given size by the noise is not micheo removed in order to prevent the laser point is erroneously recognized, the center of the x-coordinate of the selected label according to the following equation: 5 (C _x ) and the center C _y of the y coordinate (S109, S110).

In the conversion step S111 of the monitor coordinates, the previously calculated coordinates are converted into monitor coordinates (M _x , M _y ) using the horizontal size (W) and the vertical size (H) of the monitor screen. (B _W ) and the vertical size (B _H ) of the binary image, the monitor coordinates (M _x , M _y ) are converted into the following Equation (6).

As described above, preferred embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms have been used, they have been used only in a general sense to easily describe the technical contents of the present invention and to facilitate understanding of the invention , And are not intended to limit the scope of the present invention. It is to be understood by those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

100: a beam projector 200: a screen
300: laser pointer 400: camera
500: user terminal 600: IP network

Claims (7)

A presentation system in which a beam projector displays a presentation image provided from a user terminal, a laser pointer scans a laser point on a screen, and a camera captures a screen,
A first step of setting four corners of the screen full screen in the outline point setting mode;
A second step of converting the size of the image obtained from the camera to a predetermined size in the laser point recognition mode;
A third step of converting the image obtained from the camera into a monochrome image if the image is color;
A fourth step of calculating a transformation matrix H through homography calculation;
A fifth step of performing a perspective transformation of an area corresponding to the monitor screen among the images acquired using the conversion matrix in the same manner as the monitor screen;
A sixth step of converting the perspective transformed monochrome image into a binarized image using a set threshold value;
A seventh step of removing noise from the binarized image;
An eighth step of assigning a label to the connected region of the one portion in the binarized image from which the noise is removed, and adding the number of 1s of each label to calculate the size;
A ninth step of selecting a label having the largest size among labels within a specific size, and calculating a center point of the selected label; And
And converting the computed center point coordinates into monitor coordinates. The method of any preceding claim,
2. The method of claim 1, wherein the first step
Changing a camera mode to an outlier point setting mode, displaying a white dialog on a screen by scanning a monitor screen in white by a beam projector to find outline points on the screen, capturing a dialogue image displayed on the screen, Converting the dialogue image into black and white, binarizing the dialogue image, and then setting the four corner coordinates of the entire screen by obtaining the upper left, lower left, upper right, and lower right coordinates of the one-person region. Recognition method.
3. The method according to claim 1 or 2, wherein the fourth step
Using the camera pinhole model and the projective transformation,

And a transformation matrix (H) is calculated from the transformation matrix (H).
3. The method according to claim 1 or 2, wherein the sixth step
B (x, y) is a binarized image, f (x, y) is a monochrome image, and T is a threshold value,

Wherein the binarizing step binarizes the value of the laser point position coordinates on the screen of the presentation system according to the value of the position of the presentation point.
The method according to claim 1 or 2, wherein the seventh step
In the morphology, the noise is removed by erosion and expansion. In the erosion operation, the 3 * 3 size filter is used less than 3 times and the expansion operation is made 3 times smaller than 3 * 3 filter. A method of recognizing laser point position coordinates on a screen of a system.
3. The method according to claim 1 or 2, wherein the ninth step
The center (C _x ) of the x coordinate of the selected label, the center of the y coordinate (C _y )
Equation

And calculating the coordinates of the laser point position on the screen of the presentation system.
3. The method according to claim 1 or 2, wherein the step
Horizontal dimension (W) of the monitor screen, the vertical size (H), the horizontal scale (B _W) of the binary image, when said vertical dimension (B _H), the monitor coordinates _(x M, _y M) is formula

Wherein the coordinates of the laser point position on the screen of the presentation system are converted into the coordinate of the laser point position on the screen.

Images