JP2009020891A - Remote control system of computer by light rail of optical pointer, control module and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system, a control module and a method for remotely controlling a computer by tracking light of an optical pointer. <P>SOLUTION: A series of images of projected image frames and an optical cursor are captured. The captured images are stored. For each captured image, the position of the optical cursor is detected. A closed-loop light rail of the optical cursor is obtained. An image portion of one captured image is chosen as a reference sample image. By comparing the reference sample image with the image frames, an image block is determined from an image frame including at most one icon and resembling the reference image most. Based on the determined image block, a control signal is generated and forwarded to a computer 1 so as to execute a command related to the icon included in the determined image block. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a computer remote control technique, and more particularly to a system, a control module, and a method for remotely controlling a computer using an optical tracking optical pointer.

  A method of remotely controlling a computer is already known (Patent Document 1). Using this conventional method, the projector projects an image frame output from the computer onto an external screen, for example. The laser pointer generates a light cursor and projects it onto an external screen. The camera captures (captures) the projected image frame and optical cursor as an image capture device. A feature such as size, shape, intensity, etc. of at least one optical cursor is detected from the captured image, and a control command for computer control is generated.

U.S. Patent No. 6,275,214

  However, in detecting the characteristics of the optical cursor in this conventional method, coordinate conversion must be performed. Specifically, corners in the captured image (captured image) must be detected corresponding to the projected image frame. In addition, every time the projector or camera moves during use, the coordinates must be corrected, which is inconvenient.

  The present invention has been made in view of the above problems, and an object thereof is to provide a system, a control module, and a method for remotely controlling a computer by optical tracking of an optical pointer.

  According to one aspect of the present invention, a method for remotely controlling a computer is provided. A method for remotely controlling a computer according to the present invention is a method for remotely controlling a computer that projects an image frame including one or more icons on a remote screen, wherein the optical cursor generated by a handheld optical pointer is displayed on the screen. Projecting on a screen, a step b for capturing a series of images of the image frame and the light cursor, a step c for detecting the position of the light cursor in each of the captured images, and each detected Obtaining a closed-loop light rail of the light cursor based on the position of the light cursor of the light cursor; and obtaining an image portion of one captured image in the captured image based on the closed-loop light rail of the light cursor as a standard sample Image Selecting the image block from the image frame including at most one icon and most similar to the standard sample image, comparing the standard sample image with the image frame; Generating a control signal for causing a computer to execute a command related to an icon included in the image block based on the block.

  In this method, in step e, the correlation coefficient between the standard sample image and the image portion of the image frame is calculated, or the shape, color, and gradation of the image portion of the selected image frame are calculated. Preferably, the image block is selected by comparing any of the above with the standard sample image.

  In this method, the control signal is to cooperate with a control signal to the computer by the pointing device.

  In step c, it is preferable that the position of the optical cursor in each of the captured images is detected based on any of the intensity, shape, and color of the optical cursor.

  In step e, the selected image portion is preferably a minimum rectangular area confined by the closed loop light rail of the optical cursor and surrounding one captured image area.

  According to another aspect of the present invention, a system for remotely controlling a computer is provided. A system for remotely controlling a computer according to the present invention is a system for remotely controlling a computer that outputs an image frame including one or more icons. The system is electrically connected to the computer and is connected to a remote screen from the computer. A projector that projects an image frame; a hand-held optical pointer that generates a light cursor and projects the generated light cursor onto the screen; and an image that captures a series of images of the projected image frame and the light cursor. A capture device electrically connected to the computer for storing an image frame received from the computer; and a captured image received from the image capture device electrically connected to the image capture device. And a control module for storing the optical cursor, wherein the control module detects a position of the optical cursor in the captured image and a closed-loop optical rail of the optical cursor based on the detected position of the optical cursor. And selecting an image portion in the one captured image as a standard sample image based on the closed-loop light rail of the optical cursor, comparing the standard sample image with the image frame, including at most one icon and Selecting an image block from an image frame most similar to a standard sample image, and generating a control signal based on the image block in response to executing a command associated with an icon included in the image block, and sending the control signal to the computer Special To.

  In this system, the control module is preferably connected to the memory unit for storing the image frame received from the computer and an image captured by the image capture device, and the respective captured images. The position of the optical cursor is detected, and a closed loop optical rail of the optical cursor can be drawn based on the detected position of each optical cursor, and stored in the memory unit based on the closed loop optical rail of the optical cursor. A sample selection unit that selects an image portion of the captured image as the standard sample image, and the memory unit and the sample selection unit are connected to each other, and the standard sample image selected by the sample selection unit is stored in the memory unit. The image file A comparison unit for defining the image block from the image frame that includes at most one icon and is most similar to the standard sample, and is connected to the comparison unit so as to be connected to a computer; And a control signal generation unit for sending the control signal to the computer for causing the computer to execute a command related to the icon included in the image block in response to the image block determined by the unit.

  Further, in this system, the comparison unit calculates a correlation coefficient between the standard sample image and the image portion of the selected image frame, or the shape of the image portion of the selected image frame, Preferably, the image block is defined by comparing either the color or the gradation with the standard sample image.

  Further, in this system, the control signal sent out by the control signal generation unit is to cooperate with a control signal to the computer by the pointing device.

  In this system, it is preferable that the position of the optical cursor in each captured image is detected by the sample selection unit based on any of the light intensity, shape, and color of the optical cursor.

  Further, in this system, it is preferable that the image portion selected by the sample selection unit is a minimum rectangular area that is confined by the closed-loop light rail of the optical cursor and surrounds one captured image area.

  Furthermore, according to another aspect of the present invention, a control module for remotely controlling a computer is provided. A control module for remotely controlling a computer according to the present invention includes a computer including one or more icons and sending out an image frame, a projector electrically connected to the computer and projecting the image frame on a remote screen, The image frame received from the computer as a control module electrically connected to the computer and the image capture device in a system having a projected image frame and an image capture device that captures a series of images of a light cursor A memory unit for storing the image captured by the image capture device; and a memory unit connected to the memory unit to detect a position of the optical cursor in each of the captured images. A sample that draws a closed-loop light rail of the light cursor based on the position of each light cursor, and selects an image portion of the capture image stored in the memory unit as a standard sample image based on the closed-loop light rail of the light rail A selection unit, coupled to the memory unit and the sample selection unit, for comparing the standard sample image selected by the sample selection unit with the image frame stored in the memory unit and including at most one icon; A comparison unit that determines an image block from the image frame that is most similar to the standard sample image; and a comparison unit that is electrically connected to the computer and is connected to the comparison unit, and the image block defined by the comparison unit Based on the icons included in the image block. It includes a control signal generating unit that the command to generate a control signal for causing the computer to execute shipped to the computer, a.

  Furthermore, according to another aspect of the present invention, a method for remotely controlling a computer is provided. A method of remotely controlling a computer according to the present invention includes a computer including one or more icons and sending an image frame, a projector electrically connected to the computer and projecting the image frame on a remote screen, a light Generating control signals for remotely controlling the computer in a system having a handheld optical pointer that generates a cursor and projects it onto the screen, and an image capture device that captures a series of images of the projected image frame and optical cursor A method of detecting the position of the optical cursor in each of the captured images; obtaining a closed loop optical rail of the optical cursor based on the detected position of the optical cursor; ii; Closed Selecting an image portion of one capture image in the capture image as a standard sample image based on a light rail, and comparing the standard sample image with the image frame, the standard sample image including at most one icon Selecting an image block from the image frame most similar to iv; generating a control signal based on the image block for causing the computer to execute a command related to an icon included in the image block; It is characterized by having.

  The present invention compares an image portion of one captured image in a captured image selected as a standard sample image based on the closed loop light rail of the optical cursor based on the position of each detected optical cursor with the image frame, Generating a control signal for causing a computer to execute a command related to the icon included in the image block by selecting an image block from an image frame that includes one icon and is most similar to a standard sample image Thus, it is possible to remotely control the computer with the optical pointer without converting the coordinates between the captured image and the image frame as in the prior art. In particular, the closed loop light rail of the optical cursor obtained by the optical pointer can be remotely controlled to the computer for convenient use without moving the projector or image capture device such as a camera during use Workability can be improved.

  FIG. 1 is a diagram illustrating a system 100 for implementing a first embodiment of a method for remotely controlling a computer 1 according to the present invention. The computer 1 outputs an image frame including at least one icon as shown in FIG. 3b or 4b. The system 100 includes a projector 2, a handheld pointer 3, an image capture device 4, and a control module 5.

  The projector 2 is electrically connected to the computer 1 and projects an image frame onto an external screen (not shown) separated by the computer 1.

  The optical pointer 3 is used to generate an optical cursor and project the generated optical cursor onto an external screen. Examples of the optical pointer 3 include a laser pointer and an LED pointer.

  The image capture device 4 captures a projected image frame and a series of images 40 of the optical cursor of the optical pointer 3 at a predetermined image frame capture rate. In this example, the image frame capture rate is 30 frames per second (FPS). The captured image 40 preferably includes the entire image frame 20 to be projected. As the image capture device 4, for example, there is a charge-coupled device (CCD).

  In this example, the control module 5 is incorporated so as to be electrically connected to the computer 1, and receives and stores an image frame. The control module 5 is electrically connected to the image capture device 4 and reads and stores the image 40 captured by the image capture device 4. The control module 5 includes a memory unit 51, a sample selection unit 52, a comparison unit 53, and a control signal generation unit 54.

  The memory unit 51 stores the image frame received from the computer 1 and the image 40 captured by the image capture device 4.

  The sample selection unit 52 is electrically connected to the memory unit 51, detects the position of the optical cursor in each captured image, and can draw a closed-loop optical rail of the optical cursor based on the detected position of the optical cursor. Then, the image portion of the capture image 40 stored in the memory unit 51 is selected as a standard sample image based on the closed loop light rail of the optical cursor.

  The comparison unit 53 is electrically connected to the memory unit 51 and the sample selection unit 52, compares the standard sample image selected by the sample selection unit 52 with the image frame stored in the memory unit 51, and displays at most one icon. An image block is defined from the image frame most similar to the standard sample image it contains.

  The control signal generation unit 54 is electrically connected to the comparison unit 53 so as to be connected to the computer 1, generates a control signal based on the image block determined by the comparison unit 53, and sends the control signal to the computer 1. In response to the control signal, the computer 1 is caused to execute a command related to the icon included in the image block.

  FIG. 2 is a flowchart showing a method for remotely controlling the computer 1 according to the first embodiment of the present invention.

In step S1, an optical cursor is generated using the handheld pointer 3 that is an optical indicator, and the generated optical cursor is projected onto an external screen.

  In step S2, the image capture device 4 captures the projected image frame 20 and a series of images 40 of the optical cursor over a predetermined period, for example, 2 seconds. In this example, the capture time is 2 seconds and the frame capture rate is 30 FPS. Therefore, the amount of captured images is 60 in 2 seconds.

  In step S <b> 3, the image 40 captured in step S <b> 2 is stored in the memory unit 51 of the control module 5.

  In step S <b> 4, the sample selection unit 52 detects the position of the optical cursor for each captured image 40. In this example, the position of the optical cursor is detected in each captured image 40 based on any of the light intensity, shape, and color of the optical cursor. For example, if the captured image 40 is within the intensity range of 0 [lumen] to 255 [lumens], the optical cursor and capture are estimated by estimating that the intensity of the optical cursor is greater than, for example, a threshold of 255 [lumens]. The position of the optical cursor is detected by comparing the light intensities of the images.

  In step S5, the sample selection unit 52 obtains a closed loop optical rail of the optical cursor based on the detected position of the optical cursor. Specifically, the closed-loop light rail is drawn with the first to 51st captured images of 60 captured images 40 (2 seconds × 30 frames / second = 60 frames), for example. In other words, the position of the optical cursor is obtained by superimposing the 51st captured image and the first captured image within 2 seconds.

  In step S6, the sample selection unit 52 selects an image portion of one captured image as a standard sample image from the captured cha images 40 stored in the memory unit 51 based on the closed loop light rail of the optical cursor. In this example, the selected image portion is a standard sample image, eg, as shown in FIGS. 3b and 4b, of one captured image 40 confined by a closed loop light rail drawn by an optical cursor. This is the smallest square area that surrounds the image. The standard sample image includes, for example, at least one icon representing a Microsoft PowerPoint (registered trademark) document as shown in FIG. 3b, and closes the document to the standard sample image as shown in FIG. 4b. 1 icon at most. Furthermore, as already described above, this one captured image 40 is the 51st captured image of the 60 captured image images 40. In this example, including at most one icon means including one or more icons in the drawn area.

  In step S7, the comparison unit 53 compares the standard sample image selected by the sample selection unit 52 with the image frame stored in the memory unit 51, and includes at most one icon and is most similar to the standard sample image. Define an image block from an image frame. In this example, the image block is selected by calculating the correlation coefficient between the standard sample image and the image portion of the defined image frame. Specifically, the image block is defined by comparing any of the shape, color, and gradation of the image portion of the selected image frame with the standard sample image.

  In step S <b> 8, the control signal generation unit 54 generates and sends a control signal to the computer 1 based on the image block determined by the comparison unit 53. Then, the computer 1 executes a command related to the icon included in the image block in response to the control signal from the control signal generator 54. In this example, the control signal cooperates with the control signal of the computer 1 by an instruction device such as a mouse. For example, when the icon included in the image block is the Microsoft PowerPoint document shown in FIG. 3B, the computer 1 is controlled based on the control signal generated by the control signal generation unit 54. The mouse is electrically connected to the computer. This is the same as executing with the command related to opening the Microsoft PowerPoint document pointed at. When the icon included in the image block is an icon included in the image block indicating that the document according to FIG. 4A is closed (see FIG. 4B), the control signal generator 54 sends the computer 1 the control signal based on the control signal. The control is the same as the execution of a command corresponding to the closing of the Microsoft PowerPoint document shown in FIG. 4b by a mouse electrically connected to the computer 1.

  According to the system 100 according to the method of the present invention, a standard sample image is obtained based on the captured image 40 and compared with the image frame, thereby performing a total conversion between the captured image 40 and the image frame as in the prior art. In addition, the computer 1 can be remotely controlled by the optical pointer 3. In particular, the closed loop light rail of the optical cursor obtained by the optical pointer can be remotely controlled to the computer for convenient use without moving the projector or image capture device such as a camera during use Workability can be improved.

  The method and system according to the present invention is useful for remote control of a computer with an optical pointer.

  This international application claims priority based on Taiwan Patent Application No. 96125394 filed on July 12, 2007, the entire contents of which are hereby incorporated by reference.

1 is a schematic diagram of a system that implements a method for remotely controlling a computer of a preferred embodiment according to the present invention. FIG. It is a flowchart which shows the remote control method of the computer of the Example of FIG. 2 shows an example of an image frame output by the computer of the embodiment of FIG. It shows that the image is selected by drawing a circle indicated by a broken line using a pointer in one image in the image frame of FIG. 3a. Fig. 3b shows the selected image opened in Fig. 3b. It shows that the image is selected by drawing a circle indicated by a broken line using a pointer in one image in the image frame of FIG. 4a.

Explanation of symbols

100 ... System 2 ... Projector 20 ... Image frame 3 ... Pointer
4 ... image capture device, 40 ... image, 5 ... control module,
51 ... Memory unit, 52 ... Sample selection unit, 53 ... Comparison unit,
54 ... Control signal generator

Claims (5)

A method of remotely controlling a computer that projects an image frame containing one or more icons on a remote screen, comprising:
Projecting an optical cursor generated by a handheld optical pointer onto the screen; a.
Capturing a series of images of the image frame and the optical cursor; b.
Detecting a position of the optical cursor in each of the captured images;
Obtaining a closed loop optical rail of the optical cursor based on the detected position of each optical cursor; d
Selecting an image portion of one captured image as a standard sample image in the captured image based on a closed loop light rail of the light cursor; and
Comparing the standard sample image with the image frame and selecting an image block from the image frame that includes at most one icon and is most similar to the standard sample image;
A method for remotely controlling a computer, comprising: generating a control signal for causing the computer to execute a command related to an icon included in the image block based on the image block. A system for remotely controlling a computer that outputs an image frame including one or more icons,
A projector electrically connected to the computer and projecting an image frame from the computer on a remote screen;
A handheld optical pointer that generates an optical cursor and projects the generated optical cursor onto the screen;
An image capture device for capturing a series of images of the projected image frame and the optical cursor;
A control module electrically connected to the computer for storing image frames received from the computer and electrically connected to the image capture device for storing captured images received from the image capture device; Including
The control module detects the position of the optical cursor in the captured image, draws a closed-loop optical rail of the optical cursor based on the detected position of the optical cursor, and creates a closed-loop optical rail of the optical cursor. The image portion of the one captured image is selected as a standard sample image, the standard sample image is compared with the image frame, and an image from an image frame that includes at most one icon and is most similar to the standard sample image. A system for remote control of a computer comprising: selecting a block; generating a control signal for executing a command associated with an icon included in the image block in response to the block; and sending the control signal to the computer A computer including one or more icons and outputting an image frame; a projector electrically connected to the computer and projecting the image frame on a remote screen; and a series of images of the projected image frame and a light cursor. As the control module electrically connected to the computer and the image capture device in a system having an image capture device to capture,
A memory unit for storing the image frame received from the computer and the image captured by the image capture device;
Coupled to the memory unit, detecting a position of the optical cursor in each captured image, and drawing a closed-loop optical rail of the optical cursor based on the detected position of the optical cursor; A sample selection unit for selecting an image portion of the captured image stored in the memory unit based on a light rail as a standard sample image;
The standard sample image is connected to the memory unit and the sample selection unit, compares the standard sample image selected by the sample selection unit with the image frame stored in the memory unit, and includes at most one icon and the standard sample A comparison unit for determining an image block from the image frame most similar to an image;
Control connected to the comparison unit so as to be electrically connected to the computer, and causing the computer to execute a command related to an icon included in the image block based on the image block defined by the comparison unit A control signal generator for generating a signal and sending it to the computer;
A control module comprising: A computer including one or more icons and delivering an image frame; a projector electrically connected to the computer for projecting the image frame on a remote screen; and a handheld optical for generating a light cursor and projecting the screen A method of generating a control signal for remotely controlling the computer in a system having an expression pointer and a projected image frame and an image capture device that captures a series of images of a light cursor comprising:
Detecting the position of the optical cursor in each of the captured images; and
Obtaining a closed-loop optical rail of the optical cursor based on the detected position of the optical cursor; and ii.
Defining an image portion of one captured image in the captured image as a standard sample image based on a closed loop light rail of the light cursor; and
Comparing the standard sample image with the image frame and selecting an image block from the image frame that is most similar to the standard sample image including at most one icon; and
Generating a control signal for causing the computer to execute a command related to an icon included in the image block based on the image block.   5. The method according to claim 4, wherein in the step v, the control signal cooperates with a control signal of a computer by a pointing device.

Images