EP2208112A2

EP2208112A2 - Apparatus and method for tracking a light pointer

Info

Publication number: EP2208112A2
Application number: EP08847393A
Authority: EP
Inventors: Jizhang Shan; Jin Li
Original assignee: Omnivision Technologies Inc
Current assignee: Omnivision Technologies Inc
Priority date: 2007-11-07
Filing date: 2008-10-23
Publication date: 2010-07-21
Also published as: EP2218252A1; WO2009061619A3; TWI400554B; CN101896867A; EP2218252A4; TW200933452A; WO2009061620A1; TW200935164A; TWI375898B; CN101896867B; EP2208112A4; CN101919241A; CN101919241B; WO2009061619A2

Abstract

A light pointer apparatus has a light source module for projecting a light beam onto a surface. The light pointer apparatus also has an image sensor module and a transmitter module. The image sensor module detects a position in the surface of a visible light spot generated by the projected light beam striking the surface. The transmitter module transmits the position of the visible light spot to a remote device for remote control of a device.

Description

-

APPARATUS AND METHOD FOR TRACKING A LIGHT POINTER

BRIEF DESCRIPTION OF THE INVENTION

[0001] This invention relates generally to tracking a light pointer. More particularly, this invention relates to an apparatus and method for tracking a light pointer position to emulate a remote control device.

BACKGROUND OF THE INVENTION

[0002] Light pointers are devices that project a light beam onto a surface area to highlight an item of interest. Their use has become commonplace at meetings, lectures, conferences, and other venues where presentation materials are shown on a display screen. The presentation materials are typically projected from a computer onto the display screen using a projection device connected to the computer, such as a liquid crystal projector. A presenter may use a light pointer at any time during the presentation to draw attention to particular locations or items of interest shown on the display screen with a light spot that is visible to the audience.

[0003] Most common light pointers are handheld, pen-shaped devices that hold a battery set, a control circuit, a light source module and a lens. The light source module includes a light source, e.g., a laser or Light Emitting Diode ("LED"), for emitting light at a given wavelength, for example wavelengths corresponding to red, green, or blue light. [0004] Light pointers may also include switches or buttons for enabling remote control of basic computer functions. For example, United States Patent Publication Number 2006/0197756 describes a light pointer with switches mounted thereon to enable remote initiation of various computer keyboard commands and/or pointing device (i.e., mouse, touch pad, track ball) operations, such as select, move, left click, right click and double click. The light pointer emits a light beam onto an image projected onto a display screen to produce a visible light spot superimposed on the image. The light spot functions as a control cursor for a presentation computer. A video camera in an image processor connected to the presentation computer is used to scan the image projected onto the screen. The image processor then analyzes the scanned image to identify the position of the control cursor (i.e., light spot) emitted by the light pointer. A user may direct the control cursor to a position in the screen and emulate position-dependent computer commands via the switches. [0005] Light pointers may also include controllers and wireless modules for transmitting control signals to the presentation computer. In United States Patent Publication -

Number 2006/0227107, for example, a light pointer is provided with a wireless operation module for transmitting control signals to a presentation computer indicating the pressing of a given switch mounted on the light pointer. One switch may be used to transmit a page-up signal to the computer and another may be used to transmit a page-down signal to the computer. The page-up and page-down signals indicate to the computer that a new page of presentation material is to be displayed and projected onto the screen.

[0006] Light pointers may also be integrated with other devices. For example, a light pointer may be integrated into a wireless mouse, as described in United States Patent No. 6,417,840. The integrated device is capable of selectively communicating control signals to a computer when operating as a mouse and transmitting a light beam to a projection screen when operating as a light pointer.

[0007] In addition, light pointers may be adapted to generate a modulated output that serves as a control signal for a projection system. In United States Patent Publication No. 2007/0030460, a light pointer has a modulated output that is used to provide commands to a projection system. The projection system has an image sensor integrated into it for detecting the position of light spots emitted by the light pointer. In one example, a user may press a button mounted on the light pointer to cause it to pulse at a given frequency. The pulsation indicates to the projection system that the light pointer position is to be monitored for a mouse "dragging" event until the user presses the button again to stop the pulsation. Different frequencies may be used to activate different mouse events.

[0008] These and other currently-available light pointers can only function as remote control devices with the use of mechanical buttons or switches. A user must manually operate the switches to control the presentation computer. This may be cumbersome and difficult to perform in practice as the user must change focus from giving the presentation to operating the switches. The switches may also be prone to error or fail entirely during a presentation.

[0009] Accordingly, it would be desirable to provide a light pointer apparatus that is capable of emulating a remote control device such as a wireless mouse without the use of manual switches. In particular, it would be desirable to provide a light pointer apparatus and method for tracking the position of a light pointer and using the position to control a presentation computer.

SUMMARY OF THE INVENTION

[0010] A light pointer apparatus has a light source module for projecting a light beam onto a surface. An image sensor module detects a position in the surface of a visible light - spot generated by the projected light beam striking the surface. A transmitter module transmits the position of the visible light spot to a remote device for remote control of a device.

[0011] An embodiment of the invention includes an image sensor apparatus for use in a light pointer projecting a light beam onto a surface. An image sensor module detects a position in the surface of a visible light spot generated by the projected light beam striking the surface. A transmitter module transmits the position of the visible light spot to a remote device for remote control of a device.

[0012] Another embodiment of the invention includes a method for emulating a remote control device with a light pointer. A light beam is projected onto a surface with the light pointer to generate a visible light spot on the surface. A position in the surface corresponding to the visible light spot is detected with an image sensor integrated into the light pointer. The position of the visible light spot is transmitted to a remote device. The remote device is controlled based on the position of the projected light beam.

BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The invention is more fully appreciated in connection with the following detailed description taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:

[0014] FIG. 1 illustrates a light pointer apparatus constructed according to an embodiment of the invention;

[0015] FIG. 2 illustrates a more detailed view of the light pointer apparatus of FIG. 1;

[0016] FIG. 3 illustrates a further detailed view of the light pointer apparatus of FIG. i;

[0017] FIG. 4 illustrates a flow chart for emulating a remote control device with a light pointer apparatus in accordance with an embodiment of the invention;

[0018] FIG. 5 illustrates an exemplary environment in which the light pointer apparatus operates in accordance with an embodiment of the invention;

[0019] FIG. 6 illustrates exemplary images detected by the light pointer apparatus in the exemplary environment of FIG. 5 in accordance with an embodiment of the invention;

[0020] FIG. 7 illustrates another exemplary environment in which the light pointer apparatus operates in accordance with an embodiment of the invention; -

[0021] FIG. 8 illustrates a remote device being controlled by the light pointer apparatus in the exemplary environment of FIG. 7 in accordance with an embodiment of the invention; and

[0022] FIG. 9 illustrates a further exemplary embodiment in which the light pointer apparatus operates in accordance with an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

[0023] A light pointer apparatus is provided. As generally used herein, a light pointer may be a device for projecting a light beam onto a surface area to highlight an item of interest. The surface area may be, for example, a display screen showing images projected thereon by a projector connected to a computer generating the images. The light beam may be projected as a visible light spot superimposing the images on the display screen. A light source module is included in the light pointer for emitting the light beam. [0024] The light pointer apparatus also includes an image sensor module having an image sensor and a processor. An image sensor, as generally used herein, may be a device or circuitry having an array of pixels for capturing and processing an optical image into electronic signals. The electronic signals may be processed by the processor to determine a position corresponding to the visible light spot emitted by the light pointer. [0025] A light pointer apparatus constructed according to an embodiment of the invention is illustrated in FIG. 1. Light pointer 100 has a light source module 105, an image sensor module 110, and a transmitter module 115. Light source module 105 includes a light source, e.g., a laser or Light Emitting Diode ("LED"), for emitting a light beam at a given wavelength, for example, wavelengths corresponding to red, green, or blue light. The light beam may be projected onto a surface, such as surface 120, to produce a visible light spot 125 on the surface 120. The visible light spot 125 can appear in different patterns and shapes, such as a circular block, the shape of a butterfly, the shape of a cursor, and so on. [0026] Light is reflected from surface 120 when light pointer 100, facing surface 120, projects a light beam onto it. The reflected light is sensed by image sensor module 110, which is positioned in light pointer 100 to capture images in the same direction as the light beam. Image sensor module 110 sees the surface 120 and the visible light spot 125 thereon and detects its position. The position of the light spot 125 may be detected automatically or computed by processing and analyzing the captured image.

[0027] The position of the light spot 125 is transmitted to remote device 130 by transmitter module 115. Transmitter module 115 may be any wireless transmitter capable of - interfacing with image sensor module 110, such as a WiFi or Bluetooth wireless transmitter. The position transmitted to remote device 130 may be a 2D (x,y) coordinate with a corner of surface 120 serving as the reference (0,0) point. Alternatively, the position may be a displacement between the current position and a previous position, with the initial position determined when light pointer 100 is first powered up.

[0028] A power ON/OFF button or switch (not shown) is included in light pointer

100 for powering it up. It is appreciated that the power switch may be the only switch included in light pointer 100. It is also appreciated that transmitter module 115 may also be used to transmit the image captured by the image sensor module 110 to remote device 130. [0029] Remote device 130 receives the position of the light spot from transmitter module 115 and uses the position to control one or more of its functions. It is appreciated that remote device 130 may have a receiver module (not shown) capable of receiving signals wirelessly from transmitter module 115. It is also appreciated that remote device 130 may include control routines for controlling the one or more of its functions based on the position of light spot 125.

[0030] In one exemplary embodiment, remote device 130 may be a computer generating images for display at surface 120, which may be a display screen for displaying images for an audience, e.g., during a presentation. Light pointer 100 may be used to send commands to computer 130 depending on the position of the light spot in the display screen. [0031] In this exemplary embodiment, remote computer 130 may have a calibration routine to scale the position on surface 120 to a position on the image displayed on surface 120. For example, the light spot 125 may be positioned to superimpose a region of the images to indicate to computer 130 that a new image or slide of the presentation is to be displayed, such as an icon on the images indicating an image or page number of the presentation. The position of the light spot 125 on surface 120 is determined with 2D coordinates at the scale of surface 120. Upon receiving the position of light spot 125, remote device 130 maps the received position into 2D coordinates at the scale of the image displayed on surface 120.

[0032] It is appreciated that remote computer 130 may have a pre-determined mapping from display screens into presentation images. It is also appreciated that remote computer 130 may determine the mapping upon projecting the images onto the display screen.

[0033] Alternatively, light pointer 100 may also be moved continuously across display screen 120 to indicate to computer 130 that a new image or slide is to be displayed. -

Positioning the light spot 125 on the icon or moving the light pointer 100 continuously has the same effect as clicking a button on a mouse associated with computer 130 to go to the next slide. In this exemplary embodiment, light pointer 100 effectively operates as a wireless mouse associated with computer 130.

[0034] In another exemplary embodiment, remote device 130 may be a toy or a game.

Light pointer 100 may be used as a remote control for the toy or game by positioning the light spot according to a desired command. For example, the toy 130 may be a remote control car and the light pointer 100 may be used to guide the remote control car to a given location. When the position of the light spot 125 is transmitted to the car by transmitter module 115, the car is directed to move to that position.

[0035] In yet another exemplary embodiment, remote device 130 may be an electronic appliance, for example, remote device 130 may be a vacuum cleaner. Light pointer 100 may be used as a remote control for the vacuum cleaner by positioning the light spot 125 on a surface to be cleaned. When the position of the light spot 125 is transmitted to the vacuum cleaner by transmitter module 115, the vacuum cleaner is directed to the surface to clean it.

[0036] In a further exemplary embodiment, remote device 130 may be a controller connected to a network of appliances. Light pointer 100 may be used as a remote control for the appliances by positioning the light spot 125 directly on a button of the appliance, e.g., the power button. When the image sensor module 110 detects that the light spot 125 is on the power button and its position is transmitted to controller 130, controller 130 may send a signal to the appliance to turn it ON or OFF.

[0037] It is appreciated that these exemplary embodiments are given for purposes of illustration only and additional exemplary embodiments are within the scope of the invention. It is also appreciated that remote device 130 may include control routines to control one or more of its functions based on the position of the light spot according to each exemplary embodiment. It is also appreciated that remote device 130 may use the position of light spot 125 to control another device connected to remote device 130, such as, for example, in the exemplary embodiment of the controller described above.

[0038] It is further appreciated that remote device 130 may include training and adaptation routines to interpret different positions of light spots generated by light pointer 100 as different commands. For example, remote device 130 may interpret a position on the upper left corner of surface 120 to correspond to a page down command, a position on the - lower left corner of surface 120 to correspond to a page up command, a position on the upper right corner of surface 120 to correspond to an animation command, and so on. [0039] FIG. 2 illustrates a more detailed view of a light pointer apparatus constructed according to an embodiment of the invention. Image sensor module 205 in light pointer 200 includes image sensor 210 and processor 215. Image sensor 210 and processor 215 as shown may be mounted on separate chips. Image sensor 210 may be a CMOS or CCD image sensor, or any other type of image sensor capable of capturing optical images. Processor 215 may be a Digital Signal Processor ("DSP"), an Image Signal Processor ("ISP"), or any other kind of processor capable of interfacing with image sensor 210.

[0040] In one exemplary embodiment, image sensor 210 captures an image of the surface 120 with light spot 125 and transmits the image to processor 215 for analysis and processing. Processor 215 may then execute image processing routines to detect and locate the light spot 125 on the captured image. As described above, the 2D coordinates or displacement of the light spot 125 on surface 120 can be computed by processor 215 for use in remote device 130 for remote control of remote device 130.

[0041] Another detailed view of a light pointer apparatus constructed according to an embodiment of the invention is illustrated in FIG. 3. Image sensor module 305 in light pointer 300 includes integrated image sensor/processor chip 310. Integrated chip 310 has both an image sensor and a processor on the same chip. The image sensor may be similar to image sensor 210 and the processor may be similar to processor 215, both shown in FIG. 2. In this exemplary embodiment, a single integrated chip 310 is used to both capture an image showing the light spot 125 and to detect the position of the light spot 125 in the captured image.

[0042] Referring now to FIG. 4, a flow chart for emulating a remote control device with a light pointer apparatus in accordance with an embodiment of the invention is described. A light pointer apparatus may be used to project a light beam onto a surface and generate a light spot visible on the surface in step 400. The position of the light spot may be detected by an image sensor module in the light pointer in step 405 and transmitted to a remote device by a transmitter module in the light pointer in step 410. Optionally, depending on the remote device, the position of the light spot in the surface may be mapped into a position at the scale of the remote device in step 415. The position of the light spot is then used to control the remote device as described above with reference to FIG. 1 in step 420. [0043] An exemplary environment in which the light pointer apparatus operates in accordance with an embodiment of the invention is illustrated in FIG. 5. Light pointer 500 - may be used to project a light beam into light spot 505 visible on surface 510. Surface 510 may be a display screen to display images generated by remote computer 515 during a presentation. A projector 520 connected to remote computer 515 may be used to project the images onto display screen 510.

[0044] According to an embodiment of the invention, an image sensor module in light pointer 500 sees the image displayed on display screen 510 and the light spot 505 superimposed on the image. A processor in the image sensor module in light pointer 500 processes the captured image to detect and locate the position of light spot 505. As described above, the position may be a 2D coordinate or a displacement in the display screen 510. [0045] The position of light spot 505 is wirelessly transmitted to remote computer

515 by a transmitter module in light pointer 500. The remote computer 515 may use the position to control one or more of its functions. For example, the remote computer 515 may use the position of light spot 505 to direct an operation to be performed on the presentation, e.g., to go to a previous or next image of the presentation, to start an animation, to superimpose another image on top of the current slide, and so on.

[0046] It is appreciated that remote computer 515 may have a calibration routine 525 to map the position of light spot 505 on display screen 510 into a position at the scale of the image displayed on display screen 510. For example, a 2D coordinate (xi,yi) in the display screen 510 may correspond to a 2D coordinate (x₂,y₂) in the image being displayed. It is also appreciated that remote computer 515 may have a training and adaptation routine 530 to interpret different positions of light spots generated by light pointer 500 as different commands as described above.

[0047] FIG. 6 shows exemplary images captured and processed by light pointer 500 shown in FIG. 5. Image 600 shows the optical image that the image sensor in the image sensor module of light pointer 500 sees when light pointer 500 is projecting light spot 505 onto display screen 510. The light spot 505 may be seen by the image sensor as a region 605, which may be a colorful region in the screen that appears different from its neighborhood. For example, light spot 505 may appear as a monochromatic (e.g., red, green, or blue), typically nearly round region 605 at a given diameter and luminance. A processor in the image sensor module of light pointer 500 processes the image 600 to detect and locate a 2D coordinate or displacement for the light spot 505. The processed image 610 may include a marking such as cross 615 to indicate the position of the light spot 505. [0048] It is appreciated that the processor in the image sensor module of light pointer

500 may include image processing routines for identifying region 605 as light spot 505 and - for computing the 2D coordinate or displacement of the light spot 505. These routines may include, for example, object and pattern recognition routines, edge detection routines, and other computer vision and image processing routines for differentiating the region 605 as light spot 505 from the rest of the image and locating its position.

[0049] Another exemplary embodiment in which the light pointer apparatus operates in accordance with an embodiment of the invention is shown in FIG. 7. Light pointer 700 may be used as a remote control for remote computer 720. Light pointer 700 may be used to project a light beam into light spot 705 that, for example, superimposes icon 710 on the image displayed on display screen 715 during a presentation. Projector 725 connected to remote computer 720 may be used to project images from the presentation into display screen

715.

[0050] An image sensor module in light pointer 700 sees the image displayed on display screen 715 with the light spot 705 superimposed onto icon 710 and detects the position of light spot 705. A transmitter module in light pointer 700 transmits the position to remote computer 720. When remote computer 720 receives the position of light spot 705 and determines with a calibration routine that the light spot 705 is within icon 710, remote computer 720 automatically displays the next image of the presentation for projection onto display screen 715 by projector 725.

[0051] FIG. 8 illustrates the response of remote computer 720 when controlled by light pointer 700 shown in FIG. 7. Upon receiving the position of light spot 705 on icon 710 indicating that a new image is to be displayed for presentation onto display screen 715, remote computer 720 advances to the next image of the presentation. The next image is then projected onto display screen 715 by projector 725 as image 800.

[0052] Image 800 includes page icon 805 indicating the new page number 2 of the presentation displayed onto display screen 715. A presenter may then use light pointer 700 to highlight items of interest on image 800, such as by pointing light pointer 700 to display screen 715 to generate light spot 810.

[0053] It is appreciated that light pointer 700 operates as a wireless mouse for remote computer 720. It is also appreciated that remote computer 720 may include routines for interpreting the position of light spots generated by light pointer 700 to correspond to different mouse operations, such as select, right click, left click, drag, and so on.

[0054] Another exemplary embodiment in which the light pointer apparatus operates in accordance with an embodiment of the invention is shown in FIG. 9. In this exemplary - embodiment, light pointer 900 is used as a remote control for remote control toy car 905. Remote control car 905 is directed to move to different positions by light pointer 900. [0055] For example, light pointer 900 may be used to project a light beam into light spot 910. The position of light spot 910 is detected by the image sensor module in light pointer 900 and transmitted to remote control car 905 by the transmitter module in light pointer 900. Remote control car 905 is then directed to move to the position corresponding to light spot 910.

[0056] It is appreciated that remote control car 905 is shown for illustration purposes only. Other remote devices may be controlled by the light pointer apparatus without deviating from the scope and principles of the invention.

[0057] Advantageously, the light pointer apparatus enables a user to control a device remotely by simply pointing the light pointer apparatus to a desired position. This enables the light pointer apparatus to operate as both a light pointer and a wireless mouse without the use of any buttons or switches to trigger different mouse commands.

[0058] The foregoing description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to one skilled in the art that specific details are not required in order to practice the invention. Thus, the foregoing descriptions of specific embodiments of the invention are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed; obviously, many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications; they thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the following claims and their equivalents define the scope of the invention.

Claims

-WHAT IS CLAIMED IS:

1. A light pointer apparatus, comprising: a light source module for projecting a light beam onto a surface; an image sensor module for detecting a position in the surface of a visible light spot generated by the projected light beam striking the surface; and a transmitter module for transmitting the position of the visible light spot on the surface to a remote device for remote control of a device.

2. The light pointer apparatus of claim 1, wherein the image sensor module comprises an image sensor and a processor.

3. The light pointer apparatus of claim 2, wherein the position is determined by the processor.

4. The light pointer apparatus of claim 2, wherein the image sensor and the processor are mounted on separate chips.

5. The light pointer apparatus of claim 2, wherein the image sensor and the processor are integrated into a single chip.

6. The light pointer apparatus of claim 1, wherein the light source module comprises a light source selected from the list comprising: a laser; and a light emitting diode.

7. The light pointer apparatus of claim 1, wherein the position comprises a displacement position.

8. The light pointer apparatus of claim 7, wherein the displacement position is determined with reference to a reference point in the surface.

9. The light pointer apparatus of claim 7, wherein the displacement position is determined with reference to a previous position of the projected light beam. -

10. The light pointer apparatus of claim 1, wherein the surface comprises a display screen.

11. The light pointer apparatus of claim 10, wherein the remote device comprises a computer generating images for display onto the display screen.

12. The light pointer apparatus of claim 11, wherein the computer comprises a calibration routine and a training routine.

13. An image sensor apparatus for use in a light pointer projecting a light beam onto a surface, comprising: an image sensor module for detecting a position in the surface of a visible light spot generated by the projected light beam striking the surface; and a transmitter module for transmitting the position of the visible light spot to a remote device for remote control of a device.

14. The image sensor apparatus of claim 13, wherein the surface comprises a display screen.

15. The image sensor apparatus of claim 14, wherein the light beam is projected to superimpose an image projected onto the display screen.

16. The image sensor apparatus of claim 13, wherein the image sensor module comprises an image sensor and a processor.

17. The image sensor apparatus of claim 16, wherein the processor is configured to determine the position of the projected light beam.

18. The image sensor apparatus of claim 16, wherein the image sensor and the processor are mounted on separate chips.

19. The image sensor apparatus of claim 16, wherein the image sensor and the processor are integrated in a single chip. -

20. The image sensor apparatus of claim 13, wherein the transmitter module comprises a wireless transmitter.

21. A method for emulating a remote control device with a light pointer, comprising: projecting a light beam onto a surface with the light pointer to generate a visible light spot on the surface; detecting a position in the surface corresponding to the visible light spot with an image sensor integrated into the light pointer; transmitting the position of the visible light spot to a remote device; and controlling the remote device based on the position of the visible light spot.

22. The method of claim 21 , wherein detecting a position in the surface comprises detecting a two-dimensional coordinate and a displacement position.

23. The method of claim 21, further comprising displaying an image onto the surface.

24. The method of claim 23, further comprising mapping the position of the visible light spot to a position in the image.

25. The method of claim 23, wherein controlling the remote device comprises controlling the display of the image onto the surface.