WO2010007566A1

WO2010007566A1 - Camera device and screen device

Info

Publication number: WO2010007566A1
Application number: PCT/IB2009/053006
Authority: WO
Inventors: Galileo J. Destura; Jan Kneissler; Jose A. Liso; Dirk T. G. Castelijns; Henri A. M. Roosen; Tom Limerkens
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2008-07-18
Filing date: 2009-07-10
Publication date: 2010-01-21

Abstract

Camera devices (1) for interaction with screen devices (2) comprise cameras (10) for capturing images of the screen devices (2) and detectors (11) for detecting light sources (21-24) of the screen devices (2) in said images and derivors (12) for, in different situations, deriving position information from different pairs of light sources (21-22) (23-24). Some distances between different pairs of light sources (21-22) (23-24) are different. At different ranges, the camera devices (1) can distinguish different pairs of light sources (21- 22) (23-24). Control signals allow control of the light sources (21-24). Interaction signals allow determination of pointing positions (35) at screens (20) of the screen devices (2). The screen devices (2) comprise the screens (20) for displaying video and the different pairs of light sources (21-22) (23-24) for providing position indications to the camera devices (1). The light sources (21-24) may each comprise two or more light emitting diodes and one or more tilts and individualizations.

Description

Camera device and screen device

FIELD OF THE INVENTION

The invention relates to a camera device for interaction with a screen device. The invention also relates to a screen device for interaction with a camera device, and relates to a system and to a method. Examples of such a camera device are pointers comprising cameras, and examples of such a screen device are displays and monitors.

BACKGROUND OF THE INVENTION

US 2007 / 0080940 discloses a remote control system and a display device. The remote control system includes a remote control transmitter with a light-emitting element that emits position detection light signals, a remote control receiver with a light-reception signal processing unit that detects position detection reception signals from the position detection light signals, an output detection unit that obtains output signals as position detection output signals, and an arithmetic processing unit that calculates a displacement of the remote control transmitter in a first and second axis direction based on an amplitude correlation of the output signals, and a cursor control unit that controls a position of a cursor based on the displacement of the remote control transmitter in the first and second axis direction. As described in paragraph 0501, the display device detects a distance between the display device and the control device and transmits distance information to the control device, to allow the control device to make a correction and an adjustment.

SUMMARY OF THE INVENTION

It is an object of the invention to provide a camera device that does not require the screen device to detect a distance between the screen device and the camera device and to transmit distance information to the camera device.

Further objects of the invention are to provide a screen device that does not need to detect a distance between the screen device and the camera device and to transmit distance information to the camera device, and to provide a system and a method.

According to a first aspect of the invention, a camera device for interaction with a screen device is provided, the camera device comprising: a camera for capturing an image of at least a part of the screen device, a detector for detecting light sources of the screen device in said image, and a derivor for, in a first situation, deriving position information from a first pair of light sources and not deriving position information from at least a part of a second pair of light sources and for, in a second situation, not deriving position information from at least a part of the first pair of light sources and deriving position information from the second pair of light sources, at least one of the first pair of light sources being different from at least one of the second pair of light sources, and a distance between the first pair of light sources being different from a distance between the second pair of light sources. A camera device is introduced comprising a camera for capturing an image of at least a part of a screen device. This screen device comprises a first pair with first and second light sources and comprises a second pair with third and fourth light sources, whereby at least one of the first and second light sources has to be different from at least one of the third and fourth light sources. So, in a minimum case, the screen device comprises three light sources, and in other cases the screen device comprises four or more light sources. A real distance between the first and second light sources has to be different from a real distance between the third and fourth light sources. As a result, a virtual distance between the first and second light sources in said image will be different from a virtual distance between the third and fourth light sources in said image. By providing the camera device with a detector for detecting light sources of the screen device in said image and with a derivor for, in a first situation (that for example corresponds with the camera device and the screen device being located relatively closely to each other), deriving position information from the first pair of light sources (that for example are located at a relatively small distance from each other) and not deriving position information from at least a part of the second pair of light sources (that for example are located at a relatively large distance from each other) and for, in a second situation (that for example corresponds with the camera device and the screen device being located relatively far away from each other), not deriving position information from at least a part of the first pair of light sources (that for example are located at a relatively small distance from each other) and deriving position information from the second pair of light sources (that for example are located at a relatively large distance from each other), it is no longer necessary to transmit distance information from the screen device to the camera device.

When the camera device and the screen device are located relatively closely to each other, the camera device will be able to distinguish the first pair of light sources that are located at a relatively small distance from each other and the camera device does not need and/or cannot see the second pair of light sources that are located at a relatively large distance from each other. When the camera device and the screen device are located relatively far away from each other, the camera device will no longer be able to distinguish the first pair of light sources that are located at a relatively small distance from each other but the camera device will be able to see and distinguish the second pair of light sources that are located at a relatively large distance from each other.

In case of the screen device comprising four light sources, the first and second light sources may for example be located a little bit left and a little bit right from a vertical axis going through the center of a screen of the screen device, for example at the top or at the bottom or in the middle of the screen. The third and fourth light sources may then for example be located at two different corners of the screen. In case of the screen device comprising three light sources, the first light source may for example be located at a vertical axis going through the center of the screen, for example at the top or at the bottom or in the middle of the screen. The third and fourth light sources may then for example be located at the left side and the right side of the screen, for example at a horizontal axis going through the first light source, whereby one of the third and fourth light sources forms part of both pairs etc. Other and/or further locations are not to be excluded.

As a camera, any kind of camera can be used, such as a relatively low- resolution camera and a relatively high-resolution camera and any kind of light detection circuit or light detection mechanism. As a detector, any kind of detecting circuit or detecting mechanism can be used. As a derivor, any kind of deriving circuit or deriving mechanism can be used. A pair of light sources does not exclude more than two light sources per pair.

According to an embodiment, the camera device is defined by further comprising: a control signal transmitter for transmitting a control signal to the screen device for controlling one or more of the light sources.

Such controlling may comprise switching a light source on/off and may comprise changing a brightness, an intensity, a color, a modulation, a filter or another kind of individualization of the light source.

According to an embodiment, the camera device is defined by said control signal comprising detected information from the detector or derived information from the derivor or control information calculated from the detected information or the derived information. In case of the control signal comprising the detected information or the derived information, a calculation is to be performed in the screen device. In case the calculation is performed in the camera device, the control signal will comprise the calculated control information.

According to an embodiment, the camera device is defined by further comprising: an interaction signal transmitter for transmitting an interaction signal to the screen device for determining a pointing position at a screen of the screen device.

The interaction signal indicates to which pointing position at the screen the camera device is pointing. According to an embodiment, the camera device is defined by the interaction signal comprising the position information or comprising pointing information calculated from the position information. In case of the interaction signal comprising the position information, a calculation is to be performed in the screen device. In case the calculation is performed in the camera device, the interaction signal will comprise the calculated pointing information.

According to a second aspect of the invention, a screen device for interaction with a camera device is provided, the screen device comprising: a screen for displaying video, and first and second pairs of light sources for providing position indications to the camera device, at least one of the first pair of light sources being different from at least one of the second pair of light sources, and a distance between the first pair of light sources being different from a distance between the second pair of light sources.

According to an embodiment, the screen device is defined by each light source comprising two or more light emitting diodes, at least some of the two or more light emitting diodes per light source having mutually tilted positions and/or at least some of the light sources having mutually tilted positions and/or at least some of the two or more light emitting diodes per light source radiating individualized light and/or at least some of the light sources radiating individualized light. Two different and distinguishable light sources are required to determine a pointing position at the screen. Three different and distinguishable light sources are required to determine a location of the camera device.

According to an embodiment, the screen device is defined by further comprising: a control signal receiver for receiving a control signal from the camera device for controlling one or more of the light sources. According to an embodiment, the screen device is defined by said control signal comprising detected information or derived information or control information calculated from the detected information or the derived information.

According to an embodiment, the screen device is defined by further comprising: an interaction signal receiver for receiving an interaction signal from the camera device for determining a pointing position at the screen.

According to an embodiment, the screen device is defined by the interaction signal comprising position information or comprising pointing information calculated from the position information.

According to a third aspect of the invention, a system is provided, and according to a fourth aspect, a method is provided. Embodiments of the system and of the method correspond with the embodiments of the camera device and the screen device.

An insight might be that a camera device should not require a screen device to detect a distance between the screen device and the camera device and to transmit distance information to the camera device.

A basic idea might be that two or more pairs of light sources at a screen device should provide position indications to a camera device and that the camera device should derive position information from one of the pairs or from another one of the pairs depending on the situation.

A problem to provide a camera device that does not require the screen device to detect a distance between the screen device and the camera device and to transmit distance information to the camera device and a problem to provide a screen device that does not need to detect a distance between the screen device and the camera device and to transmit distance information to the camera device are solved.

An advantage might be that inexpensive light sources are used to replace an expensive distance detecting and transmitting circuit in the screen device.

These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiment(s) described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings:

Fig. 1 shows a system comprising a camera device and a screen device that are located relatively closely to each other, Fig. 2 shows a system comprising a camera device and a screen device that are located relatively far away from each other,

Fig. 3 shows a camera device and a screen with a pointing position and position indications, and Fig. 4 shows geometrical relationships of a camera device and a screen device.

DETAILED DESCRIPTION OF EMBODIMENTS

In the Fig. 1, a system 3 is shown comprising a camera device 1 and a screen device 2 that are located relatively closely to each other, at for example 1 meter or 2 meters from each other. The camera device 1 comprises a camera 10 for capturing an image of at least a part of the screen device 2, a detector 11 connected to the camera 10 for detecting light sources 21-24 of the screen device 2 in said image, and a derivor 12 connected to the detector 11 for deriving position information from (the detections of) these light sources 21- 24. The derivor 12 is further connected to a controller 17. The controller 17 is connected to a control signal transmitter 15 for transmitting a control signal to the screen device 2 and is connected to an interaction signal transmitter 16 for transmitting an interaction signal to the screen device 2 and is connected to a calculator 18.

The screen device 2 comprises a screen 20 for displaying video connected to a controller 27. The controller 27 is further connected to the light sources 21-24 and to a control signal receiver 25 for receiving the control signal from the camera device 1 and to an interaction signal receiver 26 for receiving the interaction signal from the camera device 1 and to a calculator 28.

Further shown in the Fig. 1 are a camera viewing area 4 that only covers the light sources 21-22, a control signal link 5 and an interaction signal link 6. Said links 5-6 may be one and the same link or different links and may be realized through any kind of wired or wireless link.

In the Fig. 2, a system 3 is shown comprising a camera device 1 and a screen device 2 that are located relatively far away from each other, at for example 5 meters or 10 meters from each other. As a result, contrary to the system 3 shown in the Fig. 1 , a camera viewing area 4 is here covering each one of the light sources 21-24.

In the Fig. 3, a camera device 1 is shown pointing to a screen 20 with a pointing position 35 and position indications 31-34. A first (second, third, fourth) light source 21 (22, 23, 24) provides a first (second, third, fourth) position indication 31 (32, 33, 34). In the Fig. 4, geometrical relationships of a camera device 1 and a screen device 2 are shown. A width 41 and a depth 42 of a camera viewing area define this viewing area together with an opening angle 43. At the screen device 2, two light sources are located at a real distance 44 from each other, and as a result in a captured image at the camera device 1 , a virtual distance 45 between (detections of) the two light sources is present.

In a first situation, as shown in the Fig. 1 , when the camera device 1 and the screen device 2 are relatively close to each other, the camera viewing area 4 only sees the first pair of light sources 21-22 and the detector 11 can only detect the first pair of light sources 21-22 and the derivor 12 can only derive position information from the first pair of light sources 21-22 and cannot derive position information from the second pair of light sources 23-24.

In a second situation, as shown in the Fig. 2, when the camera device 1 and the screen device 2 are relatively far away from each other, the camera viewing area 4 sees all four light sources 21-24. However, owing to the fact that a distance between the first pair of light sources 21-22 is smaller than a distance between the second pair of light sources 23-24, the detector 11 cannot detect the first and second light sources 21-22 separately from each other and/or the detector 11 cannot distinguish the first and second light sources 21-22 from each other. As a result, the derivor 12 can only derive position information from the second pair of light sources 23-24 and cannot derive position information from each one of the first and second light sources 21-22.

According to a first option, the control signal is used for controlling one or more of the light sources 21-24. For example in the first situation, the camera device 1 may command or inform the screen device 2 to switch off the second pair of light sources 23-24 or to reduce their brightness etc. For example in the second situation, the camera device 1 may command or inform the screen device 2 to switch off the first pair of light sources 21-22 or to reduce their brightness etc. Thereto, said control signal comprises detected information from the detector 11 or derived information from the derivor 12 for informing the screen device 2 (such that the calculator 28 can make a calculation) or comprises control information calculated from the detected information or the derived information via the calculator 18 for commanding the screen device 2.

According to a second option, the interaction signal is used for determining the pointing position 35 at the screen 20. Thereto, said interaction signal comprises the position information for informing the screen device 2 (such that the calculator 28 can make a calculation) or comprises pointing information calculated from the position information via the calculator 18 for commanding the screen device 2.

According to a third option, each light source 21-24 may comprise two or more light emitting diodes, at least some of the two or more light emitting diodes per light source 21-24 having mutually tilted positions and/or at least some of the light sources 21-24 having mutually tilted positions and/or at least some of the two or more light emitting diodes per light source 21-24 radiating individualized light and/or at least some of the light sources 21-24 radiating individualized light. Especially (inorganic and organic and laser) light emitting diodes can be controlled well. A control may comprise switching on/off and may comprise changing a brightness / intensity / color / modulation / filter or another kind of individualization.

The camera device 1 may further comprise audio and/or video and/or interface components. The screen device 2 may further comprise audio and/or further video and/or interface components. Each device may have several modes each corresponding with one of the situations, whereby for example one of the modes is activated automatically after being switched on from standby and whereby an other mode is activated when necessary, automatically after some time or in response to a user action or in response to an internal detection or in response to an external message etc.

In other words, the camera device or pointer device (uWand) consists of a camera for determining the pointed position. Light sources such as for example LED clusters embedded in for example the frame of the screen device or TV are viewed by the camera embedded in the pointer device. A pointed position is calculated based on the relative position of the LED clusters in the camera's field of view. Thereto, two or more LED clusters need to be clearly separated in the camera's field of view so that other information needed by the TV application, such as a tilt of the pointer device and a distance of the pointer device to the TV, can be determined.

The pointer device is envisaged to be used on operating a large area TV as this will allow users to operate the TV in a more user friendly way and to access more features in a simple manner. When a user is at a close distance to the TV, certain LED clusters are distinguishable from each other in the camera's field of view. When the user is at the extremes of the operational area, neighboring LED clusters are merged in the camera's field of view. Thus, at the extremes of the operational area, information needed for interaction with a TV content is no longer present. The camera of the pointer device has a field of view of θ degrees (indicated by the angle 43 in the Fig. 4) and is at a distance y (indicated by 42 in the Fig. 4) from the LED clusters of the TV. L is the distance (indicated by 44 in the Fig. 4) between the two LED clusters. The camera observes an equivalent of x dimension (indicated by 41 in the Fig. 4) along the TV's plane. The two LED clusters are imaged at a distance d (indicated by 45 in the Fig. 4) on the camera plane. For the two LED clusters not to merge as captured by the camera image, the following relationships hold: Eq. 1 : L < 2 * y * tan θ

Eq. 2 : L > 2 * y * tan θ * C with C defining a camera property, C < 1, for d ≠ 0 (the two LED clusters are still observed by the camera).

The indication that the LED clusters' distance should not be outside the camera's field of view and that the LED clusters should have a minimum distance from each other is described in Eq. 1 and Eq. 2, respectively. At a given distance y = y_max from the TV, the LED clusters, as imaged by the camera, are about to merge. Beyond y_max the two LED clusters are not distinguishable from the image captured by the camera. Thus, for a given constant camera property, L has to change depending on where the user is standing within the operational area.

The above implies that at a close distance y < y_max to the screen, more LED clusters can be switched on and at least two of the switched on LED clusters can be used. At - y > y_max neighboring switched on LED clusters will merge in the image captured by the camera and as such only LED clusters that satisfy Eq. 2 will be switched on and only LED clusters satisfying Eq. 1 will be used to determine the pointed position and other needed information. So, a method is employed of using LED clusters defined in a geometrical manner. Depending on where the user is located within the operating area, LED clusters are switched that satisfy Eq. 2. The new set is further apart as compared to the previous set of LED cluster used when y < y_max. The solution provides a non-mechanical and inexpensive way of changing the L without a need to change the camera's field of view (i.e. mechanical zooming) and/or increasing a camera resolution (cost implication given the increased silicon area for detection and processing power needed).

An additional feature of the method may be that if more than two LED clusters are used at a given time, the true location of the user within the operational area can be determined. This information gives applications additional information in order to enrich and experience delivery, such as a directional sound production. The tilt of the pointer device can be determined with a minimum presence of two LED clusters. The set of LED clusters that are in use to determine the pointed position can be dynamically changed to make a user interaction angle effectively larger than the camera's physical field of view. Summarizing, camera devices 1 for interaction with screen devices 2 comprise cameras 10 for capturing images of the screen devices 2 and detectors 11 for detecting light sources 21-24 of the screen devices 2 in said images and derivors 12 for, in different situations, deriving position information from different pairs of light sources 21-22, 23-24. Some distances between different pairs of light sources 21-22, 23-24 are different. At different ranges, the camera devices 1 can distinguish different pairs of light sources 21-22, 23-24. Control signals allow control of the light sources 21-24. Interaction signals allow determination of pointing positions 35 at screens 20 of the screen devices 2. The screen devices 2 comprise the screens 20 for displaying video and the different pairs of light sources 21-22, 23-24 for providing position indications to the camera devices 1. The light sources 21- 24 may each comprise two or more light emitting diodes and one or more tilts and individualizations .

While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. For example, it is possible to operate the invention in an embodiment wherein different parts of the different disclosed embodiments are combined into a new embodiment.

Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measured cannot be used to advantage. A computer program may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:

1. A camera device (1) for interaction with a screen device (2), the camera device

(1) comprising: a camera (10) for capturing an image of at least a part of the screen device (2), a detector (11) for detecting light sources (21-24) of the screen device (2) in said image, and a derivor (12) for, in a first situation, deriving position information from a first pair of light sources (21-22) and not deriving position information from at least a part of a second pair of light sources (23-24) and for, in a second situation, not deriving position information from at least a part of the first pair of light sources (21-22) and deriving position information from the second pair of light sources (23-24), at least one of the first pair of light sources (21-22) being different from at least one of the second pair of light sources (23-24), and a distance between the first pair of light sources (21-22) being different from a distance between the second pair of light sources (23-24).

2. The camera device (1) as claimed in claim 1, further comprising: a control signal transmitter (15) for transmitting a control signal to the screen device (2) for controlling one or more of the light sources (21-24).

3. The camera device (1) as claimed in claim 2, said control signal comprising detected information from the detector (11) or derived information from the derivor (12) or control information calculated from the detected information or the derived information.

4. The camera device (1) as claimed in claim 1, further comprising an interaction signal transmitter (16) for transmitting an interaction signal to the screen device (2) for determining a pointing position (35) at a screen (20) of the screen device (2).

5. The camera device (1) as claimed in claim 4, the interaction signal comprising the position information or comprising pointing information calculated from the position information.

6. A screen device (2) for interaction with a camera device (1), the screen device

(2) comprising: a screen (20) for displaying video, and first and second pairs of light sources (21-24) for providing position indications (31-34) to the camera device (1), at least one of the first pair of light sources (21- 22) being different from at least one of the second pair of light sources (23-24), and a distance between the first pair of light sources (21-22) being different from a distance between the second pair of light sources (23-24).

7. The screen device (2) as claimed in claim 6, each light source (21-24) comprising two or more light emitting diodes, at least some of the two or more light emitting diodes per light source (21-24) having mutually tilted positions and/or at least some of the light sources (21-24) having mutually tilted positions and/or at least some of the two or more light emitting diodes per light source (21-24) radiating individualized light and/or at least some of the light sources (21-24) radiating individualized light.

8. The screen device (2) as claimed in claim 6, further comprising: a control signal receiver (25) for receiving a control signal from the camera device (1) for controlling one or more of the light sources (21-24).

9. The screen device (2) as claimed in claim 8, said control signal comprising detected information or derived information or control information calculated from the detected information or the derived information.

10. The screen device (2) as claimed in claim 6, further comprising: - an interaction signal receiver (26) for receiving an interaction signal from the camera device (1) for determining a pointing position (35) at the screen (20).

11. The screen device (2) as claimed in claim 10, the interaction signal comprising: position information or comprising pointing information calculated from the position information.

12. A system (3) comprising the camera device (1) as claimed in claim 1 and/or comprising the screen device (2) as claimed in claim 6.

13. A method for interaction between a camera device (1) and a screen device (2), the method comprising the steps of: via first and second pairs of light sources (21-24), providing position indications (31-34) from the screen device (2) to the camera device (1), at least one of the first pair of light sources (21-22) being different from at least one of the second pair of light sources (23-24), and a distance between the first pair of light sources (21-22) being different from a distance between the second pair of light sources (23-24), via the camera device (1), capturing an image of at least a part of the screen device (2) and detecting light sources (21-24) of the screen device (2) in said image and, in a first situation, deriving position information from the first pair of light sources (21-22) and not deriving position information from at least a part of the second pair of light sources (23- 24) and for, in a second situation, not deriving position information from at least a part of the first pair of light sources (21-22) and deriving position information from the second pair of light sources (23-24).