US20070147828A1 - Image element identifier - Google Patents
Image element identifier Download PDFInfo
- Publication number
- US20070147828A1 US20070147828A1 US11/438,088 US43808806A US2007147828A1 US 20070147828 A1 US20070147828 A1 US 20070147828A1 US 43808806 A US43808806 A US 43808806A US 2007147828 A1 US2007147828 A1 US 2007147828A1
- Authority
- US
- United States
- Prior art keywords
- image information
- communication device
- portable communication
- presented
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/20—Input arrangements for video game devices
- A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
- A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
- A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
- A63F13/426—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle involving on-screen location information, e.g. screen coordinates of an area at which the player is aiming with a light gun
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/53—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
- A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/53—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
- A63F13/537—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game using indicators, e.g. showing the condition of a game character on screen
- A63F13/5372—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game using indicators, e.g. showing the condition of a game character on screen for tagging characters, objects or locations in the game scene, e.g. displaying a circle under the character controlled by the player
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
- A63F2300/1087—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/30—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device
- A63F2300/303—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device for displaying additional data, e.g. simulating a Head Up Display
- A63F2300/306—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device for displaying additional data, e.g. simulating a Head Up Display for displaying a marker associated to an object or location in the game field
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/40—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterised by details of platform network
- A63F2300/406—Transmission via wireless network, e.g. pager or GSM
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/60—Methods for processing data by generating or executing the game program
- A63F2300/6045—Methods for processing data by generating or executing the game program for mapping control signals received from the input arrangement into game commands
Definitions
- the invention relates in general to controlling presentation of image information, and in particular to providing positioning information related to the presentation of image information.
- Portable communication devices such as mobile phones, may be found having memories of 0.5 Gb or more.
- Gaming using large screens is well known and was performed in the early days of electronic games, such as in television games using TV-sets as screens or in personal computer games using the computer cathode ray tubes as screens.
- the television games typically provided a photo sensitive gun capable of sensing large bright target symbols on the screen of a television receiver.
- the targets were distinctly displayed on the screen when imaged using a gun-mounted electro-optical sensor resulting in a series of output pulses indicating that the gun was indeed lined up properly with the target and a hit was scored. None of these attempted to show where the user was aiming when he pulled the trigger preceding a miss or for that matter, how closely to the center of the target on screen he was aiming when he made a hit.
- a television precision target shooting apparatus and method simulate shooting at displayed images on the screen of a television set using a simulated gun that provides indication on the screen of where the gun was pointed at the time of trigger pull. This is achieved by flashing the screen white subsequent a trigger pull in order for a photo sensor within the gun to respond to short segments of horizontal raster lines such that by counting vertical lines and time intervals with respect to vertical and horizontal synchronization pulses, respectively, identification of where on the screen the gun was pointing is achieved. This permits a symbol to be displayed upon a screen at that location.
- a gyro is used which is built in the directing device and which reacts to lateral and longitudinal motions of the directing device. Any lateral and longitudinal motions causing lateral and longitudinal displacements are related to a reference position, by which an updated position can be calculated.
- the application of such a directing device in gaming may allow the position of a cursor or symbol to be determined and displayed on a screen.
- An alternative to using a gyro to determine an updated position of a cursor or symbol on a screen is to use an accelerometer included in the directing or pointing device. Any motions of the directing or pointing device are detected by the accelerometer, and then converted to a displacement of the location where the directing or pointing device is pointing. A cursor may therefore also be displayed on the screen by using this technique.
- aspects of the invention are directed towards solving the problem of interactively controlling an image presentation as displayed on a large screen, using a portable communication device.
- aspects consistent with the invention provide a method for comparing image information being registered with image information being presented using a light registering camera unit of the portable communication device.
- a method of providing positioning information related to a presentation of image information for a portable communication device comprises providing first image information to be presented by an image information presentation unit, obtaining the first image information by the portable communication device under control of the user of the portable communication device, registering second image information by the camera of the portable communication device, mapping presented first image information and registered second image information, recognizing at least one image element of the presented first image information as being included in the registered second image information, and determining the position of a predefined area of the registered second image information in relation to the recognized at least one image element, where the determined relative position of the predefined area is used for controlling purposes of the first image information being presented.
- a second aspect of the invention is directed towards a method including the features of the first aspect, further comprises the step of sending predefined area position information for displaying the predefined area position on the image information presentation unit.
- a third aspect of the invention is directed towards a method including the features of the first aspect, wherein the step of determining the position of the predefined area comprises determining the position of the center area of second registered image information.
- a fourth aspect of the invention is directed towards a method including the features of the first aspect, further comprising obtaining updated directing of the camera of the portable communication device under the control of the user of the portable communication device.
- a fifth aspect of the invention is directed towards a method including the features of the first aspect, further comprising the step of receiving user input by the portable communication device, which user input is used as input data for controlling the first image information being presented.
- a sixth aspect of the invention is directed towards a method including the features of the fourth aspect, further comprising the step of controlling the first image information being presented based on at least one of the obtained updated directing or received user input.
- a portable communication device for providing positioning information related to a presentation of a first image information, comprises an image information providing unit, arranged to provide a first image information to an image information presentation unit, a light registering camera unit, arranged to register second image information when being directed towards the image presentation unit, a mapping unit, connected to the light registering camera unit and arranged to map the first image information and the second image information, and a control unit arranged to control providing of the first image information to an image information unit, to control registering second image information by the light registering camera unit, and to control mapping of the first image information and the second image information, and further arranged to recognize at least one image element of the presented first image information as being included in the second image information, to determine the position of a predefined area of the second registered image information in relation to the recognized at least one image element and use the determined relative position of the predefined area to control the first image information being presented.
- An eighth aspect of the invention is directed toward a portable communication device including the features of the seventh aspect and wherein the control unit is further configured to use the determined relative position of the predefined area to control the first image information being presented.
- a ninth aspect of the invention is directed towards a portable communication device including the features of the seventh aspect, wherein the portable communication device comprises a mobile phone.
- Still other aspects of the invention provide a computer program product comprising a computer readable medium, having thereon computer program code means, to make a portable communication device or a computer execute, comparing image information being registered with image information being presented, using a light registering camera unit of the portable communication device.
- a computer program product comprising a computer readable medium, having thereon computer program code for execution by a portable communication device or a computer, when said computer program code is loaded in the portable communication device or the computer, to cause the portable communication device or computer perform a method comprising: providing of first image information from the portable communication device or the computer, to be presented by an image information presentation unit, registering second image information by the camera of the portable communication device, mapping presented first image information and registered second image information, recognizing at least one image element of the presented first image information as being included in the second registered image information, and determining the position of a predefined area of the second registered image information in relation to the recognized at least one image element.
- An eleventh aspect of the invention is directed to a computer program product including the features of the tenth aspect, where the method performed by the portable communication device or computer further comprises using the determined relative position of the predefined area for controlling the first image information being presented.
- aspects of the invention provide, among other advantages, positioning of an image center relative to recognized surrounding image elements, which enables more advanced gaming using mobile phones in connection with external image presentation units.
- the invention may also provide a flexible positioning tool, which also can be used for controlling objects based on relative positioning of cursor movements.
- FIG. 1 schematically illustrates an exemplary portable communication device consistent with the invention
- FIG. 2 is a flow chart illustrating processing according to one embodiment of the invention
- FIG. 3 schematically presents one simple example of first and second image information according to the invention
- FIG. 4 illustrates a portable communication device in the form of a cellular phone, according to one embodiment of the invention.
- FIG. 5 illustrates a computer program product having stored thereon computer program code to make a portable communication device or a computer execute steps of a method according to one embodiment of the invention.
- aspects consistent with the invention are directed towards solving the problem of interactively controlling an image presentation as displayed on a large screen, using a portable communication device.
- FIG. 1 schematically showing a portable communication device 100 , according to one embodiment of the present invention, the invention is now explained in more detail.
- the portable communication device 100 comprises an image information providing unit 102 , a light registering camera unit 104 , a mapping unit 106 , a control unit 108 , a memory unit 110 , and a user input unit 112 .
- the control unit 108 is connected to all other units. That is, control unit 108 is connected to the image information providing unit 102 , the light registering unit 104 , the mapping unit 106 , the memory unit 110 and the user input unit 112 .
- the light registering camera unit 104 and the memory unit 110 are connected to the mapping unit 106 .
- the image information providing unit 102 is further connected to the memory unit 110 .
- aspects consistent with the invention use image information that is registered by a light registering camera unit, compare the registered image information with the image information that is being presented, and based on recognized image elements or image objects, to position a predetermined part of the camera image in the image information being presented.
- the user of the portable mobile phone being one example of the communication device 100 , can utilize the position of the predetermined part of the camera image relative to the displayed image elements in the image information being presented, to control the image information that is being presented.
- FIG. 2 presenting a flow chart of a method according to one embodiment of the present invention, the method will be described in more detail.
- This exemplary method starts with sending video or image information for presentation (step 202 ). This step is performed by the image information providing unit 102 under control of the control unit 108 .
- Video information is herein understood as comprising image information that may be registered by a camera unit.
- the portable communication device 100 Prior to sending the video information, the portable communication device 100 is connected to an image information presentation unit, such a TV-screen, a TV-set, PC-screen, a video projector or the like, to be used for the presentation of the first image information.
- an image information presentation unit such as a TV-screen, a TV-set, PC-screen, a video projector or the like, to be used for the presentation of the first image information.
- the connection between the mobile phone, being one example of the portable communication device 100 , and the image information presentation unit is performed using, for example, a universal serial bus (USB) connector, according to one implementation of the present invention.
- USB universal serial bus
- connection may be established using Bluetooth technique using a first Bluetooth module either built-in in the mobile phone or associated with it, and a second Bluetooth module connected to or associated with the image information presentation unit.
- the mobile phone may send video information, as one example of first image information, to the image information presentation unit.
- the method of providing positioning information may also include directing the light registering camera unit 104 of the mobile phone 100 towards the image information presentation unit, under control of the user of the mobile phone 100 , and obtaining video information (step 204 ). That is, the user of mobile phone 100 may use the camera function of mobile phone 100 (i.e., light registering camera unit 104 ) as a pointer and point the mobile phone 100 in the direction of the image information presentation unit. This step is included since the light registering camera unit 104 , within the method of the present invention, is arranged to register light from the image information presentation unit.
- the light registering camera unit 104 may detect a change of the direction of the camera included in mobile phone 100 with respect to the image information presentation unit. This change of direction may be caused by the user of the mobile phone 100 , such as would occur by the user moving mobile phone 100 , or by other influences.
- the camera image may then be registered (step 206 ). That is, the light registering camera unit 104 may register image information, such as image information associated with the video information being presented on the image information presentation unit, as an example of image information that is being presented. The light that is registered by the light registering camera unit 104 may then form the second image information, that will be used to compare to the first image information, as described in more detail below.
- the step of registering is performed by the light registering camera unit 104 under the control of the control unit 108 .
- the subsequent step is mapping presented first image information and registered second image information (step 208 ).
- mapping unit 106 may be performed by mapping unit 106 , being provided with the second image information from the light registering camera unit 104 and first image information from the memory unit 110 , under control of the control unit 108 .
- mapping of image or picture information is well known to the skilled person, this mapping step will not be discussed in full detail to not unduly obscure the thrust of the invention. However, it is to be understood that a mapping may include relative size adjustments, rotations and translations of the two kinds of image information.
- projection of the first image information at an angle may be used in the step of mapping when mapping the first image information with the registered second image information, since the second image information may have been registered at an angle relative to the presentation of the first image information using the image information presentation unit.
- the step of mapping is performed taking into account shaking of the mobile phone 100 , including the light registering camera unit 104 , caused by unsteady user operation when directing the light registering camera unit 104 towards the image information presentation unit.
- a next step of the method of providing positioning information is recognizing at least one image element of the first image information as being included in the registered second image information (step 210 ). For example, suppose that an image of a person is included in the first image information. In this case, this step may include recognizing the image of a person in the second image information. This step of recognizing may be performed by the control unit 108 being provided with mapping information by the mapping unit 106 .
- a complete image object may be recognized by the control unit 108 in step 210 , as being present in both first and second image information.
- the relative position of the camera image center with respect to the recognized person may then be determined (act 212 ), as one example of determining the relative position of a predefined area of the registered second image information in the first image information, with respect to a recognized image element. Determining the relative position of the camera image center is a step that may be performed by the control unit 108 , according to one embodiment of the present invention.
- mapping first image information such as the presented video information and the second image information, such as the registered second image information, as well as the step of recognizing image elements as present in both these image information, may be performed in various ways. Mapping may be made at various time intervals following different criteria. Such variations would be known to one ordinary skill in the art and will therefore not be discussed hereinafter.
- image information providing unit 102 may provide the camera image center position to an image presentation unit on which the center position is displayed together with the video information, under the control of the control unit 108 .
- displaying the camera image position with the video information is followed by determining whether the obtained camera image (i.e., the second information obtained at step 204 ) is adequate (step 216 ).
- control unit 108 with assistance of the mapping unit 106 may determine whether any substantial direction change of the mobile phone 100 occurred based on, for example, the degree of recognition of image elements and objects in step 210 .
- displaying the camera image position with the video information may be followed by determining whether a change of direction is detected by the camera under the control of the control unit 108 .
- processing may return to step 204 . That is, new video information (e.g., a camera image) from mobile phone 100 may be obtained, under the influence of the user of the mobile phone 100 .
- new video information e.g., a camera image
- the subsequent step detects whether user interaction has occurred (step 218 ).
- User interaction may be exemplified by the removal of an element or the graphical disintegration of an image object, or the displacement of an object, to mention a few examples only. Such interactions may be performed by use of an application program, such as a video game (e.g., a shooting game), being executed by mobile phone 100 .
- a video game e.g., a shooting game
- processing may return to step 216 , according to this embodiment of the present invention.
- the method may end (step 220 ).
- the relative position of a predetermined area of the second image information on the first image information may be used for obtaining direction orders in aviation games, wherein the position of the camera center (e.g., a cursor) relative to the position of the object may be used to alter the altitude of the object in the aviation game, make the object perform turns, prepare for touch down and landing, etc., to mention a few possibilities only.
- the position of the camera center e.g., a cursor
- the camera center position may be associated with one or more recognized objects in order to change the speed, direction or conditions for the objects.
- One example would be to associate the camera center with a car or becoming your own car, such as in a racing game, after which any position of the camera center relative to the associated car may be used for steering and speed regulating purposes, etc.
- Many more variations along similar lines may also be provided.
- An external image information presentation unit 302 in the form of a screen is being used for presenting the first image information, here in the form of an adult 308 to the right in the presentation and a child 306 , to the adult's right positioned close to the center of the presentation.
- Portable communication device 304 also referred to herein as mobile phone 304 , may include the same or similar elements as portable communication device 100 described above with respect to FIG. 1 .
- the camera registers the second image information, illustrated as image 310 , containing the adult 314 to the right and the child 312 to the adult's right positioned close to the center of the presentation.
- mapping unit 106 the step of mapping of the first and second image information is performed, as described above by mapping unit 106 , in FIG. 1 .
- the camera center position is determined and information is provided to the image presentation unit to display the camera center position.
- the first image presentation as displayed on screen 316 now comprises the camera center, denoted by the star 322 .
- the position of the star 322 thus corresponds to the camera image center.
- This image center 322 on the screen 316 is positioned to the left of the child 318 and even further away from the adult 320 .
- a new image associated with the movement of the mobile phone 100 may be obtained (step 204 ).
- the second image information of the screen as viewed by the camera is denoted by 332 , which shows the adult 336 close to the center of the camera picture and the child 334 to the left in the camera picture, being one example of the second image information.
- the view as denoted by 338 is provided as the first image information.
- the adult 342 and the child 340 have not changed their position in the first image information, but a cursor indicating the camera center is now displayed as the star 344 at the position of the adult 342 . This is due to the directing of the camera towards the position of the adult 330 in the first image information of the screen 324 .
- Screen 346 denotes the screen after the adult image has been wiped out or removed after the receipt of user input via mobile phone 304 .
- the adult may be removed via an input associated with a user playing a video game executed by mobile phone 304 .
- the child 348 is still present and so is the star 350 denoting the camera center.
- the portable communication device 304 may be used as a pointing device and a symbol, such as a star in this example, may be used to indicate the center position at which the camera is pointed.
- the portable communication device 304 acts as a pointing device associated with images on an external image presentation unit, such as the displays illustrated in screens 316 , 338 and 346 .
- the symbol e.g., the star
- the user of portable communication device 304 may perform an action, such as remove the image, during execution of a video game being played by the user of portable communication device 304 .
- Described above is one example of how positioning information may be used in a simple way for controlling the video information being presented by an external image information presentation unit.
- a mobile phone may be used as a pointer of the game.
- This pointer may be displayed as a gun or shooting device, allowing the user to control the virtual gun to shoot in the direction at which the camera of the mobile phone is pointed.
- the pointer may also be used to move images within the display.
- buttons of the mobile phone are associated with special functions. Buttons may be used to activate certain actions, for instance to set a password simply by entering a certain digit while directing the camera to a specific part of the monitor/TV set.
- the combination of directing the mobile phone to a particular part of a monitor or display and entering a specific key may thus form a unique pattern that can be used as a secret key or for general security purposes.
- FIG. 4 illustrates a portable communication device 40 in the form of a mobile phone, according to one embodiment of the present invention.
- Portable communication device 40 may include the same or similar elements as portable communication device 100 described above with respect to FIG. 1 .
- FIG. 5 schematically depicts a computer program product 50 according to one embodiment of the invention, having thereon computer program code means.
- the computer program code means comprised on the computer program product, is loaded in a computer or portable communication device
- said computer or portable communication device executes providing of first image information from the portable communication device of the computer, to be presented by an image information presentation unit, registering second image information by the camera of the portable communication device under control of the user of the portable communication device, mapping presented first image information and registered second image information, recognizing at least one image element of the presented first image information as being included in the second registered image information, and determining the position of a predefined area of the second registered image information in relation to the recognized at least one image element, in order to enable using the determined relative position of the predefined area for controlling purposes of the first image information being presented.
- the computer program product may be provided as a CD-ROM disc according to one embodiment of the invention.
- the computer program product 50 can alternatively be provided as another type of disc such as a digital video disc (DVD), a hard disc, or be provided as a memory or other storage capacity, such as a flash-based memory, for example a memory stick or a USB (Universal Serial Bus) memory or even a memory of the type being volatile.
- DVD digital video disc
- hard disc or be provided as a memory or other storage capacity, such as a flash-based memory, for example a memory stick or a USB (Universal Serial Bus) memory or even a memory of the type being volatile.
- USB Universal Serial Bus
- the steps of the method as described above may be executed in a different order, without deviating from the scope of the invention.
- some steps may be formed concurrently.
- some steps of the method may be omitted, following yet a different embodiment of the invention.
- the number of steps may moreover be changed, by for instance incorporating a few steps in others or dividing the function of certain steps in other steps, such that novel steps are created but without deviating from the overall function of the steps.
- the units as comprised in the portable communication device, as shown in FIG. 1 may at least partly be comprised in other performing units, without deviating from the invention.
- Other units may co-exist according to other embodiments of the invention.
- aspects of the invention provide, among other advantages, positioning of an image center relative to recognized surrounding image elements, which enables, for example, more advanced gaming using mobile phones in connection with external image presentation units.
- the invention may also provide a flexible positioning tool, which also can be used for controlling objects based on relative positioning of cursor movements.
Abstract
Description
- This application claims priority under 35 U.S.C. § 119 based on U.S. Provisional Application Ser. No. 60/749,621, filed Dec. 13, 2005, the disclosure of which is incorporated herein by reference.
- The invention relates in general to controlling presentation of image information, and in particular to providing positioning information related to the presentation of image information.
- Electronic games within portable electronic devices have recently developed into advanced and complex games requiring large memory space. Portable communication devices, such as mobile phones, may be found having memories of 0.5 Gb or more.
- Whereas the games have increased in complexity, the size of the screen of the mobile phones, being one example of portable communication devices, has not had the same development. Screens are still relatively small, which sets limitations to the user experience when gaming.
- Gaming using large screens is well known and was performed in the early days of electronic games, such as in television games using TV-sets as screens or in personal computer games using the computer cathode ray tubes as screens.
- Interactions between the user of the game and the displayed gaming information were allowed in television target aiming and shooting games. The television games typically provided a photo sensitive gun capable of sensing large bright target symbols on the screen of a television receiver. The targets were distinctly displayed on the screen when imaged using a gun-mounted electro-optical sensor resulting in a series of output pulses indicating that the gun was indeed lined up properly with the target and a hit was scored. None of these attempted to show where the user was aiming when he pulled the trigger preceding a miss or for that matter, how closely to the center of the target on screen he was aiming when he made a hit.
- In U.S. Pat. No. 4,395,045, a television precision target shooting apparatus and method simulate shooting at displayed images on the screen of a television set using a simulated gun that provides indication on the screen of where the gun was pointed at the time of trigger pull. This is achieved by flashing the screen white subsequent a trigger pull in order for a photo sensor within the gun to respond to short segments of horizontal raster lines such that by counting vertical lines and time intervals with respect to vertical and horizontal synchronization pulses, respectively, identification of where on the screen the gun was pointing is achieved. This permits a symbol to be displayed upon a screen at that location.
- These techniques to identify a flashing item by using a photo optical sensor may, however, not be used for a plurality of identical targets, since each individual position is associated with unique position descriptors, such as vertical lines and time intervals.
- There are also other techniques with which a user could directly interact with an image presentation. By using a directing or pointing device, the direction in which such a device is directed or pointed can be determined.
- In one technique, a gyro is used which is built in the directing device and which reacts to lateral and longitudinal motions of the directing device. Any lateral and longitudinal motions causing lateral and longitudinal displacements are related to a reference position, by which an updated position can be calculated. The application of such a directing device in gaming may allow the position of a cursor or symbol to be determined and displayed on a screen.
- An alternative to using a gyro to determine an updated position of a cursor or symbol on a screen is to use an accelerometer included in the directing or pointing device. Any motions of the directing or pointing device are detected by the accelerometer, and then converted to a displacement of the location where the directing or pointing device is pointing. A cursor may therefore also be displayed on the screen by using this technique.
- These two directing techniques are based on motions relative to a reference direction, for which reason these techniques become inapplicable for determining a plurality of positions of targets on a screen.
- There is hence a need to overcome the limitations of prior art techniques and to provide a technique that is more flexible and applicable to the positioning of a plurality of targets.
- Aspects of the invention are directed towards solving the problem of interactively controlling an image presentation as displayed on a large screen, using a portable communication device.
- Aspects consistent with the invention provide a method for comparing image information being registered with image information being presented using a light registering camera unit of the portable communication device.
- According to one aspect of the invention, a method of providing positioning information related to a presentation of image information for a portable communication device, comprises providing first image information to be presented by an image information presentation unit, obtaining the first image information by the portable communication device under control of the user of the portable communication device, registering second image information by the camera of the portable communication device, mapping presented first image information and registered second image information, recognizing at least one image element of the presented first image information as being included in the registered second image information, and determining the position of a predefined area of the registered second image information in relation to the recognized at least one image element, where the determined relative position of the predefined area is used for controlling purposes of the first image information being presented.
- A second aspect of the invention is directed towards a method including the features of the first aspect, further comprises the step of sending predefined area position information for displaying the predefined area position on the image information presentation unit.
- A third aspect of the invention is directed towards a method including the features of the first aspect, wherein the step of determining the position of the predefined area comprises determining the position of the center area of second registered image information.
- A fourth aspect of the invention is directed towards a method including the features of the first aspect, further comprising obtaining updated directing of the camera of the portable communication device under the control of the user of the portable communication device.
- A fifth aspect of the invention is directed towards a method including the features of the first aspect, further comprising the step of receiving user input by the portable communication device, which user input is used as input data for controlling the first image information being presented.
- A sixth aspect of the invention is directed towards a method including the features of the fourth aspect, further comprising the step of controlling the first image information being presented based on at least one of the obtained updated directing or received user input.
- Other aspects consistent with the invention provide a device for comparing image information being registered with image information being presented, using a light registering camera unit of the portable communication device.
- According to a seventh aspect of the invention, a portable communication device for providing positioning information related to a presentation of a first image information, comprises an image information providing unit, arranged to provide a first image information to an image information presentation unit, a light registering camera unit, arranged to register second image information when being directed towards the image presentation unit, a mapping unit, connected to the light registering camera unit and arranged to map the first image information and the second image information, and a control unit arranged to control providing of the first image information to an image information unit, to control registering second image information by the light registering camera unit, and to control mapping of the first image information and the second image information, and further arranged to recognize at least one image element of the presented first image information as being included in the second image information, to determine the position of a predefined area of the second registered image information in relation to the recognized at least one image element and use the determined relative position of the predefined area to control the first image information being presented.
- An eighth aspect of the invention is directed toward a portable communication device including the features of the seventh aspect and wherein the control unit is further configured to use the determined relative position of the predefined area to control the first image information being presented.
- A ninth aspect of the invention is directed towards a portable communication device including the features of the seventh aspect, wherein the portable communication device comprises a mobile phone.
- Still other aspects of the invention provide a computer program product comprising a computer readable medium, having thereon computer program code means, to make a portable communication device or a computer execute, comparing image information being registered with image information being presented, using a light registering camera unit of the portable communication device.
- According to a tenth aspect of the invention, a computer program product comprising a computer readable medium, having thereon computer program code for execution by a portable communication device or a computer, when said computer program code is loaded in the portable communication device or the computer, to cause the portable communication device or computer perform a method comprising: providing of first image information from the portable communication device or the computer, to be presented by an image information presentation unit, registering second image information by the camera of the portable communication device, mapping presented first image information and registered second image information, recognizing at least one image element of the presented first image information as being included in the second registered image information, and determining the position of a predefined area of the second registered image information in relation to the recognized at least one image element.
- An eleventh aspect of the invention is directed to a computer program product including the features of the tenth aspect, where the method performed by the portable communication device or computer further comprises using the determined relative position of the predefined area for controlling the first image information being presented.
- Aspects of the invention provide, among other advantages, positioning of an image center relative to recognized surrounding image elements, which enables more advanced gaming using mobile phones in connection with external image presentation units.
- The invention may also provide a flexible positioning tool, which also can be used for controlling objects based on relative positioning of cursor movements.
- It should be emphasized that the term “comprises/comprising” when used in this specification is taken to specify the presence of stated features, integers, steps or components, but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.
- The present invention will now be described in more detail in relation to the enclosed drawings, in which:
-
FIG. 1 schematically illustrates an exemplary portable communication device consistent with the invention; -
FIG. 2 is a flow chart illustrating processing according to one embodiment of the invention; -
FIG. 3 schematically presents one simple example of first and second image information according to the invention; -
FIG. 4 illustrates a portable communication device in the form of a cellular phone, according to one embodiment of the invention; and -
FIG. 5 illustrates a computer program product having stored thereon computer program code to make a portable communication device or a computer execute steps of a method according to one embodiment of the invention. - Aspects consistent with the invention are directed towards solving the problem of interactively controlling an image presentation as displayed on a large screen, using a portable communication device.
- With reference to
FIG. 1 , schematically showing aportable communication device 100, according to one embodiment of the present invention, the invention is now explained in more detail. - The
portable communication device 100 comprises an imageinformation providing unit 102, a lightregistering camera unit 104, amapping unit 106, acontrol unit 108, amemory unit 110, and auser input unit 112. As is shown inFIG. 1 , thecontrol unit 108 is connected to all other units. That is,control unit 108 is connected to the imageinformation providing unit 102, thelight registering unit 104, themapping unit 106, thememory unit 110 and theuser input unit 112. Moreover, the light registeringcamera unit 104 and thememory unit 110 are connected to themapping unit 106. The imageinformation providing unit 102 is further connected to thememory unit 110. - Aspects consistent with the invention use image information that is registered by a light registering camera unit, compare the registered image information with the image information that is being presented, and based on recognized image elements or image objects, to position a predetermined part of the camera image in the image information being presented. By displaying this position, the user of the portable mobile phone, being one example of the
communication device 100, can utilize the position of the predetermined part of the camera image relative to the displayed image elements in the image information being presented, to control the image information that is being presented. - By referring to
FIG. 2 presenting a flow chart of a method according to one embodiment of the present invention, the method will be described in more detail. - This exemplary method starts with sending video or image information for presentation (step 202). This step is performed by the image
information providing unit 102 under control of thecontrol unit 108. Video information is herein understood as comprising image information that may be registered by a camera unit. - Prior to sending the video information, the
portable communication device 100 is connected to an image information presentation unit, such a TV-screen, a TV-set, PC-screen, a video projector or the like, to be used for the presentation of the first image information. - The connection between the mobile phone, being one example of the
portable communication device 100, and the image information presentation unit is performed using, for example, a universal serial bus (USB) connector, according to one implementation of the present invention. - According to one alternative of the present invention, the connection may be established using Bluetooth technique using a first Bluetooth module either built-in in the mobile phone or associated with it, and a second Bluetooth module connected to or associated with the image information presentation unit.
- The mobile phone may send video information, as one example of first image information, to the image information presentation unit.
- The method of providing positioning information may also include directing the light registering
camera unit 104 of themobile phone 100 towards the image information presentation unit, under control of the user of themobile phone 100, and obtaining video information (step 204). That is, the user ofmobile phone 100 may use the camera function of mobile phone 100 (i.e., light registering camera unit 104) as a pointer and point themobile phone 100 in the direction of the image information presentation unit. This step is included since the light registeringcamera unit 104, within the method of the present invention, is arranged to register light from the image information presentation unit. - According to an alternative embodiment of the invention, the light registering
camera unit 104 may detect a change of the direction of the camera included inmobile phone 100 with respect to the image information presentation unit. This change of direction may be caused by the user of themobile phone 100, such as would occur by the user movingmobile phone 100, or by other influences. - The camera image may then be registered (step 206). That is, the light registering
camera unit 104 may register image information, such as image information associated with the video information being presented on the image information presentation unit, as an example of image information that is being presented. The light that is registered by the light registeringcamera unit 104 may then form the second image information, that will be used to compare to the first image information, as described in more detail below. The step of registering is performed by the light registeringcamera unit 104 under the control of thecontrol unit 108. - Having registered the camera image as the second image information by the light registering
camera unit 104 and having access to the video information (as one example of the first image information), the subsequent step is mapping presented first image information and registered second image information (step 208). - This mapping may be performed by
mapping unit 106, being provided with the second image information from the light registeringcamera unit 104 and first image information from thememory unit 110, under control of thecontrol unit 108. - Since mapping of image or picture information is well known to the skilled person, this mapping step will not be discussed in full detail to not unduly obscure the thrust of the invention. However, it is to be understood that a mapping may include relative size adjustments, rotations and translations of the two kinds of image information.
- In an alternative embodiment of the present invention, projection of the first image information at an angle may be used in the step of mapping when mapping the first image information with the registered second image information, since the second image information may have been registered at an angle relative to the presentation of the first image information using the image information presentation unit.
- According to another embodiment the step of mapping is performed taking into account shaking of the
mobile phone 100, including the light registeringcamera unit 104, caused by unsteady user operation when directing the light registeringcamera unit 104 towards the image information presentation unit. - Having mapped the first image information and the second image information, a next step of the method of providing positioning information is recognizing at least one image element of the first image information as being included in the registered second image information (step 210). For example, suppose that an image of a person is included in the first image information. In this case, this step may include recognizing the image of a person in the second image information. This step of recognizing may be performed by the
control unit 108 being provided with mapping information by themapping unit 106. - According to another embodiment of the present invention, a complete image object may be recognized by the
control unit 108 instep 210, as being present in both first and second image information. - The relative position of the camera image center with respect to the recognized person may then be determined (act 212), as one example of determining the relative position of a predefined area of the registered second image information in the first image information, with respect to a recognized image element. Determining the relative position of the camera image center is a step that may be performed by the
control unit 108, according to one embodiment of the present invention. - It is noted that image information and especially motion pictures, that is video information, may be formatted in various different ways. For this reason the steps of mapping first image information, such as the presented video information and the second image information, such as the registered second image information, as well as the step of recognizing image elements as present in both these image information, may be performed in various ways. Mapping may be made at various time intervals following different criteria. Such variations would be known to one ordinary skill in the art and will therefore not be discussed hereinafter.
- Next, a symbol, such as a cursor may be displayed on the presented video information at the camera image center position (step 214). In one implementation, image
information providing unit 102 may provide the camera image center position to an image presentation unit on which the center position is displayed together with the video information, under the control of thecontrol unit 108. - In one embodiment of the present invention, displaying the camera image position with the video information is followed by determining whether the obtained camera image (i.e., the second information obtained at step 204) is adequate (step 216). For example, in one implementation,
control unit 108 with assistance of themapping unit 106 may determine whether any substantial direction change of themobile phone 100 occurred based on, for example, the degree of recognition of image elements and objects instep 210. - Various other criteria for determining whether the obtained camera image is adequate (e.g., whether a substantial change in direction of the
mobile phone 100 has occurred) may be used in alternative embodiments of the invention. - For example, according to one embodiment of the present invention, displaying the camera image position with the video information (i.e., step 214) may be followed by determining whether a change of direction is detected by the camera under the control of the
control unit 108. - If it is determined in
step 216 that the obtained camera image is not adequate, or according to the alternative embodiment that a change of direction has been detected, processing may return to step 204. That is, new video information (e.g., a camera image) frommobile phone 100 may be obtained, under the influence of the user of themobile phone 100. - If, however, it is determined in
step 216 that the image is adequate or according to an alternative embodiment that a change of direction has not been detected, the subsequent step detects whether user interaction has occurred (step 218). User interaction may be exemplified by the removal of an element or the graphical disintegration of an image object, or the displacement of an object, to mention a few examples only. Such interactions may be performed by use of an application program, such as a video game (e.g., a shooting game), being executed bymobile phone 100. - If it is determined in
step 218 that user interaction is received, processing may return to step 216, according to this embodiment of the present invention. - If, on the other hand, it is determined that no user interaction is received in
step 218, the method may end (step 220). - According to another embodiment of the invention, the relative position of a predetermined area of the second image information on the first image information, such as, for example, the camera image center of the video presentation may be used for obtaining direction orders in aviation games, wherein the position of the camera center (e.g., a cursor) relative to the position of the object may be used to alter the altitude of the object in the aviation game, make the object perform turns, prepare for touch down and landing, etc., to mention a few possibilities only.
- According to yet another embodiment of the invention, the camera center position may be associated with one or more recognized objects in order to change the speed, direction or conditions for the objects. One example would be to associate the camera center with a car or becoming your own car, such as in a racing game, after which any position of the camera center relative to the associated car may be used for steering and speed regulating purposes, etc. Many more variations along similar lines may also be provided.
- With reference to
FIG. 3 , an example of using the invention for controlling purposes of an electronic game is schematically illustrated. An external imageinformation presentation unit 302 in the form of a screen is being used for presenting the first image information, here in the form of anadult 308 to the right in the presentation and achild 306, to the adult's right positioned close to the center of the presentation. - A
portable communication device 304 containing a light registering camera unit, illustrated asbox 305 inFIG. 3 , is directed towards the screen of imageinformation presentation unit 302.Portable communication device 304, also referred to herein asmobile phone 304, may include the same or similar elements asportable communication device 100 described above with respect toFIG. 1 . By registering incident light, the camera registers the second image information, illustrated asimage 310, containing theadult 314 to the right and thechild 312 to the adult's right positioned close to the center of the presentation. - Having obtained first and second image information, the step of mapping of the first and second image information is performed, as described above by mapping
unit 106, inFIG. 1 . After having recognized both the adult and the child as being included in both images (i.e., images displayed by imageinformation presentation unit 302 and image 310), the camera center position is determined and information is provided to the image presentation unit to display the camera center position. - As a result, the first image presentation as displayed on
screen 316 now comprises the camera center, denoted by thestar 322. The position of thestar 322 thus corresponds to the camera image center. Thisimage center 322 on thescreen 316 is positioned to the left of thechild 318 and even further away from theadult 320. - After having determined that the camera image information was not adequate or that the camera was moved in
step 216, a new image associated with the movement of themobile phone 100 may be obtained (step 204). - This is schematically depicted by the
mobile phone 304 being directed in a new direction towardsscreen 324, as compared to the previous exemplary picture. That is,mobile phone 304 has been pointed in a new direction. As result, theadult 330 appears to be located close to the center of the camera view and thechild 328 is located to the left in the camera view. - Consequently, the second image information of the screen as viewed by the camera is denoted by 332, which shows the
adult 336 close to the center of the camera picture and thechild 334 to the left in the camera picture, being one example of the second image information. - By performing the steps of mapping, recognizing and displaying the camera center position on the first image presentation, the view as denoted by 338 is provided as the first image information. As expected, the
adult 342 and thechild 340 have not changed their position in the first image information, but a cursor indicating the camera center is now displayed as thestar 344 at the position of theadult 342. This is due to the directing of the camera towards the position of theadult 330 in the first image information of thescreen 324. -
Screen 346 denotes the screen after the adult image has been wiped out or removed after the receipt of user input viamobile phone 304. For example, the adult may be removed via an input associated with a user playing a video game executed bymobile phone 304. Thechild 348 is still present and so is thestar 350 denoting the camera center. - Therefore, in this example, the
portable communication device 304 may be used as a pointing device and a symbol, such as a star in this example, may be used to indicate the center position at which the camera is pointed. In this manner, theportable communication device 304 acts as a pointing device associated with images on an external image presentation unit, such as the displays illustrated inscreens portable communication device 304 may perform an action, such as remove the image, during execution of a video game being played by the user ofportable communication device 304. - Described above is one example of how positioning information may be used in a simple way for controlling the video information being presented by an external image information presentation unit.
- In another example, other interactions between the user of a mobile phone and a game during gaming applications are enabled according to the invention. For example, during gaming sessions using large displays, such as TV sets, monitors, etc., a mobile phone may be used as a pointer of the game. This pointer may be displayed as a gun or shooting device, allowing the user to control the virtual gun to shoot in the direction at which the camera of the mobile phone is pointed. The pointer may also be used to move images within the display.
- In yet another example of the application of the invention, a combination of the camera as a pointing and registering tool, and other user interfaces is provided. For example, when pointing the camera at specific positions on the screen, certain buttons of the mobile phone are associated with special functions. Buttons may be used to activate certain actions, for instance to set a password simply by entering a certain digit while directing the camera to a specific part of the monitor/TV set. The combination of directing the mobile phone to a particular part of a monitor or display and entering a specific key may thus form a unique pattern that can be used as a secret key or for general security purposes.
-
FIG. 4 illustrates aportable communication device 40 in the form of a mobile phone, according to one embodiment of the present invention.Portable communication device 40 may include the same or similar elements asportable communication device 100 described above with respect toFIG. 1 . - In addition,
FIG. 5 schematically depicts acomputer program product 50 according to one embodiment of the invention, having thereon computer program code means. When the computer program code means, comprised on the computer program product, is loaded in a computer or portable communication device, said computer or portable communication device, executes providing of first image information from the portable communication device of the computer, to be presented by an image information presentation unit, registering second image information by the camera of the portable communication device under control of the user of the portable communication device, mapping presented first image information and registered second image information, recognizing at least one image element of the presented first image information as being included in the second registered image information, and determining the position of a predefined area of the second registered image information in relation to the recognized at least one image element, in order to enable using the determined relative position of the predefined area for controlling purposes of the first image information being presented. - The computer program product may be provided as a CD-ROM disc according to one embodiment of the invention. However, the
computer program product 50 can alternatively be provided as another type of disc such as a digital video disc (DVD), a hard disc, or be provided as a memory or other storage capacity, such as a flash-based memory, for example a memory stick or a USB (Universal Serial Bus) memory or even a memory of the type being volatile. - It is emphasized that this invention can be varied in many ways, of which the embodiments discussed above only are examples of a few. These different embodiments are hence non-limiting examples. The scope of the invention, however, is only limited by the subsequently following claims.
- According to one alternative embodiment of the invention, the steps of the method as described above may be executed in a different order, without deviating from the scope of the invention. In addition, some steps may be formed concurrently. Still further, some steps of the method may be omitted, following yet a different embodiment of the invention.
- The number of steps may moreover be changed, by for instance incorporating a few steps in others or dividing the function of certain steps in other steps, such that novel steps are created but without deviating from the overall function of the steps.
- The units as comprised in the portable communication device, as shown in
FIG. 1 , may at least partly be comprised in other performing units, without deviating from the invention. Other units may co-exist according to other embodiments of the invention. - Aspects of the invention, as described above, provide, among other advantages, positioning of an image center relative to recognized surrounding image elements, which enables, for example, more advanced gaming using mobile phones in connection with external image presentation units.
- The invention may also provide a flexible positioning tool, which also can be used for controlling objects based on relative positioning of cursor movements.
- The scope of the invention is defined by the claims and their equivalents.
Claims (12)
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/438,088 US7513700B2 (en) | 2005-12-13 | 2006-05-22 | Image element identifier |
PCT/EP2006/066187 WO2007068511A1 (en) | 2005-12-13 | 2006-09-08 | Image element identifier |
EP06793370A EP1960070A1 (en) | 2005-12-13 | 2006-09-08 | Image element identifier |
JP2008544901A JP2009519066A (en) | 2005-12-13 | 2006-09-08 | Image element identifier |
KR1020087016968A KR20080080631A (en) | 2005-12-13 | 2006-09-08 | Image element identifier |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US74962105P | 2005-12-13 | 2005-12-13 | |
US11/438,088 US7513700B2 (en) | 2005-12-13 | 2006-05-22 | Image element identifier |
Publications (2)
Publication Number | Publication Date |
---|---|
US20070147828A1 true US20070147828A1 (en) | 2007-06-28 |
US7513700B2 US7513700B2 (en) | 2009-04-07 |
Family
ID=37116087
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/438,088 Expired - Fee Related US7513700B2 (en) | 2005-12-13 | 2006-05-22 | Image element identifier |
Country Status (5)
Country | Link |
---|---|
US (1) | US7513700B2 (en) |
EP (1) | EP1960070A1 (en) |
JP (1) | JP2009519066A (en) |
KR (1) | KR20080080631A (en) |
WO (1) | WO2007068511A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130169820A1 (en) * | 2011-03-15 | 2013-07-04 | David Alexander Stewart | Camera device to capture and generate target lead and shooting technique data and images |
US20140078319A1 (en) * | 2007-02-22 | 2014-03-20 | Panasonic Corporation | Imaging apparatus |
US9267761B2 (en) * | 2011-03-15 | 2016-02-23 | David A. Stewart | Video camera gun barrel mounting and programming system |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8217997B2 (en) * | 2010-03-16 | 2012-07-10 | Interphase Corporation | Interactive display system |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4395045A (en) * | 1980-06-16 | 1983-07-26 | Sanders Associates, Inc. | Television precision target shooting apparatus and method |
US5138304A (en) * | 1990-08-02 | 1992-08-11 | Hewlett-Packard Company | Projected image light pen |
US20050107160A1 (en) * | 2003-11-14 | 2005-05-19 | Zeroplus Technology Co., Ltd. | [photographic pointer positioning system and its operation process] |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1175345C (en) | 1999-03-31 | 2004-11-10 | 皇家菲利浦电子有限公司 | Remote control for display apparatus |
US6287200B1 (en) | 1999-12-15 | 2001-09-11 | Nokia Corporation | Relative positioning and virtual objects for mobile devices |
DK1573498T3 (en) | 2002-11-20 | 2012-03-19 | Koninkl Philips Electronics Nv | User interface system based on a pointing device |
-
2006
- 2006-05-22 US US11/438,088 patent/US7513700B2/en not_active Expired - Fee Related
- 2006-09-08 KR KR1020087016968A patent/KR20080080631A/en not_active Application Discontinuation
- 2006-09-08 JP JP2008544901A patent/JP2009519066A/en not_active Withdrawn
- 2006-09-08 WO PCT/EP2006/066187 patent/WO2007068511A1/en active Application Filing
- 2006-09-08 EP EP06793370A patent/EP1960070A1/en not_active Withdrawn
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4395045A (en) * | 1980-06-16 | 1983-07-26 | Sanders Associates, Inc. | Television precision target shooting apparatus and method |
US5138304A (en) * | 1990-08-02 | 1992-08-11 | Hewlett-Packard Company | Projected image light pen |
US20050107160A1 (en) * | 2003-11-14 | 2005-05-19 | Zeroplus Technology Co., Ltd. | [photographic pointer positioning system and its operation process] |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140078319A1 (en) * | 2007-02-22 | 2014-03-20 | Panasonic Corporation | Imaging apparatus |
US20130169820A1 (en) * | 2011-03-15 | 2013-07-04 | David Alexander Stewart | Camera device to capture and generate target lead and shooting technique data and images |
US8908045B2 (en) * | 2011-03-15 | 2014-12-09 | David Alexander Stewart | Camera device to capture and generate target lead and shooting technique data and images |
US9267761B2 (en) * | 2011-03-15 | 2016-02-23 | David A. Stewart | Video camera gun barrel mounting and programming system |
US9546846B2 (en) | 2011-03-15 | 2017-01-17 | David A. Stewart | Video camera gun barrel mounting system |
Also Published As
Publication number | Publication date |
---|---|
US7513700B2 (en) | 2009-04-07 |
WO2007068511A1 (en) | 2007-06-21 |
KR20080080631A (en) | 2008-09-04 |
EP1960070A1 (en) | 2008-08-27 |
JP2009519066A (en) | 2009-05-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11195307B2 (en) | Image processing apparatus, image processing method, and program | |
US10864433B2 (en) | Using a portable device to interact with a virtual space | |
US8477099B2 (en) | Portable data processing appartatus | |
US9084938B2 (en) | Handheld device for spectator viewing of an interactive application | |
US9504920B2 (en) | Method and system to create three-dimensional mapping in a two-dimensional game | |
CN101189049B (en) | Method of executing an application in a mobile device | |
US9262015B2 (en) | System for portable tangible interaction | |
JP5448073B2 (en) | Information processing apparatus, information processing program, information processing system, and selection target selection method | |
CN111566596B (en) | Real world portal for virtual reality displays | |
JP2015187884A (en) | Pointing device with camera and mark output | |
CN104870941A (en) | Display of separate computer vision based pose and inertial sensor based pose | |
US10537814B2 (en) | Screen coding methods and camera based game controller for video shoot game | |
US10979700B2 (en) | Display control apparatus and control method | |
JP2015118556A (en) | Augmented reality overlay for control devices | |
US7513700B2 (en) | Image element identifier | |
CN114706489A (en) | Virtual method, device, equipment and storage medium of input equipment | |
US20170354866A1 (en) | Non-transitory storage medium having game program stored therein, information processing apparatus, information processing system, game processing method | |
KR101914660B1 (en) | Method and apparatus for controlling displaying of augmented reality contents based on gyro sensor | |
CN104580967A (en) | Map projection method based on portable projector, and device for projection | |
US20100053080A1 (en) | Method For Setting Up Location Information On Display Screens And A Recognition Structure Thereof | |
EP4311584A2 (en) | Data processing apparatus and method | |
JP6935921B2 (en) | Terminal devices and programs | |
CN101330948A (en) | Image element identifier | |
CN115463409A (en) | Frame supplementing method for picture information of handheld device and handheld device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SONY ERICSSON MOBILE COMMUNICATIONS AB, SWEDEN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NELSON, JOAKIM;REEL/FRAME:017916/0565 Effective date: 20060522 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees | ||
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20170407 |