EP2888712A1 - Intégration d'un ou de plusieurs dispositifs informatiques mobiles dans un système robotique - Google Patents

Intégration d'un ou de plusieurs dispositifs informatiques mobiles dans un système robotique

Info

Publication number
EP2888712A1
EP2888712A1 EP13834203.5A EP13834203A EP2888712A1 EP 2888712 A1 EP2888712 A1 EP 2888712A1 EP 13834203 A EP13834203 A EP 13834203A EP 2888712 A1 EP2888712 A1 EP 2888712A1
Authority
EP
European Patent Office
Prior art keywords
physical
agents
agent
state
virtual representations
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP13834203.5A
Other languages
German (de)
English (en)
Other versions
EP2888712A4 (fr
Inventor
Boris Sofman
Hans TAPPEINER
Mark Matthew Palatucci
Patrick Lee Deneale
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anki Inc
Original Assignee
Anki Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anki Inc filed Critical Anki Inc
Publication of EP2888712A1 publication Critical patent/EP2888712A1/fr
Publication of EP2888712A4 publication Critical patent/EP2888712A4/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
    • A63F13/65—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor automatically by game devices or servers from real world data, e.g. measurement in live racing competition
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/55—Controlling game characters or game objects based on the game progress
    • A63F13/57—Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game
    • A63F13/573—Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game using trajectories of game objects, e.g. of a golf ball according to the point of impact
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63H—TOYS, e.g. TOPS, DOLLS, HOOPS OR BUILDING BLOCKS
    • A63H17/00—Toy vehicles, e.g. with self-drive; ; Cranes, winches or the like; Accessories therefor
    • A63H17/26—Details; Accessories
    • A63H17/32—Acoustical or optical signalling devices
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63H—TOYS, e.g. TOPS, DOLLS, HOOPS OR BUILDING BLOCKS
    • A63H30/00—Remote-control arrangements specially adapted for toys, e.g. for toy vehicles
    • A63H30/02—Electrical arrangements
    • A63H30/04—Electrical arrangements using wireless transmission
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/25—Output arrangements for video game devices
    • A63F13/28—Output arrangements for video game devices responding to control signals received from the game device for affecting ambient conditions, e.g. for vibrating players' seats, activating scent dispensers or affecting temperature or light
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/30—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device
    • A63F2300/308—Details of the user interface
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/60—Methods for processing data by generating or executing the game program
    • A63F2300/69—Involving elements of the real world in the game world, e.g. measurement in live races, real video
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
    • A63F2300/8017—Driving on land or water; Flying
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63H—TOYS, e.g. TOPS, DOLLS, HOOPS OR BUILDING BLOCKS
    • A63H2200/00—Computerized interactive toys, e.g. dolls

Definitions

  • the present invention relates to robotic systems capable of being integrated with mobile computing devices.
  • a particular source of such expense is the use of dedicated hardware and firmware for such robotic systems, including control systems and autonomous components. Given the cost-competitive nature of commodity-based product markets, in particular those related to entertainment products, it can be unfeasible to produce and sell products for such robotic applications in a profitable manner.
  • mobile computing devices such as smartphones and/or tablets are leveraged as a platform to support and control robotic systems.
  • the rapid adoption of such devices among consumers has made it feasible and realistic to assume that a sufficient number of such devices are available for use by a consumer in connection with a robotic system.
  • a robotic system is integrated with one or more mobile computing devices.
  • mobile computing devices may include, for example, one or more smartphones, tablet computers, laptop computers, gaming consoles, kiosks, and/ or the like.
  • the techniques of the present invention can be implemented using any other suitable computing devices, such as desktop computers, web-based computing architectures, and/ or the like.
  • Such computing components can be networked with each other, and/ or with physical components such as agents under control of a user or users; such networking can make use of Bluetooth, Wifi, and/ or other wireless networking technologies.
  • the system of the present invention is implemented as an application in entertainment, such as the manifestation of a video game in physical space. Games and entertainment are an appealing use of the invention for its inclusion of interactivity among multiple users.
  • the system of the present invention provides an environment wherein users control one or more agents of the system, while one or more other agents may be controlled by artificial intelligence.
  • Fig. 1 is a block diagram depicting an architecture for implementing the present invention according to one embodiment.
  • Fig. 2 depicts an example of integration of events and functions in physical and virtual space, according to one embodiment.
  • Fig. 3 is a block diagram depicting aspects of the role of the host device (base station) according to at least one embodiment, and illustrating various components and structures that implement the relationship between users controlling vehicles and the vehicles themselves.
  • Fig. 4 depicts an example in which body covers on vehicles are used to reflect a difference in characteristics of corresponding representations of the vehicles in a virtual environment, thus reinforcing the relationship between the physical and virtual environments, according to one embodiment.
  • FIG. 5 depicts an example of the use of a virtual accessory to implement a digitally rendered element within a physical space, according to one embodiment.
  • FIG. 6 depicts an example of an embodiment of the present invention for implementing a gameplay environment wherein vehicles race on a race track, according to one embodiment.
  • a host device 108 which may be implemented on any suitable computing device, whether mobile or stationary, such as for example a smartphone, tablet, laptop computer, or the like, and/ or any combination thereof.
  • host device 108 supports and runs various algorithms contained in software which implement game operations.
  • Host device 108 and associated software are collectively referred to herein as a base station or central control unit.
  • Any of a variety of different devices can serve as host device 108; examples include smartphones, tablet computers, laptop computers, desktop computers, video game consoles, and/ or any other computing device capable of supporting the control software for the system.
  • a device can use any suitable operating system, including for example and without limitation: iOS or MacOS, available from Apple Inc. of Cupertino, California; Android, available from Google, Inc. of Mountain View, California; or Windows, available from Microsoft Corporation of Redmond, Washington.
  • host device 108 is an iPhone or iPad, available from Apple Inc. of Cupertino, California, running a suitable software application ("app").
  • software for controlling host device 108 may be provided via any suitable means, such as a downloadable application ("app") that includes the appropriate functionality and gameplay structure to operate agents 104 A through 104F in physical space and to plan, coordinate and execute gameplay according to rules, user-controlled actions, and/ or artificial intelligence.
  • host device 108 maintains the state of agents 104 A through 104F, and sends and receives commands to and from agents 104A through 104F.
  • Host device 108 may also include a suitable user interface for facilitating user interaction with the system.
  • agents 104 A through 104F are mobile agents such as vehicles, and are referred to herein as such, although they may be other objects or components.
  • host device 108 is the central node for all activity and control commands sent to agents 104 A through 104F and/ or other components such as accessories 105, 106, whether the commands originate from algorithms running on host device 108 or are routed through host device 108 but originate from control devices 101D through 101K controlled by users 109D through 109K who are physically present or remotely located.
  • agents 104 A through 104F and/ or other components such as accessories 105, 106, whether the commands originate from algorithms running on host device 108 or are routed through host device 108 but originate from control devices 101D through 101K controlled by users 109D through 109K who are physically present or remotely located.
  • a more distributed architecture may be implemented wherein host device 108 need not be the central node for all activity and control commands.
  • Fig. 1 includes a specific number of controllers 101D through 101K, agents 104B through 104H, accessories 105, 106 (which may also be considered a type of agent), AI-controlled vehicles 104J (which may also be considered a type of agent), and other components.
  • controllers 101D through 101K agents 104B through 104H
  • accessories 105, 106 which may also be considered a type of agent
  • AI-controlled vehicles 104J which may also be considered a type of agent
  • system 100 is implemented in a centralized manner, wherein controllers 101D through 101K and agents (including vehicles 104 A through 104F), along with other components, communicate with host device 108.
  • controllers 101D through 101K and agents including vehicles 104 A through 104F
  • agents including vehicles 104 A through 104F
  • host device 108 communicates with host device 108.
  • multiple users 109 or players
  • other agents/ vehicles 104J may be controlled by means of artificial intelligence.
  • any number of external devices may be connected to host device 108 via any suitable communications protocol, such as for example a cellular/ Internet connection 107.
  • the various external devices may or may not be identical to host device 108. Some or all of the external devices serve as player controllers.
  • Fig. 1 depicts various examples of devices that can be used as player controllers, including: game console 101B with any number of controllers 101 J, 101K (controlled by users 109J, 109K, respectively): laptop computer 101D (controlled by user 109D); stand-alone controller 101E (controlled by user 109E); and smartphones 101F, 101G, and 101H (controlled by users 109F, 109G, and 109H, respectively).
  • controllers 101 can be an iPhone or iPad, available from Apple Inc. of Cupertino, California, running a suitable software application ("app"). Controllers 101J, 101K, 101E can be of any suitable type, including for example controllers that are commonly used with console game devices.
  • a game is hosted on host device 108.
  • Host device 108 supports gameplay in physical space in a physical environment (such as a race track) as well as in a virtual environment under the direction of software; the state of the virtual environment is maintained in memory on host device 108 and/ or elsewhere.
  • FIG. 6 there is shown an example of an embodiment of the present invention for implementing a gameplay environment wherein vehicles 104 (race cars) race on a drivable surface 601 (such as a race track), according to one embodiment.
  • vehicles 104 race cars
  • a drivable surface 601 such as a race track
  • the system can be implemented in an entirely different physical environment, with agents other than vehicles, and/ or with different types of tracks or no track at all.
  • drivable surface 601 is, in at least one embodiment, a physical model of one or more roads, and can include objects such as stop signs, traffic lights 105, railroad crossings, and/ or the like.
  • Vehicles 104 are mobile agents capable of independent motion. Vehicles 104 can be physically modeled after cars, trucks, ambulances, animals, or any other desired form.
  • each vehicle includes one or more sensors 604 that can read information from drivable surface 601 and a communication module (not shown) that can send and receive commands and/ or other information to/ from host device 108, for example via wireless means.
  • drivable surface 601 can include any number of segments 602, although such a segmented arrangement is optional. Such segments 602 may connect at specified connection points and can be reconfigured to construct any desired structure. This structure is referred to as drivable surface 601.
  • Road segments 602 include continuous regions that one or more vehicles 104 can navigate called drivable sections and connect to each other end-to-end using a simple click-in mechanism present at each connection point.
  • 602 can also optionally transmit power to an adjacent road piece 602 and can optionally include a microcontroller for advanced functionality, such as traffic lights
  • Drivable surface 601 can also be provided as a single piece.
  • Drivable surface 601 (and/ or segments 602) can be foldable, collapsible, Tollable, or otherwise able to be made compact for storage.
  • base station software running on host device 108, operates a virtual version of the physical game that continuously maintains parity with events in the physical environment by updating stored information relating to vehicle 104 position, direction, velocity and other aspects characterizing game events.
  • host device 108 ensures that at any point in time the game states in the physical environment and the virtual environment are identical (or substantially identical), or at least that the game state in the virtual environment is a representation of the physical state to at least a sufficient degree of accuracy for gameplay purposes.
  • artificial intelligence software runs on host device 108 and issues commands (via wireless communication mechanisms or other mechanisms) to control one or more vehicles 104J operating on track 601.
  • software for controlling vehicles 104J may be located elsewhere, and/ or may run on vehicles 104J themselves.
  • host device 108 can simultaneously serve as a control unit for a human user 109 A controlling a vehicle 104 (in the depicted example, human user 109A uses host device 108 to control vehicle 104A). Such functionality can be provided on host device 108 while host device 108 also serves as a conduit and interpreter for control commands incoming from other devices 101D through 101K controlling other vehicles 104B through 104F. In another embodiment, host device 108 does not serve as a control unit for a human user 109, but rather operates as a dedicated central control unit.
  • agents under user control do not need to be consistent in form or function.
  • users 109 may be given the opportunity to control objects or elements other than vehicles (such as traffic lights, railway crossings, gun turrets, drawbridges, pedestrians, and/ or the like).
  • Player controllers 101D through 101K may communicate directly with host device 108 or they may communicate via intermediary devices. For example, in Fig. 1, controllers 101J and 101K communicate with host device 108 via game console 101B. Similarly, any number of tiers of connections can be configured between player controllers and the host device, such as one or more smart- phones connecting to the host device through a succession of devices networked back to the host device.
  • Fig. 1 depicts an example in which vehicles 104B through 104F are controlled by human users 109B through 109F, respectively. Additional agents, referred to as accessories 105, 106, may also be controlled by human users 109, or they may operate automatically (for example, under the direction of artificial intelligence software running at host device 108 or elsewhere). Each accessory 105, 106 may be a physical or virtual item that can be powered or passive, and that can be used to affect aspects of the gameplay environment and/ or other agents 104 directly. In this example, accessory 105 is a physical traffic light such as shown in Fig. 6.
  • Smart accessories 105 can also exist as passive elements in physical space.
  • An example of a passive smart accessory is a decal intended to represent a potential hazard on drivable surface 601 placed in an arbitrary location at some point before or during play.
  • Such a decal can contain an encoding scheme that permits its unique identification by vehicles 104 (as they pass over it, for instance).
  • a user 109 in control of one or more such passive smart accessories can selectively activate or deactivate the accessories individually or in groups. While passive in the physical environment, in the virtual environment such accessories can represent a range of different features such as, for example:
  • passive accessories 105 may not necessarily indicate a state change in a physical way
  • host device 108 can report changes in virtual state, and can transmit such changes to users 109, via controllers 101, in real time.
  • Behavior of vehicles 104 may also change in response to changes in virtual state of accessories 105.
  • Accessory 106 is an example of a virtual accessory, which has no physical component other than a computing device (such as a smartphone or tablet computer or the like) with an appropriate output device (such as a display screen).
  • Virtual accessory 106 can be physically placed at a particular location in the physical game environment to render the accessory appropriately in both appearance and state.
  • a virtual accessory (tablet 106) that implements a digitally rendered element (an image 503 of a gun turret) within a physical space, according to one embodiment.
  • Image 503 can move and animate so that it appears to interact with physical vehicles 104 in the gameplay area; for example, it can be made to appear to shoot physical vehicle 104, and physical vehicle 104 can be made to react as though it has been shot. These behaviors and interactions mirror the virtual environment, as also shown in Fig. 5, in which turret 501 is shooting at virtual representation 502 of vehicle 104.
  • accessories 105, 106 need not rely on a human user for operation but can operate under the control of artificial intelligence software running on host device 108 and/ or elsewhere.
  • the system is implemented in a distributed environment, wherein, for example, host device 108 has the capacity to distribute portions of its logic to any number of devices to which it is connected and which are capable of supporting execution of said logic. Examples of these include smart- phones, tablet computers, laptops, game consoles, and/ or the like, but can also be any suitable devices capable of providing the necessary support to run the logic assigned to it.
  • some of the processing tasks associated with operating system 100 can be distributed to one or more controllers 101D through 101H.
  • logic can be distributed to, for instance, one or more remotely located servers.
  • a modular design to the structure of host device 108 can lend itself to convenient distribution of logic, and the type of logic processes offloaded from host device 108 need not be of one particular type of function or process.
  • the distribution of logic can be prioritized according to computational and memory demand, such that those most taxing of host device's 108 resources are the first to be allocated elsewhere.
  • the wireless interface employed to communicate with and/ or among controllers 101D through 101H be identical to that used to connect to agents 104A through 104F under the users' 109 control.
  • host device 108 communicates with controllers 101D through 101H via Wi-Fi
  • host device 108 communicates with agents 104A through 104F via Bluetooth.
  • host device 108 can serve as a bridge between a high-power protocol (such as Wi-Fi) and a low-power protocol (such as Bluetooth).
  • Bluetooth in particular Bluetooth Low Energy (BTLE or BLE) or similarly capable wireless protocol
  • agents 104 can use the wireless protocol to communicate with similarly enabled BTLE/ wireless devices.
  • BTLE Bluetooth Low Energy
  • a user 109 wishing to assume control of a particular vehicle 104 or active smart accessory 105 can bring the intended controller 101 (e.g., a BTLE-equipped smartphone) in proximity to the desired vehicle 104.
  • the intended controller 101 e.g., a BTLE-equipped smartphone
  • a user 109 Leveraging BTLE's capability of determining relative distance or proximity to another BTLE-enabled device, a user 109 can bring two BTLE-equipped devices within a threshold range of distance.
  • this can prompt a data exchange between the smartphone (e.g., 101F) and vehicle 104, presenting the user 109 with the option of selecting vehicle 104 for play.
  • the selection is subsequently relayed to host device 108 indicating the pairing between vehicle 104 and the user's 109 smartphone 101, now designated as vehicle's 104 control device.
  • BTLE data exchanges among vehicles 104 and/ or similarly wirelessly-enabled agents can be used in other ways.
  • users or observers can receive information about the status of an agent 104 with respect to gameplay, overall lifetime usage, and/ or historic achievements, and/ or they can perform diagnostics or customize the unit.
  • controllers 101D through 101H can be implemented using any suitable devices. Again, less sophisticated controllers 101J, 101K can be used, such as wireless gamepads or joysticks. In instances in which a gamepad or joystick 101J, 101K is used which is not equipped with a wireless communication module supporting direct communication with host device 108, the connection to host device 108 can be achieved through a game console 101B or other intermediary, or through the use of a dongle (not shown) that plugs into an appropriate port on host device 108. Such a dongle links wirelessly to controller 101 and passes communications through the port into which it is plugged.
  • Alternative embodiments of the dongle can include units that implement a bridge between a wireless protocol compatible with controller 101 and a wireless protocol compatible with host device 108.
  • controllers 101D through 101H can also receive updates from host device 108 that reflect the current state of the game.
  • some or all controllers 101D through 101H can be equipped with one or more output devices (such as a display, speaker, haptic output mechanisms, and/ or the like), so as to be able to enhance the play experience based on such received state information.
  • Such enhancements can include, for example, renderings, haptic output (e.g., vibration) and/ or audio representing action on the game track and/ or augmenting such action to increase the sense of realism or provide details otherwise not visible to a user 109.
  • the visual, haptic, and/ or audio information presented to users 109 through some or all controllers 101D through 101H can be unique to each device.
  • Various agents 104 may be in different states at any given time (with respect to, for example, position, speed, status, action, and the like); in addition, in the case of multiple users 109 controlling a single agent 104, user roles or control may be different. Therefore, the various cues and data presented to each user 109 can be tailored to the current state of the agent 104 and the user's 109 particular role. For example, in at least one embodiment, a racing game might present renderings of the race course from the perspective of the vehicle 104 while displaying vehicle-specific data to the user 109.
  • controllers 101 may be appropriate that the various forms of information transmitted to controllers 101 would differ in one or more aspects for each user 109 associated with said agent 104.
  • controllers 101D through 101H may be physically present, so that controllers 101D through 101H communicate directly (via wireless protocol such as Bluetooth) with host device 108.
  • users of controllers 101D through 101H may be remotely located and connected via a host network (such as network 107). Controllers 101D through 101H may rely on information reported back from host device 108 regarding game status.
  • the tight coupling and parity maintenance between the states of the virtual play environment and the physical play environment enable users 109 to control vehicles 104 on a remotely located track 601.
  • Such an architecture allows users who are not locally present to participate through reliance on the virtual representation of the game.
  • remote users 109 may have the same degree of control of their vehicles 104 as do local users 109, and they may have a view of events that is functionally identical to that of local users 109.
  • spectators can observe game events from a remote location by making use of an external network connection.
  • a spectator, or observer 110 observes game events via tablet 103 that communicates via cellular/ Internet connection 107 with host device 108.
  • tablet 103 or any other device used to display gameplay activity
  • a remote user 109 can additionally use an auxiliary device (not shown) to display the game separate from the controller 101 he or she is using to control a physical agent 104.
  • an auxiliary device not shown
  • the remote user's controller 101 has a small screen, it may be useful to have such an auxiliary device for use as a second screen, thus providing a more complete and immersive game experience.
  • user-specific information can be displayed on one display (such as the display on controller 101), while a visualization of gameplay may be shown on the auxiliary display.
  • Using a separate device or devices as a monitor can alleviate potential display limitations and provide an experience more familiar to console gamers who are accustomed to using a hand-held gamepad to control action that they watch on a separate screen.
  • Host device 108 and/ or other components may provide additional functionality.
  • one or more controller (s) 101 can provide supplemental information related to events occurring in the game, and/ or can serve as a direct platform for additional gameplay.
  • Such a device may have any suitable components to provide a desired level of interactivity, including for example:
  • a user interface which may be responsive to touch, button control, orientation relative to cardinal directions or device tilt/ acceleration, and/ or any other suitable user input;
  • Non- actionable information include, for example, audio, video, and/ or haptic output, separately or in any suitable combination.
  • Specific types of output may include, for example, sound effects, music, vibrations, images, animations, film clips, and/ or the like. Any of these can be used, singly or in any suitable combination, to improve the sense of realism or otherwise enrich the sensory aspects of the game experience.
  • these enhancements can assume the form of data that provides summary or supplemental information regarding aspects of the game and/ or its users 109 such as car performance data, racer rankings by time, competitive standings, and/ or other data.
  • highly interactive game controllers 101 such as smartphones and tablets can be used to provide improvement to the game experience by leveraging the control and connective capabilities that these devices afford to create play scenarios outside the direct action occurring in the physical environment, and / or to permit modifications to aspects of the game or agents operating within it, whether during a game or outside gameplay.
  • a vehicle 104 may pull into a pit stop on the physical racetrack; gameplay may then switch to controller 101 where a user 109 faces a challenge of servicing vehicle 104 as a member of the pit crew in an entirely digitally context more akin to a conventional video game format.
  • a vehicle 104 can drive to a particular point in physical space, such as a building located on a play set, at which point the gameplay switches to the display on the game controller where the vehicle 104 driver's or team's exploits continue inside the building in a digitally rendered environment.
  • a digital environment can be supported by a highly interactive controller 101 and/ or a controller 101 complemented by a separate display console (not shown) that communicates with controller 101 and/ or with host device 108.
  • control can also be provided for virtual accessories 106, such as those hosted by an Android- or iOS-based device played in the physical play environment, which interact with physical agents 104 and/ or affect their behavior.
  • a controller 101 may provide functionality for manipulating a vehicle's 104 speed and steering. Making greater use of the capacity of a smartphone or similarly capable device as controller 101, however, could enable broader game controls.
  • vehicles 104 can be equipped with weapons that can be directed at competing vehicles 104.
  • a highly interactive controller 101 can be used to support functions beyond basic vehicle speed and steering. Additional functions can be provided, such as for example aiming weapons mounted on a vehicle 104 or selecting a target vehicle 104 among an active representation of all vehicles 104 racing on drivable surface 601.
  • multiple users 109 can control a single agent 104 in a game.
  • three users 109 might control a single vehicle 104 in which one user 109 is providing steering and speed control, another user 109 aims and shoots a front- facing gun, and a third user 109 operates a rear-facing gun.
  • controllers 101 provide information tailored to the role filled by each user 109.
  • the user 109 driving vehicle 104 might find the display information and control schemes similar to those of a typical racing game, while the users 109 operating the guns might see a rendered view of the physical play environment in which the displayed perspective is from a position on vehicle 104 and includes virtual representations of track elements and competing vehicles 104 in locations and on trajectories relative to their actual positions in the physical play environment.
  • the resulting control and interactive experience the smart device affords would be similar to a first-person shooter game.
  • the guns need not be (but may be) represented by any physical structure or element on physical vehicles 104 themselves; rather, they may be represented in the virtual environment instead of (or in addition to) having such physical structure.
  • a rendering of such firing may be presented in the virtual environment, and optionally physical vehicles 104 may provide some visual and/ or auditory indications of such weapons activation (such as flashing lights, movement of physical gun barrels, sounds, and/ or the like).
  • the target of the gunfire e.g.
  • another vehicle 104) may respond as if hit, for example by changing its course, flipping over, becoming disabled, and/ or the like; visual feedback may be provided in the virtual and/ or physical environments, for example by the target emitting lights that make it appear as though it is in flames or otherwise disabled. LEDs or other visual and/ or auditory components can be installed on vehicles 104 to provide the muzzle flash and sounds recreating the noise of a machine gun; such output can be synchronized with the firing of the corresponding gun in virtual space.
  • vehicles 104 may be designed to exhibit (simulated) damage in physical space.
  • an LED can indicate that a vehicle 104 is sustaining damage, or the change in color of an LED array can indicate a vehicle's 104 current damage status.
  • more complex approaches may be used to replicate or simulate the damage incurred to a vehicle 104 whether through weapon strikes, impacts or other means. These may include parts that fall away from the vehicle 104 or mechanisms that, when triggered, impart kinetic energy such as might be the result of collision or explosive forces acting on the vehicle 104.
  • vehicles 104 exhibit simulated damage by changing their behavior, for example to simulate a flat tire, damaged steering or engine components, or the like.
  • the system of the present invention maintains parity in a game space that occurs simultaneously in virtual and physical space by reconciling events at a low level, so as to permit bi-directional influence between the physical and the virtual environments.
  • Fig. 2 there is shown an example of such integration of events and functions in both physical space and virtual space.
  • a racing game is implemented in which the mobile agents are vehicles competing in physical space; the vehicles are also equipped with virtual weapons.
  • the Figure depicts a series of events that involve virtual events impacting physical ones and vice versa. While the cars are racing on a physical course, the base station maintains a virtual representation of the race state in real time, so that the position, velocity, acceleration, course and other metrics characteristic of moving vehicles are continuously tracked in a re-creation in memory that mirrors the changing state of the physical world.
  • the virtual representations of the vehicles may have a number of functionalities or characteristics that are not present on the physical cars.
  • One example is a gun that a user 109 can use to shoot at other vehicles in virtual space, but which does not exist in physical space. Because the virtual states and physical states are tightly coupled, events happening in one can affect the state of the other.
  • Vehicle 104K is represented in virtual environment 202 by vehicle representation 204K.
  • vehicle representation 204K shoots a virtual gun at vehicle representation 204L, which represents vehicle 104L in physical environment 201.
  • vehicle 104K in physical environment 201 may have no actual gun that discharges a projectile
  • host device 108 operating the game may determine that, given the relative positions of the vehicle representations 204K, 204L, as well as the orientation of the gun, physics would yield a strike on vehicle representation 204L at position 2 in the Figure.
  • Host device 108 also determines that the strike and resulting virtual energy imparted on impact displaces target vehicle representation 204L from its course in virtual environment 202.
  • the system maintains parity between the virtual and physical environments.
  • control algorithms of host device 108 recreate the virtual displacement of vehicle representation 204L in physical environment 201.
  • physical vehicle 104L is artificially propelled to move in a manner that mimics the displacement of vehicle representation 204L in virtual environment 202.
  • physical vehicle 104L having been struck by a virtual weapon at position 2, is artificially deflected from its current course in physical space. This results in a collision in physical space with another vehicle 104M at position 3.
  • the impact between physical vehicles 104L and 104M at position 3 involves a transfer of real energy that physically displaces vehicle 104M off its original course vector as shown at position 4.
  • host device 108 Upon detection of this change of course in physical environment 201, host device 108 causes virtual environment 202 to be correspondingly adjusted so that vehicle representations 204L and 204M continue to mirror the motion of vehicles 104L and 104M in physical environment 201.
  • vehicle representation 204M is made to veer off course in an identical fashion.
  • the system of the present invention does not necessarily maintain a precise balance of mutual influence across states, but can also be set to maintain a preferred dominant state.
  • the system can be configured so that virtual environment 202 dominates physical environment 201, and physical environment 201 simply mirrors the events occurring in virtual environment 202; in at least one embodiment, the opposite configuration may be implemented. Any appropriate priority scheme can be established between the physical and virtual environments 201, 202.
  • the system of the present invention provides further advantages through the use of a controller 101 and/ or host device 108 that has the capacity to connect to external server networks (not shown), thus providing an improved user experience.
  • a user 109 controlling one or more agents 104 via controller(s) 101 can download virtual goods for use in the game, and/ or can download digital content such as sound effects or light pattern sequences for LEDs active on the agents.
  • Any suitable virtual accessories or digital content can be made available via any suitable e- commerce mechanism such as the downloading of apps and/ or resources.
  • Such content can be made available for download via remote servers or installation through other means that enable users to tailor or improve their experiences.
  • any of a variety of aspects of the game experience can be modified or augmented through such methods, including, for example, personalities for AI-controlled vehicles, new commentators, new scenarios for existing race courses, content schemes for rendering events occurring in virtual space, and/ or customizations and enhancements of the vehicles.
  • host device 108 reconciles commands from users 109 with the operating framework of the game, and also reconciles events occurring in virtual environment 202 with those occurring in physical environment 201 involving physical agents 104.
  • FIG. 3 there is shown a block diagram depicting some aspects of the role of host device 108 (base station) according to at least one embodiment, and illustrates the components and structures that can be implemented between users 109 controlling vehicles 104 (using controllers 101) and vehicles 104 themselves.
  • Fig. 3 also illustrates some mechanisms by which, in at least one embodiment, users 109 can cooperate to control a vehicle 104, either with other users 109, or assisted by artificial intelligence (AI) planners 305 A, 305B running on host device 108, or both.
  • AI artificial intelligence
  • four users 109M through 109Q use controllers 101M through 101Q to control four agents (three vehicles 104M, 104N, 104P and one smart accessory 105).
  • Two AI planners 305A, 105B are provided as well. Between the controllers 101 and the agents 104M, 104N, 104P, 105 are a number of structures operating within the framework of host device 108 (base station). Arrows indicate the direction of information flow. [0072] For illustrative purposes, and for clarity, Fig. 3 does not necessarily describe all structures or the complete flow of information, but rather emphasizes information relevant to vehicle control only. For example, information is shown as traveling in one direction from controllers 101 to filters 301. One skilled in the art will recognize that the control mechanisms used in connection with the present invention can also include the transfer of additional data not depicted in Fig. 3, including data sent from host device 108 to various other components concerning, for example, race performance reporting, sounds, images, animations, and/ or other non-control information.
  • vehicle commands originating from a user 109 are sent via controller 101 to a filter 301; filter 301 can refine the instructions received from controller 101 according to current circumstances concerning the vehicle 104 under the user's 109 control or other aspect of the game.
  • a velocity filter 301 can be implemented. Under normal circumstances, a vehicle 104 can have a permissible range of speeds at which it can travel. A velocity filter 301 can reduce this range by eliminating the upper end of this range, for example if the vehicle 104 had suffered (virtual) damage to it as part of gameplay.
  • filters 301 can also be provided, so that each vehicle 104 may have a number of different filters 104 affecting different facets of its performance; filters 104 can control or modify such parameters as velocity, lane changing rate, and/ or equipment usage (e.g., ammunition in an onboard weapon), and/ or the like.
  • an effects system 302 is provided, which may send input directly related to specific agents 104M, 104N, 104P, 105. Effects system 302 operates as a global actor, having broad capacity to influence vehicle behavior. It may do so in any of several ways. In various embodiments, effects system 302 simulates the potential consequences of interplay of vehicles 104 with one another, as well as external factors that may affect the performance of one or more vehicles 104. [0075] In at least one embodiment, a game engine 303 is provided, which contains rule sets and scenarios, and which generally guides play. Game engine 303 is tightly bound to effects system 302, often triggering effects.
  • effects system 302 can inform game engine 303 of events that have initiated actions on its part and which may bear consequence to the course of events as determined by game engine 303.
  • Fig. 3 omits lines connecting users 109 to effects system 302; however, one skilled in the art will recognize that actions taken by users 109 can directly or indirectly trigger effects system 302.
  • a user 109 can deploy a virtual oil slick behind his or her vehicle 104 (more accurately, since the oil slick does not exist in physical space, it is actually deployed behind the vehicle representation 204 in virtual environment 202).
  • a trailing vehicle 104 passing over the virtual oil slick would experience a (presumably temporary) loss of control; this is implemented, for example, by the effects system 302 reducing or eliminating (temporarily) the ability of user 109 controlling the trailing vehicle 104 to steer or to brake.
  • An example of an indirect trigger to effects system 302 is as follows: in at least one embodiment, if a user 109 crashes his vehicle 104 or that of another user 109 such that, under typical auto racing rules, game engine 303 indicates to effects system 302 that yellow flag conditions are in effect, effects system 302 can put speed limitations into place according to the parameters of the yellow flag conditions. Such limitations may remain in place, for example, until game engine 303 subsequently lifts the yellow flag conditions after the track has been cleared.
  • an arbiter 304 effectively issues the direct instructions to agents, whether vehicles 104 or accessories 105.
  • arbiter 304 optimizes communications directly to the agent or agents 104M, 104N, 104P, 105 to which it is connected.
  • arbiter 304 can share some of filter's 301 role in moderating or altering user commands.
  • arbiter 304 can serve to reduce or eliminate redundant commands coming from a user 109.
  • Arbiter 304 is also valuable in situations such as that depicted for vehicle 104N, in which two users 109N, 109P and an AI planner 305A are controlling a single vehicle 104N. This is an example of a situation in which more than one user 109 is controlling a vehicle 104.
  • a vehicle 104 may be equipped with weapons that permit aiming and firing, with one user 109 controlling vehicle 104 and the second user 109 operating its weapons.
  • arbiter 304 integrates and sequences the separate command sets appropriate to execute action(s) in agent 104 as desired by the separate users 109.
  • a user 109 can pair with an AI planner 305, so that AI planner 305 is the driver and user 109 operates vehicle's 104 weaponry, or vice versa.
  • AI planner 305 The capacity to rely on AI planner 305 to assist a user 109 controlling a vehicle 104 offers a number of benefits even in situations where a single user 109 might otherwise have the capability of operating all the systems on a vehicle 104.
  • planner 305 can be relied upon to provide steering or to smartly maneuver around slower opponents on a course, leaving a user 109 to control vehicle's 104 speed.
  • planner 305 can provide assistance to users 109 in their control of a vehicle 104, and it may do so in a number of different ways according to the game rules and/ or user 109 preference.
  • planner 305 assuming steering control of a vehicle 104 can be reversed, such that planner 305 controls velocity while user 109 retains control of steering.
  • a user 109 may be comfortable with steering and controlling speed on an open course, but less confident about passing or avoiding obstacles on a crowded course; the system can thus be configured to prompt planner 305 to intercede when passing situations or obstacles requiring active avoidance are encountered.
  • a single user 109 may control multiple agents 104 (which may include vehicles 104 and/ or accessories 105).
  • a user 109 might control one agent 104 while a series of others are configured to follow said agent.
  • user 109 commands might be duplicated across a number of agents 104 such that they respond in unison (e.g., all changing direction or speed simultaneously).
  • a user 109 may provide higher-level commands to a group of agents 104.
  • multiple vehicles 104 under such control may respond to a user's 109 high-level goal-oriented direction, such as to crash another user's vehicle 104 off the course.
  • agents 104 might position themselves around the targeted vehicle 104 so as to surround it partly or fully and together push it off the course.
  • arbiter 304 moderates the separate and potentially independent commands intended for vehicle 104, selecting among them and/ or sequencing them in its relay to vehicle 104.
  • arbiter 304 moderates the separate and potentially independent commands intended for vehicle 104, selecting among them and/ or sequencing them in its relay to vehicle 104.
  • Numerous aspects of behavior can be imparted to AI planner 305 controlling vehicle 104.
  • AI planners 305 can embody personality characteristics that influence their general behavior.
  • AI planners 305 can simulate aggressive drivers that are inclined toward forcing other vehicles 104 off drivable surface 601 or, alternatively, drivers that avoid conflict and instead focus on navigating a course in avoidance of other users 109.
  • AI schemes can initiate messages to other users 109 to, for example, taunt other drivers, either in association with particular events occurring on drivable surface 601 or in keeping with an intended character disposition.
  • some capacity for assistive control resides on vehicles 104 themselves.
  • vehicles 104 can have the capacity to maintain their relative lateral position on drivable surface 601 through curves and turns without requiring active steering from a user 109.
  • vehicles 104 can have the capacity to moderate their speed throughout a course according to velocities that are prescribed as part of the track identification (for example, encoded in readable form on track segments 602). It is also possible, by the same means of localization through recognition of information encoded on segments 602, for vehicles 104 to determine that they have left the course or are otherwise not on the course by the absence of such information in their field of detection for said information.
  • vehicle 104 it may still be possible for user 109 to control vehicle 104, though such control can be limited to a subset of the total capabilities of vehicle 104 when localized on drivable surface 601.
  • vehicle's 104 response to user 109 control may be similar to that of a conventional remote-controlled car.
  • host device 108 can control the game in other ways apart from actual gameplay.
  • host device 108 can control various features of vehicles 104 that operate in a physical environment as defined in software. These features can include, for example, performance characteristics and vehicle capabilities.
  • a vehicle's acceleration profile, top speed, handling and other physical aspects of its performance can be governed in software rather than dictated by the physical limits that exist for it in the real world.
  • the system takes into account how those definitions bear consequences in gameplay in both physical and virtual spaces. Differences among Agents in Virtual Environment
  • vehicles 104 may be advantageous to manufacture vehicles 104 with identical hardware components, which enabling differences in how vehicles 104 drive, behave, and respond.
  • one vehicle 104 may be intended to perform like a high- performance sports car while another may be a heavy truck with poor acceleration but high inertia or light-weight compact car with a tight turning radius, and so on.
  • differences may or may not be visually suggested by different bodies or covers that fit onto the chassis (but which may be solely cosmetic and, in at least one embodiment, do not physically affect the performance characteristics of the vehicle in any meaningful way).
  • software such traits can be ascribed easily to the vehicles 104, so as to implement a system wherein the underlying capacities of the physical agents are identical, while the vehicles 104 perform and respond according to the characteristics ascribed to them in the virtual space.
  • the system of the present invention uses the physics of vehicle representations 204 and their interactions in virtual environment 202 to control movement and behavior of vehicles 104 in physical environment 201.
  • two vehicles 104 having identical or similar physical mobility profiles may be provided, so that their weight, acceleration, maneuvering capabilities and like metrics are the same or comparable one another in the physical world.
  • the primary difference between the vehicles 104 is that, in the virtual environment 202, one is intended to represent an armored tank (slow and very heavy) and the other represents a sedan (fast and light).
  • some visual distinction can be made to physical vehicles 104 to reinforce the difference in the characteristics of the corresponding virtual representations 204; for example, body covers on the chassis can have different markings, designs, and/ or colors to reflect the difference in virtual characteristics.
  • FIG. 4 there is shown an example in which body covers on vehicles 104R, 104S in physical environment 201 are used to reflect a difference in characteristics of corresponding vehicle representations 204R, 204S in virtual environment 202, thus reinforcing the relationship between the physical and virtual environments 201, 202, according to one embodiment.
  • Vehicle 104S is given a cover to make it look like a tank, while vehicle 104R is made to look like a sedan.
  • vehicle representations 204R, 204S are configured to interact with one another in a manner that simulates and reflects the fact that a full-sized, heavily armored vehicle such as a tank may outweigh a full- sized automobile by several times, even though physical vehicles 104R, 104S are, in actuality, comparable in mass and size. While both vehicles 104R, 104S depicted in Fig. 4 would operate in physical environment 201, in order to maintain consistency in their interactions, the governing moving body dynamics in a collision are those defined in virtual environment 202. [0091] In the example of Fig.
  • vehicle 104R (representing a sedan) is traveling at a high speed on a path that causes it to collide with vehicle 104S (representing a tank).
  • vehicle 104S (representing a tank).
  • the physical and virtual environments 201, 202 are tightly coupled to one another. Because the mass properties of the two physical vehicles 104R, 104S are identical or closely similar, Newtonian mechanics would dictate that a collision at position 2 would result in an outcome as shown at position 3, in which vehicle 104S (representing the tank) is displaced and rotated and vehicle 104R (representing the sedan) continues traveling on a course similar to its pre-crash heading, albeit at a diminished speed, since the collision involved a transfer of energy to vehicle 104S.
  • the system maintains consistency with the properties ascribed to the two vehicles' representations 204R, 204S in virtual environment 202, so that the results of the impact of the two vehicles 104R, 104S in the physical environment 201 are artificially made to follow the physics of the collision in the virtual environment 202.
  • vehicle representation 204S since the mass of vehicle representation 204S is significantly larger than that of vehicle representation 204R, the result shown at position 4 results, wherein vehicle representation 204R (the sedan) effectively bounces off of vehicle representation 204S (the tank), while having little effect on the momentum and position of vehicle representation 204S As shown in position 5 of physical environment 201, physical vehicles 104R, 104S are artificially made to follow the trajectories of corresponding vehicle representations 204R, 204S, thus extending the simulation to the physical space.
  • vehicle representation 204R the sedan
  • vehicle representation 204S the tank
  • vehicle 104R experiences a significantly greater alteration of speed and heading since the inertia of vehicle representation 204S dominates the interaction. If left to the actual physics of the collision between the actual vehicles 104R, 104S, the result would be a collision of equals, with the sedan displacing the tank in a way that would not be realistic considering the mass differences between an automobile and an armored military vehicle. Accordingly, in at least one embodiment, in order to maintain the differences ascribed to the vehicles in virtual space, the consequences of a collision are dictated by the vehicle parameters defined in software and the physics governing interactions in the virtual environment.
  • the specific details and sequence of events in the collision are determined according to algorithms governing motion in virtual environment 202.
  • coherence is maintained between the physical and virtual environments.
  • One possible approach to enforce such coherence is to allow the algorithms determining the results of the collision in virtual environment 202 to directly influence the control of vehicles 104 in physical environment 201, so as to cause vehicles 104 to move through a collision event in the same (or similar) manner as do vehicle representations 204.
  • the system of the present invention enforces protocols regarding priority between the virtual and physical environments 202, 201, so as to maintain a desired degree of parity between events happening in virtual environment 202 and those in physical environment 201; in this manner, the system of the present invention provides bi-directional influence of events, as described above.
  • vehicles 104R, 104S in physical environment 201 respond in tandem with their counterparts in virtual environment 202.
  • Vehicles 104R, 104S thereby perform and respond according to parameters defined in virtual environment 202 and according to laws of physics as applied to such parameters in virtual environment 202.
  • a collision occurs in both the virtual and real environments 202, 201. Since the mass properties assigned to vehicle representations 204 in virtual environment 202 do not match those of physical vehicles 104, the outcome of the collision would be different in the two environments.
  • the operating protocol gave priority to the physics of impact as determined within virtual environment 202; the physical vehicles 104 were made to respond in accordance with those physics. In other situations, however, it might be desir- able to give priority to the physics and behaviors of the physical environment 201, and to cause the virtual environment 202 to be adjusted accordingly.
  • any set of rules can be defined to govern the motion of virtual bodies in collision or any other aspect of movement otherwise dictated by the physics of the real world.
  • Other instances might include, for example, frictionless surfaces in parts of virtual environment 202, causing physical vehicles 104 driving over the corresponding portions of drivable surface 601 to lose the ability to steer or to stop and to drift according to inertia; or simulated inclines where vehicle speed and acceleration capacity is diminished as if ascending even if physical drivable surface 601 remains flat.
  • Virtual forces of numerous types can be introduced arbitrarily and can influence the motion of vehicles 104 differently than would the forces of the real world acting solely according to real-world physics.
  • the system of the present invention can simulate and implement behaviors that do not follow real-world laws of physics, but that may follow other rules.
  • the system can be configured to correct for such errors, based on an enforced priority scheme that specifies whether the errors should be corrected in favor of the physical or virtual environments.
  • the system of the present invention may establish protocols to assure parity is maintained between the physical and the virtual environments. It may be most convenient to understand how these protocols operate to assure parity between the two environments through an examination of the processes governing localization and control of vehicles 104 across both environments.
  • vehicle 104 will have no operable information regarding the nature of the track layout or its position on its course. It is also possible that host device 108 which controls the events occurring in virtual environment 202 will have no information regarding drivable surface 601 that vehicle 104 is on or vehicle's 104 current location on it.
  • means are provided to enable a user 109 to provide track and location information as part of a setup process, either by direct input or through a means that enables host device 108 to determine it, such as a digital photo of drivable surface 601 that includes vehicle 104.
  • the information for establishing track and position can be gathered by vehicle 104 itself.
  • vehicle 104 can drive along the course for a distance sufficient for it to read the data regarding the identification of a particular segment 602 of drivable surface 601 and the identification of vehicle's 104 location on the segment 602.
  • vehicle 104 Once vehicle 104 has obtained this information, the information is communicated to host device 108, which uses it to reference an index of segments 602 to match the unique segment 602 identification to a segment 602 and the location identification. Host device 108 thereby generates an approximation of vehicle's 104 position on drivable surface 601.
  • Identification of segment 602 by host device 108 further yields a set of data regarding vehicle's 104 operable limits, which may in turn be communicated to vehicle 104.
  • This data may be a combination of localization information (e.g., in the case of a race track, vehicle's 104 position along the track and its horizontal offset from the track's centerline) and data regarding operating limits such as the maximum permissible speeds along various sections of the track or whether turning is permitted at intersections on the track.
  • a benefit of providing such data to vehicle 104 is that it enables vehicle 104 to maintain, at a low level, some ability to operate independently of commands from host device 108, such as maintaining its lateral position on a course and moderating its speed in response to changes in course geometry.
  • host device 108 can employ a motion model to track vehicle's 104 position on an ongoing basis from the initial state established by vehicle's 104 position in physical space. Recognizing the aforementioned sources of error that yield differences between vehicle's 104 actual position and those predicted by a motion model in virtual environment 202, vehicle 104 in physical environment 201 may provide updates of its location via regular communication with host device 108.
  • the update data may include, for example, current information regarding roadpiece ID and location ID, as well as direct location data such as the precise lateral position of vehicle 104 on the track. Using this data either directly or as inputs to calculations that serve to update virtual environment 202, host device 108 maintains coherence between the movement of vehicle 104 in physical environment 201 and movement of corresponding vehicle representation 204 in virtual environment 202.
  • the motion of physical vehicle 104 takes priority over that of corresponding vehicle representation 204, since the updates are based on the characteristics of motion of physical vehicle 104 and provided to the virtual model.
  • Other embodiments may operate with the reverse protocol.
  • the motion of virtual vehicle representation 204 may have priority, and the updates can be made to assure that physical vehicle 104 is matching its correspondent's movement.
  • Such circumstances may include scenarios other than live operation of vehicle 104 or vehicles 104 or even ones in which the events unfolding in virtual environment 202 are based on a model motion.
  • An example might be one in which vehicle movements and interactions in a previously played game are captured and stored for subsequent replay.
  • recreating that gameplay in physical environment 201 can involve replaying the recorded events in a virtual form that guides corresponding physical vehicle 104 or vehicles 104.
  • priority may depend on circumstances and how the assignment of priority in any given situation supports the underlying structure and goals of gameplay. In other embodiments, priority might not be assigned to one side or the other, but rather the system might seek compromise between them in a state that is partly between what one or the other might separately dictate. In such cases, behavior and movement in both virtual and physical environments 202, 201 can be adjusted to conform to the compromise state.
  • the maintenance of two states in parity, one physical and one virtual allows for capture, storage, distribution, and/ or subsequent recreation of gameplay or entire games.
  • the ability for host device 108 to make adjustments to physical environment 201 to match virtual environment 202 can also be applied to re-creating past games or running of se- quences of play recorded or altogether fabricated and replayed as a progression of events and actions occurring in virtual environment 202 and directly duplicated in physical environment 201.
  • the action in the re-creation of a recorded game scenario or during the course of a game played live, the action can be accompanied by a commentator providing narration of the events.
  • the coordinated sequences in virtual environment 202 can enable an automated commentator to provide relevant information regarding the changing state of action in gameplay (as well as color commentary).
  • agents 104 can take a number of forms, whether under direct user control or that of an AI planner 305 or some combination thereof.
  • a virtual accessory 106 can be provided, to implement a digitally rendered element in physical space.
  • FIG. 5 there is shown an example of a virtual accessory 106 in the context of the invention embodied in a racing and shooting game.
  • virtual accessory 106 represents an actively aimed and fired gun turret 503. Similar to other agents in the game space, virtual accessory 106 maintains a presence in both the physical and virtual environments 201, 202. For the physical presence, however, a host device provides the rendering of accessory 106.
  • a host device provides the rendering of accessory 106.
  • the depiction of turret 503 is provided by a tablet 504 (such as an iPad, although any other electronic device can be used), which displays gun turret 503 in the position and in the state corresponding to its presence 501 in virtual environment 202.
  • virtual accessory 106 may be integrated into the gameplay through placement of tablet 504 at a designated position and/ or orientation marked on the medium. Alternatively, any arbitrary placement may be permitted relative to the physical game space.
  • a tablet 504 equipped with a back-facing camera can be used to host virtual accessory 106. During placement of tablet 504, the camera can be switched on and recording video. As tablet 504 is positioned, the shifting perspective of the physical game space provides input data from which the tablet's position and orientation relative to the other elements of the game space can be deduced. This approach eliminates restriction on placement of tablet 504 with respect to position and orientation.
  • the rendering of virtual accessory 106 can be done against a background image of that portion of the printed media it overlays. This may be achieved by host device 108 providing an image of the covered portion of the media directly to tablet 504 or by tablet 504 taking an image of the footprint area beneath it during placement. In the latter case, this approach is useful if the virtual accessory 106 is intended for placement beyond the game space such as that defined by the geometric limits of a printed medium.
  • a virtual accessory 106 in terms of its integration into gameplay is similar to other agents 104.
  • vehicle representation 204T passes near gun turret representation 501 in virtual environment 102; correspondingly, physical vehicle 104T passes gun turret 503 rendered on virtual accessory 106 (tablet 504) in physical environment 201.
  • gun turret representation 501 discharges, resulting in an impact with vehicle representation 204T.
  • the full shot and impact are visible.
  • rendering of discharge from the turret gun 503 may be limited to the screen of virtual accessory 106, in this case tablet 504.
  • the result of vehicle 104T and vehicle representations 204T being struck would be same, with virtual vehicle representation 204T experiencing damage and, in this instance, a loss of control leading to its veering off course.
  • Corresponding vehicle 104T in physical environment 201 may have the capacity to display the impact through sounds, and/ or by illuminating LEDs and/ or other means.
  • vehicle's 104T movement and behavior mirror the resulting loss of control as seen in virtual environment 202.
  • the present invention can be implemented as a system or a method for performing the above-described techniques, either singly or in any combination.
  • the present invention can be implemented as a computer program product comprising a nontransitory computer-readable storage medium and computer program code, encoded on the medium, for causing a processor in a computing device or other electronic device to perform the above-described techniques.
  • Certain aspects of the present invention include process steps and instructions described herein in the form of an algorithm. It should be noted that the process steps and instructions of the present invention can be embodied in software, firmware and/ or hardware, and when embodied in software, can be downloaded to reside on and be operated from different platforms used by a variety of operating systems.
  • the present invention also relates to an apparatus for performing the operations herein.
  • This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computing device.
  • a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, flash memory, solid state drives, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus.
  • the computing devices referred to herein may include a single processor or may be architectures employing multiple processor designs for increased computing capability.
  • the present invention can be implemented as software, hardware, and/ or other elements for controlling a computer system, computing device, or other electronic device, or any combina- tion or plurality thereof.
  • an electronic device can include, for example, a processor, an input device (such as a keyboard, mouse, touchpad, trackpad, joystick, trackball, microphone, and/ or any combination thereof), an output device (such as a screen, speaker, and/ or the like), memory, long-term storage (such as magnetic storage, optical storage, and/ or the like), and/ or network connectivity, according to techniques that are well known in the art.
  • Such an electronic device may be portable or nonportable.
  • Examples of electronic devices that may be used for implementing the invention include: a mobile phone, personal digital assistant, smartphone, kiosk, server computer, enterprise computing device, desktop computer, laptop computer, tablet computer, consumer electronic device, television, set-top box, or the like.
  • An electronic device for implementing the present invention may use any operating system such as, for example: Linux; Microsoft Windows, available from Microsoft Corporation of Redmond, Washington; Mac OS X, available from Apple Inc. of Cupertino, California; iOS, available from Apple Inc. of Cupertino, California; and/ or any other operating system that is adapted for use on the device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Toys (AREA)
  • Information Transfer Between Computers (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)

Abstract

La présente invention concerne l'intégration d'un ou de plusieurs dispositifs informatiques mobiles dans un système robotique. Selon l'invention, des configurations physiques de composants individuels du système dans un espace physique, ou d'agents, sous la commande d'un utilisateur ou d'utilisateurs, sont dupliquées en une représentation dans un espace virtuel. Un certain degré de parité en temps réel est maintenu entre les espaces physique et virtuel de façon à définir un environnement virtuel qui reflète l'environnement physique. Des événements se produisant dans un environnement peuvent influencer directement le cours des événements se produisant dans l'autre environnement et avoir des conséquences sur ce dernier. Les éléments de l'espace virtuel deviennent donc réellement interdépendants et unifiés, sur un pied d'égalité, avec les éléments de l'espace physique. Dans au moins un mode de réalisation, le système de la présente invention est mis en œuvre en tant qu'application de divertissement, telle que la manifestation d'un jeu vidéo dans un espace physique.
EP13834203.5A 2012-08-27 2013-08-09 Intégration d'un ou de plusieurs dispositifs informatiques mobiles dans un système robotique Withdrawn EP2888712A4 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261693687P 2012-08-27 2012-08-27
PCT/US2013/054388 WO2014035640A1 (fr) 2012-08-27 2013-08-09 Intégration d'un ou de plusieurs dispositifs informatiques mobiles dans un système robotique

Publications (2)

Publication Number Publication Date
EP2888712A1 true EP2888712A1 (fr) 2015-07-01
EP2888712A4 EP2888712A4 (fr) 2016-09-28

Family

ID=50184133

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13834203.5A Withdrawn EP2888712A4 (fr) 2012-08-27 2013-08-09 Intégration d'un ou de plusieurs dispositifs informatiques mobiles dans un système robotique

Country Status (10)

Country Link
EP (1) EP2888712A4 (fr)
JP (2) JP6067120B2 (fr)
KR (1) KR101793189B1 (fr)
CN (1) CN104662578B (fr)
AU (1) AU2013309312B2 (fr)
CA (1) CA2882099C (fr)
DE (1) DE112013004190T5 (fr)
GB (1) GB2519903A (fr)
HK (2) HK1207459A1 (fr)
WO (1) WO2014035640A1 (fr)

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2482119B (en) 2010-07-19 2013-01-23 China Ind Ltd Racing vehicle game
CN106292356B (zh) * 2015-06-09 2019-03-29 联想(北京)有限公司 一种机器人控制方法及电子设备
DE102015121892A1 (de) * 2015-12-15 2017-06-22 Viessmann Modelltechnik Gmbh Fahrregler für eine Modelleisenbahn, Regelvorrichtung und Modelleisenbahn
CN108780612B (zh) * 2016-02-02 2021-03-26 德卡产品有限公司 模块化机电实体
CN106474743B (zh) * 2016-12-28 2022-05-06 赵红领 智能轨道扫码竞速玩具车的控制方法
US10686883B2 (en) * 2017-07-07 2020-06-16 Hewlett Packard Enterprise Development Lp Bluetooth low energy devices
CN107368029B (zh) * 2017-08-24 2019-08-27 关磊 重力赛车射击系统
JP7064513B2 (ja) 2018-01-16 2022-05-10 株式会社ソニー・インタラクティブエンタテインメント ロボット装置、ロボット装置の制御方法およびプログラム
JP6961723B2 (ja) 2018-01-16 2021-11-05 株式会社ソニー・インタラクティブエンタテインメント 移動体および移動体制御方法
WO2019142228A1 (fr) * 2018-01-16 2019-07-25 株式会社ソニー・インタラクティブエンタテインメント Dispositif de traitement d'informations et procédé de génération d'image
SG10201804153SA (en) * 2018-05-16 2018-11-29 Voyager Innovations Inc System and method for determining an outcome associated with an event
CN108854070A (zh) * 2018-06-15 2018-11-23 网易（杭州）网络有限公司 游戏中的信息提示方法、装置及存储介质
WO2020032207A1 (fr) 2018-08-10 2020-02-13 川崎重工業株式会社 Dispositif de traitement d'informations, système d'exploitation de robot, et procédé de fonctionnement de robot
CN109782737A (zh) * 2019-01-28 2019-05-21 平安科技（深圳）有限公司 无人车模拟测试方法和系统
GB201906813D0 (en) * 2019-05-16 2019-06-26 Roboraca Ltd Metaverse
KR102324845B1 (ko) * 2019-10-02 2021-11-11 (주)케이시크 사용자 게임 연계 자율주행 방법 및 시스템
WO2023031155A1 (fr) * 2021-08-30 2023-03-09 Lego A/S Système et procédé de réalité augmentée pour un objet mobile du monde réel
WO2023186290A1 (fr) * 2022-03-30 2023-10-05 Telefonaktiebolaget Lm Ericsson (Publ) Agencement de module logiciel informatique, agencement de circuits, agencement et procédé pour fournir un affichage virtuel pour un affichage simultané de représentations d'objets de vie réelle à différents emplacements physiques

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10244A (en) * 1853-11-15 Improvement in uniting shovel-blades to the handle-straps
JP2001022264A (ja) * 1999-07-12 2001-01-26 Sony Corp シミュレーション装置
US8160994B2 (en) * 1999-07-21 2012-04-17 Iopener Media Gmbh System for simulating events in a real environment
DE69937058T2 (de) * 1999-10-26 2008-05-29 Sony France S.A. Verfahren und System zur Agentsübertragung für einen Roboter
US6491566B2 (en) * 2001-03-26 2002-12-10 Intel Corporation Sets of toy robots adapted to act in concert, software and methods of playing with the same
GB2385238A (en) * 2002-02-07 2003-08-13 Hewlett Packard Co Using virtual environments in wireless communication systems
US7090576B2 (en) * 2003-06-30 2006-08-15 Microsoft Corporation Personalized behavior of computer controlled avatars in a virtual reality environment
US7704119B2 (en) * 2004-02-19 2010-04-27 Evans Janet E Remote control game system with selective component disablement
US20060223637A1 (en) * 2005-03-31 2006-10-05 Outland Research, Llc Video game system combining gaming simulation with remote robot control and remote robot feedback
US8287372B2 (en) * 2006-09-28 2012-10-16 Mattel, Inc. Interactive toy and display system
KR100842566B1 (ko) * 2007-02-01 2008-07-01 삼성전자주식회사 휴대단말기의 움직임을 이용하여 로봇을 제어하기 위한방법 및 장치
GB2449694B (en) * 2007-05-31 2010-05-26 Sony Comp Entertainment Europe Entertainment system and method
US20090265642A1 (en) * 2008-04-18 2009-10-22 Fuji Xerox Co., Ltd. System and method for automatically controlling avatar actions using mobile sensors
CN101732858B (zh) * 2008-11-11 2013-10-30 盛乐信息技术（上海）有限公司 现实与虚拟结合的联网游戏系统及实现方法
DK2435149T3 (en) * 2009-05-28 2015-09-21 Anki Inc Distributed system for autonomous control of toy cars
US20120190453A1 (en) * 2011-01-25 2012-07-26 Bossa Nova Robotics Ip, Inc. System and method for online-offline interactive experience

Also Published As

Publication number Publication date
DE112013004190T5 (de) 2015-07-16
HK1207460A1 (en) 2016-01-29
EP2888712A4 (fr) 2016-09-28
CN104662578A (zh) 2015-05-27
JP6154057B2 (ja) 2017-06-28
HK1207459A1 (en) 2016-01-29
JP2015533534A (ja) 2015-11-26
AU2013309312B2 (en) 2017-04-20
WO2014035640A1 (fr) 2014-03-06
CA2882099A1 (fr) 2014-03-06
GB2519903A (en) 2015-05-06
KR20150046302A (ko) 2015-04-29
CA2882099C (fr) 2017-10-24
KR101793189B1 (ko) 2017-11-02
AU2013309312A1 (en) 2015-03-05
CN104662578B (zh) 2019-01-01
JP2017080455A (ja) 2017-05-18
JP6067120B2 (ja) 2017-01-25
GB201503471D0 (en) 2015-04-15

Similar Documents

Publication Publication Date Title
US10874952B2 (en) Virtual representation of physical agent
US9067145B2 (en) Virtual representations of physical agents
AU2013309312B2 (en) Integration of a robotic system with one or more mobile computing devices
US20200254353A1 (en) Synchronized motion simulation for virtual reality
EP3856380B1 (fr) Attraction de manège à réalité augmentée et virtuelle modulaire
EP2105178B1 (fr) Dispositif de divertissement et jouet commandé à distance
JP2020511218A (ja) ゲームプレー型乗り物車両システム及び方法
JP6615910B2 (ja) シミュレータ乗り物
US9463379B1 (en) Ride vehicle mounted interactive game system
US20190105572A1 (en) Drivable vehicle augmented reality game
KR101097761B1 (ko) 완구용 로봇 제어 시스템 및 이동통신단말기를 통해 제어되는 완구용 로봇
JP6957218B2 (ja) シミュレーションシステム及びプログラム
CN106457047A (zh) 交互游戏地面系统和方法
EP4114543B1 (fr) Systèmes et procédés de traitement pour robot de spectacle cartographié par projection réactive
US20220083055A1 (en) System and method for robot interactions in mixed reality applications
JP2022530727A (ja) 仮想レーシングカーの制御方法と装置、デバイスおよびコンピュータプログラム
JP2024509342A (ja) 別個の装置を使用してインテリジェント車両を操作するための装置、システム、および方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20150302

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1207460

Country of ref document: HK

RA4 Supplementary search report drawn up and despatched (corrected)

Effective date: 20160826

RIC1 Information provided on ipc code assigned before grant

Ipc: A63F 13/65 20140101AFI20160822BHEP

Ipc: A63F 13/573 20140101ALI20160822BHEP

17Q First examination report despatched

Effective date: 20180921

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20190402

REG Reference to a national code

Ref country code: HK

Ref legal event code: WD

Ref document number: 1207460

Country of ref document: HK