GB2510811A

GB2510811A - Augmented reality systems

Info

Publication number: GB2510811A
Application number: GB1222786.4A
Authority: GB
Inventors: Ian Saunter
Original assignee: Optricks Media Ltd
Current assignee: Optricks Media Ltd
Priority date: 2012-12-18
Filing date: 2012-12-18
Publication date: 2014-08-20
Also published as: GB201222786D0

Abstract

An augmented reality game, comprising at least one playing piece and an electronic device with a touch screen and camera. The playing piece has a plurality of visually distinguishable orientations and, a plurality of touch patterns for identifying the orientation of the piece when the piece touches the touch screen. The electronic device includes game play software to detect the orientation of the playing piece via its camera and display an augmented reality representation, it also detects the orientation of said playing piece (or another) when it is touched to the touch screen of the device and controls the game play of the game in response to the orientation of the piece, changing the augmented display. The piece may include conductive material to work with a capacitative touch screen. The playing piece may be 3-D with a number of facets all being visually or touch distinguishable. There may be an arena with at least one mount for an electronic device. Further disclosed is a method of controlling the augmented reality game on an electronic device, with a touch screen and a camera.

Description

Augmented Reality Systems

FIELD OF THE INVENTION

This invention relates to apparatus, methods and computer program code for implementing augmented reality systems, in particular systems in which physical items interact with software and hardware of electronic devices such as smart mobile phones. Embodiments of the techniques we describe have applications, for example, for implementing games whose apparatus resides in a combination of physical objects which the player or players manipulate, and software.

BACKGROUND TO THE INVENTION

Broadly speaking augmented reality techniques combine real-world information, generally an image captured by a camera of the physical, real-world, with computer-generated subject matter, typically an image, which may be in 3D and/or animated.

The combination is displayed on the screen of an electronic device such as a mobile phone giving the impression that the computer-generated imagery, for example a character, is living within the real-world. The end result can bring a magical combination of virtual and real-worlds to life.

It is known to use augmented reality in smart phone-based games. However these are limited and improvements are desirable.

SUMMARY OF THE INVENTION

According to the present invention there is therefore provided an augmented reality game, the game comprising: at least one playing piece having a plurality of visually distinguishable orientations and a plurality of separately visually identifiable features for identifying a said orientation of said playing piece when the playing piece is on a playing field; and an electronic device with a touch screen and a camera; wherein said playing piece further comprises a plurality of touch patterns for identifying a said orientation of said playing piece to said electronic device when said playing piece is touched to said touch screen; and wherein said electronic device includes gameplay software to: detect said playing piece using said camera when said playing piece is on the playing field; display an augmented reality representation of said playing piece on said touch screen; detect a touch orientation of the or another said playing piece on said touch screen; and control gameplay of said game in response to said detected touch orientation, wherein said control of said gameplay S includes at least displaying a modified said augmented reality representation.

Thus embodiments of the invention enable physical playing pieces, such as dice, to interact with and control game play in an augmented reality environment, in particular by interacting both with the camera via a captured image and the touch screen of the electronic device.

Embodiments of the system may employ one or multiple playing pieces -for example the or each player may have one or more dice such as a character dice and an action dice. (In this specification, dice' includes the singular and thus means for one or more dice). In some preferred embodiments the camera of the electronic device captures an image which is employed to determine a visual orientation of a playing piece and the touch screen is used to determine the touch orientation of the same or a different playing piece. The orientation of the playing piece may for a dice, say, simply comprise a determination of which face is uppermost -there is no necessity to determine a rotational orientation of such a face, although the system software may do so. The augmented reality representation of a playing piece on the touch screen may but need not necessarily correspond with a physical location of a playing piece in the playing field. For example in the case of a dice thrown into the playing field a character may be created where the dice lands and then walk to a more central location in order to do battle with a second character.

In some preferred embodiments the touch screen is a capacitive touch screen and the touch patterns on a playing piece comprise patterns of conductive material, preferably invisible, for example hidden beneath an overlying graphic. In a simple embodiment the system need only distinguish between two different orientations of the playing piece, for example between two different faces of a dice, but preferably in the case of a dice the system is able to distinguish each facet.

The conductive material may be provided in many different ways, for example by an etched or otherwise patterned conductive layer or layer of conductive material, or by employing patterns of conductive, for example carbon-loaded plastic. Although most touch screens are capacitive touch screens, for a resistive touch screen a pattern of bumps may be employed.

In still other approaches, instead of either or both of the camera and touch screen determined orientations, an RFID-determined orientation may be employed -for example a pair of antennas on opposite sides of an object enables an EFID reader to distinguish between two different orientations such as up and down. With such an approach an REID reader may be attached to the electronic device or, for example, to a computer (say via a USB interface) in communication with the electronic device -for example an RFID reader may be integrated into a connector which plugs into the device and/or computer, and bearing a cradle to hold the playing piece so that the orientation of the playing piece can be determined. In the case of USB cradle communication with the electronic device, for example smart phone, may be via Bluetooth (RIM) and/or WiEi (RIM).

In some preferred embodiments the playing piece has a plurality of facets, such as a dice with 4, 5, 6 or more faces.

In embodiments the features on the playing piece identified by the camera/image processing comprise markers for surface regions of the playing piece, for example faces of the dice. These markers may comprise a pattern around a boundary of each separately identifiable surface region (facet or face of a dice), for example a barcode-type pattern around the edge of each region to be identified. This facilitates processing of the captured image to identify orientation of a playing piece.

Preferred embodiments of the system include a playing arena component, for example having a base region defining the playing field and an arm to support the electronic device (smart phone) to direct the camera of the device downwards at an angle towards the playing field. Embodiments of such a component may have two or more arms to facilitate play by two or more players. In preferred embodiments the mount for the electronic device is adjustable to cater for different fields of view, resolution and like of the camera's in different phones.

Preferably the base, playing field component bears a graphic to enable the software to display an augmented reality game play environment at the location of the captured image. Preferably the displayed environment is in three dimensions.

Optionally the graphic can be identified by the software and the augmented reality environment selected in response to this identification from amongst one or more stored environments, either stored locally on the phone or downloaded from a remote server.

In one implementation the augmented reality game comprises at least two of the playing pieces either shared between the players or for each player. One may be employed in the playing field, for example to animate an augmented reality character; another may be employed on the touch screen to control the behaviour of the character according to the orientation or face applied to the touch screen of the electronic device. This second piece may, for example, control a strength of attack, defence or the like.

In embodiments the system comprises a set of at least one of the playing pieces, for example a character playing piece. In preferred embodiments each orientation of the character piece defines data for a different augmented reality character to employ in the game, and, in addition, the system is configured to enable the software to distinguish between different playing pieces of the set, which may define different sets of characters. In this way, for example, in a Hobbit" set one playing piece may allow selection between various "Hobbit" characters whilst another may include an additional Gandalf" character, thus incentivising players to acquire more playing pieces. In terms of technical implementation, this requires that the system is able to distinguish between orientations, more particularly faces or facets, of different playing pieces of the set. Thus in embodiments the separately visually identifiable features and/or touch patterns identify not only an orientation of a playing piece, but also identify the selected playing piece from a set of possible playing pieces.

In some preferred implementations the game play software is configured to coordinate game play across a plurality of the electronic devices, typically mobile phones, so that at least the touch screens of the devices depict coordinated views of the playing field and/or augmented reality game play environment. Thus, for example, the same character behaviour may be observed on each device screen; optionally different devices may apply a different viewpoint to a 3D augmented reality environment in which the game play occurs.

As previously mentioned in some embodiments the system further comprises a remote server stirring data defining (animated and/or 3D) characters and/or augmented reality representations/environments. The game play software may then be configured to download this stored data, for example when needed and/or in response to a payment control system authenticating an upgrade or extension to the permitted game play and/or in response to an activation code, or in some other manner.

In a related aspect the invention provides a method of controlling an augmented reality game on an electronic device with a touch screen and a camera, the method comprising: capturing an image of a playing field for said game with said camera; rendering an augmented reality gameplay environment on said touch screen, wherein said augmented reality gameplay environment comprises a software generated image dependent on said capture image; capturing an image of a first playing piece in said playing field with said camera; determining a first orientation of said first playing piece from said captured image; capturing a touch pattern of a second playing piece on said touch screen; determining a second orientation of said playing piece on said touch screen; and controlling gameplay of said augmented reality game responsive to said detected first and second orientations.

In embodiments the first and second playing piece may be the same playing piece, although preferably these have different functions -for example one controlling a selected character, another an action or behaviour of the character and so forth.

The invention further provides processor control code to implement the above-described systems and methods, for example on a smart phone or other consumer electronic device. The code is provided on a computer program product, that is a physical data carrier such as a disk, CD-or DVD-ROM, programmed memory such as non-volatile memory (e.g. Flash) or read-only memory (Firmware). Code and/or data to implement embodiments of the invention may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as C, or assembly code. As the skilled person will appreciate such code and/or data may be distributed between a plurality of coupled components in communication with one another.

The invention also provides a playing piece, in particular a three-dimensional playing piece as described above; a playing arena component as described above; and a method of providing a playing arena using the playing arena component.

In accordance with a still further aspect of the present invention there is provided an apparatus for playing a game, comprising; a 3D object comprising a plurality of faces, each having a distinguishable pattern; an electronic readable device comprising a camera and a touch screen; and a software tool configured to run on the electronic readable device. In use, the electronic readable device is configured to identify a pattern on a said face of the object through the camera and compare said pattern with a pattern on a face of the object which is touched on the touch screen of the electronic device so that the software tool can perform a rule of the game.

In accordance with a related aspect of the present invention, there is a provided a method of controlling a game using an apparatus comprising a 3D object comprising a plurality of faces, an electronic readable device comprising a camera and a touch screen, and a software tool being run on the electronic device, the method comprising: identifying a pattern on a face of the object touched on the touch screen of the electronic device; identifying a pattern on an upper face of the or another object, positioned or thrown by a player, using the camera; comparing the pattern of the positioned/thrown object with the pattern of the touched object; and performing a rule of the game based on the comparison result.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention will now be further described, by way of example only, with reference to the accompanying figures in which: Figure 1 shows powering up an action dice using touch screen; Figure 2 shows a character coming to life on screen using augmented reality; Figure 3 shows mockup arena designs, showing device holders, viewing angles and

playfield view as may be seen through the devices;

Figure 4 shows a mockup game arena built around the game box and packaging; Figure 5 shows an overview of an example game, illustrating flow of the gameplay and operation of the software; Figure 6 shows capacitive ink patterns printed on the dice; Figure 7 shows a frame marker with a discrete edge pattern; Figure 8 shows an example rig showing two device holders and central viewing area into which Dig/Dice have been thrown; Figure 9 shows a mobile device and software for the system; Figure 10 shows a game system with multiple players, an optional computer (PC) player, and an optional remote server; and Figure 11 shows a flow diagram of software for the game.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The Dig/dice platform enables physical dice to interact with the digital elements of the game. All the dice in the game are treated with an invisible layer of capacitive ink, preferably configured uniquely to each face.

i) Touchscreen The player can hold a face of a dice to the touchscreen of the phone or tablet. The App will recognise that face and use this information to modify some aspects of the gameplay (Fig 1).

ii) Augmented reality Each face of the dice can be recognised by the App, using marker recognition technology. Once a dice had been thrown the App recognises its uppermost face and the player sees, on-screen, an animated 3D model of the game character come magically to life (Fig.2). The visual recognition system may alternately/also trigger an appropriate action in the game.

Dig/Dice thus enables a sophisticated integration and close interaction between the physical world of the dice, and the digital world of the phone/tablet App.

A DigiDice system typically comprises: 1. DigiDice 2. Arena

3. Background Markers

4. Instructions 5. Application 1) DigiDice-enabled Dice Sets of dice for each player, denoting characters, actions or events. For example: Character Dice Each Character Dice can have a different symbol representing a game character on each of its 6 side.

Action Dice Each Action Dice can have up to 6 actions on opposing faces denoting the moves will be performed in the present game round.

Obiect or Even Dice May be used to modify a pre or post action/outcome in the game.

2) Arena This may take the form of either a plastic-moulded, clip-together playset or a cardboard engineered Arena formed from game packaging. The Arena has a gantry at each end where the players' smariphones are mounted; between them is the enclosed combat field into which dice are thrown and contained (Figs 3. and 4).

3) Arena marker sheets and/or transfers A selection of inlay sheets would be included or otherwise made available which can be placed into the floor of the Arena to create different gaming environments in which the action will be played out. These sheets are recognised as by the application, which responds by displaying a corresponding 3D model as the centre of the play field. For example, one marker might trigger an open setting in a jungle clearing, whilst another creates a more coliseum-styled, enclosed arena.

4) Instructions Instructions for play include the URL from which a game App can be downloaded according to the user's device (iOS, iPhone, iPad Touch, Android etc). The App may be free to download although subsequent, incremental add-on games, characters or upgrades may be charged for.

5) The Application (App) The App handles and controls the flow of the game -making it easy for users to follow gameplay steps in the right order and keep count of scores, character selections, victories and so forth.

Usually a game will be played by two players with their own devices, both having the App installed. The Apps handle communications between the two devices via Bluetooth(RTM) or via WiFi(RTM) in order to keep both devices in sync, with one device acting as host, the other as client.

Embodiments of the system combine physical play patterns with COl and special effect; and are extensible -the approach enables design and development of hundreds of rulesets.

There is scope for detailing a wide range of rule sets and characters with which to play additional game characters; objects or environments may be provided as incremental downloads alongside new dice featuring any of the characters.

We will now describe an example game in more detail.

PHYSICAL COMPONENTS

The Dig/Dice system comprises the following components: 1. 3D objects, such as a six-sided dice, which have been treated with a capacitive ink printed in such a way as to enable a touch screen dice to detect which face is being presented to it via the pattern of capacitance of each face (Fig 6). Patterns are (preferably uniquely) configured to each face and capable of carrying messages through the dice to interact in various ways with a Mobile Application. The capacitive ink print layer is not exposed to the human eye.

Such objects (dice) are also printed with a coded marker (or bar code') on (around the edge of) each face which can be detected and recognised via a camera-enabled smart device (Smartphone or tablet) for the purpose of rendering Augmented Reality assets (graphics or data)on the Smartphone's screen (Fig 7). If the software needs to distinguish between dice and multiple different sets, e.g. 4 sets of frame markers may be used. Preferably up to 3 sides of a cube are recognised at one time in order to determine which side is oriented with its face up.

2. An adjustable, physical rig or harness capable of holding between one (1) and four (4) mobile devices or tablets such that their respective cameras can focus forwards and downwards onto a common field of view before them, upon which Augmented reality content will be rendered on the device (Fig 8).

3. One or more camera-enabled mobile device with a capacitive touch screen, such as a Smartphone or tablet computer, running an application capable of detecting capacitive ink patterns, recognising markers and natural features, and communication via Internet, WiFi or Bluetooth services (Fig 9). Devices are able to share status and data locally or remotely via Internet and/or USB connection to PCs and servers (Fig 10).

SOFTWARE SYSTEM COMPONENTS

1) Device Operating System which shall include a Multi-touch detection system or API 2) Augmented Reality Software, for example Vuforia by Qualcomm, which may include with or both of: i) Frame marker detection which may be used to determine up to 3 frame markers on the face of a cube, detect which face is uppermost and optionally its orientation.

U) Natural feature tracking which may be used either for recognising faces of a cube or identifying any background features in the camera's field of view which are not cubes (e.g. field of play).

3) Touch recognition algorithm to determine specific patterns presented to the touch screen, via the device touch OS.

4) 3D rendering engine and, optionally, an animation system, such as Unity, to handle representation of AR assets and other graphical or user-interface components as may be required of the application.

5) Application software to manage assets, rules, logic and I/O.

6) Stored data as may be required by the application which may include characters and environments and their properties; frame marker data and updates; touch patterns and updates.

EXAMPLE FLOW using a generic game example (Fig 11) Phase 1 Launch Application and step through introduction and set up options.

Choose number of devices Choose number of participants Load rule set for the type of game selected Users throw DigiDice into the play field to see which user starts AR module identifies winning roll Application identifies background marker type and renders appropriate 3D Augmented Reality scene in place of the background marker.

Phase 2 The first player (P1) rolls first DigiDice AR module identifies dice and which face is oriented face up AR module graphical assets appropriate to identified dice, as determined by its rule set.

Graphical assets may represent characters, objects, events, or other actions such as special effects, animations or user interface items Application prompts Al to any further actions as may be required to complete this phase and once these have been actioned, prompts P1 for touch input, which commences Phase 3.

Phase 3 P1 takes either the same Dig/Dice from the play field or another DigiDice and touches the face they wish to have detected to their device's touch screen.

Application detects the capacitive pattern embedded on that face and identifies that dice and/or which face of the dice is being presented to it.

Application determines appropriate action, as determined by its rule set, for example, assigning different rewards if that face is thrown face up on the next throw, and stores such data, presents outcomes to user and prompts user to take next step accordingly.

Once the outcome for P1 is determined P2 (player 2) may be prompted to take their turns, unless these have been taking place simultaneously.

Phase 4 Al (and P2) now throw the dice they had just touched to the device.

As at Phase 2, the AR module identifies the upward orientated face of the dice thrown and displays an outcome according to the game rules. However, such outcome may now be moderated, enhanced or otherwise modified as a result of the data stored about the outcome, according to the user's actions at Phase 3, and the face detected by the application during that phase.

For example, had P1 effectively told' the application that P1 expected Face X to be shown on the next throw of their DigiDice, and this was indeed the case, the result of that throw might indicate a different (improved) score or trigger a win condition.

Equally, P2's actions at Phase 3 could affect a different outcome to their own or both players' throws.

With Phase 4 complete, the process returns to Phase 2 until such time as Phase 4 resolves a game's outcome (win, lose, draw or otherwise) at which point users can return to Phase 1 to restart the entire playing process.

Although we have described techniques for use with augmented reality games, the skilled person will appreciate that the invention may also be adapted to other applications, for example training systems and the like. Thus the invention also contemplates replacing a playing piece of the invention with an interactive user token with corresponding properties. In a corresponding manner the invention contemplates substituting references to the augmented reality game/game play/environment with more general references to an augmented reality system.

Similarly, instead of capacitive ink a capacitive core and/or a near field arrangement (e.g. with a USB base unit or cradle), and/or some other mechanism (e.g. an optical mechanism, such as an LED-based mechanism) may be used to identify a dice/face to an electronic device.

No doubt many other effective alternatives will occur to the skilled person. It will be understood that the invention is not limited to the described embodiments and encompasses modifications apparent to those skilled in the art lying within the spirit and scope of the claims appended hereto.

Claims

CLAIMS: 1. An augmented reality game, the game comprising: at least one playing piece having a plurality of visually distinguishable orientations and a plurality of separately visually identifiable features for identifying a said orientation of said playing piece when the playing piece is on a playing field; and an electronic device with a touch screen and a camera; wherein said playing piece further comprises a plurality of touch patterns for identifying a said orientation of said playing piece to said electronic device when said playing piece is touched to said touch screen; and wherein said electronic device includes gameplay software to: detect said playing piece using said camera when said playing piece is onthe playing field;display an augmented reality representing of said playing piece of said touch screen; detect a touch orientation of the or another said playing piece on said touch screen; and control gameplay of said game in response to said detected touch orientation, wherein said control of said gameplay includes at least displaying a modified said augmented reality representation.
2. An augmented reality game as claimed in claim 1 wherein said game-play software is further configured: to identify a visual orientation of said playing piece on said playing field; and to control gameplay of said game in response to said detected visual and touch orientations.
3. An augmented reality game as claimed in claim 1 or 2 wherein said touch screen is a capacitative touch screen and wherein said touch patterns comprise patterns of conductive material.
4. An augmented reality game as claimed in claim 1, 2 or 3 wherein said playing piece has a plurality of facets, wherein an orientation of a said facet defines a said visually distinguishable orientation, and wherein said separately visually identifiable features comprise markers for each said facet.
5. An augmented reality game as claimed in any preceding claim wherein said S separately visually identifiable features comprise markers for surface regions of the playing piece, and wherein a said marker comprises a pattern around a boundary of a said surface region.
6. An augmented reality game as claimed in any preceding claim further comprising a playing arena component defining said playing field, wherein said playing arena component includes a mount for said electronic device to hold said electronic device to direct said camera downwards towards said playing field.
7. An augmented reality game as claimed in claim 6 wherein said playing arena component includes a playing field component bearing a playing field graphic, and wherein said gameplay software is configured to capture an image of said playing field graphic using said camera, and to display an augmented reality gameplay environment on said screen dependent upon said captured image.
8. An augmented reality game as claimed in any preceding claim comprising at least two said playing pieces, a first said playing piece for use in said playing field and a second said playing piece for use on said touch screen, wherein said gameplay software is configured to animate an augmented reality character dependent on an identified visual orientation of said first playing piece on said playing field and to control a behaviour of said augmented reality character dependent on an identified touch location and/or orientation of said second playing piece on said touch screen.
9. An augmented reality game as claimed in any preceding claim, further comprising a plurality of said electronic devices, wherein a said electronic device comprises a handheld mobile device, and wherein said gameplay software is configured to coordinate gameplay across said handheld mobile devices such that at least said touch screens of said mobile devices depict coordinated views of said playing field or augmented reality gameplay environment.
10. An augmented reality game as claimed in any preceding claim, further comprising a remote server storing data defining one or more of characters and gameplay environments for said augmented reality representation/environment, and wherein said gameplay software is configured to download stored data to said electronic device for said game.
11. A method of controlling an augmented reality game on an electronic device with a touch screen and a camera, the method comprising: capturing an image of a playing field for said game with said camera; rendering an augmented reality gameplay environment on said touch screen, wherein said augmented reality gameplay environment comprises a software generated image dependent on said capture image; capturing an image of a first playing piece in said playing field with said camera; determining a first orientation of said first playing piece from said captured image; capturing a touch pattern of a second playing piece on said touch screen; determining a second orientation of said playing piece on said touch screen; and controlling gameplay of said augmented reality game responsive to said detected first and second orientations.
12. A computer program product storing processor control code for controlling a processor to implement the method of claim 11.
13. A three-dimensional (3D) playing piece for the augmented reality game of any one of claims 1 to 10 wherein said playing piece comprises a plurality of touch patterns for identifying an orientation of said playing piece to said electronic device when said playing piece is touched to said touch screen, wherein said touch screen is a capacitative touch screen, and wherein said touch patterns comprise patterns of conductive material.
14. A 3D playing piece as claimed in claim 13 wherein said playing piece is a dice.
15. A playing arena component for the augmented reality game of any one of claims 1 to 10, said component defining said playing field, wherein said playing arena component includes a mount for said electronic device to hold said electronic device to direct said camera downwards towards said playing field, and wherein S said playing arena component includes a playing field component bearing a playing field graphic for use by said gameplay software in displaying an augmented reality gameplay environment on said screen dependent upon said playing field graphic.
16. A method of providing a playing arena for an augmented reality game, the method comprising providing a playing arena component as recited in claim 15.
17. An apparatus for playing a game, comprising; a 3D object comprising a plurality of faces, each having a distinguishable pattern; an electronic readable device comprising a camera and a touch screen; and a software tool configured to run on the electronic readable device; wherein, in use, the electronic readable device is configured to identify a pattern on a said face of the object through the camera and compare said pattern with a pattern on a face of the object which is touched on the touch screen of the electronic device so that the software tool can perform a rule of the game.
18. A method of controlling a game using an apparatus comprising a 3D object comprising a plurality of faces, an electronic readable device comprising a camera and a touch screen, and a software tool being run on the electronic device, the method comprising: identifying a pattern on a face of the object touched on the touch screen of the electronic device; identifying a pattern on an upper face of the or another object, positioned or thrown by a player, using the camera; comparing the pattern of the positioned/thrown object with the pattern of the touché object; and performing a rule of the game based on the comparison result.