KR20220105119A - Technology configured to enable monitoring of user engagement with physical prints via an augmented reality delivery system - Google Patents

Abstract

Described herein are techniques configured to enable monitoring of user engagement with physical prints via an augmented reality delivery system. Although the application of the technology is described as a primary example, it should be understood that the technology has additional applications. Some demonstrative embodiments enable the collection of analytical information primarily about user engagement with print-based advertising, and in some cases, prevent users from purchasing goods and/or services directly through engagement with print-based advertising. It relates to the use of AR technology as a means of enhancing the effectiveness of print-based advertising by enabling it. A wide range of forms of AR experience delivery and engagement monitoring are available.

Description

Technology configured to enable monitoring of user engagement with physical prints via an augmented reality delivery system

The present invention, in various embodiments, relates to a technology configured to enable monitoring of user engagement with a physical print via an augmented reality delivery system. While some embodiments will be described herein with particular reference to such applications, it will be understood that the present invention is not limited to such fields of use, but is applicable in a broader context.

Any discussion of background throughout this specification should in no way be considered as an admission that such background is widely known or forms part of the common general knowledge in the field.

It is known to monitor user involvement in digital content delivered via the Internet, for example digital advertising. Such monitoring is of considerable value in the context of performing analysis to determine the relationship between user engagement and other factors. There is a technical problem in the sense that the existing engagement monitoring method is limited to digitally distributed content, and the application of the existing engagement monitoring method cannot be extended to content distributed through physical print.

It is an object of the present invention to overcome or improve at least one of the disadvantages of the prior art or to provide a useful alternative.

Exemplary embodiments are described below in the section entitled “Claim.”

One embodiment provides a computer-implemented method for enabling monitoring of user engagement with printed content, including printed content distributed through magazines, newspapers, wrappers and booklets, the method comprising:

distributing a plurality of articles carrying respective copies of a common printed content item, each copy of the printed content item comprising a common graphical artifact, the common graphical artifact comprising augmented reality (AR) in a cloud hosted system ) associated with a set of content data;

executing an engaging application on a plurality of user mobile devices, each running instance of the engaging application comprising:

process the image data captured via the camera module of the mobile device to identify the presence of the graphic artifact when the printed content item is brought into the field of view of the camera module;

obtain from the cloud hosted system at least a subset of the set of augmented reality (AR) content data associated with the graphical artifact;

Based on the obtained AR content data, causing a display, on the display screen of the mobile device, of rendering of the interactive AR content superimposed on the image capture based representation of the physical substrate, superimposed on the image capture based representation of the physical substrate wherein the rendered animation being rendered is configured to be displayed at a defined position and orientation relative to an image capture based representation of the physical substrate;

At the server device, data indicative of user involvement in the rendered interactive content from augmented reality (AR) content data associated with the graphical artifact is collected from a plurality of user mobile devices, such that the user to multiple copies of a common printed content item generating analytical information indicative of engagement.

One embodiment provides a device configured to enable monitoring of user engagement with a physical substrate delivering a print, the device comprising:

a camera module configured to capture image data of the capture area;

display screen;

microprocessor;

A memory module coupled to a microprocessor, when executed through the microprocessor:

(i) identify the presence of a graphic artifact having predetermined properties printed on the physical substrate; (ii) an image identifier extraction module configured to process image data captured via the camera module to extract an image identifier code from the graphic artifact;

an augmented reality (AR) data access module configured to retrieve the set of executable AR data associated with the extracted image identifier code from the AR data storage module;

An AR rendering module configured to cause, based on the retrieved AR data, a display, on a display screen, of a rendered animation superimposed on the image capture based representation of the physical substrate, wherein the AR rendering module is configured to superimpose on the image capture based representation of the physical substrate. an AR rendering module wherein the rendered animation is displayed at a defined position and orientation relative to the image capture based representation of the physical substrate;

(i) monitor user interactions associated with display of the rendered animation; (ii) compiling data representing user interactions; (iii) configure the device to provide an engagement monitoring module configured to communicate data indicative of user interactions to the server device, wherein the data indicative of user interactions is associated with an image identifier code, thereby associating engagement interactions with a particular printed matter. and a memory module for maintaining computer-executable code.

One embodiment provides a device wherein the rendered animation is an interactive animation that is variably rendered in response to inputs by a user of the device.

One embodiment is that the AR data storage module maintains a plurality of sets of AR data associated with a given image identifier code, and wherein certain of the sets of AR data are assigned to the device based on one or more attributes of a user associated with the device. It provides the device of choice for delivery.

One embodiment provides a device in which one or more attributes of a user associated with the device are defined in a user profile, wherein the user profile is stored in a memory module and/or cloud-hosted data store of the device.

One embodiment provides a device in which data storage is maintained by a cloud-hosted server AR data management system.

One embodiment provides a system configured to enable monitoring of user engagement with a physical substrate delivering a print, the system comprising:

A communication module that enables communication between a system and a plurality of user devices, each user device comprising:

a camera module configured to capture image data of the capture area;

display screen;

microprocessor;

A memory module coupled to a microprocessor, when executed through the microprocessor:

(i) identify the presence of a graphic artifact having predetermined properties printed on the physical substrate; (ii) an image identifier extraction module configured to process image data captured via the camera module to extract an image identifier code from the graphic artifact;

an augmented reality (AR) data access module configured to retrieve the set of executable AR data associated with the extracted image identifier code from the AR data storage module;

An AR rendering module configured to cause, based on the retrieved AR data, a display, on a display screen, of a rendered animation superimposed on the image capture based representation of the physical substrate, wherein the AR rendering module is configured to superimpose on the image capture based representation of the physical substrate. an AR rendering module wherein the rendered animation is displayed at a defined position and orientation relative to the image capture based representation of the physical substrate;

(i) monitor user interactions associated with display of the rendered animation; (ii) compiling data representing user interactions; (iii) configure the device to provide an engagement monitoring module configured to communicate data indicative of user interactions to the server device, wherein the data indicative of user interactions is associated with an image identifier code, thereby associating engagement interactions with a particular printed matter. a memory module for maintaining computer-executable code that

The system includes a communication module that provides an AR storage module.

One embodiment provides a system wherein the rendered animation is an interactive animation that is rendered variably in response to inputs by a user of the device.

One embodiment is that the AR data storage module maintains a plurality of sets of AR data associated with a given image identifier code, and wherein certain of the sets of AR data are assigned to the device based on one or more attributes of a user associated with the device. It provides the system of choice for delivery.

One embodiment provides a system in which one or more attributes of a user associated with a device are defined in a user profile, wherein the user profile is stored in a memory module and/or cloud-hosted data store of the device.

One embodiment provides a system in which one or more attributes of a user are updated in response to monitoring user interactions associated with display of a rendered animation.

One embodiment is the rendering of AR data for a given image ID code where updating of the user's one or more attributes results in changes in AR data selection and/or execution on successive interactions with the common image ID code. A system is provided that results in a customized selection and/or execution.

One embodiment provides a system comprising a module for providing access to an e-commerce platform to enable the system to enable a user of a user device to purchase goods and/or services based on interaction with rendered AR data. .

One embodiment provides a system in which data storage is maintained by a cloud-hosted server AR data management system.

One embodiment provides a method for enabling monitoring of user engagement with a physical substrate delivering a print, the method comprising:

defining computer-executable code representing a plurality of interactive augmented reality animation data sets, each data set comprising: on a display screen of a user device; executable by the AR rendering module of the user device to cause the display, wherein the rendered animation superimposed on the image capture based representation of the physical substrate is displayed at a position and orientation defined relative to the image capture based representation of the physical substrate. ;

associating each AR data set with an identifier code;

printing a given one of the identifier codes on a plurality of physical substrates, wherein each of the physical entities has applied to the given one of the identifier codes a print medium designed to complement the AR data set associated with the identifier code. step;

identifying that the user device executing the defined software application is capturing image data of the physical substrate from which a given one of the identifier codes was extracted;

causing the user device to display a rendered animation of an AR data set associated with an identifier code such that the animation is rendered superimposed on the image capture based representation of the physical substrate, wherein the rendered animation is superimposed on the image capture based representation of the physical substrate. is displayed at a defined position and orientation relative to an image capture based representation of the physical substrate;

data indicative of user interactions is associated with the image identifier code to monitor user interactions associated with the presentation of the rendered animation to associate the engagement interactions with a particular print; and

and compiling data representing user interactions.

One embodiment provides a method for enabling monitoring of user engagement with a physical substrate delivering a print, the method comprising:

dispensing a plurality of physical substrates carrying a common printed content item, wherein the common printed content item is associated with a designated set of AR content in a database;

At the first mobile device, read the graphic artifact from a first of the physical substrates carrying the printed content item, and in response, obtain server data from the designated set of AR content, and interact with the AR based on the data. rendering the game;

At the second mobile device, read the graphic artifact from a second of the physical substrates carrying the printed content item, and in response, obtain server data from the specified set of AR content, and interact with the AR based on the data. rendering the game;

the server receives data representative of an interaction with one instance of the mobile devices of the AR interactive game on one of the mobile devices and, in response, the data representative of the interaction executes on the other of the mobile devices and the cloud-hosted game server device communicating with the first mobile device and the second mobile device to cause a change of state of the AR interactive game.

One embodiment provides a method comprising maintaining a record of interactions with AR content at a first mobile device and a second mobile device to enable analysis of engagement with printed content items.

One embodiment provides a method for enabling user engagement with a physical substrate carrying a print, the method comprising:

dispensing a plurality of physical substrates carrying a common printed content item, wherein the common printed content item is associated with a designated set of AR content in a database;

At the first mobile device, read the graphic artifact from a first of the physical substrates carrying the printed content item, and in response, obtain server data from the designated set of AR content, and interact with the AR based on the data. rendering the animation;

rendering an interaction target representing a function of purchasing goods and/or services through the rendering of the AR interaction animation;

and in response to a predetermined interaction with an interaction object representing the ability to purchase goods and/or services, triggering a process for causing a purchase transaction for the user for goods and/or services.

One embodiment provides a method wherein a process for causing a purchase transaction for a user for goods and/or services includes a process for completing a purchase transaction using pre-stored user data including payment information and delivery information .

One embodiment provides a method in which a process for causing a purchase transaction for a user for goods and/or services includes a process for redirecting the user to an indication that renders a user interface that facilitates completion of the purchase transaction.

References throughout this specification to “one embodiment,” “some embodiments,” or “an embodiment,” refer to specific features, structures, and/or descriptions in connection with the embodiments. or that the feature is included in at least one embodiment of the present invention. Accordingly, the phrases “in one embodiment”, “in some embodiments”, or “in an embodiment” in various places throughout this specification The appearances of these are not necessarily all referring to the same embodiment, but may be the same. Moreover, the particular features, structures, or characteristics may be combined in any suitable manner, as will be apparent to those skilled in the art from the present invention in one or more embodiments.

As used herein, unless otherwise specified, the use of the ordinal adjectives “first,” “second,” “third,” etc. to describe a common object is It merely indicates that similar objects in different instances are recited, and thus is not intended to suggest that the objects being described must be in a given sequence, in order or in any other way, either temporally, spatially, or in any other way.

In the following claims and the description herein, any of the terms comprising, consisting of or which comprises includes at least the following elements/features, but , is an open term meaning not excluding others. Accordingly, the term comprising, when used in the claims, should not be construed as limiting to the means or elements or steps listed thereafter. For example, the scope of the expression a device comprising A and B should not be limited to devices comprising only elements A and B. As used herein, any of the terms including or which includes or that includes also includes at least the elements/features that follow the term, but , is an open term that also means not to exclude others. Thus, including is synonymous with and means comprising.

As used herein, the term “exemplary” is used in the sense of providing examples as opposed to indicating quality. That is, an “exemplary embodiment” is an embodiment provided as an example as opposed to necessarily being an embodiment of exemplary quality.

Embodiments of the present invention will now be described by way of example only with reference to the accompanying drawings.
1A shows a description scheme according to an embodiment.
1B shows a description scheme according to an embodiment.
1C shows a description scheme according to an embodiment.
2A illustrates a method according to an embodiment.
2B illustrates a method according to an embodiment.
2C illustrates a method according to an embodiment.
2D illustrates a method according to an embodiment.

Described herein are techniques configured to enable monitoring of user engagement with physical prints via an augmented reality delivery system. Although the application of the technology is described as a primary example, it should be understood that the technology has additional applications. Some demonstrative embodiments enable the collection of analytical information primarily about user engagement with print-based advertising, and in some cases, prevent users from purchasing goods and/or services directly through engagement with print-based advertising. It relates to the use of AR technology as a means of enhancing the effectiveness of print-based advertising by enabling it. A wide range of forms of AR experience delivery and engagement monitoring are available.

In summary, some embodiments of the techniques discussed herein enable monitoring of user engagement with physical prints and, in some cases, to those who experience such prints (including “shared experiences” interactive content). It is configured to deliver interactive content. For example, this is:

• Advertisements printed in newspapers or magazines (eg, to enable analysis in a manner similar to Internet-delivered advertisements);

• advertisements printed in leaflets, booklets, etc. (again, allowing analysis in a manner similar to, for example, Internet-delivered advertisements);

• business cards (eg, conveying multimedia messages);

• Non-advertising content in newspapers or magazines (eg, enhancing the content being provided);

• an item or wrapping paper for the items (eg, providing multimedia content and/or a “Terms and Conditions” document for user consent);

• May involve involvement in any one or more of the other physical entities that convey the print.

In some embodiments, the technique is applied to virtual copies of printed matter, for example 2D content displayed on a computer screen. One example is a magazine that is experienced in digital form through a tablet computer or the like. It will be appreciated that the technology described herein operable with a printed material may function the same as digital versions of such printed material rendered on two-dimensional display screens.

Through inducing the user to interact with the physical substrate through an AR-enabled device (e.g., a mobile device such as a smartphone or tablet, and an AR headset/glasses, etc.) to experience augmented reality animations designed to complement printed materials. Engagement is tracked. User interactions (eg, time spent watching animation, interactions with animation in the form of a video game, user's interactions with device during animation, etc.) are monitored and recorded for analysis do. It is optionally used to update data in the user profile. Interactions may be performed via input devices of the smartphone/tablet (eg, via buttons, touch screen, microphone, camera, etc.) and/or interactions performed (eg, via smartwatch, biometric monitor, game controller, etc.) ) may include interactions observed through external input devices.

As used herein, the term “augmented reality animation” refers to any augmented reality content rendered on a screen via augmented reality technology, which in fact is the content (as identified and tracked using a camera module). It means to be displayed at a location defined with respect to an object or objects identified in real-world space. “Animation” may include three-dimensional non-interactive content, interactive content (eg, a video game rendered as a three-dimensional augmented reality object), a three-dimensional map object rendered as an augmented reality object, and other forms of content. can In some embodiments, the Unity augmented reality platform is utilized to enable creation of augmented reality content (and alternative technologies and/or platforms are used in further embodiments).

The augmented reality animation is customized based on data in the user profile in some embodiments. Such a profile may include demographic data about the user, known interests/habits, along with other information. When designing AR animation, designers preferably take these factors into account to enable delivery of a tailored experience. In some embodiments, the user profile is updated in response to monitoring of user engagement. This is implemented such that, in some cases, for an example of a given print (eg, a given advertisement), this AR data provided may differ between successive interactions for a given user. For example, in one embodiment, the user engages with a printed advertisement via AR, and the user profile is updated in response to such engagement. The user is then engaged with the same printed advertisement (optionally printed in different locations, eg, in different magazines, different media types, etc.) via AR, and is presented with a different AR experience due to changes in user profile data. It is optionally implemented to provide on-the-fly AR-based delivery of material in a manner similar to an automated email tracking engine. The material presented to the user via AR data is tailored based on the user's previous engagement, delivering a limited logical path of content in response to user engagement actions.

In some embodiments, the augmented reality animation is configured to convey a “shared experience,” whereby a given user's experience of the content is affected by the experience of one or more additional users with the content. For example, in some embodiments, the augmented reality content is such that one player's interaction with the augmented reality content on the user's device is another player's experience of the augmented reality content on the user's device (or multiple on the user's respective device). It is an interactive multiplayer game that affects the experiences of two other players. Interactive multiplayer games are real-time multiplayer online gaming (e.g., real-time multiplayer online gaming where the game server maintains a central simulation based on input from the players' devices, and players view each local simulation based on data from the server. , sports games, first-person shooter games, etc.), or turn-based multiplayer gaming (e.g., Scrabble or chess type games), in which input from one player is provided to the server, and the server provides data to other players in response in response. ) may be included.

In some embodiments, the augmented reality animation includes content available from one or more web servers, eg, content in the form of an e-commerce function (e.g., enabling purchase of a product through interaction with the animation); and display and provide access to social media content (eg, enabling making and/or viewing of comments) and the like. Exemplary embodiments are discussed below:

• In an exemplary embodiment, the AR content includes an interactive object that facilitates a purchase transaction. A purchase transaction may be a direct purchase transaction (eg, a user interacts with an object rendered as an AR component or with an AR animation, thereby making a purchase using pre-stored payment information), or an indirect purchase transaction ( for example, the user interacts with an object being rendered as an AR component or with AR animation, and, in so doing, is directed to a web page that enables completion of a purchase, etc.).

• In another example, an AR animation, or object provided with an AR animation, provides functionality for viewing of third-party comments, and/or input of user-defined comments. This may leverage third-party content commenting platforms.

User interactions used to generate analytics data indicative of user engagement include any one or more of the following:

• Time spent on engagement. For example, the length of time a user views a particular AR animation, the length of time the user interacts with a particular component of the AR animation in a particular way, the number of repeated access events to the AR animation (or other attribute(s) )) and the like.

ㆍLocation tracking of involvement. For example, this may be determined based on readings from the GPS module (or other location tracking module) of the smartphone.

• Characteristics of engagement with interactive modules (eg engagement within an AR video game such as number of games played, time spent playing, playing performance characteristics, etc.).

• Engagement preferences (eg, playing a game or viewing information).

• Repeat Engagement (and Characteristics of Repeat Engagement).

• Digital infrared heat maps for interaction with AR objects and/or virtual environment (eg, to help determine whether a user is engaged by a particular advertisement).

• Digital infrared heat maps for viewing of portions of print and/or AR content (eg, based on retinal tracking via front-facing cameras).

• Asset interactions, eg inputs to interacting objects, “likes” (as in the social media concept of “likes”), (enabling the attention of downstream electronic marketing methods to find future conversions) Identification of non-conversions for sale, and “longer retention-based” promotion of goods/services (potential interest in purchasing goods and/or services is interactive infrared heat maps, AR content status/location analysis, and/or retinal determined based on a longer lingering view of an object representing goods and/or services that are selectively derived from tracking).

• Conversion rates (eg, users accessing e-commerce platforms and/or purchasing products as a result of engagement with AR data).

This may be used to collect information, such as demographic data, for example, by enabling analytical monitoring of engagement with a particular print example (eg, advertisements delivered through one or more types of distribution media). It is useful to understand the effect of advertising campaigns delivered through

An added benefit is that advertising campaigns delivered in physically printed form, etc. can be improved and updated based on intelligence gathered (via engagement monitoring), optimizing effectiveness even after physical distribution. In addition, AR data delivery logic selectively enables split testing where multiple AR experiences are tested simultaneously across a user base.

According to one embodiment, a method for enabling monitoring of user engagement with a physical substrate carrying a print includes the processes presented below.

First, the method includes defining computer executable code representing a plurality of interactive augmented reality (AR) animation data sets. Each data set is executable by the AR rendering module of the user device to cause the display of AR data. In this regard, each data set includes: (i) a print on the substrate on which the AR animation will be superimposed; and (ii) AR experience objectives (eg, based on viewer profiles, etc.).

The AR rendering module of the user device is configured to cause, on a display screen of the user device, rendering of the rendered animation superimposed on the image capture based representation of the physical substrate, the rendering superimposed on the image capture based representation of the physical substrate Animated animation is displayed at a defined position and orientation relative to an image capture-based representation of the physical substrate. The rendered animation is preferably an interactive animation that is rendered variably in response to inputs by a user of the device (eg, to present a video game in combination with game performance logic).

Each AR data set is associated with an identifier code. These codes are printed on physical substrates and enable the computer system to identify the correct AR animation for display. That is, each of the physical entities has applied to the identifier code a print medium designed to complement the AR data set associated with that identifier code.

The method includes printing a given one of the identifier codes on a plurality of physical substrates and distributing the substrates to users. The method then identifies that the user device executing the defined software application is capturing image data of the physical substrate from which a given one of the identifier codes was extracted, and sends a rendered animation of the AR data set associated with the identifier code to the user device. including the step of making it appear. User interactions associated with the presentation of the rendered animation are monitored, and data representative of the user interactions is compiled. Such data is preferably maintained on a server for later analysis.

In some embodiments, data derived from monitoring of user engagement is used to update data maintained in a user profile. This enables customization of AR data presented to a given user in response to data derived from previous interactions. Personalization of AR data may include the presentation of suggestions based on identified historical user engagement attributes, and the like.

1A shows a scheme according to an embodiment. It will be understood that various components shown in these examples may be substituted for other components for further embodiments.

The example of FIG. 1A and other examples herein will be described with reference to a plurality of “modules”. The term “module” refers to a logically separable software component (computer program) or hardware component. A module of an embodiment refers not only to a module in a computer program but also to a module in a hardware configuration. The discussion of embodiments also discusses computer programs that cause modules to function (including a program that causes a computer to perform each step, a program that causes the computer to function as a means, and a program that causes the computer to implement each function) , and serves as a discussion of systems and methods. For convenience of explanation, the phrases “stores information”, “causes information to be stored”, and other equivalents are used. If the embodiment is a computer program, these phrases can be "causes a memory device to store information" or "controls a memory device to cause the memory device to store information)”. Modules may correspond to functions in a one-to-one correspondence. In a software implementation, one module may form one program, or multiple modules may form one program. One module can form multiple programs. Multiple modules may be executed by a single computer. A single module may be executed by multiple computers in a distributed or parallel environment. One module can contain other modules. In the discussion that follows, the term “connection” refers not only to a physical connection, but also to a logical connection (such as the exchange of data, instructions, and data reference relationships). The term “predetermined” means that something is predetermined in advance of the process of interest. Accordingly, the term “predetermined” is intended to refer to something predetermined in advance of the process of interest in an embodiment. Even after the process in the embodiment has started, the term “predetermined” refers to the process of interest depending on the condition or state of the embodiment at the present time, or depending on the condition or state that continues up to the present time. It refers to something determined in advance. When “predetermined values” is plural, the predetermined values may be different from each other, or two or more of the predetermined values (including all values) may be equal to each other. The statement “if A, B is to be performed” means “that it is determined whether something is A, and if something is determined as A, that action B must be performed.” it is determined whether something is A, and that if something is determined as A, an action B is to be carried out). If no judgment is made as to whether something is A, the statement becomes meaningless.

The term “system” refers to a configuration in which multiple computers, hardware configurations, and devices are interconnected via a communications network (including one-to-one communications connections). The terms “system” and “device” also refer to a single computer, a configuration comprising a hardware configuration, and a device. Systems do not include social systems, which are social “institutions” expressed by people.

In each process performed by the module, or one of the processes performed by the module, information as a process target is read from the memory device, the information is then processed, and process results are written on the memory device. Descriptions relating to the reading of information from the memory device before the process and the writing of the processed information onto the memory device after the process may be omitted as appropriate. Memory devices may include a hard disk, random access memory (RAM), an external storage medium, a memory device connected through a communication network, and a ledger within a CPU (central processing unit).

The scheme of FIG. 1 includes an exemplary user device in the form of a smartphone 120 (other types of devices including AR headsets/glasses may exist in further embodiments). The smartphone 120 includes a display screen 121 (preferably a touch screen, for example a capacitive or resistive type touch screen), and (as a rear-facing camera) that captures image data in the zone 122 not shown. commonly known, in this example an image capture device (such as a digital camera pointed in the opposite direction to the display screen).

Smartphone 120 also includes a memory module coupled to the microprocessor, the memory module being a computer executable that, when executed through the microprocessor, configures the smartphone to run a mobile app, shown as mobile app module 130 . Maintains possible code. The mobile app module 130 includes:

• Image processing module 131, configured to process image data collected by the image capture device to identify image artefacts. As used herein, the term “image artefact” is described as a graphic artefact that can be identified in an image that enables a determination that a particular image (or a particular category of images) is being viewed.

• (i) identify the presence of a graphic artifact having predetermined properties printed on the physical substrate; (ii) an image identifier extraction module 132, configured to process image data captured via the camera module to extract an image identifier code for the graphic artifact. This may include reading a graphic code comprising data representing the identifier code, or using data derived from graphic artifacts to identify the identifier code from a database.

• an augmented reality (AR) data access module 133 configured to retrieve, from the AR data delivery module 141 of the AR file management system 140 , a set of executable AR data associated with the extracted image identifier code. This AR data contains the code that enables the rendering of AR animations, for example AR video games.

- AR rendering module 138, configured to cause, on the display screen, display of the rendered animation superimposed on the image capture based representation of the physical substrate, based on the retrieved AR data. Rendered animations are superimposed in defined positions and orientations as discussed in further detail below. This causes the animation to appear, in some embodiments, as if it were drifting in three-dimensional space on, or resting on, a physical substrate.

• (i) monitor user interactions associated with display of the rendered animation; (ii) compiling data representing user interactions; (iii) an engagement monitoring module 135 configured to communicate data indicative of user interactions to the server device 160 . The compiled data may include, for example, data indicative of a location of engagement based on data retrieved from a mobile device GPS module.

• A user profile data module 134 that maintains data defining the user's attributes, including demographics, historical activity, and the like. In some embodiments, such user profile data is alternatively included in the cloud hosted storage and/or supplemented by additional profile data included in the cloud hosted storage.

1 also shows an AR-enhanced image file management system 140 with which smartphone 120 (via module 130) interacts via the Internet. In overview, system 140 is configured to facilitate delivery of AR data to module 130 for animations to be displayed in response to the identified image. In some embodiments (as described below), system 140 transmits the AR data itself; In other embodiments, system 140 transmits a link or other data that enables module 130 to download AR data from additional networked locations.

System 140 includes an AR data storage module 143 containing AR data sets defined by AR data generation system 170 . AR data sets include animations and associated logic for interactive animations (eg, video games based on interactive animations). System 140 also includes an image data storage module 144 that maintains data associated with printable media associated with AR data sets (eg, each print media data set is associated with a given one of the AR data sets). (including graphic artifacts that embed an image ID code associated with one or more).

An AR data generation system may be virtually any system or group of systems used to generate AR data. For example, in some embodiments, the AR data generation system includes a system that provides AR authoring tools (eg, Vuforia) and interactive content authoring tools (eg, Unity, Unreal, etc.). interactive content authoring tools provide authoring tools for logically programming an interactive experience involving three-dimensional objects in a three-dimensional virtual space; AR authoring tools enable the transformation of such space into a frame of reference defined by the physical substrate to provide AR data at a predetermined position and orientation with respect to the physical substrate.

System 140 also includes an AR delivery rules module that enables customization of AR data to be delivered in a particular instance based on rules that have defined AR data set selection criteria in addition to image ID code. For example, customization of AR data may enable customized delivery of AR content based on known attributes to a viewer, including user attributes maintained in user profile data. Customization of AR data includes: selecting a profile-fit AR data set; selecting profile fitting logic; and identifying additional data (eg, stored user values, etc.) for presentation along with the AR data.

Personalization based on a user profile in some embodiments may include: (i) monitoring user engagement; (ii) updating the user profile based on such user engagement; and (iii) thereafter delivering the AR content in a changed manner based on the updated user profile. This makes AR possible.

Personalization based on a user profile in some embodiments may include: (i) monitoring user engagement; (ii) updating the user profile based on such user engagement; and (iii) thereafter delivering the AR content in the altered manner. Personalization based on a user profile in some embodiments may include: (i) monitoring user engagement; (ii) updating the user profile based on such user engagement; and (iii) thereafter delivering the AR content in a changed manner based on the updated user profile. This enables AR customization based on updated user profiles.

In some embodiments, a logical event structure that causes delivery of specific AR data for a given image ID is defined based on the identified historical user engagement data. In this way, whenever a user engages with a particular piece of print (e.g. an ad) (as opposed to delivering the same experience each time the user engages with the same ad), the AR experience is based on a logical event structure. to be changed It is optionally used to actually mimic the techniques known from automated email marketing, which also uses logical events to automatically send emails containing specific content based on user interactions with previous emails. use structures. As an example in the context of the present technology, a logical event structure can be controlled via AR material delivered to a given user based on past interactions with the given advertisement (and/or other advertisements, such as advertisements for a common product/service). makes it possible Such control may be achieved by delivery of different selections of AR data, or by changing the execution of code that implements an interactive AR experience (eg, changing the execution of a video game delivered via AR data). can

The printing system 150 causes printing of image data from the storage module 144 on a plurality of physical substrates. An exemplary physical substrate 110 is shown. Exemplary physical substrate 110 includes a main print area 111 (including artwork, margins, etc.) and one or more graphic artifacts 112A and 112B that have embedded image ID codes. In some embodiments, multiple artefacts are printed in defined locations to facilitate determination of substrate position and orientation by module 131 such that the AR animation is displayed at the correct location and orientation. In that regard, in FIG. 1 , display screen 121 provides an image capture-based representation of a physical substrate 123 (essentially a live image of image capture by an image capture device), and a physical substrate 121 in a controlled/defined position and orientation. The AR animation 124 superimposed on the display of 123 is shown. AR animation is three-dimensional (ie, extending out of the substrate using the substrate as ground level over which the animated objects are moving) or (eg, substrate 111 shown at 123 on screen 121 ). toe dimensions) to provide a rendering showing the additional material on the surface of the

Although two graphic artefacts 112A and 112B are shown, in some embodiments there may be fewer or more graphic artefacts providing access to the image ID code. In some embodiments, there is a single artifact that can inform the mobile device of the location and orientation of the physical substrate. In some embodiments, the graphic artefact is embedded in the artwork (eg, photograph).

In some embodiments, as shown in FIG. 1B , the mobile app module 130 may be configured with one or more additional networked systems, such as an e-commerce platform 191 , a social networking platform 192 and other online data sources ( 193) and is configured to communicate. Although the example of FIG. 1B shows a module 130 directly accessing such platforms, in some embodiments, accessing is via a module in the system 140 .

In the case of e-commerce platform 191, in some embodiments, the system may also be coupled to one or more e-commerce platforms so that users can interact with the AR experience, either directly through the software application, or through external hyperlinks issued within the software application. make it possible to purchase associated goods and/or services. This provides a technology that allows users to make online purchases from engagement with printed materials (eg, printed advertisements). Moreover, in some embodiments, interactions with the e-commerce platform are monitored to update the user's user profile, allowing customization of AR data in future interactions. In a preferred embodiment, module 130 (or system 140, or platform 191) is preconfigured to store payment and shipping information for a user, thereby streamlining the purchase process (eg, direct AR content, or a single interaction or optimized interaction process in which a user can purchase desired goods and/or services through interaction with objects rendered on another screen displayed together with AR content). The use of AR content as a means of driving digital engagement in physical print (such as newspaper and magazine advertisements), including engagement in the form of online purchases of goods and/or services, is a far more efficient way to direct printed advertisements. It should be understood that making it possible to drive online sales. Many advertisers have devalued print media over online advertising due to the benefits of link-click purchases made possible from online advertising; Through the technology disclosed herein, there is the ability to drive connected click purchases from print media advertisements.

In the case of social networking platform 192, this is a social networking platform that provides the ability for users to enter comments and/or reviews and/or likes, and to read comments and/or reviews and/or likes entered by other users. may include any social networking platform including networking platforms. Module 130 downloads content from such a platform that enables input and/or viewing of comments and/or reviews and/or likes as part of the AR animation, or via user interface objects displayed in conjunction with the AR animation.

Other online data sources may include:

• Data values (eg climate values, temperature values, sports scores, etc.).

• Map data (eg, via Google Maps, etc.) provided as an AR animation, eg, to provide a location and/or direction based on the user's current location. For example, the map data may be used to provide an AR map for an advertisement to indicate to the viewer the nearest location to obtain the advertiser's products or services.

Use of map data within (or launched in response to interaction with AR data) map data to validate proximal locations where goods and/or services are available for purchase and/or post-purchase collection support e-commerce functions. to be used in various embodiments. It will be appreciated that the primary benefit of the technology described herein includes enabling advertisers to drive online sales via print media advertisements (eg, via AR content with e-commerce capabilities); Some embodiments additionally/alternatively seek to support “offline store” physical retailers through a “buy and collect” approach and/or by using map data to drive in-store visits.

1C depicts a further embodiment representing a “shared experience” AR environment. In the overview, two (or more) users typically experience a common piece of AR content based on having the same triggering imprint, and one user's interaction with such content is different from another user (or multiple other users). ), a “shared experience” AR environment influencing the experience by

In the example of FIG. 1C , there are two smartphones 120 and 120' running respective instances of mobile app modules 130 and 130', respectively. Both smartphones were used in this example to capture image data that included the same print (eg, the same advertisement printed in two different physical magazines at different locations). Both instances of the mobile app module get AR data from a common cloud hosted AR data delivery module 194 . The delivery module 194 is configured to send respective smartphone data and instructions controlling delivery of AR content by smartphones, and to receive data representing interactions with the AR content (and optionally concurrently displayed content). do. Module 194 is in communication with a shared experiences execution module 195 that is configured to coordinate sharing of AU experiences among multiple users. In a further embodiment, smartphones communicate with module 195 in a manner that bypasses module 194 (eg, module 194 provides data constituting a local execution of AR content), and execution of the content. includes processes for sending data to and receiving data from module 195 to facilitate shared experience functions.

Some example use cases for shared usage AR content are considered below. They claim that they provide new ways to enable user engagement with physically printed content (eg, newspaper and magazine advertisements) where the ability to share the experience of content with users at remote locations has previously been inherently hampered. point will be understood.

• In one example, module 195 is configured to simulate a multiplayer AR game played by both smartphone 120 and users of smartphone 120' (and optionally one or more additional players) at the same time. run it Module 195 maintains an overall game context representing the current state of both players in the game via upstream data packets in this way, and the AR simulation rendered on each smartphone mimics the overall game context. downstream data packets may be provided to It will be appreciated that this is a common approach for managing online multiplayer games, and that a variety of known techniques for managing online multiplayer games may be used. However, at a practical level in the present technical framework, it enables multiple users to have a shared interactive experience with a print medium (eg, engagement between readers of the same magazine or advertisement).

• In another example, module 195 maintains current game state for a plurality of turn-based games, including AR turn-based games played on smartphone 120 and smartphone 120'. For example, the AR turn-based game may be an AR game based on a board game such as Scrabble or Chess. In this example, module 195 is configured to facilitate passage of player turn data (eg, chess moves) between players in a common game to enable turn-based player-to-player interaction. A practical example is that a scrabble board is printed on a physical substrate where AR animation superimposes the movements of a player and an opponent on the scrabble board during a competitive game. Games are preferably coordinated by a matchmaker module that identifies groups (usually pairs) of players with the players launching AR content and ready to join the game.

• In another example, module 195 manages a cooperative game in which multiple players each contribute to content in the AR animation, and the addition of each player becomes visible to all players. In some embodiments, this includes AR animations involving tasks that require teamwork from multiple players to complete. In some cases, a matchmaker module is provided to help coordinate the process in which groups of users are dynamically managed so that each group has up to a predetermined maximum number of users.

In a further example, the module 195 may allow multiple users at remote locations to create a common piece of AR content by, for example, entering data and/or changing graphical objects in a way that becomes observable by other users. It provides a cooperation module that can cooperate through In some cases, a matchmaker module is provided to help coordinate the process in which groups of users are dynamically managed so that each group has up to a predetermined maximum number of users.

• In a further example, module 195 provides a communication module that enables users to communicate (eg, via text-based messaging and/or audio-based messaging). The communication module may utilize a matchmaker module, which in some embodiments matchmaking (eg, to match users with shared characteristics, eg, in terms of language, interests, age, etc.) Match users based on analysis of user demographic information based on rules engine. In some embodiments, communication is limited to users already connected through an existing social networking platform, such that users only communicate with people in the users' existing social networks.

• In a further example, module 195 may be configured for multiple users (eg, stored demographic information and/or criteria such as social network commonality), provided that a threshold number of users make a common purchase during a common shared AR experience. provides a cooperative purchasing module in which a group constituted by a matchmaking module that creates groups based on

It will be understood that these are merely examples, and that other approaches may be used.

2A shows a method 200 according to one embodiment. This method describes an exemplary embodiment in which AR content is used as a means to enable monitoring of engagement with printed advertisements in print, eg, magazines, newspapers, and the like.

Block 201 represents a process that includes defining printable content for application to a physical substrate using, for example, a static content authoring application (such as Adobe Photoshop, etc.). Content is defined to include AR perceptual artefacts (graphical artefacts perceived by an AR software application that processes captured image data of an image to enable a determination that AR content should be displayed), which AR perceptual artefacts are Represents an AR unique identifier (UID) uniquely associated with the enablement content.

Block 202 represents a process including generating AR content associated with the AR UID at block 203 . This includes defining code, animations, rules, game execution software, web-to-data binding features, etc. for AR content that should be displayed as an overlay on the printable content. The AR content is made available at the server for download in block 204 (eg, such that the mobile device can request the download of the AR content based on the AR UID).

Block 205 represents the printing and dispensing of physical substrates, eg, printing and dispensing of a magazine, newspaper, etc., containing pages on which the printable content of block 201 is printed.

Block 206 allows a user of a mobile device (eg, a smartphone, an AR headset, a smartphone connected to an AR headset, etc.) to identify AR graphic artifacts, download the AR content based on the AR UID, and the AR content represents the process used to enable the execution of and interaction with AR content, and to execute monitoring of the interaction so that the server device receives analysis information of user interactions with the printed content.

2B shows a method 210 associated with block 206 of FIG. 2A . The user launches a specific smartphone app at 211 , which triggers the AR artifact identification process at 212 . The process of block 212 includes processing the image data collected via the camera module of the smartphone (or connected AR headset) to search for AR artifacts in the camera field of view. Such an artifact is identified at block 213 , which triggers a request (or identification of locally stored AR content) for AR content data to be sent to the server based on the AR UID extracted from the image. AR content is received and executed at 214 , and during execution, for example, as further described above, user interaction with AR animation provides a proxy for user engagement (ie, monitoring of user interaction with printed content). action) is monitored. Data derived from such monitoring is transmitted to the server at block 216 .

2C illustrates a method 230 associated with block 214 of method 210 . Block 231 indicates that AR content is received from a server, and includes objects bound to hyperlinked data (eg, AR content is a content download form including data values, images, website content, etc.) Web-based locations contain objects that serve as containers to be rendered) process. Block 232 represents configuring the access modules to fetch (pull) and/or receive (via push) data from web locations. The configuration of these modules may enable control over shorter play times for dynamically changing web content, or longer play times for generally static web content. Then at block 233, the AR content is rendered with live (ie, current) data from web sources. In one embodiment, the web content includes objects that facilitate the purchase of goods and/or services.

2D shows a method 240 that also relates to block 214 of the method 210 . In this example, block 241 indicates that the user's interaction with the AR-connected rendering on the user's device (eg, a smartphone, but in other examples, an AR headset) triggers a request to participate in a multiplayer online game. represents the process. Block 242 represents the game server performing the matchmaking process to either place the user into an existing game having an empty player slot, or to create a new game in which the user takes a player slot. Block 243 represents the process for AR content to present a multiplayer game in an ongoing two-way data exchange with a game server.

It will be appreciated that the techniques described above provide a means for users to interact with physical substrates containing printed matter, such as via a smartphone device, and such engagement to be monitored and measured for later analytical purposes.

As will be appreciated by one of ordinary skill in the art, aspects of the invention may be embodied as a system, method, or computer program product. Accordingly, aspects of the present invention are referred to herein as entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or all generally as “circuitry”, “module” or “system”. It may take the form of an embodiment combining software and hardware aspects that may be implemented. Moreover, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable media having computer readable program code embodied therein.

Any combination of one or more computer readable media may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples (not exhaustive list) of computer-readable storage media are: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory). ), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal having computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium is not a computer-readable storage medium, but can be any computer-readable medium that can communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. have.

The program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The computer program code for performing the operations for aspects of the present invention is an object-oriented programming language such as Java, Smalltalk, C++, etc. and conventional procedural programming languages such as the “C” programming language or similar programming languages, Perl, VBS, etc. or in any combination of one or more programming languages, including scripting languages, such as similar languages, and/or functional languages, such as Lisp and ML, and logic-oriented languages, such as Prolog. The program code may run entirely on the user's computer as a standalone software package, partly on the user's computer, partly on the user's computer and partly on the remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer is connected to any type of network, including a local area network (LAN) or a wide area network (WAN), or via a connection that may be made to an external computer (e.g., over the Internet using an Internet service provider). through) can be connected to the user's computer.

Aspects of the invention are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device creating a machine, such that the instructions executed through the processor of the computer or other programmable data processing device are displayed in flowcharts and/or A block diagram also creates a block or means for implementing the functions/acts specified in the blocks.

These computer program instructions may be stored on a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored on the computer readable medium are presented in flowcharts and/or A block diagram creates an article of manufacture comprising instructions implementing the function/action specified in the block or blocks.

The computer program instructions may be loaded into a computer, other programmable data processing apparatus, or other devices to cause a series of operating steps to be performed on the computer, other programmable apparatus, or other devices generating a computer implemented process such that the computer or Instructions executed on another programmable device provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the drawings illustrate the architecture, functionality and operation of possible implementations of systems, methods and computer program products in accordance with various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that includes one or more executable instructions that implement the specified logical function(s). It should also be noted that, in some alternative implementations, the functions recited in the blocks may occur out of the order recited in the figures. Depending on the functionality involved, for example, two blocks shown one after another may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Each block in the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, performs the specified functions or acts, or combinations of special-purpose hardware and computer instructions, or special-purpose hardware. It will also be noted that it may be implemented by infrastructure systems.

A computer program product may include all individual features that enable implementation of the methodologies described herein and, when loaded into a computer system, perform the methods. A computer program, software program, program, or software in this context may be directly or (a) translated into another language, code or notation; and/or (b) any language, code or notation in any language, code or notation of a set of instructions intended to cause a system having information processing capabilities to perform a particular function after either or both of its representation in a different material form. means expression.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are also intended to include the plurals unless the context clearly dictates otherwise. When the terms “comprises” and/or “comprising” are used herein, they specify the stated feature, integer, step, act, element, and/or component, but include one or more of them. It will be further understood that this does not exclude the presence or addition of other features, integers, steps, acts, elements, components, and/or groups thereof.

All means or steps in the following claims, as well as corresponding structures, materials, acts and equivalents of functional elements, if present, represent any and all elements that perform a function in combination with other claimed elements as specifically claimed. It is intended to include any structure, material or act. The description of the invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the present invention. The embodiments were chosen and described in order to best explain the principles and practical application of the invention, and to enable those skilled in the art to understand the invention, with various modifications suitable for the particular use contemplated.

Various aspects of the present invention may be embodied as a program, software or computer instructions embodied in a computer or machine-usable or readable medium that, when executed on a computer, processor and/or machine, cause the computer or machine to perform the steps of a method. have. A machine-readable program storage device is also provided that tangibly embodies a program of instructions executable by the machine to perform the various functions and methods described herein.

The systems and methods of the present invention may be implemented and executed on general purpose computers or special purpose computer systems. The terms “computer system” and “computer network,” as may be used herein, refer to various combinations of fixed and/or portable computer hardware, software, peripherals, and storage devices. may include A computer system may include a plurality of individual components that are networked or connected to perform cooperatively, or may include one or more stand-alone components. The hardware and software components of the computer system of the present application may include and be incorporated into fixed and portable devices such as desktops, laptops and/or servers. A module may be a component of a device, software, program, or system that implements some “function” that may be implemented as software, hardware, firmware, electronic circuitry, or the like.

While specific embodiments of the invention have been described, it will be understood by those skilled in the art that there are other embodiments equivalent to the described embodiments. Accordingly, it is to be understood that the invention should be limited only by the scope of the appended claims and not by the specific illustrated embodiments.

In the preceding description of exemplary embodiments of the invention, it is noted that various features of the invention are sometimes grouped together in a single embodiment, figure, or description of the invention in order to simplify the invention and aid understanding of one or more of the various aspects of the invention. point must be understood. This method of invention, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, aspects of the invention lie within all the features of a single aforesaid disclosed embodiment. Accordingly, the claims that follow the Detailed Description are hereby expressly incorporated into this Detailed Description, each claim standing on its own as a separate embodiment of the invention.

Moreover, although some embodiments described herein include some features that are not other features included in other embodiments, combinations of features of different embodiments are within the scope of the present invention and as will be understood by those skilled in the art. , which is meant to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

Moreover, some of the embodiments are described herein as a method or combination of elements of a method that may be implemented by a processor of a computer system or other means for performing the functions. Accordingly, a processor having the instructions necessary to perform such a method or element of a method forms the means for performing a method or element of a method. Moreover, the elements described herein of device embodiments are examples of means for performing the functions performed by the elements for carrying out the present invention.

In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail in order not to obscure an understanding of this description.

Likewise, it should be noted that when the term coupled is used in the claims, it should not be construed as limited to direct connections only. The terms “coupled” and “connected” may be used along with their derivatives. It should be understood that these terms are not intended as synonyms for each other. Accordingly, the scope of the expression a device A coupled to a device B will not be limited to devices or systems where the output of device A is directly coupled to the input of device B. This means that there is a path between A's output and B's input, which can be a path comprising other devices or means. “Coupled” may mean that two or more elements are in direct physical or electrical contact, or that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

Accordingly, while what has been described as being believed to be preferred embodiments of the present invention, those skilled in the art can make other and additional changes thereto without departing from the spirit of the invention, and all such changes and modifications are included within the scope of the present invention. It will be appreciated that it is intended to be claimed as being. For example, any formula given above represents only procedures that can be used. Functions may be added or deleted from the block diagrams and operations may be interchanged among functional blocks. Steps may be added to or deleted from the methods described within the scope of the present invention.

Claims (20)

A computer-implemented method that enables monitoring of user engagement with printed content, including printed content distributed through magazines, newspapers, wrappers, and booklets, comprising:
distributing a plurality of articles carrying respective copies of a common printed content item, each copy of the printed content item comprising a common graphical artifact, the common graphical artifact comprising augmented reality in a cloud hosted system (AR) associated with a set of content data;
executing an engaging application on a plurality of user mobile devices, each running instance of the engaging application comprising:
process image data captured via the camera module of the mobile device to identify the presence of the graphic artifact when the printed content item is brought into the field of view of the camera module;
obtain from the cloud-hosted system at least a subset of the set of augmented reality (AR) content data associated with the graphical artifact;
cause, on a display screen of the mobile device, a display of rendering of interactive AR content superimposed on an image capture based representation of the physical substrate, based on the obtained AR content data, and capturing the image of the physical substrate wherein a rendered animation superimposed on a base representation is configured to be displayed in a defined position and orientation relative to the image capture based representation of the physical substrate;
At a server device, data indicative of user involvement in the interactive content rendered from the augmented reality (AR) content data associated with the graphical artifact is collected from the plurality of user mobile devices, the plurality of common printed content items being generating analytical information indicative of user involvement in a copy of A device configured to enable monitoring of user engagement with a physical substrate carrying a print, comprising:
a camera module configured to capture image data of the capture area;
display screen;
microprocessor;
A memory module coupled to the microprocessor, when executed through the microprocessor:
(i) identify the presence of a graphic artifact having predetermined properties printed on the physical substrate; (ii) an image identifier extraction module configured to process the image data captured via the camera module to extract an image identifier code from the graphic artifact;
an augmented reality (AR) data access module configured to retrieve a set of executable AR data associated with the extracted image identifier code from an AR data storage module;
an AR rendering module configured to cause, on the display screen, display of a rendered animation superimposed on an image capture-based representation of the physical substrate, based on the retrieved AR data, wherein the image capture of the physical substrate an AR rendering module wherein the rendered animation superimposed on a base representation is displayed at a defined position and orientation relative to the image capture based representation of the physical substrate;
(i) monitor user interactions associated with display of the rendered animation; (ii) compiling data representative of the user interactions; (iii) communicate the data indicative of the user interactions to a server device, wherein the data indicative of the user interactions is associated with the image identifier code to provide an engagement monitoring module configured to associate engagement interactions with a particular printed matter; A device comprising: a memory module that maintains computer-executable code constituting the device. 3. The method of claim 2,
wherein the rendered animation is an interactive animation that is variably rendered in response to inputs by a user of the device. 4. The method of claim 2 or 3,
The AR data storage module maintains a plurality of sets of AR data associated with a given image identifier code, and wherein certain of the sets of AR data are assigned to the device based on one or more attributes of the user associated with the device. A device selected for delivery. 5. The method of claim 4,
wherein the one or more attributes of the user associated with the device are defined in a user profile, wherein the user profile is stored in the memory module and/or cloud-hosted data store of the device. 6. The method according to any one of claims 2 to 5,
The data store is maintained by a cloud hosted server AR data management system. A system configured to enable monitoring of user engagement with a physical substrate carrying a substrate, comprising:
A communication module that enables communication between the system and a plurality of user devices, each user device comprising:
a camera module configured to capture image data of the capture area;
display screen;
microprocessor;
A memory module coupled to the microprocessor, when executed through the microprocessor:
(i) identify the presence of a graphic artifact having predetermined properties printed on the physical substrate; (ii) an image identifier extraction module configured to process the image data captured via the camera module to extract an image identifier code from the graphic artifact;
an augmented reality (AR) data access module configured to retrieve a set of executable AR data associated with the extracted image identifier code from an AR data storage module;
an AR rendering module configured to cause, on the display screen, a display of a rendered animation superimposed on an image capture based representation of the physical substrate, based on the retrieved AR data, wherein the image capture of the physical substrate an AR rendering module wherein the rendered animation superimposed on a base representation is displayed in a defined position and orientation with respect to the image capture based representation of the physical substrate;
(i) monitor user interactions associated with display of the rendered animation; (ii) compiling data representative of the user interactions; (iii) communicate the data indicative of the user interactions to a server device, wherein the data indicative of the user interactions is associated with an image identifier code to provide an engagement monitoring module configured to associate the engagement interactions with a particular printed matter; a memory module that maintains computer-executable code constituting a device;
wherein the system includes a communication module that provides the AR storage module. 8. The method of claim 7,
wherein the rendered animation is an interactive animation that is variably rendered in response to inputs by a user of the device. 9. The method according to claim 7 or 8,
The AR data storage module maintains a plurality of sets of AR data associated with a given image identifier code, and wherein certain of the sets of AR data are assigned to the device based on one or more attributes of the user associated with the device. system selected for delivery. 10. The method of claim 9,
wherein the one or more attributes of the user associated with the device are defined in a user profile, wherein the user profile is stored in the memory module and/or cloud-hosted data store of the device. 11. The method of claim 10,
and the one or more attributes of the user are updated in response to monitoring of the user interactions associated with display of the rendered animation. 12. The method of claim 11,
The updating of the one or more attributes of the user results in changes in AR data selection and/or execution on successive interactions with a common image ID code; customizing the AR data for rendering with respect to a given image ID code. A system that results in a selected selection and/or execution. 13. The method according to any one of claims 7 to 12,
and a module for providing access to an e-commerce platform to enable the user of the user device to purchase goods and/or services based on interaction with rendered AR data. 14. The method according to any one of claims 7 to 13,
The data store is maintained by a cloud hosted server AR data management system. A method for enabling monitoring of user engagement with a physical substrate carrying a print, comprising:
defining computer-executable code representing a plurality of sets of interactive augmented reality animation data, each data set of a rendered animation superimposed on an image capture-based representation of the physical substrate on a display screen of the user device. , executable by an AR rendering module of a user device to cause a display, wherein the rendered animation superimposed on the image capture-based representation of the physical substrate has a defined position and orientation relative to the image capture-based representation of the physical substrate. A step that is represented in the;
associating each AR data set with an identifier code;
printing a given one of the identifier codes on a plurality of physical substrates, applying to a given one of the identifier codes a print medium each of which is designed to complement the AR data set associated with that identifier code; What was done;
identifying that the user device executing a defined software application is capturing image data of the physical substrate from which the given one of the identifier codes was extracted;
causing the user device to display the rendered animation of the AR data set associated with the identifier code such that the animation is rendered superimposed on the image capture based representation of the physical substrate, the image capture based representation of the physical substrate wherein the rendered animation superimposed on the representation is displayed in a defined position and orientation relative to the image capture based representation of the physical substrate;
data indicative of user interactions is associated with an image identifier code to monitor user interactions associated with the presentation of the rendered animation to associate engagement interactions with a particular print; and
and compiling data representing the user interactions. A method for enabling monitoring of user engagement with a physical substrate carrying a print, comprising:
dispensing a plurality of physical substrates carrying a common printed content item, the common printed content item being associated with a designated set of AR content in a database;
At a first mobile device, read a graphic artifact from a first of the physical substrates carrying the printed content item, and in response, obtain server data from the designated set of AR content, based on the data rendering an AR interactive game;
At a second mobile device, read the graphic artifact from a second of the physical substrates carrying the printed content item, and in response, obtain the server data from the designated set of AR content, and add to the data rendering the AR interactive game based on;
a server receives data indicative of an interaction with one of the mobile devices of the AR interactive game on one of the mobile devices, and in response, the data indicative of the interaction is and a cloud-hosted game server device communicating with the first mobile device and the second mobile device to cause a change of state of the AR interactive game executing in another. 17. The method of claim 16,
maintaining a record of the interaction with the AR content at the first mobile device and the second mobile device to enable analysis of engagement with the printed content items. A method of enabling user engagement with a physical substrate carrying a print, comprising:
dispensing a plurality of physical substrates carrying a common printed content item, the common printed content item being associated with a designated set of AR content in a database;
At a first mobile device, read a graphic artifact from a first of the physical substrates carrying the printed content item, and in response, obtain server data from the designated set of AR content, based on the data rendering the AR interactive animation;
rendering an interaction target representing a function of purchasing goods and/or services through the rendering of the AR interaction animation;
in response to a predetermined interaction with the interaction object representing the ability to purchase the goods and/or services, triggering a process for causing a purchase transaction for the user for the goods and/or services; How to. 19. The method of claim 18,
wherein the process of initiating the purchase transaction to the user for the goods and/or services comprises completing the purchase transaction using pre-stored user data including payment information and delivery information. 19. The method of claim 18,
wherein the process of causing the purchase transaction to the user for the goods and/or services comprises redirecting the user to an indication that renders a user interface that facilitates completion of the purchase transaction.

Images