EP3019955A1

EP3019955A1 - Method of optimization for an application

Info

Publication number: EP3019955A1
Application number: EP14744619.9A
Authority: EP
Inventors: Alasdair John Joseph BAILEY; Iurii SYNYCHENKO; Sergii OVCHARENKO
Original assignee: Maxymiser Ltd
Current assignee: Maxymiser Ltd
Priority date: 2013-07-09
Filing date: 2014-07-09
Publication date: 2016-05-18
Also published as: CN105683912B; JP6865252B2; GB201412190D0; GB2518936A; WO2015004452A1; JP2016528608A; GB201312280D0; CN105683912A; JP2019215895A

Abstract

The field of the invention relates to methods of optimization for a native application, to systems including a computing device on which such a native application is executable, to computing devices on which such a native application is executable, and to servers configured to communicate with computing devices on which such a native application is executable. According to an aspect of the invention, there is provided a method of optimization for a native application whose software is written in a reflection-based programming language, the native application executable on a computing device including a display and a processor, the native application configured to display a user interface on the display of the computing device, the method including the steps of: (i) applying generic code to the native application to produce a modified native application which is configured to communicate with a server external to the computing device, and (ii) remotely configuring the modified native application from the server to examine and to modify the structure and behaviour of the modified native application at run time so as to be capable of altering one or more of: the data, functions or content of a user interface generated by the modified native application.

Description

METHOD OF OPTIMIZATION FOR AN APPLICATION BACKGROUND OF THE INVENTION 1. Field of the Invention

The field of the invention relates to methods of optimization for a native application, to systems including a computing device on which such a native application is executable, to computing devices on which such a native application is executable, and to servers configured to communicate with computing devices on which such a native application is executable.

2. Technical Background Optimization of applications on computing devices is challenging, as when an application is written, it is usually not possible to envision all the uses to which such an application may be desired for use in the future. Hitherto, applications have been updated in response to various occurrences, such as to bugs which are discovered after software release, to the release of new services with which it may be desirable for an application to interact, or to protect against newly discovered security flaws in the software. However, typically the updating of applications has involved the supply of an updated software version from the supplier of the software, for installation on an associated computing device in place of the original software version. A further aspect is that the developer of the application doesn't have the possibility to compare application performance side-by- side to assess which of two or more versions of the application has the better performance eg. for sales or actions.

3. Discussion of Related Art EP1352322(A2) and EP1352322B1 disclose time-optimized replacement of a software application, and in particular a method for replacing old software that is in use with new software, which permits the maximum availability of the software. The method is subdivided into a preparation phase and an active phase. The preparation phase takes place during the operation of the old software. The active phase is merely characterized by the execution of a MOVE command.

FR2841014A1 and FR2841014(B1) disclose a method for optimizing a computer program that causes a transition between first and second predetermined states in response to an event occurring that belongs to a group of events, in which the method has the following steps: reading of a sub-group of events chosen from a group of events; modeling of a second decision graph using registers and equivalent to a first decision graph when the events of the event group do not belong do a self generating group; and generation of a software function for said transition modeling. There is further disclosed a device for optimizing a computer program.

However, it is desirable to provide greater speed or flexibility in optimizing a software application. It is also desirable to achieve a precise measurement of the response of users to a change in an application.

SUMMARY OF THE INVENTION

According to a first aspect of the invention, there is provided a method of optimization for a native application whose software is written in a reflection-based programming language, the native application executable on a computing device including a display and a processor, the native application configured to display a user interface on the display of the computing device, the method including the steps of:

(i) applying generic code to the native application to produce a modified native application which is configured to communicate with a server external to the computing device, and

(ii) remotely configuring the modified native application from the server to examine and to modify the structure and behaviour of the modified native application at run time so as to be capable of altering one or more of: the data, functions or content of a user interface generated by the modified native application.

An advantage is that the modified native application can be further modified without performing an installation of a new version of a native application. A further advantage is that the modified native application can be further modified very quickly, because it is remotely configurable from a remote server. A further advantage is that the modified native application can be further modified in response to factors determined at a remote server, because the modified native application is remotely configurable from the remote server. A further advantage is that the native application is readily modified to provide the modified native application. A further advantage is that a precise measurement of the response of users to a change in an application may be provided. There is a technical effect on a process outside the computing device, because the server can remotely configure the modified native application. The computer operates in a new way, because an altered user interface is produced by the modified native application without installation of a new version of the native application. The computer is a better computer because the modified native application may be further modified without its operation being interrupted in the way that is typically associated with installation of a new software version. The method may be one wherein the generic code is the only code needed to be applied to source code of the native application to enable optimization of the modified native application. An advantage is that modification of the native application is simply made. The method may be one wherein the generic code enables many kinds of optimizations to be remotely configured in an optimization system's user interfaces, without changing the code within multiple versions and instances of the native application. An advantage is that multiple versions and instances of the native application do not need to be modified. The method may be one wherein modifying the structure and behavior of the modified native application at runtime includes modifying one or more of the values, meta-data, properties, images and functions of the modified native application. An advantage is that the modified native application can be modified in many ways. The method may be one wherein the computing device is a mobile device, a touch screen device, a smart phone, a video game console, a computer, a smart TV, a TV set-top-box, or a device connectable to the internet. An advantage is that the method may be applied to many types of devices, including devices which are rarely updated, if ever. The method may be one wherein the computing device is connectable to the internet over a wired or wireless wide area network (WAN) or over a mobile phone network to a server. An advantage is that the method can be used for different types of connections.

The method may be one wherein optimization includes adapting visual experiences for each user on the fly in order to increase the probability of a user completing a desired action. An advantage is that increased user interaction may be obtained.

The method may be one wherein a desired action is a click or an order placement. An advantage is that increased user interaction may be obtained.

The method may be one wherein a reflection based programing language is a common reflection based programing language. An advantage is that such a language is easily used. The method may be one wherein the method further includes collecting one or more attributes about the user/device for use in targeting, profiling or analyzing user behavior. An advantage is that user-customized user interface output may be provided. The method may be one wherein the collected attributes are one of more of: Connection type to the device (e.g. 2G, 3G, 4G, Wi-Fi); Device Type; application version; Operating System type and version; Location (e.g. by coordinates, city, country, continent, postal code); Internet service provider; Recency and details of past visits to the app; Recency and details of past actions; Custom user defined attributes such as past interaction with a specific product category; Referral source where a user has the opportunity to launch the app from a variety of locations; Time of visit by date, day, hour, and Data imported from other locations such as CRM databases. An advantage is that user-customized user interface output may be provided. The method may be one further including the step of: including or excluding visitors /devices from a campaign using their attributes. An advantage is that improved focus on relevant users may be obtained.

The method may be one further including the step of: creating segmentation rules to target content to visitors /devices according to their attributes in combination with rules for other user segment information. An advantage is that improved focus on relevant users may be obtained.

The method may be one wherein native application performance is optimized during a campaign as well as after the campaign. An advantage is that the benefits produced are produced over a longer time period.

The method may be one wherein alterable user interface content includes one or more of:

a) Simple text or image block (banners, headers of the pages) movement, resizing, hiding, reformatting;

b) Buttons texts, movement, resizing, reformatting, and hiding;

c) 'Push notification' frequency and content;

d) Displayed picture (picture could be stored on external server); e) New text or image block;

Lists & tables (ordering, how many items to display);

g) Navigation paths within the application (e.g. does 'add to cart' direct the user to the cart page or back to a category page to make further selections), and h) Displaying a navigation based upon the last user choice.

An advantage is that a wide range of user interface content may be altered.

The method may be one wherein the native application includes a library. An advantage is that content can be locally stored on the computing device for repeat use, or for use during intermittent connectivity.

The method may be one wherein the library is static for all applications.

The method may be one wherein the library is configured to send requests to, and to receive responses from, a content generator. An advantage is that the library may use the most recent configuration data. An advantage is that library updating may be automated.

The method may be one wherein the library interacts with a native application cache that is included in the native application. An advantage is that content and a unique identifier for the user interacting with it can be locally stored on the computing device for repeat use, or for use during intermittent connectivity.

The method may be one wherein the native application cache is configured to receive content pushed from the external server.

The method may be one wherein the library makes a request to a content generator on native application start. An advantage is that the library's data is as up-to-date as possible, and that the user will see only the most up-to-date user interface content.

The method may be one wherein the library is arranged to make a request to a content generator, the request including a location and user attributes in parameters. An advantage is improved customization. The method may be one wherein the library includes one or more of the following modules:

CG request module;

External assets module (image transaction);

Tracking module (actions tracking, sending results to the CG);

Application object identification module;

MVT module (implement alternative content visualization);

Personalization module;

Segmentation module, and

Administrative console module.

The method may be one wherein the library can modify only specified objects and properties in code instructions. According to a second aspect of the invention, there is provided use of a method according to any aspect of the first aspect of the invention, to optimize a selectable button displayed on the display from among at least two alternatives for the selectable button. An advantage is that a most appropriate button may be displayed. According to a third aspect of the invention, there is provided use of a method according to any aspect of the first aspect of the invention, to optimize a displayed product on the display from among at least two alternatives for the displayed product, based on a most recently used filter option. An advantage is that a most appropriate product may be displayed.

According to a fourth aspect of the invention, there is provided use of a method according to any aspect of the first aspect of the invention, to optimize a displayed food product on the display from among at least two alternatives for the displayed food product, based on a most recently viewed food category. An advantage is that a most appropriate product may be displayed.

According to a fifth aspect of the invention, there is provided use of a method according to any aspect of the first aspect of the invention, to optimize a displayed advertisement on the display from among at least two alternatives for the displayed advertisement, based on recorded click through rates for the displayed advertisements. An advantage is that a most appropriate advertisement may be displayed.

According to a sixth aspect of the invention, there is provided a system comprising a server, a computing device including a display and a processor, and a modified native application executable on the computing device, the modified native application software being written in a reflection-based programming language, the modified native application configured to display a user interface on the display of the computing device, a native application having been modified by applying generic code to the native application to produce a modified native application which is configured to communicate with the server which is external to the computing device, wherein the modified native application is remotely configurable by the server to examine and to modify the structure and behaviour of the modified native application at run time so as to be capable of altering one or more of: the data, functions or content of a user interface generated by the modified native application.

An advantage is that the modified native application can be further modified without performing an installation of a new version of a native application. A further advantage is that the modified native application can be further modified very quickly, because it is remotely configurable from the remote server. A further advantage is that the modified native application can be further modified in response to factors determined at the remote server, because the modified native application is remotely configurable from the remote server. A further advantage is that a precise measurement of the response of users to a change in an application may be provided.

The system may be configured to perform a method according to any aspect of the first aspect of the invention.

The system may be one wherein the server is further configured to collect one or more attributes about the user/ device for use in targeting, profiling or analyzing user behavior.

The system may be one wherein the server is further configured to filter reports, including conversion and revenue reports, by attribute values. The system may be one wherein the server is further configured to discover segments that may require a different experience to increase conversion rates based upon the attributes. The system may be one wherein the server is configured to automatically detect higher performing content variants and then to arrange to serve those higher performing content variants to more future visitors.

The system may be one wherein the system includes the ability to switch a concluded campaign to a mode where winners only' are served, meaning that the native application supplier does not need to make an update to the code in order to apply winning variations of content or user experience.

The system may be one wherein the system includes a self-learning, predictive algorithm which drives each user to the best offer for that individual, automatically.

The system may be one wherein the system includes a segment discovery process to automate the analysis process to discover the user segments which require a different experience than the overall winner to increase performance against measures for that segment.

The system may be one wherein the system provides multivariate testing.

The system may be one wherein the server is configured to send segmentation rules to the native application.

The system may be one wherein the system is configured to track one or more of:

a) Views of a specific screen, for example the screen shown at completion of an order;

b) Button clicks or finger/ stylus taps;

c) Tracking on the dependent website pages;

d) Audio/video players playing time, and

e) Action attribute and action value opportunities (e.g. to measure sales value in £/$/€ or other currency). The system may be one wherein the system is configured to measure a response of users to a change in the application. An advantage is that a precise measurement of the response of users to a change in an application may be obtained.

According to a seventh aspect of the invention, there is provided use of the system of any aspect of the sixth aspect of the invention, wherein historical customer information, such as products held or a customer's propensity to buy a product, can be used to dynamic drive the best, personalized experience for each customer on native applications.

According to an eighth aspect of the invention, there is provided a computing device including a display and a processor, the computing device including a modified native application whose software is written in a reflection-based programming language, the modified native application executable on the device, the modified native application configured to display a user interface on the display of the computing device, the modified native application having been modified by applying generic code to a native application to produce the modified native application which is configured to communicate with a server external to the computing device, wherein the modified native application is remotely configurable by the server to examine and to modify the structure and behaviour of the modified native application at run time so as to be capable of altering one or more of: the data, functions or content of a user interface generated by the modified native application. According to a ninth aspect of the invention, there is provided a server configured to communicate with a computing device, the computing device including a display and a processor, the computing device including a modified native application whose software is written in a reflection-based programming language, the modified native application executable on the computing device, the modified native application configured to display a user interface on the display of the computing device, the modified native application having been modified by applying generic code to a native application to produce the modified native application which is configured to communicate with the server which is external to the computing device, wherein the server is arranged to remotely configure the modified native application to examine and to modify the structure and behaviour of the modified native application at run time so as to be capable of altering one or more of: the data, functions or content of a user interface generated by the modified native application. According to a tenth aspect of the invention, there is provided a system comprising a server, a plurality of computing devices each including a display and a processor, and a modified native application executable on the plurality of computing devices, the modified native application software being written in a reflection-based programming language, the modified native application configured to display a user interface on the display of each of the plurality of computing devices, a native application having been modified by applying generic code to the native application to produce a modified native application which is configured to communicate with the server which is external to the plurality of computing devices, wherein the modified native application is remotely configurable by the server to examine and to modify the structure and behaviour of the modified native application at run time so as to be capable of altering one or more of: the data, functions or content of a user interface generated by the modified native application, wherein the server is arranged to configure a plurality of modified native applications on the plurality of computing devices in a plurality of different configurations and to record measurements of user actions on the plurality of computing devices in relation to the plurality of different configurations. An advantage is that a precise measurement of the response of users to a change in an application may be provided.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects of the invention will now be described, by way of example only, with reference to the following Figures, in which:

Figure 1 shows how an example of how a test works inside an application.

Figure 2 shows an example of a creation of an optimization system, in table form.

Figures 3 to 14 show examples of specifications.

Figure 15 shows an example of states which, once a campaign is setup, can be gone through starting from implementation through to live when it is published.

Figure 16 shows examples of attributes.

DETAILED DESCRIPTION

There is provided a method of optimization for native applications. Maxymiser has created a new approach to optimize native software applications which exist on many types of internet connected devices ranging from smart phones to video game consoles, PCs, smart TVs and TV set-top-boxes. In an example, optimization means adapting visual experiences for each user on the fly in order to increase the probability of a user completing a desired action, like a click or an order placement.

The invention relates to a method of optimization for native applications whose software is written in a reflection based programing language, for example any common reflection based programing language. In computer science, reflection is the ability of a computer program to examine and modify the structure and behavior (e.g. the values, meta-data, properties and functions) of the program at runtime. The native application may run on a computer, a video game station, a smart phone or any other kind of device connected to internet, typically over a wired or wireless wide area network (WAN) or mobile phone network to a server. The method of optimization for a native application may comprise the step of integrating a native application optimization system with a native application by applying generic code to the native application, that being the only code needed to be applied to the source code of the native application to enable the native application optimization system to optimize the native application by altering one or more of: the data, functions or content assets of a user interface (UI) in the native application. This is the first time that an optimization system has been able to be integrated using a generic code for integration which can then enable many kinds of optimizations to be remotely configured in the optimization system's user interfaces, without changing the code within multiple versions and instances of the native application. Previous methods required new code to be applied for each optimization campaign, creating the requirement for a new release version to be downloaded by the user from a server before the optimization could commence. This new method removes that requirement, thereby making the optimization program faster and more sophisticated. The method may be able to collect the following attributes about the user/ device for use in targeting, profiling or analyzing user behavior:

• Device Type, application version and Operating System version

• Location (e.g. by coordinates, city, country, continent, postal code)

· Internet service provider and connection type

• Recency and details of past visits to the app

• Recency and details of past actions

• Custom user defined attributes such as past interaction with a specific product category

· Referral source where a user has the opportunity to launch the app from a variety of locations

• Time of visit by date, day, hour, day-part

• Data imported from other locations such as CRM databases The new system may include enhancements to Maxymiser's platform to optimize a native application. These enhancements may include the following features:

• Include or exclude visitors/ devices from a campaign using their attributes.

• Create segmentation rules to target content to visitors/devices according to their attributes in combination with rules for other user segment information.

· Filter reports, including conversion and revenue reports, by attribute values.

• Discover segments that may require a different experience to increase conversion rates based upon the attributes.

Tests can be configured to automatically detect higher performing content variants and then serve those to more future visitors. This means that overall app performance is optimized during the campaign as well as after it. This is known as Dynamic Optimization.

The system may include the ability to switch a concluded campaign to a mode where winners only' are served, meaning that the application owner does not need to make an update to the code in order to apply winning variations of content or user experience. Making an update to app code is often a very large investment for an app developer since it is necessary to test the app, submit it for approval to the application distributor (e.g. Apple Appstore) and then distribute it which in the case of smart televisions can involve waiting a number of months until the next planned software update distribution, the timing of which is in the hands of the hardware manufacturer, not the application developer.

In addition, Maxymiser's Full Managed Services provides support for native applications, including creative design, web development, report analysis and quality control across mobile devices. Testing with Native Applications is a development by Maxymiser. Since native application devices do not support Silverlight, the On Page Console is not provided for native application testing.

Testing for native applications may include:

• The OneTouch platform which creates an IT-less environment for marketers to make changes to their native application.

• A breadth of platform capability that can be applied to a native application, including recommendations based upon affinity patterns and a self-learning, predictive algorithm which drives each user to the best offer for that individual, automatically.

• Segment discovery to automate the analysis process to discover the user segments which require a different experience than the overall winner to increase performance against agreed measures for that segment. This now also applies to segments that may be identified by native applications.

• Use of historical customer information, such as products held or a customer's propensity to buy a product, can be used to dynamic drive the best, personalized experience for each customer on native applications.

Specification

Examples of what it is possible to test:

a) Simple text or image block (banners, headers of the pages) movement, resizing, hiding, reformatting.

b) Buttons texts, movement, resizing, reformatting, and hiding. c) 'Push notification' frequency and content

d) Displayed picture (picture could be stored on external server).

e) New text or image block.

f) Lists & tables (ordering, how many items to display).

g) Navigation paths within the application (e.g. does 'add to cart' direct the user to the cart page or back to a category page to make further selections)

h) Displaying a navigation based upon the last user choice.

i) Segmentation and personalization of the above (tailoring these items to user segments rather than the whole user base).

Examples of what it is possible to track:

a) Views of a specific screen, for example the screen shown at completion of order.

b) Button clicks or finger/stylus taps.

c) Tracking on the dependent website pages.

d) Audio/video players playing time.

e) Action attribute and action value opportunities (e.g. to measure sales value £/$/€ or other currency).

In an example of how a test may work inside an application, a mobile device can send requests to and receive responses from a content generator, which includes a content generation service. The mobile device includes a mobile application. The mobile application includes a library which is static for all applications. The library is configured to send the requests to and to receive the responses from the content generator. The library interacts with a mobile application cache that is included in the mobile application. An external server with content assets (eg. images) can push content to the mobile application cache. Figure 1 shows how an example of how a test works inside an application.

Multivariate Testing (MVT) Example

1) 'mobile version' library makes a request to the content generator (CG) on application start. 2) CG creates response with current domains' content in JavaScript Object Notation (JSON) format:

• Locations list.

• Generated variants for each location.

· Content for the required variants.

• Action list with settings and location mappings.

3) User going to open view with action.

• 'mobile version' library finds corresponding location.

• 'mobile version' library reads linked actions.

· 'mobile version' library adds events trackers if it's necessary.

• 'mobile version' library calculates necessary values and attributes.

• 'mobile version' library makes cached request to the CG with the location and the actions in parameters.

4) User going to open view without any actions or elements.

· 'mobile version' library finds corresponding location.

• 'mobile version' library makes cached request to the CG with the location.

5) User going to open view with elements.

• 'mobile version' library finds corresponding location.

· 'mobile version' library reads linked variants.

• 'mobile version' library shows elements' content.

• 'mobile version' library make cached request to the CG with the location and shown variants in parameters.

An example is shown in Figure 1.

Personalization Example

1) 'mobile version' library make request to the CG on application start.

2) CG create response with current domains' content in JSON format: · Locations list.

• User attributes with settings and location mappings.

3) User going to open view with user attribute.

• 'mobile version' library finds corresponding location. • 'mobile version' library reads linked user attributes.

• 'mobile version' library calculates necessary values and puts it to the corresponding user attributes.

• 'mobile version' library make cached request to the CG with the location and user attributes in parameters.

An example is shown in Figure 1.

Segmentation Example a) 'mobile version' library makes a request to the CG on application start.

b) CG create response with current domains' content in JSON format:

• Locations list.

• User attributes with values.

• Segmentation rules with settings.

• Generated variants for each location.

• Content for the required variants.

c) User going to open view with elements.

• 'mobile version' library finds corresponding location.

• 'mobile version' library check segmentation rules and choose the segment.

• 'mobile version' library specify necessary variants.

• 'mobile version' library shows elements' content.

• 'mobile version' library make cached request to the CG with the location, segment, and shown variants in parameters.

An example is shown in Figure 1.

Examples of what the manager can do at the administrative console

a) For the console running, user should register device ID at the Maxymiser (MM) UI.

Technical requirements / details examples.

'mobile version' library modules:

1) CG request module. 2) External assets module (image transaction).

3) Tracking module (actions tracking, sending results to the CG).

4) Application object identification module.

5) MVT module (implement alternative content visualization).

6) Personalization module.

7) Segmentation module.

8) Administrative console module.

Commercial positioning for Mobile Version

Mobile smartphone adoption is increasing at a rapid rate. Significant investments are being made in mobile applications to keep up with the fast-moving market. Extending your reach to mobile devices is an opportunity, but it requires an optimized experience to truly be effective with such a small screen.

Maxymiser's mobile version has revolutionized mobile optimization— a must for any brand investing time and budget into mobile marketing. The mobile version enables the marketer to effectively test and target content and create personalized experiences for mobile devices to increase conversion rates. Quickly optimize all of your mobile content including banner ads, data submission forms, site layout, promotions or any other content to drive the most relevant experiences for each individual user and increase revenue for your brand.

Don't you need a mobile marketing strategy to include mobile apps? The on-the-go consumer enjoys having their favorite apps at their fingertips. With the mobile version you can perform A/B and multivariate tests for mobile apps to improve the user experience without requiring end-user updates to the app. Identify a new winning experience as seasonal changes occur and dynamically change the user experience without alienating the end-user with excessive updates to the app. Or, let the mobile version adapt and determine new winners automatically, driving the most relevant experience over time.

The mobile version's reporting and analytics gives you the insight into the experiences of mobile users, identifying the winning experience for each user segment and identifying new segments that require a different experience. You'll identify the right experience and measure the overall campaign effectiveness across web sites and mobile devices.

In an example of a creation of an optimization system, in stage zero for creating a SDK specification, a specification is created, with a project timeline and requirements gathering. In stage one, for SDK developing and for UI adjustments, a library is created, MM UI modifications and web service developing occur, and interactions with an app store occur. In stage two for debugging, MM UI & SDK integration is tested with a smartphone app, internal quality assurance and bug fixing occurs, usability testing occurs, improvements and change requests occur, and drafts of production process build-up are performed. In stage three, beta testing occurs. A test spec is created, integration within the beta app and test creation are performed, beta client testing is performed, improvements and change requests are generated based on beta feedback, and a production process build-up is finalized. Figure 2 shows an example of a creation of an optimization system, in table form.

Example with iPhone SDK, or other smartphone SDK General iPhone SDK (or another smartphone SDK) functionality should allow easy library integration into customer's application. Integration instruction should occupy no more than 1 page. It should correspond to existing Maxymiser test campaign creation conception. It should allow production and sandbox modes. It should remember user personalization data (similar to the browser cookies). It should allow offline measurement.

Mobile administration console should allow:

• Changing mode from sandbox to production and back;

· Viewing and selecting interface objects and properties;

• Changing property value;

• Memorizing selected properties and values;

• Exporting selected properties and values to Maxymiser UI; Framework. Call web-service library Example

For generating web-service classes, in an example http://sudzc.com/ is used. There may be provided MM CG web-service calls. There may be provided offline calls (calls queue and postponed calls on getting online).

Framework. Load images library Example There may be provided images loading functionality. There may be provided image caching (should load only changed images).

MMCore library Example There may be provided a set of methods and properties similarly to mmcore.js:

MMCore.CGRequestO;

MMCore.setActionO;

MMCore.renderLocationElements Q ;

MMCore . s etUs er Attribute 0 ;

MMCore.setUserDefaultsO;

MMCore. setLocationQ;

MMCore.addMVTConstramtO;

MMCore. domain;

MMCore. server;

MMCore.CGRequestCallback;

Encapsulate call web-service library, load images library, MVT library, MAConsole library. There may be provided different load content modes: sync, async. May allow reading, setting user personalization criteria MVT library Example

There may be provided changing properties values according to the CG content. There may be provided property types and properties existence validation. There may be provided implementation of user constraints: • Library can modify nothing;

• Library can modify everything;

• Library can modify only specified objects and properties (in code instructions).

• Library can modify only specified objects and properties (in Admin UI).

Element variant should contain set of properties and their values.

If content not ready (hasn't yet loaded from CG) default content should be shown.

MAConsole Library Example MAConsole (mobile administration console) should provide finding, viewing and selecting interface objects and properties. There may be provided exporting selected properties and values to Admin UI. MAConsole should be shown when user is in company IP and makes some specific touches, or when debug mode set in application. MM CG web service Example

CG web-service which should return data and implement interaction with application (similar to the CG JSRenderer) Admin UI Example

New Admin UI functionality:

1. "Mobile version" campaign type.

2. "Object oriented" element type.

3. Editor for variant of "Object oriented" element type.

4. Campaign level property constraints.

Integration Example Example of how to integrate Maxymiser MVT library into a simple view-based application. Before integration you need to make sure that necessary campaigns have already been created within the Maximyser UI tool.

1. Copy MMCore classes files to the Classes folder under the "Group & files" list in your Xcode project. The classes files can be downloaded . . . 2. Include the MMCore class header to the appDelegate class header.

#import "MMCore.h"

3. Add the following to the applicationDidFinishLaunching event handler of your appDelegate implementation file.

[MMCore init: self domain: @"YOUR_DOMAIN"];

[MMCore setServer: @" cg_global.maxymiser.com/cg_v5/"];

[MMCore setConstramtMode: @"allowAll"];

[MMCore CGRequestJ;

4. Add the following to the viewDidApear event handler for all of your viewContorller implementation file.

[MMCore setLocation: self];

5. Each time you want to set the additional location, use the function below.

[MMCore setLocation: @"YourLocationName"];

6. You can use the following methods to report additional data.

[MMCore setAction: @"YourActionName"];

7. Content variations implements when location is changing. You can call element rendering at any time by the following function.

[MMCore renderLocationElements: @"YourLocationName"];

8. You can add properties scope which will accessible for modification.

[MMCore setConstramtMode: @"allowSpecified"];

[MMCore addMVTConstramt: @"objectl/*"];

[MMCore addMVTConstramt: @"object2/*"];

9. Build and run application.

Product Scenario Example

Example of how to create Maxymiser smartphone (eg. iPhone) MVT.

In Maxymiser UI

1. Choose domain.

2. Create 'mobile version' campaign.

3. Add own IP to the company IP.

In smartphone (eg. iPhone) application 4. Run necessary smartphone (eg. iPhone) application with 'mobile version' libraries.

5. Find the interface element you want to include to the campaign.

6. Switch to the MAConsole and authorize. Go to the "Sandbox" mode.

7. Use marking tool.

8. Choose interface element and its property. Add the property to the bookmark.

9. Do 5-8 again for other properties.

You can use bookmarks for creating element. Do the following:

In smartphone (eg. iPhone) application

1. Go to the bookmark list. Select bookmarks from list. Press "Create element".

2. If your session is over you need to authorize. Then choose campaign, specify element name, and press "Save". When you save element system automatically creates location (if location doesn't exist) and links the element to the location.

You can use bookmarks for creating a variant for the element. Do the following:

In smartphone (eg. iPhone) application

1. Find the interface element you want to modify.

2. Switch to the MAConsole and authorize. Go to the "Sandbox" mode.

3. Chose element from list (system will add properties from element to the bookmark).

4. Choose property from bookmark, change property value.

5. Do 3 for other properties.

6. Press "Create variant".

7. If your session is over you need to authorize. Then specify variant name and press "Save".

You can preview the variant in application. Do the following:

In smartphone (eg. iPhone) application

1. Run necessary smartphone (eg. iPhone) application with 'mobile version'.

2. Find the interface element you want to preview.

3. Switch to the MAConsole and authorize. Go to the "Sandbox" mode.

4. Choose the element from list, choose the variant, and close MAConsole. After that you will see the variant on the screen.

Examples of Specifications In an example of a specification, a primary action is "Order", which tracks an order placed through the application. The 'Value' captures the value in GBP of the order. Attributes are "delivery" = order was placed for a pizza to be delivered, and "collection" = order was placed for a pizza to be collected from a pizza store. In a secondary action, "Clicks" tracks clicks on a 'customize' button. An example is shown in Figure 3. In test variants, different button styles and colours are tested for the 'customize' button. An example is shown in Figure 4.

In an example of a specification, a primary action is "Order", which tracks an order placed through the application. The 'Value' captures the value in GBP of the order. Attributes are "delivery" = order was placed for a pizza to be delivered, and "collection" = order was placed for a pizza to be collected from a pizza store. In a secondary action, "Add to Cart" tracks whether a customized pizza has been added to a user's cart. An example is shown in Figure 5.

In an example of a specification, a primary action is "Purchase", which tracks an order placed through the application. The 'Value' captures the value in GBP of the order. In a secondary action, "ProdViews" tracks views of a product page within the application. An example is shown in Figure 6. In test variants, when a user navigates to a category browsing page, the results are filtered by the last used filter option. Variants may be by title, by price or by release, for example. Examples are shown in Figure 7.

In an example of a specification, a primary action is "FoodViews", which tracks views of one of the application food packages. In an example, both food category pages and food detail pages should be tracked. Examples are shown in Figure 8. The left hand side of Figure 8 shows an example of a food category page. The right hand side of Figure 8 shows an example of a food detail page. In an example of personalisation criteria, the last viewed food category is stored as a personalisation criteria "FoodPreference". Possible values are: cakes, breakfast, lunch, muffins, salads, sandwiches, yoghurt or blank. Examples are shown in Figure 9.

In an example of a specification, a primary action is "RadioPlay", which tracks if the user listened to a radio station. Attributes track the number of minutes of radio listened to, which may grouped into the following possible values: 0-30, 30-60, >60. In a secondary action, "BannerClicks" tracks clicks on an external banner at the top of the application page. An example is shown in Figure 10. In an example, three banner variants are included in a test. The three banner variants may be 'no advert', 'one advert' and 'two adverts'. An example is shown in Figure 11.

In an example of a specification, a primary action is "BookClicks", which tracks the clicks on a T3ook' button. The 'Value' captures the cost of the booking. Attributes tracks the agent of the journey booked, eg. Opodo. In a secondary action, "BookingComplete" tracks completed bookings through the web-in app site. In a secondary action, "Search" tracks searches performed in the application — a view of the search results page. Examples are shown in Figure 12.

In an example of a specification, a primary action is "Purchase", which tracks the completed purchases made through the application. The 'Value' captures the value of the purchase. Attributes tracks the city that the deal was purchased for. An example is shown in Figure 13.

In an example of a test page, a test page may be any of category browsing pages. In an area of a test page, a test element may be a buy colour test element. In an area of a test page, a test element may be a button test element. In an area of a test page, a test element may be an image test element. In an area of a test page, a test element may be a layout test element. An example is shown in Figure 14.

Mobile App Testing: Instructions for Site Creation

Example of How to Create a Site for Mobile App Testing (API Method)

We provide a guide to setting up a Site for a client who is licensed for Mobile App Testing. Campaign creation is described elsewhere.

The intended audience is the administrators who regularly set up Sites for clients.

Perform the following steps to prepare a site for mobile app testing: 1. Add a new site through "Admin/Site Administration" section in Maxymiser Admin UI.

The name of the site should be set according to this pattern:

<Client's name without domain>.app

For example Maxymiser.app

Next the following parameters for the new Site should be set:

"Visual Campaign Builder" - Disabled;

"Use first-party cookies" - checked;

"Mobile app testing site" - checked.

2. Select this site in the "Campaigns" tab. Go to "Site Settings/Site Browser Rules". 3. Open a "Define custom Browser Rule" tab and add a custom inclusive browser rule with the next parameters:

Browser— Any, OS— Any, Device type— Any, Version— Empty

4. Go to "Site Settings/Personalization Criteria".

5. Add three personalization criteria with the following names:

OSVersion, AppVersion, ConnectionType

6. Go to "Site Settings/Site Pages".

7. Add two site pages:

A page called "GenerationPage" with one included mask "Generations";

A page called "ActionsPage" with one included mask "Actions";

The field "Preview URL" is not used and can be populated with any URL to pass validation.

8. Add only the page "GenerationPage" to the campaign once created (see further description below). Maxymiser Mobile App Testing API and SDK Implementation Guide for Developers - Example

1 Introduction

This implementation guide provides steps for integrating the Maxymiser Mobile Application Testing solution 'MMTApp'. It describes how an A/B testing campaign can be implemented and run within native mobile applications developed for a smartphone operating system eg. iOS. This guide assumes that you are familiar with creating apps for a smartphone operating system eg. iOS.

2 Getting Started

Prerequisites for using and integrating the API with an App:

• A development environment with XCode installed.

• If you run your app on devices for testing purposes you will need to have an Apple Development Program account.

· A proficiency in development using Obj ective-C and native iOS Apps.

Note: Our SDK supports iOS 7 and onwards. For iOS6 and below the SDK can be integrated into your app, but the SDK will be inactive and you will not be able to use the

Maxymiser features.

3 Integration Guide

Our Integration Guide takes you through the six steps to complete the setup of an A/B test within an iOS App.

3.1 Campaign Creation

We have provided a number of links to our current Maxymiser UI documentation. This is where you will setup and maintain Mobile App tests. See Appendix 1 for our Terminology Notes and our glossary can be viewed in Appendix 3.

3.1.1 Setup a campaign for testing.

1. Create a Max Test campaign— full documentation can be found in our Help guide.

2. Map your campaign to the Page that has been created in the Maxymiser UI for App testing— full documentation can be found in our Help guide. Please ensure that you do not map your campaign to other Pages because the predefined Page has special properties set for App testing. 3. Create at least one Action that you would like to track inside your application— full documentation can be found in our Help guide. Please see Appendix 2 for a list of user attributes that are supported for Mobile App Testing.

4. Create an Element which will contain variants of your user experience - full documentation can be found in our Help guide. In an example, multivariate tests are not supported so please ensure that only one element is created per campaign.

5. Create at least one Variant for an Element of your user experience — full documentation can be found in our Help guide.

6. Specify Variant Weight and control— full documentation can be found in our Help guide.

For example:

We would like to change the background color and verify which one has the higher conversion rate. Create two variants of your user experience in addition to Default as this is created automatically. The Default variant (Control variant) has a white background. Variant 1 — has a green background and Variant 2 has a blue background.

Now we need to set up variant weightings.

Default - 50%.

Variant 1 - 25%.

Variant 2 - 25%.

This means: half of all users do not see any changes in UI, and won't participate in the campaign. A quarter of users will see the green background and a quarter the blue background.

3.2 SDK Integration

Now that a campaign has been configured in the Maxymiser UI we can start the process of integrating the SDK.

1. Locate and extract the MMTApp library zip file which is distributed alongside this documentation.

2. Extract the content of the SDK package.

3. Launch XCode and open the iOS application to be tested.

4. Add the following files from the SDK package to your app:

MMTApp. h

MMTApi.h

MMTExperiences.h MMTLogger.h

libMMTApp.a

5. Link your app's target with the following libraries:

CoreData.framework

SystemConfiguration. framework

libMMTApp.a

6. Insert '#import "MMTApp.h"' directive to the top of the working files in your project. The MMTApp is an entry point to the API.

7. Create and configure API:

a) In your application delegate insert the following:

/ / Create API for 'myapp' site

id <MMTApi> api = [MMTApp apiForSite:@"myapp"];

/ / Use Sandbox configuration, not Production

[api useSandbox:YES];

b) Replace 'myapp' with the name of your Site in the Maxymiser UI.

c) Later you can refer to the first created API by [MMTApp defaultApi], so there is no need to keep the reference to the created API.

Please see API Reference for more configuration options. 3.3 Experience Fetching

The code below will fetch and cache all campaigns and their variants for later use.

/ / Show interstitial page

[[MMTApp defaultApi] fetchExpenences:^{^} {

/ / Hide interstitial page and display Home page

} ];

The first argument is a block that we will invoke once experiences are fetched. It is safe to update UI inside this block as it will be invoked on the main thread.

Note: Please be aware that due to latency there may be a short delay in receiving a response from Maxymiser with details of the variants to display. Therefore, we recommend fetching experiences in a place in your App where a user may naturally experience a delay, for example an interstitial page or a search term/login input page. We recommend fetching experiences at the App launch.

3.4 Content Transformation When experiences are fetched you can use them to transform content.

There are two methods you can choose from to implement content transformations.

3.4.1 Variant Name

Base the decision logic of your application on the variant name. In this case, Variants of the user experience for the View element use the format shown in the following example. When the Variant Name is Default, the Variant Content is Empty.

When the Variant Name is Variantl, the Variant Content is Empty.

When the Variant Name is Variant2, the Variant Content is Empty. - (void)viewWillAppear:(BOOL)animated {

[super viewWillAppeananimated];

UlColor * color = nil;

NSString *variantName = [[[MMTApp defaultApi] getExperiences]

getVariantNameForCampaign:@"Campaign"

element:®" View"];

if ([@" Variantl" isEquaiToString:variantName]) {

color = [UlColor greenColor];

} else if ([@"Variant2" isEqualToString:variantName]) {

color = [UlColor blueColor];

} else {

color = [UlColor whiteColor];

}

self.view.backgroundColor = color;

[[MMTApp defaultApi] trackContentSeen:@"Campaign"];

}

3.4.2 Variant Content

You can describe content transformation in variant content. Variants of the user experience for the View element would use the format shown in the following example. When the Variant Name is Default, the Variant Content is whiteColor.

When the Variant Name is Variantl, the Variant Content is greenColor.

When the Variant Name is Variant2, the Variant Content is blueColor. - (void)viewWillAppear:(T300L)animated {

[super viewWillAppeananimated];

NSStnng *vanantContent = [[[MMTApp defaultApi] getExp

getVariantContentForCampaign:@''Campaign^l

element:®" View"];

if(variantContent != nil) {

self.view.backgroundColor = [UlColor performSelector:NSSelectorFromString(variantContent)];

} else {

self.view.backgroundColor = [UlColor whiteColor];

[[MMTApp defaultApi] trackContentSeen:@"Campaign"]; Important: When a user is shown content from a campaign, Maxymiser platform must be notified by calling

[[MMTApp defaultApi] trackContentSeen: %CampaignName%

Failure to perform this step will mean that reporting will be corrupted since no users will be recognized as entering campaign.

3.5 Action Tracking

All testing campaigns require at least one action so that you can track the impact of altering a user experience - for example, to track a conversion to sale. Use the following code to track an Action:

[[MMTApp defaultApi] trackAction:@"Purchase" value: 100 attribute:nil];

If the connection to Maxymiser is not available, action will be cached and sent once connection resumes. 3.6 QC & go-live

Environments

The Maxymiser UI supports two environments: Sandbox and Production.

3.6.1 Sandbox Sandbox is a development test environment where campaigns are created and configured. All testing on your application should be carried out in Sandbox. Sandbox traffic is served from pre-defined Company IPs.

To preview a specific Variant in-app for Quality Control (QC) testing purposes:

1. Use the Maxymiser UI - Campaign Settings > Campaign Content > Elements & Variants page to set weightings of all variants to Ό'.

2. Set the weight of the variant that you want to preview to ΊΟΟ'. Run your application and make sure that the transformation is applied correctly and that other functionality is working as expected.

3. Repeat these steps for all variants in the user experience.

4. Complete the necessary stages to trigger all configured actions.

5. Check the Action Log report to ensure that action's value and action's attribute are being logged correctly.

Note: Data captured in Sandbox mode does not appear in the Maxymiser UI general reports. This mode exists for QC testing purposes. An App must not be released with tests connected to Sandbox environment because Service Level Agreements (SLAs) are not supported and reporting will not be available.

3.6.2 Production

Production is the environment used to serve production (live user) traffic.

Data gathered from this environment is made available in the reporting area (see Maxymiser UI - Analytics tab) for analysis to determine which variant is the best performer.

The MMTApp allows the environment to be switched as follows:

#ifdef DEBUG

[api useSandbox:YES];

#endif

Important: Never release an app to end users containing a campaign which is still connected to the Sandbox environment.

3.6.3 Publish

A Publish is necessary to move a campaign between the Sandbox and Production states — full documentation can be found in our Help guide.

Once a Campaign is setup it can go through states starting from implementation through to live when it is published. The States are described in the table shown in Figure 15. 3.6.4 Ending a campaign

At the end of a campaign (when a winning user experience variant is identified) the winning variant can be weighted to 100 in the Maxymiser UI prior to republishing the campaign — this ensures that only the winning variant is displayed to users. It is recommended that the campaign is switched to the 'Not Collecting Data' state once the campaign has ended so that the performance data is frozen for future reference.

The campaign-specific code should be removed from the App when the winning user experience variant has been coded to the App codebase (e.g. at the next release).

Please see Appendix 4 for our Troubleshooting Guide

4 API Reference

MMTApp

SDK top-level class. Provides facilities to create API for a Site and contains methods for

SDK configuration.

Tasks

Managing APIs

+ defaultApi

+ setDefaultApi:

+ apiForSite:

Configuration

+ setLogger:

+ setLogLevel:

Class Methods

+ defaultApi

Returns default API.

+ (id <MMTApi>)defaultApi;

Return Value

API object set explicitly with +setDefaultApi: or first API object created with +apiForSite:, otherwise return nil.

+ setDefaultApi:

Sets default API.

+ (void)setDefaultApi:(id <MMTApi>)api;

Parameters

api

An API object to be set as default. + apiForSite:

Creates new API for the given Site.

+ (id <MMTApi>)apiForSite:(NSString *)site;

Parameters

site

The name of the Site.

Return Value

API object linked with a given Site.

+ setLogger:

Sets logger to be used by SDK.

+ (void) setLogger: (id <MMTLogger>)logger;

Parameters

logger

Logger to use.

+ setLogLevel:

Sets logging level.

+ (void)setLogLevel:(MMTLogLevel) level;

Parameters

level

Logging level.

MMTApi

Protocol to be used to fetch experiences and track actions. Tasks

Configuration

- setRequestTimeOut:

- setWiFiOnly:

- useSandbox:

Experiences

- fetchExperiences:

- fetchExperiences: forPage:

- getExperiences

Tracking

- trackAction: value: attribute:

- trackAction: value: attribute: forPage: - trackContentSeen:

Instance Methods

- setRequestTimeOut:

Sets the maximum time which can be spent making a request to our servers. Default is three seconds. When the time is exceeded, the attempt to fetch experiences will cease and the previous user experience will be selected.

- (void) setRequestTimeOut: (NSTimelnterval)timeout;

Parameters

timeout

Timeout to use.

- setWiFiOnly:

Specifies whether new experiences can be fetched only when user is connected to a Wi- Fi network. By default any available network will be used.

- (void)setWiFiOnly:(BOOL)wifiOnly;

Parameters

wifiOnly

YES to use only Wi-Fi network, NO to use any available network.

- useSandbox:

Specifies whether Sandbox configuration should be used. By default Production configuration will be used.

- (void)useSandbox:(BOOL)sandbox;

Parameters

sandbox

YES to use Sandbox configuration, NO to use Production configuration.

- fetchExperiences:

Fetches new experiences from Maxymiser platform for a default Page named 'Generations'. Fetching will be performed on a background thread.

- (void)fetchExperiences:(void (^Λ)0) handler;

Parameters

handler

A block that will be invoked once experiences are fetched.

- fetchExperiences: forPage:

Fetches new experiences from Maxymiser platform for a specified Page. Fetching will be performed on a background thread. (void)fetchExperiences:(void (^{^})Q)handler forPage:(NSString *)page;

Parameters

handler

A block that will be invoked once experiences are fetched.

page

A Page for which you want to fetch experiences.

- getExperiences

Returns fetched experiences that should be used for content transformations.

- (id <MMTExperiences>)getExperiences;

Return Value

Experiences previously fetched from Maxymiser platform, nil if experiences were not fetched.

- trackAction: value: attribute:

Tracks a user action for a default Page named 'Actions'.

- (void) trackAction: (NSString *)name value: (NSInteger)value attribute: (NSString *)attr; Parameters

name

Name for the action which should exist in the site configuration in the Maxymiser UI.

value

Integer value of the action. This can be used for the tracking of various parameters such as amount and sum.

attr

The action attribute. This can be used for the tracking of various additional information related with the action.

- trackAction: value: attribute: forPage:

Tracks a user action for a specified Page.

- (void) trackAction: (NSString *)name value: (NSInteger)value attribute: (NSString *)attr forPage: (NSString *)page;

Parameters

name

value Integer value of the action. This can be used for the tracking of various parameters such as amount and sum.

attr

page

The Page for which you want to track the action

- trackContentSeen:

Tracks a special type of action for confirmation that a user has been served the content. - (void)trackContentSeen:(NSString *)campaign;

Parameters

campaign

Campaign name which content was served.

MMTExperiences

Protocol to be used to retrieve generated variants for the fetched experiences.

Tasks

Variants

- getVariantNameForCampaign: element:

- getVariantContentForCampaign: element:

Instance Methods

- getVariantNameForCampaign: element:

Retrieves a generated variant name for a campaign and the element which is related to a test configuration in the Maxymiser UI.

- (NSString *)getVariantNameForCampaign:(NSString *) campaign element: (NSString *) element;

Parameters

campaign

Name of the campaign for which you want to get the variant name,

element

Name of the element in the campaign for which you want to get the variant name.

Return Value

The name of the generated variant for the element in the campaign. Returns nil for the following cases: the campaign is not started, the campaign is finished, the campaign is paused or the site is disabled. In any of these situations a process to render default content should be put in place.

- getVariantContentForCampaign: element:

Retrieves a generated variant content for a campaign and the element which is related to a test configuration in the Maxymiser UI.

- (NSString *)getVariantContentForCampaign:(NSString *)campaign element: (NSString *) element;

Parameters

campaign

Name of the campaign for which you want to get the variant content, element

Name of the element in the campaign for which you want to get the variant content.

Return Value

The content of the generated variant for the element in the campaign.

Returns nil for the following cases: the campaign is not started, the campaign is finished, the campaign is paused or the site is disabled. In any of these situations a process to render default content should be put in place.

MMTLogger

Protocol to be used for logging messages from the SDK. Implementation of this protocol can be provided to the MMTApp class.

Tasks

Logging

- debug:

- info:

- warning:

- error:

Instance methods

- debug:

Logs message with log level Debug.

- (void) debug: (NSString *)message;

Parameters

message Message to log.

- info:

Logs message with log level Info.

- (void)info:(NSString *)message;

Parameters

message

Message to log.

- warning:

Logs message with log level Warning.

- (void)warning:(NSString *)message;

Parameters

message

Message to log.

- error:

Logs message with log level Error.

- (void) error: (NSString *)message;

Parameters

message

Message to log.

5 Examples of possible System Limitations

In an example, the Maxymiser Mobile Library has the following system limitations:

• Each App can connect to one Maxymiser Site entity.

• The Site cannot contain more than 10 campaigns.

· The user interface contains some web-specific terminology, the guide above covers the settings that are supported, all others in the Maxymiser UI can be disregarded. If in doubt, Maxymiser can assist with campaign QC processes.

• A campaign must not contain more than one element,

o Multivariate testing is not currently supported.

· An Element cannot contain more than 10 variants.

• Segmentation, MaxPredict, MaxRecommend and VCB are not supported.

• In an example, not all user attributes are fully supported in this version, so some attributes will show 'null' or even meaningless values. See Appendix 2 for a list of supported attributes. • Filter Action by Referrer URL and Filter Action by URL parameter are not supported within mobile app testing.

6 Appendices

6.1 Appendix 1 - Terminology Notes

The UI for Mobile App testing has been re-purposed from web site testing so there are certain parts and terminology that still apply to web site testing.

• The Maxymiser UI uses the terminology (web) Site, this can be read as App name in mobile app testing

· Where the Maxymiser UI refers to a 'Page' this can be thought of as a container for Experiences and Actions.

6.2 Appendix 2 - Attributes

The core attributes which are captured in App campaigns differ from those which are captured in web campaigns as described below.

Note: Attributes that are not listed below may still be seen in the reporting user interface and may contain either 'null' or meaningless values. These should be disregarded and not used for analysis.

6.2.1 New to Apps only - Examples The Attribute 'Connection' which is the Connection type to the device - 2G, 3G, 4G, Wi-Fi.

The Attribute 'OS' which is the Operating System type and version.

The Attribute 'App Version' which is the App version. 6.2.2 Present in Apps and Web

Examples of these are shown in the Table in Figure 16.

6.3 Appendix 3— Glossary of Terms

A/B Testing: A method of testing changes in a single area of an app. App visitors are randomly shown content "A" or "B". The response to the content is analyzed to see if the difference between them is significant. A/B Testing is not limited to just two variants and is commonly used as shorthand for A/B/ C/D . . . [n] testing. Action: A conversion event performed by the visitor, for example purchase or click- through.

Campaign: The time period a campaign needs to run in order to get statistically reliable results.

Campaign -level Settings: Core configurations that apply to individual campaigns.

Content Generator (CG): A core component of the Maxymiser system: it is the engine that executes main operations during App testing. The Content Generator serves Variants and tracks all selected Actions and Personalization Criteria.

Content Variant: The alternative content created in the Maxymiser UI and served to visitors during the testing process. Each Element has at least two variants (one of which is designated as the Control Variant) .

Conversions: Data on how many conversion actions were made with the current Content Variant.

Default Variant: The content displayed on the page before testing. Maxymiser test reporting compares current (default/control) content with the content varied in the campaign.

Element: An identifiable part of an app, such as a landing page image, the size of a banner, or the copy of a call-to-action button that is to be tested.

Experience: An experience is composed of a specific set of variants over one or more pages. For example, a possible experience is a welcome page without an image, medium size banner, and call-to-action button saying 'View Our Best Offers'.

Implementation (status): The Campaign Status that reflects that a campaign has not yet gone live.

Live: Refers to a campaign configuration that is currently running on an App.

Maxymiser UI: A designated workspace where business users can work collaboratively to use Maxymiser's optimization solutions.

Page: A Maxymiser Site Page that corresponds to page(s) on the web site or App that contain Elements to be tested.

Production: The environment where visitors see campaign content.

Publish: The process that transfers a Campaign configuration from the Sandbox environment (QC test area) to the Production environment (web site/app) where Campaign content is visible to web site or App visitors.

Publish Center: The area of the Maxymiser UI where campaign content is transferred to the live web site. Ready for Publish: A feature used to inform the Publisher that a campaign configuration is complete and the campaign is ready to be made public.

Sandbox: The test environment where campaigns are created and configured.

Variant (Content Variant): A Variant of an Element is a possible configuration of that particular Element. There are at least two variants for every Element: a Default (or 'control') and at least one Challenger variant.

Variant Weight: A value used to determine the proportion of traffic that is served a Content Variant. Let's say we have two Content Variants "A" and "B". "A" has a weight of 10 and "B" has a weight of 30. In this case "A" gets 25% of the Campaign traffic, "B" gets 75%.

Visitor: A user who enters a campaign (and is tracked via a first-party cookie) and views content.

Visitor Attribute: A characteristic that identifies a web site or app visitor, such as their geographical location, activity on a site, operating system and personalization criteria, for example gender.

6.4 Appendix 4 - Trouble Shooting

6.4.1 Debug logging mechanism

The MMTApp library provides a debug logging mechanism based on standard NSLog output. All requests such as query construction, sending requests and response parsing are reflected by NSLog messages. It is therefore possible to see this log of library behavior in the Output window in Xcode and monitor it for errors. Generated log messages are prefixed with "MMTApp:"

6.4.2 Emergency Campaign Stop

In circumstances where a campaign needs to be stopped you can 'Pause' or 'Finish' the campaign by changing its state to 'Paused' or 'Finished' in the User Interface as referenced in the Publish section of this documentation.

For example a campaign may have been configured or programmed incorrectly leading to unexpected behavior of the application. Once the campaign is paused or finished, our API will return nil values.

Important: Ensure that the App is coded so that the nil response is processed correctly and the default user experience is shown.

6.4.3 Reporting If you don't see data in reports please ensure that MMTApp is configured to use Production configuration as Sandbox configuration does not support reporting. Also please ensure that Maxymiser platform is notified about content is being served (seen by the user) by calling [[MMTApp defaultApi] trackContentSeen: %CampaignName%]] 7 Development Support

If you require further assistance please raise a ZenDesk ticket or contact your account manager providing the following details where relevant:

1. Environment configuration.

a. Development machine— OS/XCode version.

b. Deployment target— simulator or device, if this is a device, please state the type and iOS version.

2. Steps to re -produce the problem.

3. The Debug level log from the MMTApp.

4. A copy of the relevant section of the application code if applicable.

Note

It is to be understood that the above-referenced arrangements are only illustrative of the application for the principles of the present invention. Numerous modifications and alternative arrangements can be devised without departing from the spirit and scope of the present invention. While the present invention has been shown in the drawings and fully described above with particularity and detail in connection with what is presently deemed to be the most practical and preferred example(s) of the invention, it will be apparent to those of ordinary skill in the art that numerous modifications can be made without departing from the principles and concepts of the invention as set forth herein.

Claims

1. Method of optimization for a native application whose software is written in a reflection-based programming language, the native application executable on a computing device including a display and a processor, the native application configured to display a user interface on the display of the computing device, the method including the steps of:

2. Method of Claim 1, wherein the generic code is the only code needed to be applied to source code of the native application to enable optimization of the modified native application.

3. Method of Claims 1 or 2, wherein the generic code enables many kinds of optimizations to be remotely configured in an optimization system's user interfaces, without changing the code within multiple versions and instances of the native application.

4. Method of any previous Claim, wherein modifying the structure and behavior of the modified native application at runtime includes modifying one or more of the values, meta-data, properties, images and functions of the modified native application.

5. Method of any previous Claim, wherein the computing device is a mobile device, a touch screen device, a smart phone, a video game console, a computer, a smart TV, a TV set-top-box, or a device connectable to the internet.

6. Method of any previous Claim, wherein the computing device is connectable to the internet over a wired or wireless wide area network (WAN) or over a mobile phone network to a server.

7. Method of any previous Claim, wherein optimization includes adapting visual experiences for each user on the fly in order to increase the probability of a user completing a desired action.

8. Method of Claim 7, wherein a desired action is a click or an order placement.

9. Method of any previous Claim, wherein a reflection based programing language is a common reflection based programing language.

10. Method of any previous Claim, wherein the method further includes collecting one or more attributes about the user/device for use in targeting, profiling or analyzing user behavior.

11. Method of Claim 10, wherein the collected attributes are one of more of: Connection type to the device (e.g. 2G, 3G, 4G, Wi-Fi); Device Type; application version; Operating System type and version; Location (e.g. by coordinates, city, country, continent, postal code); Internet service provider; Recency and details of past visits to the app; Recency and details of past actions; Custom user defined attributes such as past interaction with a specific product category; Referral source where a user has the opportunity to launch the app from a variety of locations; Time of visit by date, day, hour, and Data imported from other locations such as CRM databases.

12. Method of Claims 10 or 11, further including the step of: including or excluding visitors/ devices from a campaign using their attributes.

13. Method of any of Claims 10 to 12, further including the step of: creating segmentation rules to target content to visitors /devices according to their attributes in combination with rules for other user segment information.

14. Method of any previous Claim, wherein native application performance is optimized during a campaign as well as after the campaign.

15. Method of any previous Claim, wherein alterable user interface content includes one or more of:

i) Simple text or image block (banners, headers of the pages) movement, resizing, hiding, reformatting;

j) Buttons texts, movement, resizing, reformatting, and hiding;

k) 'Push notification' frequency and content;

1) Displayed picture (picture could be stored on external server);

m) New text or image block;

n) Lists & tables (ordering, how many items to display);

o) Navigation paths within the application (e.g. does 'add to cart' direct the user to the cart page or back to a category page to make further selections), and p) Displaying a navigation based upon the last user choice.

16. Method of any previous Claim, wherein the native application includes a library.

17. Method of Claim 16, wherein the library is static for all applications.

18. Method of any of Claims 16 to 17, wherein the library is configured to send requests to and to receive responses from a content generator.

19. Method of any of Claims 16 to 18, wherein the library interacts with a native application cache that is included in the native application.

20. Method of Claim 19, wherein the native application cache is configured to receive content pushed from the external server.

21. Method of any of Claims 16 to 20, wherein the library makes a request to a content generator on native application start.

22. Method of any of Claims 16 to 18, wherein the library is arranged to make a request to a content generator, the request including a location and user attributes in parameters.

23. Method of any of Claims 16 to 22, wherein the library includes one or more of the following modules:

9) CG request module;

10) External assets module (image transaction);

11) Tracking module (actions tracking, sending results to the CG);

12) Application object identification module;

13) MVT module (implement alternative content visualization);

14) Personalization module;

15) Segmentation module, and

16) Administrative console module.

24. Method of any of Claims 16 to 23, wherein the library can modify only specified objects and properties in code instructions.

25. Use of the method of any of Claims 1 to 24 to optimize a selectable button displayed on the display from among at least two alternatives for the selectable button.

26. Use of the method of any of Claims 1 to 24 to optimize a displayed product on the display from among at least two alternatives for the displayed product, based on a most recently used filter option.

27. Use of the method of any of Claims 1 to 24 to optimize a displayed food product on the display from among at least two alternatives for the displayed food product, based on a most recently viewed food category.

28. Use of the method of any of Claims 1 to 24 to optimize a displayed advertisement on the display from among at least two alternatives for the displayed advertisement, based on recorded click through rates for the displayed advertisements.

29. System comprising a server, a computing device including a display and a processor, and a modified native application executable on the computing device, the modified native application software being written in a reflection-based programming language, the modified native application configured to display a user interface on the display of the computing device, a native application having been modified by applying generic code to the native application to produce a modified native application which is configured to communicate with the server which is external to the computing device, wherein the modified native application is remotely configurable by the server to examine and to modify the structure and behaviour of the modified native application at run time so as to be capable of altering one or more of: the data, functions or content of a user interface generated by the modified native application.

30. System configured to perform a method of any of Claims 1 to 24.

31. System of Claims 29 or 30, wherein the server is further configured to collect one or more attributes about the user/device for use in targeting, profiling or analyzing user behavior.

32. System of Claim 31, wherein the server is further configured to filter reports, including conversion and revenue reports, by attribute values.

33. System of Claims 31 or 32, wherein the server is further configured to discover segments that may require a different experience to increase conversion rates based upon the attributes.

34. System of any of Claims 29 to 33, wherein the server is configured to automatically detect higher performing content variants and then to arrange to serve those higher performing content variants to more future visitors.

35. System of any of Claims 29 to 34, wherein the system includes the ability to switch a concluded campaign to a mode where winners only' are served, meaning that the native application supplier does not need to make an update to the code in order to apply winning variations of content or user experience.

36. System of any of Claims 29 to 35, wherein the system includes a self-learning, predictive algorithm which drives each user to the best offer for that individual, automatically.

37. System of any of Claims 28 to 36, wherein the system includes a segment discovery process to automate the analysis process to discover the user segments which require a different experience than the overall winner to increase performance against measures for that segment.

38. System of any of Claims 29 to 37, wherein the system provides multivariate testing.

39. System of any of Claims 29 to 38, wherein the server is configured to send segmentation rules to the native application.

40. System of any of Claims 29 to 39, wherein the system is configured to track one or more of:

f) Views of a specific screen, for example the screen shown at completion of an order;

g) Button clicks or finger/ stylus taps;

h) Tracking on the dependent website pages;

i) Audio/video players playing time, and

j) Action attribute and action value opportunities (e.g. to measure sales value in £/$/€ or other currency).

41. System of any of Claims 29 to 40, wherein the system is configured to measure a response of users to a change in the application.

42. Use of the system of any of Claims 29 to 41, wherein historical customer information, such as products held or a customer's propensity to buy a product, can be used to dynamic drive the best, personalized experience for each customer on native applications.

43. Computing device including a display and a processor, the computing device including a modified native application whose software is written in a reflection-based programming language, the modified native application executable on the device, the modified native application configured to display a user interface on the display of the computing device, the modified native application having been modified by applying generic code to a native application to produce the modified native application which is configured to communicate with a server external to the computing device, wherein the modified native application is remotely configurable by the server to examine and to modify the structure and behaviour of the modified native application at run time so as to be capable of altering one or more of: the data, functions or content of a user interface generated by the modified native application.

44. Server configured to communicate with a computing device, the computing device including a display and a processor, the computing device including a modified native application whose software is written in a reflection-based programming language, the modified native application executable on the computing device, the modified native application configured to display a user interface on the display of the computing device, the modified native application having been modified by applying generic code to a native application to produce the modified native application which is configured to communicate with the server which is external to the computing device, wherein the server is arranged to remotely configure the modified native application to examine and to modify the structure and behaviour of the modified native application at run time so as to be capable of altering one or more of: the data, functions or content of a user interface generated by the modified native application.

45. System comprising a server, a plurality of computing devices each including a display and a processor, and a modified native application executable on the plurality of computing devices, the modified native application software being written in a reflection- based programming language, the modified native application configured to display a user interface on the display of each of the plurality of computing devices, a native application having been modified by applying generic code to the native application to produce a modified native application which is configured to communicate with the server which is external to the plurality of computing devices, wherein the modified native application is remotely configurable by the server to examine and to modify the structure and behaviour of the modified native application at run time so as to be capable of altering one or more of: the data, functions or content of a user interface generated by the modified native application, wherein the server is arranged to configure a plurality of modified native applications on the plurality of computing devices in a plurality of different configurations and to record measurements of user actions on the plurality of computing devices in relation to the plurality of different configurations.