KR101784105B1 - Service Providing Method and System based on AB Testing - Google Patents

Abstract

A method and system for providing AB testing-based services are provided. A service providing method according to an exemplary embodiment of the present invention determines exposure rates of cyan for providing services to users and adjusts exposure rates based on conversion rates of cyan. As a result, it is possible to immediately and flexibly and adaptively reflect the results of AB testing, and to effectively manage the risks from testing.

Description

[0001] The present invention relates to a method and a system for providing service based on AB testing,

The present invention relates to a service providing method, and more particularly, to a method and system for providing a service through optimal cognition based on AB testing.

AB testing is a marketing technique that determines the best of two or more proposals (A, B, ...) for providing services to users (customers) and uses them to provide services.

AB testing is being used to select the timing of online advertising, the location of publishing, and the selection and listing of products in online shopping.

The best way to determine the best census in AB testing is the Conversion Rate (CVR) for each census. Conversion rate is the rate at which users take action that the service actually intended.

For example, online advertising is offered to 100 users as a banner advertisement at the bottom of the screen. When two users click on online advertising, the conversion rate is 2%. In addition, the main product was posted to the top 100 users who accessed the online shopping at the top of the screen, and the conversion rate is 1% when one user purchases the main product.

The marketing manager collects the conversion rate through AB testing and manages the service delivery to the highest conversion rate when the testing period expires. For example, if the conversion rate for A is 1% and the conversion rate for B is 3%, then B for the remaining 50% and B for the remaining 50% It is to provide services in.

So far, the AB testing and the provision of services based on it have had a problem that the reflection of the AB testing result is too fragmented and uniform. Also, testing must be completed, and the manager does it manually. There is also the problem that the results of AB testing are not immediate.

Furthermore, there is a risk that revenue / profits may be lowered due to a reduction in the conversion rate due to testing new scenarios. Risk management for this risk is impossible.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an AB testing-based service capable of immediately and flexibly applying adaptive AB testing results and effectively managing the risk of testing Method and system.

According to an aspect of the present invention, there is provided a service providing method for providing a service to a user, the method comprising: a first rate of exposure of a first cyan for providing a service to users; Determining a second exposure rate that is an exposure rate of the first exposure unit; Determining a first conversion rate as a conversion rate of the first cyan and a second conversion rate as a conversion rate of the second cyan; And adjusting the first exposure rate and the second exposure rate based on the first conversion rate and the second conversion rate.

The determining step may include receiving a minimum conversion rate for the service; And determining the first exposure rate and the second exposure rate so that the input minimum conversion rate can be secured.

Also, the second exposure rate is determined by the following equation,

B = PV * (1 - m / a)

Where B is the second exposure rate, PV is the total exposure rate, m is the minimum conversion rate, and a may be the first conversion rate.

The overall exposure rate for the service may include a first target exposure rate, which is an exposure rate for the first purpose, and a second target exposure rate, which is an exposure rate for the second purpose.

In addition, the first target exposure rate and the second target exposure rate may be determined based on the first conversion rate and the second conversion rate.

The second target exposure rate may vary depending on an absolute difference between the first conversion rate and the second conversion rate.

The first purpose may be to test the second cyan, and the second purpose may be to expose a cyan having a high conversion rate among the first cyan or the second cyan.

The first exposure rate includes an eleventh exposure rate that is an exposure rate of the first cyan for the first purpose and a 21st exposure rate that is an exposure rate of the first cyan for the second purpose, 2 exposure rate may include a twelfth exposure rate that is the exposure rate of the second cyan for the first purpose and a 22nd exposure rate that is the exposure rate of the second cyan for the second purpose.

In addition, the ratio of the eleventh exposure rate to the twelfth exposure rate may be predetermined.

One of the 21st exposure rate and the 22nd exposure rate may be large.

If the first conversion rate is larger than the second conversion rate and the 21st exposure rate is larger than the 22nd exposure rate and the second conversion rate is larger than the first conversion rate, Rate.

The first cyan may be a cyan currently exposed to the users to provide the service, and the second cyan may be a cyan developed to replace the first cyan.

According to another aspect of the present invention, there is provided a service providing system including: a providing unit for providing a service to user terminals; And a second exposure rate that is an exposure rate of a second cyan different from the first cyan, and a second conversion rate which is a conversion rate of the first cyan, And a controller for determining the second conversion rate as the conversion rate of the second cyan and adjusting the first exposure rate and the second exposure rate.

As described above, according to the embodiments of the present invention, it is possible to flexibly and adaptively reflect the results of the AB testing, thereby making it possible to switch to a better service while safely ensuring sales / profit.

In addition, according to the embodiments of the present invention, it is possible to immediately reflect the result of the AB testing even during the testing, and thus it is possible to guarantee the market preemption by quick application.

Furthermore, according to embodiments of the present invention, it is possible to manage a risk of sales / profit reduction due to a reduction in conversion rate, which may occur when AB testing of a new cyan is performed, to a desired level.

1 illustrates a service providing system according to an embodiment of the present invention;
Figure 2 is a flow chart provided in the description of the initial exposure rate calculation process for AB testing,
FIG. 3 is a flowchart provided for explaining the exposure rate adjustment process in the AB testing process.

Hereinafter, the present invention will be described in detail with reference to the drawings.

1. AB Testing  Based service provision system

1 is a diagram illustrating a service providing system according to an embodiment of the present invention. For convenience of understanding and explanation, the user terminals 10-1, 10-2, 10-3, ..., 10-n receiving services from the service providing system 100 are further shown.

The service providing system 100 according to the embodiment of the present invention provides a service based on AB testing. Concretely, while providing services (exposing A) to the user terminals 10-1, 10-2, 10-3, ..., 10-n into the current cyan A, (A) and B (B) are exposed in parallel, and the exposure rate in A and B is adjusted according to the AB test result.

1, the service providing system 100 that performs the functions includes a cyan management unit 110, a cyan DB 120, a service provider 130, and a service management unit 140 .

The cyan management unit 110 registers the newly developed cyan in the cyan DB 120, and discards the unnecessary cyan in the cyan DB 120.

For convenience of understanding and explanation, two cyan, an A, and a B are assumed in the embodiment of the present invention. A is a cyan which is currently exposed to the user terminals 10-1, 10-2, 10-3, ..., 10-n for service provision, and B is a newly developed cyan for replacing the inside of A.

For example, the letter A in the letter A is the letter M of the company M in the upper part of the page as the main article, the letter mm in the product list is the letter cyan listed before the other items of the same kind, And in the product list, the kk product can be seen as the cyan listed before other products of the same kind.

As another example, the advertisement A may be cyan placed in a banner type at the bottom of the page, and the advertisement B may be a cyan that pops up the advertisement at the center of the page.

As another example, in A, a content having a high number of views per day is cyan listed on the right side of the page, and a content B having a large number of views per week may be cyan listed on a right side of a page.

The service provider 130 provides services to the user terminals 10-1, 10-2, 10-3, ..., 10-n in the cyan registered in the cyan DB 120. [ During the AB testing period, the covariance that the service provider 130 exposes to provide services to the user terminals 10-1, 10-2, 10-3, ..., 10-n is selective .

In other words, services of some of the user terminals 10-1, 10-2, 10-3, ..., 10-n are exposed by providing the services of the user A, . For example, a service is provided in A for user terminals -1, 3, 5, ..., n-1 and a service is provided in B for user terminals 2, 4, 6, will be.

The service management unit 140 manages services to be provided to the user terminals 10-1, 10-2, 10-3, ..., 10-n through the service providing unit 130. [

Specifically, the service management unit 140 exposes the service A to the user terminals 10-1, 10-2, 10-3, ..., 10-n, exposes the service B, And controls the service providing unit 130 accordingly.

Meanwhile, during the AB testing, the service management unit 140 controls the service providing unit 130 while determining / adjusting the exposure rate in the A and the B exposure rate. If the exposure rate in A determined and adjusted by the service management unit 140 is 40% and the exposure rate in B is 60%, the service providing unit 130 determines whether the user terminals 10-1, 10-2, 10-3 , ..., 10-n), service is provided by exposing A to 40% and service B is exposed by 60%.

2. AB Testing  - Early Exposure rate  Calculation

Hereinafter, the process of performing the AB testing under the initiative of the service management unit 140 will be described in detail. Fig. 2 is a view provided for explaining an initial exposure rate calculation process for AB testing.

This is the process of setting the exposure rate in A and B and setting the AB testing to start when AB test is added to the service in A.

2, the service management unit 140 first obtains the current conversion rate information of the service (S210), provides the information to the marketing manager (S220), and receives the minimum conversion rate of the service from the marketing manager (S230) .

Since the current service is performed in A, the current conversion rate provided in step S220 is the conversion rate for A in step S210.

The minimum conversion rate to be input in step S230 is the degree to which the conversion rate can be tolerated by providing the service in parallel with the current cyan A and the alternative cyan B in order to verify the alternative cyan B.

For example, assume that the number of users who access the service per day is 100, and the current conversion rate is 2%. If it is analyzed that at least one service user is required per day, the minimum conversion rate to be input through step S230 is 1%.

Then, the service management unit 140 calculates the initial exposure rate in the B based on the current conversion rate obtained in step S210 and the minimum conversion rate input in step S230 (S240).

The method of calculating the initial exposure rate in step S240 is as follows.

The relationship between total exposure rate (PV) and exposure rate (A) in A and exposure rate (B) in B is as follows.

PV = A + B (1)

From the above equation (1), the exposure rate (B) in B is as follows.

B = PV - A (2)

If the relationship between the conversion rate (a) in A, the conversion rate in b (b), and the minimum conversion rate (m) satisfies the following formula (3) Exposure rate.

A * a + B * b = PV * m (3)

Here, the conversion rate (a) in A is the value found in step S210, the minimum conversion rate (m) is the value input in step S230, and the conversion rate (b) in B is an unknown number with no data yet.

If the equation (3) is expanded as follows, the exposure rate (A) in A is expressed as (7).

A * a + (PV-A) * b = PV * m (4)

A * a + PV * b - A * b = PV * m (5)

A * (a-b) = PV (m-b) (6)

A = PV * (m-b) / (a-b) (7)

Assuming that the conversion rate (b) in B is 0%, which is the worst case, the following relationship holds.

A = PV * (m / a) (8)

B = PV - A = PV * (1 - m / a) (9)

PV is 100%, the conversion rate (a) in A is the value found in step S210, and the minimum conversion rate (m) is the value input in step S230. The initial exposure rate (B) in B can be calculated based on equation (9) using these values.

If the conversion rate (a) in A is 2% and the minimum conversion rate (m) is 1% as in the previous example, the initial exposure rate (B) in B is calculated as 50% in step S240.

According to Eq. (9), it can be assumed that as the minimum conversion rate (m) decreases, the initial exposure rate (B) in B will increase. The minimum conversion rate (m) is preferably set low if the confidence in success in B is high and the minimum conversion rate (m) is preferably set high if the confidence in success in B is small.

If the minimum conversion rate (m) is extremely set to "0%", the initial exposure rate (B) in B becomes PV (= 100%).

When the initial exposure rate B in the B is calculated by the above method, the service management unit 140 also calculates the initial exposure rate A in A (S250). The initial exposure rate (A) in A can be calculated by the following equation (10).

A = PV - B (10)

3. EU Testing  - Exposure rate  adjustment

Hereinafter, a process in which the service management unit 140 adjusts the exposure rate A in the A and the exposure rate B in the B during the AB testing will be described in detail with reference to FIG. Fig. 3 is a diagram provided for explaining the exposure rate adjustment process in the AB testing process.

As shown in FIG. 3, the service management unit 140 controls the service providing unit 130 to expose the images A and B according to the initial exposure rate determined in FIG. 2 (S310). Step S310 corresponds to the process of performing the AB testing while ensuring the minimum conversion rate.

When the predetermined testing period (for example, one week) elapses (S320-Y), the service management unit 140 divides the cyan exposure into the search exposure and the focused exposure based on the AB testing result, And calculates the exposure rates of the images (S330).

Step S330 corresponds to a procedure of dividing the total exposure ratio (PV) by the exploration exposure rate (S) and the concentration exposure rate (F). The relationship between exposure rates is as follows.

PV = S + F (11)

The exploratory exposures are exposures in both A and B for AB testing purposes, and are exposed according to the respective exposure rates for A and B, respectively.

Focused exposures are those that focus exclusively on proven censuses (high conversion rates) through AB testing for marketing success (service revenue / revenue growth).

The concentration exposure rate (F) and the search exposure rate (S) are calculated according to the following formula.

F = k * | ab | ² (12)

S = PV - F (13)

Here, k is a correction coefficient. If the difference between the conversion rate (a) in A and the conversion rate (b) in B becomes 5% when k is "1", the concentration exposure rate F is 25% and the search exposure rate S is 75 %to be.

On the other hand, if the difference between the conversion rate (a) in A and the conversion rate (b) in B is 10%, the concentration exposure rate F is 100% [= total exposure rate PV] 0%. In this case, it is the case that the superior examination is proved, and the search exposure, that is, the AB testing is no longer performed.

Next, the service management unit 140 calculates the search exposure rate S (A) for A and the search exposure rate S (B) for B based on the AB testing result (S340).

The search exposure rate S (A) for A and the search exposure rate S (B) for B are calculated by bisecting the search exposure rate S calculated at S330.

S = S (A) + S (B) (14)

S (A) = S / 2

S (B) = S / 2

Thereafter, the service management unit 140 determines a cyan for assigning the centralized exposure rate F based on the AB testing result (S350).

As a result of AB testing, if the conversion rate (a) in A is higher, exposure for A is exposed for the exposure assigned to the intensive exposure, while exposure for B is exposed for the exposure assigned to the intensive exposure if the conversion rate in B is higher.

Therefore, the relationship between the concentration exposure rate F and the concentration exposure rate F (A) for A and the concentration exposure rate F (B) for B is as follows.

F = F (A) + F (B) (15)

If a> b, then F (A) = F, F (B) = 0

if B> a, then F (A) = 0, F (B) = F

In this manner, the concentration exposure rate F is allotted to the cyan having a higher conversion rate.

Next, the service management unit 140 exposes the images A and B according to the calculation results of steps S330 to S350, and continues the AB testing while providing the service (S360).

For example, if the centralized exposure rate F and the search exposure rate S are calculated as 40% and 60%, respectively, at step S330, and 2) the search exposure rate S (A)] and the search exposure rate [S (B)] for B are calculated as 20%, respectively, and 3) if the conversion rate (b) 60% is allocated within B.

Therefore, in this case, in the case of the service providing the AB testing in step S360, the exposure is exposed to 20% in the A test and 80% (= 20% + 60%) in the B test.

Next, step S320 is performed again, and when a predetermined testing period has elapsed (S320-Y), the procedure is repeated from step S330.

4. Variation example

Up to now, preferred embodiments of the method and system for providing AB testing-based services have been described in detail.

In the above embodiment, the search exposure rate S (A) for A and the search exposure rate S (B) for B are the same, which can be changed. For example, it is determined as exposure rates that can guarantee a minimum conversion rate.

Also, in determining the search exposure rate S and the concentration exposure rate F, it is possible to change the method presented in the above embodiment. In other words, the search exposure rate S is determined so as to ensure a minimum conversion rate.

Further, in the above embodiment, it has been assumed that the centralized exposure rate F is allotted to the higher conversion rate, which can be modified by assigning more to the higher conversion rate as follows.

F = F (A) + F (B) (16)

F (A) = 0.8 * F, F (B) = 0.2 * F

F (A) = 0.2 * F, F (B) = 0.8 * F

Furthermore, the values 0.8 and 0.2 mentioned in the above example can be changed to other values.

It goes without saying that the technical idea of the present invention can also be applied to a computer-readable recording medium having a computer program for performing the functions of the apparatus and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention may be embodied in computer-readable code form recorded on a computer-readable recording medium. The computer-readable recording medium is any data storage device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. In addition, the computer readable code or program stored in the computer readable recording medium may be transmitted through a network connected between the computers.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

100: service providing system
110:
120: Cyan DB
130: Service Offering
140:

Claims (13)

The service providing system comprising: determining a first exposure rate that is an exposure rate of a first cyan for providing services to users and a second exposure rate that is an exposure rate of a second cyan different from the first cyan;
The service providing system comprising: a step of recognizing a first conversion rate as a conversion rate of the first cyan and a second conversion rate as a conversion rate of the second cyan; And
And the service providing system adjusting the first exposure rate and the second exposure rate based on the first conversion rate and the second conversion rate,
The overall exposure rate for the service may be,
A first target exposure rate which is an exposure rate for a first purpose of testing the second cyan and a second target exposure rate which is a second target exposure rate for exposing a cyan having a high conversion rate among the first cyan or the second cyan 2-purpose exposure rate '
Wherein the first target exposure rate and the second target exposure rate are selected from the group consisting of:
Based on the first conversion rate and the second conversion rate,
The first exposure rate may be, for example,
An eleventh exposure rate as an exposure rate of the first cyan for the first purpose and a 21st exposure rate as an exposure rate of the first cyan for the second purpose,
The second exposure rate may be, for example,
A second exposure rate, which is an exposure rate of the second cyan for the first purpose, and a second exposure rate, which is an exposure rate of the second cyan for the second purpose,
The first target application exposure rate may be, for example,
The sum of the eleventh exposure rate and the twelfth exposure rate,
The second target exposure rate may be selected from the group consisting of:
The second exposure rate, the second exposure rate, and the second exposure rate.
The method according to claim 1,
Wherein,
Receiving, by the service providing system, a minimum conversion rate for the service; And
And determining the first exposure rate and the second exposure rate so that the service providing system can secure the input minimum conversion rate.
The method of claim 2,
The second exposure rate is determined by the following equation,
B = PV * (1 - m / a)
Where B is the second exposure rate, PV is the total exposure rate, m is the minimum conversion rate, and a is the first conversion rate.
delete delete The method according to claim 1,
The second target exposure rate may be selected from the group consisting of:
Wherein the difference between the first conversion rate and the second conversion rate is dependent on an absolute difference between the first conversion rate and the second conversion rate.
delete delete The method according to claim 1,
Wherein the ratio of the eleventh exposure rate to the twelfth exposure rate is predetermined.
The method according to claim 1,
Wherein one of the 21st exposure rate and the 22nd exposure rate is larger.
The method of claim 10,
If the first conversion rate is larger than the second conversion rate, the 21st exposure rate is larger than the 22nd exposure rate,
And if the second conversion rate is larger than the first conversion rate, the 22nd exposure rate is larger than the 21st exposure rate.
The method according to claim 1,
Wherein the first cyan is a cyan currently present to the users for providing the service,
Wherein the second cyan is a cyan developed to replace the first cyan.
A providing unit for providing a service to user terminals; And
Determining a first exposure rate as an exposure rate of the first cyan for providing the service and a second exposure rate as an exposure rate of the second cyan different from the first cyan; And a management unit for recognizing a second conversion rate as a conversion rate of the second cyan and adjusting the first exposure rate and the second exposure rate,
The overall exposure rate for the service may be,
A first target exposure rate which is an exposure rate for a first purpose of testing the second cyan and a second target exposure rate which is a second target exposure rate for exposing a cyan having a high conversion rate among the first cyan or the second cyan 2-purpose exposure rate '
Wherein the first target exposure rate and the second target exposure rate are selected from the group consisting of:
Based on the first conversion rate and the second conversion rate,
The first exposure rate may be, for example,
An eleventh exposure rate as an exposure rate of the first cyan for the first purpose and a 21st exposure rate as an exposure rate of the first cyan for the second purpose,
The second exposure rate may be, for example,
A second exposure rate, which is an exposure rate of the second cyan for the first purpose, and a second exposure rate, which is an exposure rate of the second cyan for the second purpose,
The first target application exposure rate may be, for example,
The sum of the eleventh exposure rate and the twelfth exposure rate,
The second target exposure rate may be selected from the group consisting of:
The twenty-first exposure rate, and the twenty-second exposure rate.

Images