CN112052153A

CN112052153A - Product version testing method and device

Info

Publication number: CN112052153A
Application number: CN201910492045.8A
Authority: CN
Inventors: 熊涛
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-06-06
Filing date: 2019-06-06
Publication date: 2020-12-08
Anticipated expiration: 2039-06-06
Also published as: CN112052153B

Abstract

The application discloses a method and a device for testing product versions, and belongs to the technical field of computers. The method comprises the following steps: dividing a plurality of user requests into Q user requests; determining P group user requests corresponding to each test group according to P sequences corresponding to each test group in a plurality of test groups; testing the product version in each test group according to the user request corresponding to each test group; each sequence comprises P elements, and Q elements in the P sequences correspond to Q user requests one by one; the intersection of any two sequences corresponding to the same test group is an empty set, and the intersection of any two sequences corresponding to different test groups comprises an element; and the P groups of user requests correspond to the P sequences one by one, and each group of user requests comprises P user requests corresponding to P elements in the corresponding sequence. The problem that the accuracy of the test result of a plurality of test groups is lower is solved to this application, and this application is used for the test of product version.

Description

Product version testing method and device

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for testing a product version.

Background

With the development of the internet, rapid iteration of internet product versions has become a norm. Before releasing a new version of a product, an operator often needs to test multiple product versions based on a large number of user requests to select a target product version which can best meet the user requirements from the multiple product versions, and then releases the target product version as the new version.

In the related art, product versions in a plurality of test groups can be tested based on a large number of acquired user requests. Wherein each test set includes a plurality of product versions. When testing product versions in multiple test groups, an allocation method may be first adopted to divide the obtained user request into multiple user requests, and a certain test group may be tested based on the currently divided user requests of different shares. Then, another allocation method is adopted to subdivide the user requests into a plurality of user requests, and the next test group is tested based on the currently divided user requests of different shares.

However, when different test groups are tested, each user request for testing any test group is required to be uniformly distributed in different user requests for testing other test groups, so that the accuracy of the test result of each test group can be ensured. However, the user request often cannot be divided according to the above requirements by adopting the division manner in the related art, so that the accuracy of the test results of the plurality of test groups is low.

Disclosure of Invention

The application provides a product version testing method and device, which can solve the problem that the accuracy of the test results of a plurality of test groups is low, and the technical scheme is as follows:

in a first aspect, a method for testing a product version is provided, where the method includes:

dividing a plurality of user requests into Q user requests, wherein Q is P, and P is a prime number;

determining P group user requests corresponding to each test group according to P sequences corresponding to each test group in a plurality of test groups, wherein each test group comprises a plurality of product versions;

testing the product version in each test group according to the user request corresponding to each test group;

each sequence comprises P elements, and Q elements in the P sequences correspond to Q user requests one by one; the intersection of any two sequences corresponding to the same test group is an empty set, and the intersection of any two sequences corresponding to different test groups comprises an element; and the P groups of user requests correspond to the P sequences one by one, and each group of user requests comprises P user requests corresponding to P elements in the corresponding sequence.

Optionally, before the P sequences corresponding to each test group in the plurality of test groups, the method further includes:

determining a target matrix comprising P rows and P columns of elements, wherein each element in the target matrix is a natural number less than or equal to Q, and any two elements are different;

determining P sequences corresponding to each test group according to the target matrix;

wherein, the kth element in the jth sequence corresponding to the ith test group in the plurality of test groups is: the k row of the target matrix has the elements of column [ (j-1) + (k-1) × (i-1) ] mod P ] +1, i is more than or equal to 1 and less than or equal to P, j is more than or equal to 1 and less than or equal to P, and [ (j-1) + (k-1) × (i-1) ] mod P represents the remainder of [ (j-1) + (k-1) × (i-1) ] on P.

Optionally, in the target matrix, the element in the mth row and the nth column is (m-1) × P + n-1 or (m-1) × P + n, 1 ≦ m ≦ P, and 1 ≦ n ≦ P.

Optionally, each user request in the plurality of user requests carries a target number, and the dividing the plurality of user requests into Q user requests includes:

and dividing the plurality of user requests into the Q user requests according to the target numbers in the plurality of user requests.

Optionally, the target number carried by the user request includes: a user identification number;

or, the target number carried by the user request includes: an identification number of the user request;

or the target number carried by the user request comprises a first number segment and a second number segment, the first number segment comprises the user identification number, and the second number segment comprises the generation date of the user request.

Optionally, the dividing the plurality of user requests into the Q user requests according to the target numbers in the plurality of user requests includes:

carrying out Hash processing and Q remainder processing on the target number carried by each user request in sequence to obtain the number of each user request;

and dividing the user requests with the same number in the plurality of user requests into the same user request.

Optionally, the hash processing includes: and (6) carrying out DJB hash processing.

processing the target number carried by each user request by adopting a method for generating a random number to obtain the number of each user request;

In a second aspect, there is provided a product version testing apparatus, comprising:

the device comprises a dividing module, a judging module and a processing module, wherein the dividing module is used for dividing a plurality of user requests into Q user requests, Q is P, and P is a prime number;

a first determining module, configured to determine, according to P sequences corresponding to each test group in multiple test groups, a P group user request corresponding to each test group, where each test group includes multiple product versions;

the testing module is used for testing the product version in each testing group according to the user request corresponding to each testing group;

Optionally, the product version testing apparatus further includes:

a second determining module, configured to determine a target matrix including P rows and P columns of elements, where each element in the target matrix is a natural number less than or equal to Q, and any two elements are different;

a third determining module, configured to determine, according to the target matrix, P sequences corresponding to each test group;

Optionally, the dividing module includes:

and the dividing submodule is used for dividing the plurality of user requests into the Q user requests according to the target numbers in the plurality of user requests.

Optionally, the partitioning sub-module is configured to:

In a third aspect, a product version testing apparatus is provided, which includes: a processor and a memory, the memory having stored therein at least one instruction that is loaded and executed by the processor to implement the product version testing method of the first aspect.

In a fourth aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the product version testing method according to the first aspect.

In a fifth aspect, there is provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the product version testing method of the first aspect.

The beneficial effect that technical scheme that this application provided brought includes at least:

in the product version testing method provided by the embodiment of the invention, after a plurality of user requests are divided into Q user requests, P group user requests corresponding to each testing group are determined according to P sequences corresponding to each testing group. Because the intersection of any two sequences corresponding to the same test group in the multiple test groups is an empty set, the user requests of each group corresponding to the same test group are different. And because the intersection of any two sequences corresponding to different test groups comprises one element, the intersection of any two groups of user requests corresponding to different test groups only comprises one piece of data, so that each group of user requests corresponding to each test group are all distributed to all groups of user requests corresponding to other test groups. In this way, the determined user requests corresponding to the test groups are orthogonal, so that the accuracy of testing the test groups according to the user requests corresponding to the test groups is higher.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram of an AB test provided in an embodiment of the present invention;

FIG. 2 is a schematic diagram of a plurality of test sets according to an embodiment of the present invention;

fig. 3 is a flowchart of a product version testing method according to an embodiment of the present invention;

FIG. 4 is a flowchart of another method for testing product versions according to an embodiment of the present invention;

FIG. 5 is a diagram illustrating a sequence corresponding to a plurality of test sets according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a product version testing apparatus according to an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of another product version testing apparatus according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a partitioning module according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a server according to an embodiment of the present invention;

fig. 10 is a block diagram of a terminal according to an exemplary embodiment of the present invention.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

With the development of the internet, rapid iteration of internet product versions has become a norm. Operators often need to test multiple product versions based on a large number of user requests to select a target product version among the multiple product versions that best meets the user's needs. The process of the above test may be referred to as an AB test.

For example, fig. 1 is a schematic diagram of an AB test according to an embodiment of the present invention, as shown in fig. 1, when testing multiple product versions (e.g., product version a and product version B), a large number of user requests need to be collected, and the user requests need to be divided into multiple groups of user requests (e.g., divided into a first group of user requests and a second group of user requests in fig. 1). Then, a first group of user requests can be adopted to test the product version A so as to determine the preference degree of the user to the product version A; and testing the product version B by adopting a second group of user requests to determine the preference of the user to the product version A. And finally, according to the preference degree of the user to each product version, screening out the target product version with the highest user preference degree from the multiple product versions.

It should be noted that the product version a and the product version B have different policies (in this case, the product version a and the product version B have differences in certain parameters). For example, the product version a and the product version B have different ways of pushing information to the user, or the product version a and the product version B have different ways of interacting with the user, or the product version a and the product version B have different interfaces, or the product version a and the product version B have different preferential activities corresponding to the product version a and the product version B. In fig. 1, the interface of product version a is different from the interface of product version B.

The preference degree of the user for the product version can be embodied by various parameters, for example, the preference degree can be embodied by the click rate of the user for the interface. And when the click rate of the user on the interface of the product version A is greater than that of the interface of the product version B, indicating that the preference of the user on the product version A is higher.

When performing AB testing, a large number of user requests are typically employed to test product versions of multiple test groups (which may also be referred to as test layers). Each test group includes a plurality of product versions, and the product versions may include different versions of one product, and may also include different versions of a plurality of products, which is not limited in this embodiment of the present invention.

For example, fig. 2 is a schematic diagram of a plurality of test groups according to an embodiment of the present invention, as shown in fig. 2, when testing a test group 1, an allocation method may be first adopted to divide an obtained user request into a plurality of user requests (e.g., a 1 st user request, a 2 nd user request, and a 3 rd user request corresponding to the test group 1 in fig. 2), and the test group 1 is tested based on different currently divided user requests. Then, another allocation method is adopted to subdivide the user requests into multiple user requests (such as the 1 st user request, the 2 nd user request, and the 3 rd user request corresponding to the test group 2 in fig. 2), and the test group 2 is tested based on the currently divided user requests of different shares.

However, when testing different test groups, it is often required that user requests for testing different test groups are orthogonal to each other. That is, each user request required to test any test group can be evenly distributed among the different user requests used to test other test groups. In this way, the accuracy of the test results of each test set can be guaranteed. For example, the 1 st user request corresponding to the test group 1 needs to be uniformly distributed in the 1 st user request, the 2 nd user request and the 3 rd user request corresponding to the test group 2. However, the user requests cannot be divided by adopting the dividing manner in the related art, for example, the 1 st user request corresponding to the test group 1 is only distributed in the 1 st user request and the 2 nd user request corresponding to the test group 2, but not distributed in the 3 rd user request corresponding to the test group 2, which results in low accuracy of the test results of the plurality of test groups.

The embodiment of the invention provides a product version testing method, which can use a product version testing device, and the dividing mode adopted in the product version testing method can divide user requests according to the requirements, so that the accuracy of the testing result of each testing group is higher.

Fig. 3 is a flowchart of a product version testing method according to an embodiment of the present invention, and as shown in fig. 3, the product version testing method may include:

step 301, dividing the plurality of user requests into Q user requests, where Q is P × P, and P is a prime number.

Step 302, determining a P-group user request corresponding to each test group according to P sequences corresponding to each test group in a plurality of test groups, each test group including a plurality of product versions.

Step 303, testing the product version in each test group according to the user request corresponding to each test group.

In step 303, the product version testing apparatus may perform an AB test on the product versions in each test group according to the user request corresponding to each test group, so as to determine the product version in each test group that best meets the user requirement.

In summary, in the product version testing method provided in the embodiment of the present invention, after dividing a plurality of user requests into Q user requests, the P group user request corresponding to each test group is determined according to the P sequences corresponding to each test group. Because the intersection of any two sequences corresponding to the same test group in the multiple test groups is an empty set, the user requests of each group corresponding to the same test group are different. And because the intersection of any two sequences corresponding to different test groups comprises one element, the intersection of any two groups of user requests corresponding to different test groups only comprises one piece of data, so that each group of user requests corresponding to each test group are all distributed to all groups of user requests corresponding to other test groups. In this way, the determined user requests corresponding to the test groups are orthogonal, so that the accuracy of testing the test groups according to the user requests corresponding to the test groups is higher.

In addition, when a plurality of test groups are tested in the related art, the user request needs to be divided for a plurality of times, which results in low efficiency of product version testing. In the product version testing method provided by the embodiment of the present invention, no matter how many test groups need to be tested, only one division of the user request needs to be performed (see step 301 for details), so that the testing efficiency of the product version testing method provided by the embodiment of the present invention is high.

Fig. 4 is a flowchart of another product version testing method according to an embodiment of the present invention, and as shown in fig. 4, the product version testing method may include:

step 401, determining a target matrix including P rows and P columns of elements, where any two elements in the target matrix are different.

It should be noted that the product version testing method provided by the embodiment of the present invention may be executed by a product version testing apparatus. In step 401, the product version testing apparatus needs to determine a prime number P first, and then determine an object matrix including P rows and P columns.

Where a prime number is also called a prime number, in a natural number greater than 1, the prime number has no other factors than 1 and the prime number itself. For example, P in the embodiment of the present invention may be 2, 3, or 5, and P in the embodiment of the present invention is taken as 3 as an example.

The target matrix may be any one of matrices including P rows and P columns of elements, and each element in the target matrix is a natural number less than or equal to Q, and each element in the target matrix is different from each other. For example, in the embodiment of the present invention, the element in the mth row and the nth column in the target matrix may be (m-1) × P + n-1, 1 ≦ m ≦ P, 1 ≦ n ≦ P, and in this case, if P ≦ 3, the target matrix may be:

alternatively, the element in the mth row and the nth column in the target matrix may also be (m-1) × P + n, and if P is 3, the target matrix may be:

step 402, determining P sequences corresponding to each test group in the plurality of test groups according to the target matrix.

Each of the plurality of test sets may include a plurality of product versions, which may include multiple versions of the same product or multiple versions of different products. Each sequence corresponding to each test group may include P elements in the target matrix, and then P sequences corresponding to each test group may include Q elements in the target matrix. Q is P x P, i.e. Q is equal to the product of P and P.

In step 402, the product version testing apparatus may sequentially calculate each element in each sequence corresponding to each test group according to the target matrix obtained in step 401. For example, if the product version testing apparatus needs to calculate the kth element in the jth sequence corresponding to the ith test set in the plurality of test sets, the product version testing apparatus may use the kth row of the target matrix as the kth element, where 1 ≦ i ≦ P, 1 ≦ j ≦ P, 1 ≦ k ≦ P, and [ (j-1) + (k-1) × (i-1) ] mod P represents the remainder of [ (j-1) + (k-1) × (i-1) ] P for P.

For example, when the product version test apparatus needs to calculate the 1 st element in the 1 st sequence corresponding to the 1 st test group, the product version test apparatus may determine that i is 1, j is 1, and k is 1, at this time, the product version test apparatus may regard the 1 st row and 1 st column element in the target matrix as the 1 st element in the 1 st sequence corresponding to the 1 st test group. When the product version testing apparatus needs to calculate the 3 rd element in the 3 rd sequence corresponding to the 2 nd test set, the product version testing apparatus may determine that i is 2, j is 3, and k is 3, at this time, the product version testing apparatus may use the 2 nd column element in the 3 rd row in the target matrix as the 3 rd element in the 3 rd sequence corresponding to the 2 nd test set.

Fig. 5 is a schematic diagram of a sequence corresponding to a plurality of test groups according to an embodiment of the present invention. It should be noted that fig. 5 illustrates an example in which the plurality of test sets includes three test sets. Optionally, the number of the plurality of test sets may also be other values, such as 2 or 4.

Referring to fig. 5, each test set may correspond to three sequences, such as test set 1 to sequence 1, sequence 2, and sequence 3, test set 2 to sequence 4, sequence 5, and sequence 6, and test set 3 to sequence 7, sequence 8, and sequence 9. Wherein, the sequence 1 comprises: element 0, element 3, and element 6, sequence 2 includes: element 1, element 4 and element 7, sequence 3 comprising: element 2, element 5 and element 8, sequence 4 comprising: element 0, element 4, and element 8, sequence 5 includes: element 1, element 5 and element 6, sequence 6 comprising: element 2, element 3 and element 7, the sequence 7 comprising: element 0, element 5, and element 7, sequence 8 comprising: element 1, element 3 and element 8, the sequence 9 comprising: element 2, element 4 and element 6.

On one hand, the intersection of any two sequences corresponding to the same test group is an empty set, for example, in the sequence 1, the sequence 2, and the sequence 3 corresponding to the test group 1 in fig. 5, any two sequences do not include the same element, and thus, the intersection of any two sequences is an empty set.

On the other hand, the intersection of any two sequences corresponding to different test sets includes one element. For example, in fig. 5, sequence 1 corresponding to test set 1 and sequence 1 corresponding to test set 2 both include element 0, and other elements are different, so that the intersection of sequence 1 corresponding to test set 1 and sequence 1 corresponding to test set 2 includes only one element. Similarly, the intersection of the sequence 1 corresponding to the test set 1 and the sequence 2 corresponding to the test set 2 only includes one element, and the intersection of the sequence 1 corresponding to the test set 1 and the sequence 3 corresponding to the test set 2 also includes only one element. It can be seen that the elements in the sequence 1 corresponding to the test group 1 are all divided into the sequences corresponding to the test group 2, and the elements in the sequence 1 corresponding to each test group are all divided into the sequences corresponding to other test groups.

It should be noted that there may be various ways to obtain the sequences corresponding to the multiple test groups provided in the embodiment of the present invention, and in the embodiment of the present invention, the product version test apparatus obtains P sequences corresponding to each test group by using the methods shown in step 401 and step 402 as an example. Alternatively, the product version testing apparatus may directly receive the P sequences corresponding to each test group input by the worker.

It should be noted that, in the above embodiment, 1 is taken as an example of the minimum count. Optionally, in the above embodiment, the count may also be minimum count of 0, at this time, the 1 st test group in the plurality of test groups may be referred to as a 0 th test group, the identifier of the 1 st sequence may be referred to as a 0 th sequence, the 1 st element in the sequence may be referred to as a 0 th element, the 1 st row of the target matrix may be referred to as a 0 th row, and the 1 st column of the target matrix may be referred to as a 0 th column. And the Kth element in the J-th sequence corresponding to the I-th test group in the plurality of test groups is: the K row of the target matrix has the elements in the [ (J + K I) mod P ] column, I is more than or equal to 0 and less than or equal to P-1, J is more than or equal to 0 and less than or equal to P-1, and K is more than or equal to 0 and less than or equal to P-1, wherein (J + K I) mod P represents the remainder of (J + K I) on P.

Optionally, the plurality of test sets in fig. 5 may include a test set 4 (not shown in fig. 5) in addition to

test sets

1, 2, and 3, and the test set 4 may correspond to sequence 10, sequence 11, and sequence 12. Wherein, the sequence 10 comprises: element 0, element 1, and element 2, the sequence 11 comprising: element 3, element 4 and element 5, the sequence 12 comprising: element 6, element 7 and element 8.

Step 403, dividing the plurality of user requests into Q user requests.

It should be noted that the multiple user requests may be acquired in advance or acquired in real time, which is not limited in the embodiment of the present invention. The product version testing apparatus needs to divide the plurality of user requests roughly into Q user requests in step 403.

Each user request may carry a target number, and when the product version testing device divides a plurality of user requests into Q user requests, the product version testing device may divide the plurality of user requests into Q user requests according to the target numbers in the plurality of user requests. Wherein, the target number may include: the user identification numbers are different from the target numbers carried by the user requests sent by different users; or, the target number requested by the user includes: the identification numbers requested by the users are different from the target numbers requested by different users; or the target number carried by the user request comprises a first number segment and a second number segment, the first number segment comprises the user identification number, and the second number segment comprises the generation date of the user request. Of course, the target number may be another number, as long as a plurality of user requests can be divided into Q user requests substantially uniformly according to the target number, which is not limited in the embodiment of the present invention.

On one hand, when dividing the plurality of user requests into Q user requests according to the target numbers in the plurality of user requests, the product version testing device may perform hash processing and Q remainder processing (remainder processing is also referred to as modulo processing) on the target number carried by each user request in sequence to obtain the number of each user request. And then, the product version testing device divides the user requests with the same number in the plurality of user requests into the same user request.

It should be noted that after performing hash processing and Q remainder processing on the target number carried by each user request, the number of each user request obtained by the product version testing apparatus is smaller than Q, and the generated numbers of all user requests collectively include Q numbers. For example, the number of user requests may be: 0. 1, 2, 3, 4, 5, 6, 7 or 8.

Optionally, the hash processing may include: the DJB hash processing may also be other hash processing methods, such as murmurmur hash processing, which is not limited in this embodiment of the present invention. The Murmur hash processing and the DJB hash processing are two different hash processing modes, and the DJB hash processing is invented by Digitalists Daniel Bernstein (Daniel J.Bernstein).

On the other hand, the product version testing device may not adopt the hash processing and the Q remainder processing when dividing the plurality of user requests into Q user requests according to the target numbers in the plurality of user requests. For example, the product version testing apparatus may process the target number carried by each user request by using a method of generating a random number to obtain the number of each user request, and then divide the user requests with the same number in the plurality of user requests into the same user request. When the number of the user request is obtained by using the method for generating the random number, the number of the generated user request may be smaller than Q, or may be smaller than or equal to Q.

Step 404, determining a P-group user request corresponding to each test group according to the P sequences corresponding to each test group in the plurality of test groups, wherein each test group comprises a plurality of product versions.

It should be noted that the P sequences determined by the product version testing apparatus in step 402 for each test group include Q elements, and the Q elements may be in one-to-one correspondence with the Q user requests obtained by dividing in step 403. For example, the numbers of the Q user requests may be 0, 1, 2, 3, 4, 5, 6, 7, and 8, and the Q elements may also be 0, 1, 2, 3, 4, 5, 6, 7, and 8, respectively, so that the number of each user request may be considered to be the same as the value of the corresponding element. For another example, if the numbers of the Q user requests may be 0, 1, 2, 3, 4, 5, 6, 7, and 8, and the Q elements may be 1, 2, 3, 4, 5, 6, 7, 8, and 9, respectively, the difference between the number of each user request and the value of the corresponding element may be considered to be-1.

The P group user requests corresponding to each test group determined in step 404 may correspond to P sequences corresponding to the test group one to one, and each group user request in the P group user requests may include: p user requests corresponding to P elements in the corresponding sequence.

Step 405, testing the product version in each test group according to the user request corresponding to each test group.

After obtaining the P groups of user requests corresponding to each test group, the product version test apparatus may perform an AB test on the product versions in the test group based on the user requests corresponding to the test group, so as to determine the product version in each test group that best meets the user requirements.

Moreover, because the intersection of any two sequences corresponding to the same test group is an empty set, the intersection of any two groups of user requests corresponding to the same test group is also an empty set. Because the intersection of any two sequences corresponding to different test groups comprises one element, the intersection of any two groups of user requests corresponding to different test groups only comprises one user request. Therefore, the orthogonality of the user requests of the plurality of test groups obtained by the grouping method in the embodiment of the invention is better, and the accuracy of testing the product versions in the test groups based on the user requests is higher.

In the method provided by the embodiment of the present invention, it is theoretically fundamental that the intersection of any two sequences corresponding to the same test group in step 402 is an empty set. The intersection of any two sequences corresponding to the same test group will be verified as an empty set.

Assuming that an element appears in two sequences corresponding to the same test group at the same time, the element must be the same row element (assumed to be the kth row element) in the target matrix, and the corresponding two elements are also the kth elements in the two sequences, respectively.

Assuming that one of the two elements is the kth element in the jth sequence corresponding to the ith test set, and the other element is the kth element in the jth sequence corresponding to the ith test set, it can be derived: and [ [ (j-1) + (k-1) × (i-1) ] mod P ] +1 [ [ (j '-1) + (k-1) × (i-1) ] mod P ] +1, and further [ (j-1) + (k-1) × (i-1) ] mod P [ [ (j' -1) + (k-1) × (i-1) ] mod P ] is obtained. Wherein j 'is more than or equal to 1 and less than or equal to P, and j' is not equal to j.

Let (j-1) + (k-1) × (i-1) ═ n × P + r, r be the value obtained by (j-1) + (k-1) × (i-1) subtracting P, and n be the integer part of the value obtained by dividing (j-1) + (k-1) × (i-1) by P. Then (j '-1) + (k-1) × (i-1) ═ n' × P + r, n 'is the integer fraction of the value obtained by dividing (j' -1) + (k-1) × (i-1) by P. From (j-1) + (k-1) ═ i-1) ═ n × P + r and (j '-1) + (k-1) · (i-1) ═ n' × P + r, it is possible to obtain: j + (k-1) × (i-1) -n × P ═ j '+ (k-1) × (i-1) -n' × P.

Simplifying this equation yields: k (i-i) ═ j '-j + (n-n')/P. From this formula it can be determined: the remainder of k (i-i) for P is the same as the remainder of j' -j for P. And k (i-i) is 0 for the remainder of P, and thus j' -j is 0 for the remainder of P. Since j '≠ j, and j' -j < P, the above assumption does not hold, and no element appears in both sequences corresponding to the same test set at the same time.

In the method provided in the embodiment of the present invention, the intersection of any two sequences corresponding to different test sets in step 402 includes an element, which also has a theoretical basis. It will be demonstrated below that the intersection of any two sequences corresponding to different test sets includes an element.

Assuming that an element is present in both sequences simultaneously, this element must be the same row element in the target matrix (assumed to be the kth row element) and the corresponding two elements are also the kth elements in the two sequences, respectively. Assuming that one of the two elements is the kth element in the jth sequence corresponding to the ith test set, and the other element is the kth element in the jth sequence corresponding to the ith' test set, it can be derived: [ [ (j-1) + (k-1) × (i-1) ] mod P ] +1 [ [ (j '-1) + (k-1) × (i' -1) ] mod P ] +1, [ (j-1) + (k-1) × (i-1) ] mod P [ [ (j '-1) + (k-1) × (i' -1) ] mod P. Wherein i ' is more than or equal to 1 and less than or equal to P, j ' is more than or equal to 1 and less than or equal to P, and i ' is not equal to i.

Let (j-1) + (k-1) × (i-1) ═ n × P + r, r be the value obtained by (j-1) + (k-1) × (i-1) subtracting P, and n be the integer part of the value obtained by dividing (j-1) + (k-1) × (i-1) by P. Then (j '-1) + (k-1) × (i' -1) ═ n '× P + r, n' is the integer fraction of the value obtained by dividing (j '-1) + (k-1) × (i' -1) by P. From (j-1) + (k-1) ═ i-1) ═ n × P + r and (j ' -1) + (k-1) · (i ' -1) ═ n ' × P + r, it is possible to obtain: (j-1) + (k-1) × (i-1) -n ═ P ═ j '-1) + (k-1) × (i' -1) -n ═ P.

Simplifying this equation yields: k (i-i ') ═ j ' -j + (n-n ')/P. From this formula it can be determined: the remainder of k (i-i ') for P is the same as the remainder of j' -j for P. i, i ', j ', j are constants, i-i ═ a, a is not equal to 0, j ' -j ═ c, then it can be considered that the remainder of a x k to P is the same as the remainder of c to P, and P is a prime number, so k has only a unique solution. Therefore, the intersection of the jth sequence corresponding to the ith test set and the jth sequence corresponding to the ith' test set has only one element.

Fig. 6 is a schematic structural diagram of a product version testing apparatus according to an embodiment of the present invention, and as shown in fig. 6, the product version testing apparatus 60 may include:

a dividing module 601, configured to divide the multiple user requests into Q user requests, where Q is P × P, and P is a prime number;

a first determining module 602, configured to determine, according to P sequences corresponding to each test group in a plurality of test groups, a P group user request corresponding to each test group, where each test group includes a plurality of product versions;

the test module 603 is configured to test the product version in each test group according to the user request corresponding to each test group;

In summary, in the product version testing apparatus provided in the embodiment of the present invention, after the dividing module divides the plurality of user requests into Q user requests, the first determining module can determine the P group user request corresponding to each test group according to the P sequences corresponding to each test group. Because the intersection of any two sequences corresponding to the same test group in the multiple test groups is an empty set, the user requests of each group corresponding to the same test group are different. And because the intersection of any two sequences corresponding to different test groups comprises one element, the intersection of any two groups of user requests corresponding to different test groups only comprises one piece of data, so that each group of user requests corresponding to each test group are all distributed to all groups of user requests corresponding to other test groups. In this way, the determined user requests corresponding to the test groups are orthogonal, so that the accuracy of testing the test groups according to the user requests corresponding to the test groups is higher.

In addition, when a plurality of test groups are tested in the related art, the user request needs to be divided for a plurality of times, which results in low efficiency of product version testing. In the product version testing method provided by the embodiment of the invention, when a plurality of testing groups are tested, the user request is divided once only by executing once, so that the testing efficiency of the product version testing method provided by the embodiment of the invention is higher.

Optionally, fig. 7 is a schematic structural diagram of another product version testing apparatus provided in an embodiment of the present invention, as shown in fig. 7, on the basis of fig. 6, the product version testing apparatus 60 may further include:

a second determining module 604, configured to determine a target matrix including P rows and P columns of elements, where each element in the target matrix is a natural number less than or equal to Q, and any two elements are different;

a third determining module 605, configured to determine, according to the target matrix, P sequences corresponding to each test group;

wherein, the kth element in the jth sequence corresponding to the ith test group in the plurality of test groups is: the k row of the object matrix has the elements in column [ [ (j-1) + (k-1) × (i-1) ] mod P ] +1, i is more than or equal to 1 and less than or equal to P, j is more than or equal to 1 and less than or equal to P, and [ (j-1) + (k-1) × (i-1) ] mod P represents the remainder of [ (j-1) + (k-1) × (i-1) ] on P.

Optionally, fig. 8 is a schematic structural diagram of a dividing module according to an embodiment of the present invention, and as shown in fig. 8, the dividing module 601 may include: the dividing submodule 6011 is configured to divide the plurality of user requests into Q user requests according to the target numbers in the plurality of user requests.

Optionally, the target number carried by the user request includes: a user identification number; or, the target number carried by the user request includes: an identification number of the user request; or the target number carried by the user request comprises a first number segment and a second number segment, the first number segment comprises the user identification number, and the second number segment comprises the generation date of the user request.

Optionally, partition submodule 6011 is configured to: carrying out Hash processing and Q remainder processing on the target number carried by each user request in sequence to obtain the number of each user request; and dividing the user requests with the same number in the plurality of user requests into the same user request.

Optionally, the hash process includes: and (6) carrying out DJB hash processing.

Optionally, partition submodule 6011 is configured to: processing the target number carried by each user request by adopting a method for generating a random number to obtain the number of each user request; and dividing the user requests with the same number in the plurality of user requests into the same user request.

The embodiment of the invention also provides a product version testing device, which comprises: the product version testing method comprises a processor and a memory, wherein at least one instruction is stored in the memory, and the instruction is loaded by the processor and executes the product version testing method provided by the embodiment of the invention, such as the product version testing method shown in fig. 3 or fig. 4.

For example, the product version testing apparatus provided by the embodiment of the present invention may be a server. Fig. 9 is a schematic structural diagram of a server according to an embodiment of the present invention. Specifically, the method comprises the following steps:

the server 800 includes a Central Processing Unit (CPU)801, a system memory 804 including a Random Access Memory (RAM)802 and a Read Only Memory (ROM)803, and a system bus 805 connecting the system memory 804 and the central processing unit 801. The server 800 also includes a basic input/output system (I/O system) 806, which facilitates transfer of information between devices within the computer, and a mass storage device 807 for storing an operating system 813, application programs 814, and other program modules 815.

The basic input/output system 806 includes a display 808 for displaying information and an input device 809 such as a mouse, keyboard, etc. for user input of information. Wherein a display 808 and an input device 809 are connected to the central processing unit 801 through an input output controller 810 connected to the system bus 805. The basic input/output system 806 may also include an input/output controller 810 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 810 also provides output to a display screen, a printer, or other type of output device.

The mass storage device 807 is connected to the central processing unit 801 through a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer-readable media provide non-volatile storage for the server 800. That is, the mass storage device 807 may include a computer-readable medium (not shown) such as a hard disk or CD-ROM drive.

Without loss of generality, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the foregoing. The system memory 804 and mass storage 807 described above may be collectively referred to as memory.

According to various embodiments of the invention, server 800 may also operate as a remote computer connected to a network through a network, such as the Internet. That is, the server 800 may be connected to the network 812 through the network interface unit 811 coupled to the system bus 805, or may be connected to other types of networks or remote computer systems (not shown) using the network interface unit 811.

The memory further includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU.

Further exemplarily, the product version testing apparatus provided by the embodiment of the present invention may be a terminal.

Fig. 10 is a block diagram of a terminal according to an exemplary embodiment of the present invention. The terminal 900 may be a portable mobile terminal such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion video Experts compression standard Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion video Experts compression standard Audio Layer 4), a notebook computer, or a desktop computer. Terminal 900 may also be referred to by other names such as user equipment, portable terminals, laptop terminals, desktop terminals, and the like.

In general, terminal 900 includes: a processor 901 and a memory 902.

Processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so forth. The processor 901 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 901 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 902 is used to store at least one instruction for execution by processor 901 to implement the video recommendation methods provided by the method embodiments herein.

In some embodiments, terminal 900 can also optionally include: a peripheral interface 903 and at least one peripheral. The processor 901, memory 902, and peripheral interface 903 may be connected by buses or signal lines. Various peripheral devices may be connected to the peripheral interface 903 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 904, a touch display screen 905, a camera 906, an audio circuit 907, a positioning component 908, and a power supply 909.

The peripheral interface 903 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, memory 902, and peripheral interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902 and the peripheral interface 903 may be implemented on a separate chip or circuit board, which is not limited by this embodiment.

The Radio Frequency circuit 904 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 904 communicates with communication networks and other communication devices via electromagnetic signals. The radio frequency circuit 904 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 904 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuit 904 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: the world wide web, metropolitan area networks, intranets, generations of mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 904 may also include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 905 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 also has the ability to capture touch signals on or over the surface of the display screen 905. The touch signal may be input to the processor 901 as a control signal for processing. At this point, the display 905 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 905 may be one, providing the front panel of the terminal 900; in other embodiments, the number of the display panels 905 may be at least two, and each of the display panels is disposed on a different surface of the terminal 900 or is in a foldable design; in still other embodiments, the display 905 may be a flexible display disposed on a curved surface or a folded surface of the terminal 900. Even more, the display screen 905 may be arranged in a non-rectangular irregular figure, i.e. a shaped screen. The Display panel 905 can be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), and other materials.

The camera assembly 906 is used to capture images or video. Optionally, camera assembly 906 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 906 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

Audio circuit 907 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 901 for processing, or inputting the electric signals to the radio frequency circuit 904 for realizing voice communication. For stereo sound acquisition or noise reduction purposes, the microphones may be multiple and disposed at different locations of the terminal 900. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 901 or the radio frequency circuit 904 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, audio circuit 907 may also include a headphone jack.

The positioning component 908 is used to locate the current geographic Location of the terminal 900 for navigation or LBS (Location Based Service). The Positioning component 908 may be a Positioning component based on the Global Positioning System (GPS) in the united states, the beidou System in china, or the galileo System in russia.

Power supply 909 is used to provide power to the various components in terminal 900. The power source 909 may be alternating current, direct current, disposable or rechargeable. When the power source 909 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, terminal 900 can also include one or more sensors 910. The one or more sensors 910 include, but are not limited to: acceleration sensor 911, gyro sensor 912, pressure sensor 913, fingerprint sensor 914, optical sensor 915, and proximity sensor 916.

The acceleration sensor 911 can detect the magnitude of acceleration in three coordinate axes of the coordinate system established with the terminal 900. For example, the acceleration sensor 911 may be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 901 can control the touch display 905 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 911. The acceleration sensor 911 may also be used for acquisition of motion data of a game or a user.

The gyro sensor 912 may detect a body direction and a rotation angle of the terminal 900, and the gyro sensor 912 may cooperate with the acceleration sensor 911 to acquire a 3D motion of the user on the terminal 900. The processor 901 can implement the following functions according to the data collected by the gyro sensor 912: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

Pressure sensors 913 may be disposed on the side bezel of terminal 900 and/or underneath touch display 905. When the pressure sensor 913 is disposed on the side frame of the terminal 900, the user's holding signal of the terminal 900 may be detected, and the processor 901 performs left-right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 913. When the pressure sensor 913 is disposed at a lower layer of the touch display 905, the processor 901 controls the operability control on the UI interface according to the pressure operation of the user on the touch display 905. The operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

The fingerprint sensor 914 is used for collecting a fingerprint of the user, and the processor 901 identifies the user according to the fingerprint collected by the fingerprint sensor 914, or the fingerprint sensor 914 identifies the user according to the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, processor 901 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings, etc. The fingerprint sensor 914 may be disposed on the front, back, or side of the terminal 900. When a physical key or vendor Logo is provided on the terminal 900, the fingerprint sensor 914 may be integrated with the physical key or vendor Logo.

The optical sensor 915 is used to collect ambient light intensity. In one embodiment, the processor 901 may control the display brightness of the touch display 905 based on the ambient light intensity collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 905 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 905 is turned down. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera assembly 906 according to the ambient light intensity collected by the optical sensor 915.

Proximity sensor 916, also known as a distance sensor, is typically disposed on the front panel of terminal 900. The proximity sensor 916 is used to collect the distance between the user and the front face of the terminal 900. In one embodiment, when the proximity sensor 916 detects that the distance between the user and the front face of the terminal 900 gradually decreases, the processor 901 controls the touch display 905 to switch from the bright screen state to the dark screen state; when the proximity sensor 916 detects that the distance between the user and the front surface of the terminal 900 gradually becomes larger, the processor 901 controls the touch display 905 to switch from the breath screen state to the bright screen state.

Those skilled in the art will appreciate that the configuration shown in fig. 10 is not intended to be limiting of terminal 900 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.

An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method for testing the product version provided by the embodiment of the present invention is implemented, such as the method for testing the product version shown in fig. 3 or fig. 4.

Embodiments of the present invention further provide a computer program product including instructions, which, when the computer program product runs on a computer, causes the computer to execute the product version testing method provided in the embodiments of the present invention, such as the product version testing method shown in fig. 3 or fig. 4.

In this disclosure, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The term "plurality" means two or more unless expressly limited otherwise.

It should be noted that, the method embodiment provided in the embodiment of the present invention can be mutually referred to a corresponding apparatus embodiment, and the embodiment of the present invention does not limit this. The sequence of the steps of the method embodiments provided by the embodiments of the present invention can be appropriately adjusted, and the steps can be correspondingly increased or decreased according to the situation, and any method that can be easily conceived by those skilled in the art within the technical scope disclosed by the present invention shall be covered by the protection scope of the present invention, and therefore, the detailed description thereof shall not be repeated.

The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for product version testing, the method comprising:

2. The method of claim 1, wherein prior to the P sequences corresponding to each of the plurality of test sets, the method further comprises:

3. The method of claim 2, wherein the elements in the mth row and nth column of the target matrix are (m-1) × P + n-1 or (m-1) × P + n, 1 ≦ m ≦ P, and 1 ≦ n ≦ P.

4. The method according to any one of claims 1 to 3, wherein each of the plurality of user requests carries a target number, and dividing the plurality of user requests into Q user requests comprises:

5. The method of claim 4,

the target number carried by the user request includes: a user identification number;

6. The method according to claim 4, wherein said dividing the plurality of user requests into the Q user requests according to the target numbers in the plurality of user requests comprises:

7. The method according to claim 4, wherein said dividing the plurality of user requests into the Q user requests according to the target numbers in the plurality of user requests comprises:

8. A product version test apparatus, characterized in that the product version test apparatus comprises:

9. A product version test apparatus, characterized in that the product version test apparatus comprises: a processor and a memory, the memory having stored therein at least one instruction that is loaded and executed by the processor to implement the product version testing method of any of claims 1-7.

10. A computer-readable storage medium having stored thereon at least one instruction which is loaded and executed by a processor to implement the product version testing method of any one of claims 1-7.