CN112052153B

CN112052153B - Product version testing method and device

Info

Publication number: CN112052153B
Application number: CN201910492045.8A
Authority: CN
Inventors: 熊涛
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-06-06
Filing date: 2019-06-06
Publication date: 2023-05-16
Anticipated expiration: 2039-06-06
Also published as: CN112052153A

Abstract

The application discloses a product version testing method and device, and belongs to the technical field of computers. The method comprises the following steps: dividing a plurality of user requests into Q user requests; according to the P sequences corresponding to each test group in the plurality of test groups, determining P group user requests corresponding to each test group; testing the product versions in each test group according to the user request corresponding to each test group; each sequence comprises P elements, and Q elements in the P sequences are in one-to-one correspondence with Q user requests; the intersection of any two sequences corresponding to the same test group is an empty set, and the intersection of any two sequences corresponding to different test groups comprises an element; the P groups of user requests are in one-to-one correspondence with the P sequences, and each group of user requests comprises P user requests corresponding to P elements in the corresponding sequences. The method and the device solve the problem that the accuracy of test results of a plurality of test groups is low, and are used for testing product versions.

Description

Product version testing method and device

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for testing a product version.

Background

With the development of the internet, rapid iterations of internet product versions have become commonplace. Operators often need to test multiple product versions based on a large number of user requests before releasing new versions of products, so as to select a target product version which can meet the requirements of users from the multiple product versions, and then release the target product version as the new version.

In the related art, product versions in a plurality of test groups may be tested based on a large number of acquired user requests. Wherein each test set includes a plurality of product versions. When testing the product versions in the multiple test groups, an allocation method can be adopted to divide the acquired user requests into multiple user requests, and test a certain test group based on the different user requests which are divided currently. Then, another allocation method is needed to subdivide these user requests into multiple user requests and test the next test group based on the different user requests currently divided.

However, when testing different test groups, each user request for testing any test group can be uniformly distributed in different user requests for testing other test groups, so that the accuracy of the test results of each test group can be ensured. However, the division manner in the related art often cannot divide the user request according to the above requirements, resulting in lower accuracy of the test results of the multiple test groups.

Disclosure of Invention

The application provides a product version testing method and device, which can solve the problem of lower accuracy of test results of a plurality of test groups, and the technical scheme is as follows:

in a first aspect, a product version testing method is provided, the method comprising:

dividing a plurality of user requests into Q user requests, wherein Q=P is P, and P is prime number;

according to P sequences corresponding to each test group in a plurality of test groups, determining P group user requests corresponding to each test group, wherein each test group comprises a plurality of product versions;

testing the product versions in each test group according to the user request corresponding to each test group;

wherein each sequence comprises P elements, and Q elements in the P sequences are in one-to-one correspondence with the Q user requests; the intersection of any two sequences corresponding to the same test group is an empty set, and the intersection of any two sequences corresponding to different test groups comprises an element; and the P groups of user requests are in one-to-one correspondence with the P sequences, and each group of user requests comprises P user requests corresponding to P elements in the corresponding sequences.

Optionally, before the P sequences corresponding to each of the plurality of test groups, the method further comprises:

Determining a target matrix comprising P rows and P columns of elements, wherein each element in the target matrix is a natural number less than or equal to Q, and any two elements are different;

according to the target matrix, determining P sequences corresponding to each test group;

the kth element in the jth sequence corresponding to the ith test group in the plurality of test groups is: the k-th row [ [ (j-1) + (k-1) ] mod P ] +1 column elements in the target matrix, i is greater than or equal to 1 and less than or equal to P, j is greater than or equal to 1 and less than or equal to P, k is greater than or equal to 1 and less than or equal to P, [ (j-1) + (k-1) ] mod P represents [ (j-1) + (k-1) ] mod P, and the remainder is obtained by taking P.

Optionally, in the target matrix, the element in the mth row and the nth column is (m-1) p+n-1 or (m-1) p+n, where m is 1-P, and n is 1-P.

Optionally, each user request in the plurality of user requests carries a target number, dividing the plurality of user requests into Q user requests, including:

dividing the plurality of user requests into the Q user requests according to the target numbers in the plurality of user requests.

Optionally, the target number carried by the user request includes: a user identification number;

or, the target number carried by the user request includes: an identification number of the user request;

Or the target number carried by the user request comprises a first number segment and a second number segment, wherein the first number segment comprises the user identification number, and the second number segment comprises the generation date of the user request.

Optionally, the dividing the plurality of user requests into the Q user requests according to the target numbers in the plurality of user requests includes:

carrying out hash processing and Q remainder processing on the target number carried by each user request in sequence to obtain the number of each user request;

and dividing the user requests with the same number in the plurality of user requests into the same user request.

Optionally, the hash processing includes: DJB hash processing.

processing the target number carried by each user request by adopting a method for generating random numbers so as to obtain the number of each user request;

In a second aspect, a product version testing apparatus is provided, the product version testing apparatus including:

The dividing module is used for dividing the plurality of user requests into Q user requests, wherein Q=P is P, and P is prime number;

the first determining module is used for determining P group user requests corresponding to each test group according to P sequences corresponding to each test group in the plurality of test groups, wherein each test group comprises a plurality of product versions;

the testing module is used for testing the product versions in each testing group according to the user request corresponding to each testing group;

Optionally, the product version testing device further includes:

the second determining module is used for determining a target matrix comprising P rows and P columns of elements, wherein each element in the target matrix is a natural number smaller than or equal to Q, and any two elements are different;

The third determining module is used for determining P sequences corresponding to each test group according to the target matrix;

Optionally, the dividing module includes:

and the dividing sub-module is used for dividing the plurality of user requests into the Q user requests according to the target numbers in the plurality of user requests.

Optionally, the dividing submodule is configured to:

Optionally, the hash processing includes: DJB hash processing.

Optionally, the dividing submodule is configured to:

In a third aspect, a product version testing apparatus is provided, the product version testing apparatus comprising: a processor and a memory, wherein at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement the product version testing method according to the first aspect.

In a fourth aspect, a computer readable storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement the product version testing method of the first aspect is provided.

In a fifth aspect, there is provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the product version testing method of the first aspect.

The beneficial effects that this application provided technical scheme brought include at least:

in the product version testing method provided by the embodiment of the invention, after a plurality of user requests are divided into Q user requests, the P group user requests corresponding to each test group are determined according to the P sequences corresponding to each test group. Because the intersection of any two sequences corresponding to the same test group in the plurality of test groups is an empty set, the user requests of the groups corresponding to the same test group are different. And because the intersection of any two sequences corresponding to different test groups comprises an element, the intersection of any two groups of user requests corresponding to different test groups only comprises one data, so that each group of user requests corresponding to each test group is evenly distributed to all groups of user requests corresponding to other test groups. In this way, the user requests corresponding to the determined test groups are orthogonal, so that the accuracy of testing the test groups according to the user requests corresponding to the test groups is higher.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of an AB test according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a plurality of test groups according to an embodiment of the present invention;

FIG. 3 is a flowchart of a method for testing a product version according to an embodiment of the present invention;

FIG. 4 is a flowchart of another method for testing product versions according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a sequence corresponding to a plurality of test groups according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a product version testing device according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of another product version testing apparatus according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a partition module according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a server according to an embodiment of the present invention;

Fig. 10 is a block diagram of a terminal according to an exemplary embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

With the development of the internet, rapid iterations of internet product versions have become commonplace. Operators often need to test multiple product versions based on a large number of user requests to select a target product version from the multiple product versions that best meets the user's needs. The process of the above test may be referred to as an AB test.

For example, fig. 1 is a schematic diagram of an AB test provided in an embodiment of the present invention, as shown in fig. 1, when testing multiple product versions (such as a product version a and a product version B), a large number of user requests need to be collected, and the user requests are divided into multiple groups of user requests (such as a first group of user requests and a second group of user requests in fig. 1). Then, the first group of user requests can be adopted to test the product version A so as to determine the preference degree of the user on the product version A; and testing the product version B with the second set of user requests to determine the user's preference for the product version A. And finally, according to the preference of the user on each product version, the target product version with the highest preference of the user can be screened out from the product versions.

It should be noted that, the policies of the product version a and the product version B are different (in this case, the product version a and the product version B have differences in some parameters). For example, the way in which the product version a and the product version B push information to the user is different, or the way in which the product version a and the product version B interact with the user is different, or the interfaces of the product version a and the product version B are different, or the preferential activities corresponding to the product version a and the product version B are different. In fig. 1, the interface of the product version a is different from the interface of the product version B.

The preference of the user to the product version can be embodied by various parameters, for example, the click rate of the user to the interface can be embodied. When the click rate of the user on the interface of the product version A is larger than that of the user on the interface of the product version B, the preference of the user on the product version A is higher.

In performing AB testing, a large number of user requests are typically used to test product versions of multiple test groups (also referred to as test layers). Wherein each test group includes multiple product versions, and the product versions may include different versions of one product, and of course may also include different versions of multiple products, which is not limited by the embodiment of the present invention.

For example, fig. 2 is a schematic diagram of a plurality of test groups according to an embodiment of the present invention, as shown in fig. 2, when testing the test group 1, an allocation method may be first adopted to divide the acquired user request into a plurality of user requests (such as the 1 st user request, the 2 nd user request and the 3 rd user request corresponding to the test group 1 in fig. 2), and test the test group 1 based on different user requests that are currently divided. Then, another allocation method is needed to divide the user requests into multiple user requests (such as the 1 st user request, the 2 nd user request and the 3 rd user request corresponding to the test group 2 in fig. 2), and test the test group 2 based on the different user requests that are currently divided.

However, when testing different test groups, orthogonality between user requests for testing different test groups is often required. That is, each user request required to test any one test set can be evenly distributed among different user requests for testing other test sets. In this way, the accuracy of the test results of each test group can be ensured. For example, the 1 st user request corresponding to the test group 1 needs to be uniformly distributed in the 1 st user request, the 2 nd user request and the 3 rd user request corresponding to the test group 2. However, the user requests cannot be divided according to the above requirements by adopting the division manner in the related art, for example, the 1 st user request corresponding to the test group 1 is only distributed in the 1 st user request and the 2 nd user request corresponding to the test group 2, but not in the 3 rd user request corresponding to the test group 2, which results in lower accuracy of the test results of the plurality of test groups.

The embodiment of the invention provides a product version testing method, which can use a product version testing device, and the dividing mode adopted in the product version testing method can divide user requests according to the requirements, so that the accuracy of the testing results of each testing group is higher.

For example, fig. 3 is a flowchart of a product version testing method according to an embodiment of the present invention, and as shown in fig. 3, the product version testing method may include:

in step 301, a plurality of user requests are divided into Q user requests, q=p×p, and P is a prime number.

Step 302, determining P groups of user requests corresponding to each test group according to P sequences corresponding to each test group in the plurality of test groups, wherein each test group comprises a plurality of product versions.

Each sequence comprises P elements, and Q elements in the P sequences are in one-to-one correspondence with Q user requests; the intersection of any two sequences corresponding to the same test group is an empty set, and the intersection of any two sequences corresponding to different test groups comprises an element; the P groups of user requests are in one-to-one correspondence with the P sequences, and each group of user requests comprises P user requests corresponding to P elements in the corresponding sequences.

Step 303, testing the product versions in each test group according to the user request corresponding to each test group.

In step 303, the product version testing device may perform an AB test on the product versions in each test group according to the user request corresponding to each test group, so as to determine the product version in each test group that best meets the user requirement.

In summary, in the product version testing method provided by the embodiment of the present invention, after dividing a plurality of user requests into Q user requests, the P group user requests corresponding to each test group are determined according to the P sequences corresponding to each test group. Because the intersection of any two sequences corresponding to the same test group in the plurality of test groups is an empty set, the user requests of the groups corresponding to the same test group are different. And because the intersection of any two sequences corresponding to different test groups comprises an element, the intersection of any two groups of user requests corresponding to different test groups only comprises one data, so that each group of user requests corresponding to each test group is evenly distributed to all groups of user requests corresponding to other test groups. In this way, the user requests corresponding to the determined test groups are orthogonal, so that the accuracy of testing the test groups according to the user requests corresponding to the test groups is higher.

In addition, when testing a plurality of test groups in the related art, multiple divisions of the user request are required, resulting in lower efficiency of product version testing. In the product version testing method provided by the embodiment of the invention, no matter how many testing groups need to be tested, the user request is only required to be divided once (see step 301 in detail), so that the testing efficiency of the product version testing method provided by the embodiment of the invention is higher.

Fig. 4 is a flowchart of another product version testing method according to an embodiment of the present invention, as shown in fig. 4, the product version testing method may include:

step 401, determining a target matrix including P rows and P columns of elements, where any two elements in the target matrix are different.

It should be noted that, the product version testing method provided by the embodiment of the invention can be executed by the product version testing device. In step 401, the product version testing device needs to first determine a prime number P, and then determine a target matrix including P rows and P columns.

Where prime numbers are also called prime numbers, in natural numbers greater than 1, the prime numbers have no other factors than 1 and the prime number itself. For example, P in the embodiment of the present invention may be 2, 3, or 5, and in the embodiment of the present invention, P is taken as 3 as an example.

The target matrix may be any matrix including P rows and P columns of elements, and each element in the target matrix is a natural number less than or equal to Q, and the elements in the target matrix are different from each other. For example, in the embodiment of the present invention, the element in the mth row and the nth column in the target matrix may be (m-1) p+n-1, where m is 1-P, n is 1-P, and if p=3, the target matrix may be:

optionally, the element in the mth row and nth column of the target matrix may also be (m-1) ×p+n, where if p=3, the target matrix may be: />

Step 402, determining P sequences corresponding to each test group in the plurality of test groups according to the target matrix.

Each of the plurality of test groups may include a plurality of product versions, which may include multiple versions of the same product or may include multiple versions of different products. Each sequence corresponding to each test set may include P elements in the target matrix, and then each sequence corresponding to each test set may include Q elements in the target matrix. Q=p×p, i.e., Q is equal to the product of P and P.

In step 402, the product version testing apparatus may sequentially calculate each element in each sequence corresponding to each test group according to the target matrix obtained in step 401. For example, if the product version testing device needs to calculate the kth element in the jth sequence corresponding to the ith test group in the multiple test groups, the product version testing device may take the element in the kth row [ [ (j-1) + (k-1)/(i-1) ] mod P ] +1 column in the target matrix as the kth element, where 1.ltoreq.i.ltoreq.p, 1.ltoreq.j.ltoreq.p, 1.ltoreq.k.ltoreq.p, [ (j-1) + (k-1)/(i-1) ] mod P represents [ (j-1) + (k-1)/(i-1) ] and takes the remainder for P.

For example, when the product version testing device needs to calculate the 1 st element in the 1 st sequence corresponding to the 1 st test group, the product version testing device may determine that the current i=1, j=1, and k=1, and at this time, the product version testing device may use the 1 st element in the 1 st row and 1 st column in the target matrix as the 1 st element in the 1 st sequence corresponding to the 1 st test group. When the product version testing device needs to calculate the 3 rd element in the 3 rd sequence corresponding to the 2 nd test group, the product version testing device can determine that i=2, j=3 and k=3 currently, and at this time, the product version testing device can take the 3 rd element in the 3 rd row and the 2 nd element in the target matrix as the 3 rd element in the 3 rd sequence corresponding to the 2 nd test group.

Fig. 5 is a schematic diagram of a sequence corresponding to a plurality of test groups according to an embodiment of the present invention. It should be noted that, in fig. 5, the plurality of test groups includes three test groups as an example. Alternatively, the number of the plurality of test groups may be other values, such as 2 or 4.

Referring to fig. 5, each test set may correspond to three sequences, for example, test set 1 corresponds to sequence 1, sequence 2, and sequence 3, test set 2 corresponds to sequence 4, sequence 5, and sequence 6, and test set 3 corresponds to sequence 7, sequence 8, and sequence 9. Wherein sequence 1 comprises: element 0, element 3, and element 6, sequence 2 includes: element 1, element 4, and element 7, sequence 3 includes: element 2, element 5, and element 8, sequence 4 includes: element 0, element 4, and element 8, sequence 5 includes: element 1, element 5, and element 6, sequence 6 comprising: element 2, element 3, and element 7, sequence 7 comprising: element 0, element 5, and element 7, sequence 8 includes: element 1, element 3, and element 8, sequence 9 includes: element 2, element 4 and element 6.

In one aspect, the intersection of any two sequences corresponding to the same test set is an empty set, for example, in sequence 1, sequence 2, and sequence 3 corresponding to test set 1 in fig. 5, any two sequences do not include the same element, and therefore, the intersection of any two sequences is an empty set.

On the other hand, the intersection of any two sequences corresponding to different test groups includes one element. For example, in fig. 5, the sequence 1 corresponding to the test group 1 and the sequence 1 corresponding to the test group 2 only include the element 0 at the same time, and the other elements are different, so the intersection of the sequence 1 corresponding to the test group 1 and the sequence 1 corresponding to the test group 2 includes only one element. Similarly, the intersection of sequence 1 corresponding to test group 1 and sequence 2 corresponding to test group 2 includes only one element, and the intersection of sequence 1 corresponding to test group 1 and sequence 3 corresponding to test group 2 also includes only one element. It can be seen that the elements in the sequence 1 corresponding to the test group 1 are equally divided into the sequences corresponding to the test group 2, and the elements in the sequence 1 corresponding to each test group are equally divided into the sequences corresponding to the other test groups.

It should be noted that, there may be various ways of obtaining sequences corresponding to multiple test groups provided in the embodiment of the present invention, and in the embodiment of the present invention, P sequences corresponding to each test group are obtained by using the product version testing device in the method shown in step 401 and step 402 as an example. Alternatively, the product version testing device may directly receive P sequences corresponding to each test group input by a worker.

In the above embodiment, 1 is taken as the minimum count as an example. Alternatively, the foregoing embodiment may also use 0 as the minimum count, where the 1 st test group of the plurality of test groups may be referred to as the 0 th test group, the 1 st sequence may be referred to as the 0 th sequence, the 1 st element in the sequence may be referred to as the 0 th element, the 1 st row of the target matrix may be referred to as the 0 th row, and the 1 st column of the target matrix may be referred to as the 0 th column. And, the Kth element in the J-th sequence corresponding to the I-th test group in the plurality of test groups is: the K row [ (J+K.I) mod P ] column element in the target matrix is more than or equal to 0 and less than or equal to P-1, more than or equal to 0 and less than or equal to J and less than or equal to P-1, more than or equal to 0 and less than or equal to K and less than or equal to P-1, and (J+K.I) mod P represents (J+K.I) remainder of P.

Alternatively, the plurality of test sets in fig. 5 may include, in addition to

test sets

1, 2, and 3, a test set 4 (not shown in fig. 5), and the test set 4 may correspond to sequences 10, 11, and 12. Wherein sequence 10 comprises: element 0, element 1, and element 2, sequence 11 includes: element 3, element 4, and element 5, sequence 12 includes: element 6, element 7 and element 8.

Step 403, dividing the plurality of user requests into Q user requests.

It should be noted that the plurality of user requests may be obtained in advance or may be obtained in real time, which is not limited in the embodiment of the present invention. The product version testing device needs to divide the plurality of user requests into approximately Q user requests in step 403.

For example, each user request may carry a target number, and when dividing the plurality of user requests into Q user requests, the product version testing device may divide the plurality of user requests into Q user requests according to the target number in the plurality of user requests. Wherein, the target number may include: user identification numbers, different user requests sent by different users carry different target numbers; alternatively, the target number of the user request includes: the identification number of the user request, and the target numbers of different user requests are different at the moment; or the target number carried by the user request comprises a first number segment and a second number segment, wherein the first number segment comprises a user identification number, and the second number segment comprises the generation date of the user request. Of course, the target number may be another number, as long as the plurality of user requests can be divided into Q user requests substantially uniformly according to the target number, which is not limited in the embodiment of the present invention.

On the other hand, when dividing the plurality of user requests into Q user requests according to the target numbers in the plurality of user requests, the product version test device may sequentially perform hash processing and Q remainder processing (remainder processing is also referred to as modulo processing) on the target numbers carried by each user request, to obtain the number of each user request. And then, the product version testing device divides the user requests with the same number in the plurality of user requests into the same user request.

After hash processing and Q surplus processing are performed on the target number carried by each user request, the number of each user request obtained by the product version testing device is smaller than Q, and the numbers of all generated user requests include Q numbers in total. For example, the number of the user request may be: 0. 1, 2, 3, 4, 5, 6, 7 or 8.

Alternatively, the above-mentioned hash processing may include: the DJB hash processing may be other hash processing methods, such as Murmur hash processing, which is not limited in the embodiment of the present invention. The Murmur hash processing and the DJB hash processing are two different hash processing modes, and the DJB hash processing is invented by math Daniel Bernstein.

On the other hand, the product version test device may not adopt the methods of hash processing and Q remainder processing when dividing the plurality of user requests into Q user requests according to the target numbers in the plurality of user requests. For example, the product version testing device may use a method of generating a random number to process a target number carried by each user request to obtain a number of each user request, and then divide user requests with the same number in the plurality of user requests into the same user request. When the number of the user request is obtained by using the method of generating the random number, the number of the generated user request may be smaller than Q, or may be smaller than or equal to Q, but it is necessary to ensure that the numbers of all the generated user requests include Q numbers in total.

Step 404, determining P groups of user requests corresponding to each test group according to P sequences corresponding to each test group in the plurality of test groups, wherein each test group comprises a plurality of product versions.

It should be noted that, the P sequences corresponding to each test group determined by the product version testing device in step 402 include Q elements, where the Q elements may correspond to the Q user requests obtained by dividing in step 403 one by one. For example, the numbers of the Q user requests may be 0, 1, 2, 3, 4, 5, 6, 7, and 8,Q elements, respectively, and may be 0, 1, 2, 3, 4, 5, 6, 7, and 8, respectively, and then the numbers of each user request may be considered to be the same as the values of the corresponding elements. For another example, the number of Q user requests may be 0, 1, 2, 3, 4, 5, 6, 7, and 8,Q elements, respectively, or 1, 2, 3, 4, 5, 6, 7, 8, and 9, respectively, and then the difference between the number of each user request and the value of the corresponding element may be considered to be-1.

The P group user requests corresponding to each test group determined in step 404 may correspond to P sequences corresponding to the test group one to one, where each group of user requests in the P group of user requests may include: p user requests corresponding to P elements in the corresponding sequence.

Step 405, testing the product versions in each test group according to the user request corresponding to each test group.

After obtaining the P-group user requests corresponding to each test group, the product version testing device can perform an AB test on the product versions in the test group based on the user requests corresponding to the test group, so as to determine the product version most meeting the user requirements in each test group.

Moreover, since the intersection of any two sequences corresponding to the same test group is an empty set, the intersection of any two groups of user requests corresponding to the same test group is also an empty set. Because the intersection of any two sequences corresponding to different test groups includes one element, the intersection of any two sets of user requests corresponding to different test groups includes only one user request. Therefore, the orthogonality of the user requests of the plurality of test groups obtained by the grouping method in the embodiment of the invention is better, and the accuracy of testing the product versions in the test groups based on the user requests is higher.

In the method provided by the embodiment of the invention, the fact that the intersection of any two sequences corresponding to the same test group in step 402 is an empty set is a theoretical basis. The intersection of any two sequences corresponding to the same test set will be proved as an empty set.

Assuming that there are elements that appear in two sequences corresponding to the same test set at the same time, the elements must be the same row element (assumed to be the kth row element) in the target matrix, and the corresponding elements are also the kth elements in the two sequences, respectively.

Assuming that one element is the kth element in the jth sequence corresponding to the ith test group and the other element is the kth element in the jth' sequence corresponding to the ith test group, the following can be obtained: [ [ (j-1) + (k-1): (i-1) ] mod P ] +1= [ [ (j '-1) + (k-1): (i-1) ] mod P ] +1, and further [ (j-1) + (k-1): (i-1) ] mod p= [ (j' -1) + (k-1): (i-1) ] mod P. Wherein, j 'is less than or equal to 1 and less than or equal to P, and j' is not equal to j.

Let (j-1) + (k-1) =n×p+r, r being the value obtained by taking the remainder of (j-1) + (k-1) ×i-1 for P, n being the integer part of the value obtained by dividing (j-1) + (k-1) ×i-1 by P. Then (j '-1) + (k-1) × (i-1) =n' ×p+r, n 'being the integer part of the value obtained by dividing (j' -1) + (k-1) ×i-1 by P. From (j-1) + (k-1) × (i-1) =n×p+r and (j '-1) + (k-1) × (i-1) =n' ×p+r, it can be derived that: j+ (k-1), (i-1) -n p=j '+ (k-1), (i-1) -n' ×p.

Simplifying this equation can result in: k (i-i) =j '-j+ (n-n')p. From this formula it can be determined that: k (i-i) is the same as j' -j is the remainder of P. And k (i-i) has a value of 0 for P, so j' -j has a value of 0 for P. Since j '+.j, and j' -j < P, the above assumption is not true and no elements are present in both sequences corresponding to the same test set.

In the method provided by the embodiment of the present invention, it is also theoretical that the intersection of any two sequences corresponding to different test groups in step 402 includes one element. The intersection of any two sequences corresponding to different test sets will be demonstrated below to include one element.

Assuming that an element is present in both sequences, this element must be the same row element (assuming the kth row element) in the target matrix, respectively, and the corresponding two elements are also the kth element in both sequences, respectively. Assuming that one element is the kth element in the jth sequence corresponding to the ith test group and the other element is the kth element in the jth sequence corresponding to the ith' test group, the following can be obtained: [ (j-1) + (k-1) × (i-1) ] mod P ] +1= [ [ (j '-1) + (k-1) × (i' -1) ] mod P ] +1, [ (j-1) + (k-1) × (i-1) ] mod p= [ (j '-1) + (k-1) × (i' -1) ] mod P. Wherein, i ' is less than or equal to 1 and less than or equal to P, j ' is less than or equal to 1 and less than or equal to P, i ' is not equal to i.

Let (j-1) + (k-1) =n×p+r, r being the value obtained by taking the remainder of (j-1) + (k-1) ×i-1 for P, n being the integer part of the value obtained by dividing (j-1) + (k-1) ×i-1 by P. Then (j '-1) + (k-1) × (i' -1) =n '×p+r, n' being the integer part of the value obtained by dividing (j '-1) + (k-1) ×i' -1 by P. From (j-1) + (k-1) × (i-1) =n×p+r and (j ' -1) + (k-1) × (i ' -1) =n ' ×p+r, it can be derived that: (j-1) + (k-1): (i-1) -n: = (j ' -1) + (k-1): (i ' -1) -n ': P.

Simplifying this equation can result in: k (i-i ')=j ' -j+ (n-n ')p. From this formula it can be determined that: the value obtained by taking the remainder of k (i-i ') for P is the same as the value obtained by taking the remainder of j' -j for P. i, i ', j', j are constants such that i-i '=a, a is not equal to 0, j' -j=c, then a x k can be considered to have the same value as c P, but P is a prime number, so k has only a unique solution. Thus, the intersection of the jth sequence corresponding to the ith test set and the jth sequence corresponding to the ith' test set has only one element.

Fig. 6 is a schematic structural diagram of a product version testing device according to an embodiment of the present invention, and as shown in fig. 6, the product version testing device 60 may include:

a dividing module 601, configured to divide a plurality of user requests into Q user requests, q=p×p, and P is a prime number;

a first determining module 602, configured to determine, according to P sequences corresponding to each test group in the plurality of test groups, P group user requests corresponding to each test group, where each test group includes a plurality of product versions;

the testing module 603 is configured to test the product versions in each testing group according to the user request corresponding to the testing group;

In summary, in the product version testing device provided by the embodiment of the present invention, after the dividing module divides the plurality of user requests into Q user requests, the first determining module may determine the P group of user requests corresponding to each test group according to the P sequences corresponding to each test group. Because the intersection of any two sequences corresponding to the same test group in the plurality of test groups is an empty set, the user requests of the groups corresponding to the same test group are different. And because the intersection of any two sequences corresponding to different test groups comprises an element, the intersection of any two groups of user requests corresponding to different test groups only comprises one data, so that each group of user requests corresponding to each test group is evenly distributed to all groups of user requests corresponding to other test groups. In this way, the user requests corresponding to the determined test groups are orthogonal, so that the accuracy of testing the test groups according to the user requests corresponding to the test groups is higher.

In addition, when testing a plurality of test groups in the related art, multiple divisions of the user request are required, resulting in lower efficiency of product version testing. In the product version testing method provided by the embodiment of the invention, when testing a plurality of test groups, the user request is only required to be divided once, so that the testing efficiency of the product version testing method provided by the embodiment of the invention is higher.

Optionally, fig. 7 is a schematic structural diagram of another product version testing device according to an embodiment of the present invention, as shown in fig. 7, on the basis of fig. 6, the product version testing device 60 may further include:

a second determining module 604, configured to determine a target matrix including P rows and P columns of elements, where each element in the target matrix is a natural number less than or equal to Q, and any two elements are different;

a third determining module 605, configured to determine P sequences corresponding to each test group according to the target matrix;

the kth element in the jth sequence corresponding to the ith test group in the plurality of test groups is as follows: the k-th row [ (j-1) + (k-1) ] mod P ] +1 column elements in the target matrix, i is greater than or equal to 1 and less than or equal to P, j is greater than or equal to 1 and less than or equal to P, k is greater than or equal to 1 and less than or equal to P, [ (j-1) + (k-1): (i-1) ] mod P represents [ (j-1) + (k-1): (i-1) ] and takes the remainder for P.

Optionally, in the target matrix, the element in the mth row and the nth column is (m-1) p+n-1 or (m-1) p+n, where 1.ltoreq.m.ltoreq.p, and 1.ltoreq.n.ltoreq.p.

Optionally, fig. 8 is a schematic structural diagram of a partition module according to an embodiment of the present invention, and as shown in fig. 8, the partition module 601 may include: the dividing submodule 6011 is configured to divide the plurality of user requests into Q user requests according to the target numbers in the plurality of user requests.

Optionally, the target number carried by the user request includes: a user identification number; or, the target number carried by the user request includes: an identification number of the user request; or the target number carried by the user request comprises a first number segment and a second number segment, wherein the first number segment comprises the user identification number, and the second number segment comprises the generation date of the user request.

Optionally, the dividing submodule 6011 is configured to: carrying out hash processing and Q surplus processing on the target number carried by each user request in sequence to obtain the number of each user request; and dividing the user requests with the same number in the plurality of user requests into the same user request.

Optionally, the hash processing includes: DJB hash processing.

Optionally, the dividing submodule 6011 is configured to: processing the target number carried by each user request by adopting a method for generating random numbers so as to obtain the number of each user request; and dividing the user requests with the same number in the plurality of user requests into the same user request.

The embodiment of the invention also provides a product version testing device, which comprises: the system comprises a processor and a memory, wherein at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor, so that the product version testing method provided by the embodiment of the invention, such as the product version testing method shown in fig. 3 or fig. 4, is realized.

The product version testing device provided by the embodiment of the invention can be a server. Fig. 9 is a schematic structural diagram of a server according to an embodiment of the present invention. Specifically, the present invention relates to a method for manufacturing a semiconductor device.

The server 800 includes a Central Processing Unit (CPU) 801, a system memory 804 including a Random Access Memory (RAM) 802 and a Read Only Memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the central processing unit 801. The server 800 also includes a basic input/output system (I/O system) 806 for facilitating the transfer of information between various devices within the computer, and a mass storage device 807 for storing an operating system 813, application programs 814, and other program modules 815.

The basic input/output system 806 includes a display 808 for displaying information and an input device 809, such as a mouse, keyboard, or the like, for user input of information. Wherein both the display 808 and the input device 809 are connected to the central processing unit 801 via an input output controller 810 connected to the system bus 805. The basic input/output system 806 may also include an input/output controller 810 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input output controller 810 also provides output to a display screen, a printer, or other type of output device.

The mass storage device 807 is connected to the central processing unit 801 through a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer-readable media provide non-volatile storage for the server 800. That is, the mass storage device 807 may include a computer readable medium (not shown) such as a hard disk or CD-ROM drive.

Computer readable media may include computer storage media and communication media without loss of generality. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the ones described above. The system memory 804 and mass storage device 807 described above may be collectively referred to as memory.

According to various embodiments of the invention, server 800 may also operate by a remote computer connected to the network through a network, such as the Internet. I.e., server 800 may be connected to a network 812 through a network interface unit 811 connected to the system bus 805, or may be connected to other types of networks or remote computer systems (not shown) using the network interface unit 811.

The memory also includes one or more programs, one or more programs stored in the memory and configured to be executed by the CPU.

Also, for example, the product version testing device provided by the embodiment of the invention may be a terminal.

Fig. 10 is a block diagram of a terminal according to an exemplary embodiment of the present invention. The terminal 900 may be a portable mobile terminal such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion picture expert compression standard audio plane 3), an MP4 (Moving Picture Experts Group Audio Layer IV, motion picture expert compression standard audio plane 4) player, a notebook computer, or a desktop computer. Terminal 900 may also be referred to by other names of user devices, portable terminals, laptop terminals, desktop terminals, etc.

In general, the terminal 900 includes: a processor 901 and a memory 902.

Processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 901 may be implemented in hardware in at least one of a DSP (Digital Signal Processing ), an FPGA (Field-Programmable Gate Array, field programmable gate array) and a PLA (Programmable Logic Array ). The processor 901 may also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit ); a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 901 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 902 is used to store at least one instruction for execution by processor 901 to implement the video recommendation methods provided by the method embodiments herein.

In some embodiments, the terminal 900 may further optionally include: a peripheral interface 903, and at least one peripheral. The processor 901, memory 902, and peripheral interface 903 may be connected by a bus or signal line. The individual peripheral devices may be connected to the peripheral device interface 903 via buses, signal lines, or circuit boards. Specifically, the peripheral device includes: at least one of radio frequency circuitry 904, a touch display 905, a camera 906, audio circuitry 907, positioning components 908, and a power source 909.

The peripheral interface 903 may be used to connect at least one peripheral device associated with an I/O (Input/Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, memory 902, and peripheral interface 903 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 901, the memory 902, and the peripheral interface 903 may be implemented on separate chips or circuit boards, which is not limited in this embodiment.

The Radio Frequency circuit 904 is configured to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 904 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 904 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 904 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. The radio frequency circuit 904 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the world wide web, metropolitan area networks, intranets, generation mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity ) networks. In some embodiments, the radio frequency circuit 904 may also include NFC (Near Field Communication ) related circuits, which are not limited in this application.

The display 905 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display 905 is a touch display, the display 905 also has the ability to capture touch signals at or above the surface of the display 905. The touch signal may be input as a control signal to the processor 901 for processing. At this time, the display 905 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 905 may be one, providing a front panel of the terminal 900; in other embodiments, the display 905 may be at least two, respectively disposed on different surfaces of the terminal 900 or in a folded design; in still other embodiments, the display 905 may be a flexible display disposed on a curved surface or a folded surface of the terminal 900. Even more, the display 905 may be arranged in an irregular pattern other than rectangular, i.e., a shaped screen. The display 905 may be made of LCD (Liquid Crystal Display ), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 906 is used to capture images or video. Optionally, the camera assembly 906 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal and the rear camera is disposed on the rear surface of the terminal. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, camera assembly 906 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

The audio circuit 907 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 901 for processing, or inputting the electric signals to the radio frequency circuit 904 for voice communication. For purposes of stereo acquisition or noise reduction, the microphone may be plural and disposed at different portions of the terminal 900. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, the audio circuit 907 may also include a headphone jack.

The location component 908 is used to locate the current geographic location of the terminal 900 to enable navigation or LBS (Location Based Service, location-based services). The positioning component 908 may be a positioning component based on the United states GPS (Global Positioning System ), the Beidou system of China, or the Galileo system of Russia.

The power supply 909 is used to supply power to the various components in the terminal 900. The power supply 909 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power source 909 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, terminal 900 can further include one or more sensors 910. The one or more sensors 910 include, but are not limited to: acceleration sensor 911, gyroscope sensor 912, pressure sensor 913, fingerprint sensor 914, optical sensor 915, and proximity sensor 916.

The acceleration sensor 911 can detect the magnitudes of accelerations on three coordinate axes of the coordinate system established with the terminal 900. For example, the acceleration sensor 911 may be used to detect components of gravitational acceleration in three coordinate axes. The processor 901 may control the touch display 905 to display a user interface in a landscape view or a portrait view according to the gravitational acceleration signal acquired by the acceleration sensor 911. The acceleration sensor 911 may also be used for the acquisition of motion data of a game or a user.

The gyro sensor 912 may detect a body direction and a rotation angle of the terminal 900, and the gyro sensor 912 may collect a 3D motion of the user on the terminal 900 in cooperation with the acceleration sensor 911. The processor 901 may implement the following functions according to the data collected by the gyro sensor 912: motion sensing (e.g., changing UI according to a tilting operation by a user), image stabilization at shooting, game control, and inertial navigation.

The pressure sensor 913 may be provided at a side frame of the terminal 900 and/or a lower layer of the touch display 905. When the pressure sensor 913 is provided at a side frame of the terminal 900, a grip signal of the user to the terminal 900 may be detected, and the processor 901 performs left-right hand recognition or shortcut operation according to the grip signal collected by the pressure sensor 913. When the pressure sensor 913 is disposed at the lower layer of the touch display 905, the processor 901 performs control of the operability control on the UI interface according to the pressure operation of the user on the touch display 905. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The fingerprint sensor 914 is used for collecting the fingerprint of the user, and the processor 901 identifies the identity of the user according to the fingerprint collected by the fingerprint sensor 914, or the fingerprint sensor 914 identifies the identity of the user according to the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, the processor 901 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying for and changing settings, etc. The fingerprint sensor 914 may be provided on the front, back or side of the terminal 900. When a physical key or a vendor Logo is provided on the terminal 900, the fingerprint sensor 914 may be integrated with the physical key or the vendor Logo.

The optical sensor 915 is used to collect the intensity of ambient light. In one embodiment, the processor 901 may control the display brightness of the touch display 905 based on the intensity of ambient light collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display brightness of the touch display 905 is turned up; when the ambient light intensity is low, the display brightness of the touch display panel 905 is turned down. In another embodiment, the processor 901 may also dynamically adjust the shooting parameters of the camera assembly 906 based on the ambient light intensity collected by the optical sensor 915.

A proximity sensor 916, also referred to as a distance sensor, is typically provided on the front panel of the terminal 900. Proximity sensor 916 is used to collect the distance between the user and the front of terminal 900. In one embodiment, when the proximity sensor 916 detects that the distance between the user and the front face of the terminal 900 gradually decreases, the processor 901 controls the touch display 905 to switch from the bright screen state to the off screen state; when the proximity sensor 916 detects that the distance between the user and the front surface of the terminal 900 gradually increases, the processor 901 controls the touch display 905 to switch from the off-screen state to the on-screen state.

Those skilled in the art will appreciate that the structure shown in fig. 10 is not limiting and that more or fewer components than shown may be included or certain components may be combined or a different arrangement of components may be employed.

The embodiment of the invention also provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and when the computer program is executed by a processor, the method for testing the product version provided by the embodiment of the invention is realized, such as the method for testing the product version shown in fig. 3 or fig. 4.

The embodiment of the invention also provides a computer program product containing instructions, which when run on a computer, cause the computer to execute the product version testing method provided by the embodiment of the invention, such as the product version testing method shown in fig. 3 or fig. 4.

In this disclosure, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The term "plurality" refers to two or more, unless explicitly defined otherwise.

It should be noted that, the method embodiment provided in the embodiment of the present invention can be referred to with the corresponding device embodiment, and the embodiment of the present invention is not limited thereto. The sequence of the steps of the method embodiment provided by the embodiment of the invention can be properly adjusted, the steps can be correspondingly increased or decreased according to the situation, and any method which is easily conceivable to be changed by a person skilled in the art within the technical scope of the disclosure of the invention is covered in the protection scope of the invention, so that the description is omitted.

The foregoing description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, since it is intended that all modifications, equivalents, improvements, etc. that fall within the spirit and scope of the invention.

Claims

1. A method for product version testing, the method comprising:

2. The method of claim 1, wherein prior to the corresponding P sequences according to each of the plurality of test groups, the method further comprises:

3. The method according to claim 2, wherein the element of the mth row and the nth column in the target matrix is (m-1) p+n-1 or (m-1) p+n, 1.ltoreq.m.ltoreq.p, 1.ltoreq.n.ltoreq.p.

4. A method according to any one of claims 1 to 3, wherein each of the plurality of user requests carries a destination number, dividing the plurality of user requests into Q user requests, comprising:

5. The method of claim 4, wherein the step of determining the position of the first electrode is performed,

the target number carried by the user request comprises: a user identification number;

6. The method of claim 4, wherein dividing the plurality of user requests into the Q user requests according to the destination numbers in the plurality of user requests comprises:

7. The method of claim 4, wherein dividing the plurality of user requests into the Q user requests according to the destination numbers in the plurality of user requests comprises:

8. A product version testing device, characterized in that the product version testing device comprises:

9. A product version testing device, characterized in that the product version testing device comprises: a processor and a memory having stored therein at least one instruction that is loaded and executed by the processor to implement the product version testing method of any of claims 1-7.

10. A computer readable storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement the product version testing method of any one of claims 1-7.