CA3168300A1 - Method for selecting member data for e-commerce platforms and system thereof - Google Patents
Method for selecting member data for e-commerce platforms and system thereofInfo
- Publication number
- CA3168300A1 CA3168300A1 CA3168300A CA3168300A CA3168300A1 CA 3168300 A1 CA3168300 A1 CA 3168300A1 CA 3168300 A CA3168300 A CA 3168300A CA 3168300 A CA3168300 A CA 3168300A CA 3168300 A1 CA3168300 A1 CA 3168300A1
- Authority
- CA
- Canada
- Prior art keywords
- data
- bitmap table
- consumption
- plural
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000013499 data model Methods 0.000 claims abstract description 30
- 238000013507 mapping Methods 0.000 claims abstract description 17
- 238000004140 cleaning Methods 0.000 claims description 16
- 230000002708 enhancing effect Effects 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 7
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000000295 complement effect Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Provided are a method and system for clustering member data on an electronic commerce platform, capable of reducing consumption of memory and computation resources while ensuring precision of clustered member consumption data, thereby significantly enhancing the efficiency of clustering member consumption data. The method comprises: synchronizing member consumption data in a data warehouse to create a multiple-data model; generating multiple mutually different integer identifiers on the basis of member serial numbers in the data model, and storing mapping relationships between the member serial numbers and the integer identifiers in a dictionary table; generating a bitmap on the basis of one-to-one correspondence relationships between the integer identifiers and multiple latitude consumption fields in the member consumption data; and using the integer identifiers to perform, according to a query command of a user, a bit operation with respect to the multiple latitude consumption fields in the bitmap, and outputting a clustered result. The system comprises the method proposed in the above solution.
Description
METHOD FOR SELECTING MEMBER DATA FOR E-COMMERCE PLATFORMS
AND SYSTEM THEREOF
BACKGROUND OF THE INVENTION
Technical Field [0001] The present invention relates to the technical field of data processing, and more particularly to a method and a system for selecting member data for e-commerce platforms.
Description of Related Art
AND SYSTEM THEREOF
BACKGROUND OF THE INVENTION
Technical Field [0001] The present invention relates to the technical field of data processing, and more particularly to a method and a system for selecting member data for e-commerce platforms.
Description of Related Art
[0002] To e-commerce platforms, findings from analysis of member consumption data are important references for designing sales promotion. During analysis of member consumption data, multi-latitudinal selection of data forms a key step for acquiring member consumption data. In the prior art, data selection is usually achieved by pre-aggregation selection (OLAP-Druid) and distributed memory computing selection (SPARK). Therein, OLAP-Druid selection mainly adopts an HLL algorithm to compute and analyze consumption data of platform members. This algorithm tends to lose precision in business scenarios related to deduplication, making the resulting member consumption data selection imprecise. As to SPARK selection, it involves acquiring original consumption data of platform members, and placing the original consumption data in memory of each distributed time node, in an extracting-detail manner, for logical computing, so as to generate the result of member consumption data selection eventually.
It has been found in practical use that since the amount of original consumption data of members is huge, while the latter approach ensures precise selection of member consumption data, it consumes more computing resources and memory resources, leading Date Regue/Date Received 2022-07-18 to low selection efficiency and poor user experience.
SUMMARY OF THE INVENTION
It has been found in practical use that since the amount of original consumption data of members is huge, while the latter approach ensures precise selection of member consumption data, it consumes more computing resources and memory resources, leading Date Regue/Date Received 2022-07-18 to low selection efficiency and poor user experience.
SUMMARY OF THE INVENTION
[0003] The objective of the present invention is to provide a method for selecting member data for e-commerce platforms and a system thereof, which reduce consumption of memory and computing resources while ensuring precise selection of member consumption data, thereby significantly improving selection efficiency of member consumption data.
[0004] In order to achieve the foregoing objective, in one aspect, the present invention provides a method for selecting member data for e-commerce platforms, which comprises:
[0005] synchronizing member consumption data from a data warehouse so as to create plural data models;
[0006] based on member codes in the data models, generating plural integer type identifiers that are different from each other, and storing mapping relationship between the member codes and the integer type identifiers into a dictionary table;
[0007] associating the integer type identifiers with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate a bitmap table; and
[0008] according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, and outputting a selection result.
[0009] Preferably, after the step of associating the integer type identifiers with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate bitmap table, the method further comprises:
[0010] regularly updating field data in the bitmap table by means of interpolation, so as to generate a bitmap table corresponding to the present time node.
[0011] Specifically, the step of regularly updating field data in the bitmap table by means of interpolation, so as to generate a bitmap table corresponding to the present time node comprises:
[0012] based on the present time node, acquiring new member consumption data from the data Date Regue/Date Received 2022-07-18 warehouse, and synchronizing the data to the data models;
[0013] according to the mapping relationship of the member codes in the dictionary table, regularly interpolating the field data corresponding to the new member consumption data into the bitmap table, thereby updating the bitmap table by means of interpolation.
[0014] More preferably, after the step of regularly updating field data in the bitmap table by means of interpolation, so as to generate a bitmap table corresponding to the present time node, the method further comprises:
[0015] cleaning the bitmap table, and eliminating irrelevant latitudinal field data.
[0016] Further, before the step of according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, outputting selection result, the method further comprises:
[0017] presetting plural query instructions, performing a bitwise operation on the cleaned bitmap table in advance so as to obtain pre-selection results matching the plural query instructions; and
[0018] storing the pre-selection results into a temporary result list for user query.
[0019] Preferably, the step of according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, outputting selection result comprises:
[0020] receiving a query instruction from the user, and determining whether the query instruction is a said preset query instruction or not; and
[0021] if said determining has a positive result, directly outputting the matching pre-selection results from the temporary result list, and if said determining has a negative result, based on the bitmap table updated by interpolation, performing a logical operation on plural said latitudinal consumption fields through integer type identifiers and then outputting a selection result.
[0022] As compared to the prior art, the method for selecting member data for e-commerce platforms of the present invention provides the following beneficial effects.
[0023] The method for selecting member data for e-commerce platforms of the present invention first acquires the member consumption data from the data warehouse to create the data Date Regue/Date Received 2022-07-18 models. Therein, the data model comprises member codes, plural latitudinal consumption fields, and consumption dates. The method converts the member codes one by one into integer type identifiers, and stores the mapping relationship between the integer type identifiers and the member codes into the dictionary table. Then the integer type identifiers, the consumption fields, and the consumption date are used to construct the bitmap table. After the query instruction of the user is acquired, the integer type identifiers are called for logical bitwise operation on the plural latitudinal consumption fields in the bitmap table, thereby obtaining the selection result eventually.
[0024] It is thus clear that in the method for selecting member data for e-commerce platforms of the present invention, the integer type identifiers are used instead of member codes and the member consumption data are represented through a bitmap table, so that selection of member data can be easily accomplished by means of bitwise operation of sets in the bitmap table, thereby reducing use of computing and storing resources while significantly enhancing computing efficiency. Therefore, the disclosed method is particularly suitable for selection of massive member data.
[0025] In another aspect, the present invention provides a system for selecting member data for e-commerce platforms to be used in the method for selecting member data for e-commerce platforms of the technical scheme described above. The system comprises:
[0026] a data model creating unit, for synchronizing member consumption data from a data warehouse so as to create plural data models;
[0027] a dictionary table creating unit, for generating plural integer type identifiers that are different from each other based on member codes in the data models, and storing mapping relationship between the member codes and the integer type identifiers into a dictionary table;
[0028] a bitmap table generating unit, for associating the integers type identifier with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate a bitmap table; and
[0029] a query outputting unit, for according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using Date Regue/Date Received 2022-07-18 the integer type identifiers, and outputting a selection result.
[0030] Preferably, it further comprises a bitmap table updating unit connected to the bitmap table generating unit;
[0031] in which the bitmap table updating unit is for regularly updating field data in the bitmap table by means of interpolation, so as to generate a bitmap table corresponding to the present time node.
[0032] More preferably, the bitmap table updating unit comprises:
[0033] a data acquiring module, for acquiring new member consumption data from the data warehouse based on the present time node, and synchronizing the data to the data models;
and
and
[0034] a bitmap table updating module, for according to the mapping relationship of the member codes into the dictionary table, regularly interpolating the field data corresponding to the new member consumption data into the bitmap table, thereby updating the bitmap table by means of interpolation.
[0035] More preferably, further comprises a data cleaning unit arranged between the bitmap table generating unit and the query outputting unit;
[0036] in which the data cleaning unit is for cleaning the bitmap table, and eliminating irrelevant latitudinal field data.
[0037] Preferably, it further comprises a pre-selection unit and a storing unit, wherein the pre-selection unit has an input end connected to an output end of the data cleaning unit, and the storing unit has an output end connected to an input end of the query outputting unit;
[0038] the pre-selection unit, for presetting plural query instructions, performing a bitwise operation on the cleaned bitmap table in advance so as to obtain pre-selection results matching the plural query instructions; and
[0039] the storing unit, for storing the plural pre-selection results into a temporary result list for user query.
[0040] Preferably, the query outputting unit comprises:
[0041] a determining module, for receiving a query instruction from the user, and determining whether the query instruction is a said preset query instruction or not; and Date Regue/Date Received 2022-07-18
[0042] an outputting module, for, when said determining has a positive result, directly finding and outputting a matching said pre-selection result from the temporary result list, and if said determining has a negative result, based on the bitmap table updated by interpolation, performing a logical operation on plural said latitudinal consumption fields through integer type identifiers and then outputting a selection result.
[0043] As compared to the prior art, the disclosed system for selecting member data for e-commerce platforms provides beneficial effects that are similar to those provided by the disclosed method for selecting member data for e-commerce platforms as enumerated above, and thus no repetitions are made herein.
BRIEF DESCRIPTION OF THE DRAWINGS
BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The accompanying drawings are provided herein for better understanding of the present invention and form a part of this disclosure. The illustrative embodiments and their descriptions are for explaining the present invention and by no means form any improper limitation to the present invention, wherein:
[0045] FIG. 1 is a flowchart of a method for selecting member data for e-commerce platforms according to Embodiment 1 of the present invention;
[0046] FIG. 2 shows exemplary selection results obtained using the method of Embodiment 1 of the present invention;
[0047] FIG. 3 is an example of a bitmap table A according to Embodiment 1 of the present invention;
[0048] FIG. 4 is an example of a bitmap table B according to Embodiment 1 of the present invention; and
[0049] FIG. 5 is a structural diagram of a system for selecting member data for e-commerce platforms according to Embodiment 2 of the present invention.
Date Regue/Date Received 2022-07-18
Date Regue/Date Received 2022-07-18
[0050] Reference numerals:
[0051] 1 ¨ data model creating unit 2¨ dictionary table creating unit
[0052] 3 ¨ bitmap table generating unit 4¨ query outputting unit
[0053] 5 ¨ bitmap table updating module 6 ¨ data cleaning unit
[0054] 7¨ pre-selection unit 8 ¨ storing unit
[0055] 41 ¨ determining module 42¨ outputting module
[0056] 51 ¨ data acquiring module 52¨ bitmap table updating module DETAILED DESCRIPTION OF THE INVENTION
[0057] To make the foregoing objectives, features, and advantages of the present invention clearer and more understandable, the following description will be directed to some embodiments as depicted in the accompanying drawings to detail the technical schemes disclosed in these embodiments. It is, however, to be understood that the embodiments referred herein are only a part of all possible embodiments and thus not exhaustive. Based on the embodiments of the present invention, all the other embodiments can be conceived without creative labor by people of ordinary skill in the art, and all these and other embodiments shall be encompassed in the scope of the present invention.
[0058] Embodiment 1
[0059] Referring to FIG. 1, the present embodiment provides a method for selecting member data for e-commerce platfouns, which comprises:
[0060] synchronizing member consumption data from a data warehouse so as to create plural data models; based on member codes in the data models, generating plural integer type identifiers that are different from each other, and storing mapping relationship between the member codes and the integer type identifiers into a dictionary table;
associating the integers type identifier with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate a bitmap table; and according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, and outputting a selection result.
associating the integers type identifier with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate a bitmap table; and according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, and outputting a selection result.
[0061] The method for selecting member data for e-commerce platforms of the present Date Regue/Date Received 2022-07-18 embodiment first acquires the member consumption data from the data warehouse to create the data models. Therein, the data model comprises member codes, plural latitudinal consumption fields, and consumption dates. The method converts the member codes one by one into integer type identifiers, and stores the mapping relationship between the integer type identifiers and the member codes into the dictionary table. Then the integer type identifiers, the consumption fields, and the consumption dates are used to construct the bitmap table. After the query instruction of the user is acquired, the integer type identifiers are called for logical bitwise operation on the plural latitudinal consumption fields in the bitmap table, thereby obtaining the selection result eventually.
[0062] It is thus clear that in the method for selecting member data for e-commerce platforms of the present embodiment, the integer type identifiers are used instead of member codes and the member consumption data are represented through a bitmap table, so that selection of member data can be easily accomplished by means of bitwise operation of sets in the bitmap table, thereby reducing use of computing and storing resources while significantly enhancing computing efficiency. Therefore, the disclosed method is particularly suitable for selection of massive member data.
[0063] Still referring to FIG. 1, in view that the member consumption data update every day, for preventing hysteresis of the bitmap table data, in the present embodiment, after the step of associating the integers type identifier with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate a bitmap table, the method further comprises: regularly interpolating and updating the field data in the bitmap table, so as to generate a bitmap table corresponding to the present time node.
[0064] Specifically, in the embodiment described previously, the step of regularly interpolating and updating the field data in the bitmap table, so as to generate a bitmap table corresponding to the time present node comprises:
[0065] based on the present time node, acquiring new member consumption data from the data warehouse, and synchronizing the data to the data models; according to the mapping relationship of the member codes in the dictionary table, regularly interpolating the field data corresponding to the new member consumption data into the bitmap table, thereby Date Regue/Date Received 2022-07-18 updating the bitmap table by means of interpolation.
[0066] In practical implementations, the member consumption data are regularly acquired from the data warehouse for updating. Then, based on the mapping relationship in the dictionary table, a designated complement function is used to perform complement operation, so as to achieve interpolation and updating of the field data in the bitmap table.
An example is explained herein for facilitating understanding of the interpolation and updating process. First, the updated member consumption data are acquired from the data models and the member codes for which the consumption data have been updated are identified and then converted using the dictionary table, so as to get their matching integer type identifiers. Then the consumption field data updated on the day (flag=1) corresponding to the integer type identifiers are acquired and stored into the bitmap table.
The consumption field data summarized on the present day (flag=1) and the consumption field data summarized on the day before the present day (flag=2) are integrated and used as the consumption field data currently summarized (flag=2) and inserted into the bitmap table, thereby accomplishing interpolation and updating of the bitmap table.
It is to be noted that flag=1 only represents the consumption field data updated on the present day and flag=2 represents all consumption field data existing currently. The latter comprises the consumption field data updated on the present day and all consumption field data of the present day and all previous days. It is known from the foregoing implementation that the present embodiment can continuously update the consumption field data in the bitmap table by accumulated integration instead of repeated computing for all historical data, and thus effectively reduce computing loads while ensuring a precise selection result.
An example is explained herein for facilitating understanding of the interpolation and updating process. First, the updated member consumption data are acquired from the data models and the member codes for which the consumption data have been updated are identified and then converted using the dictionary table, so as to get their matching integer type identifiers. Then the consumption field data updated on the day (flag=1) corresponding to the integer type identifiers are acquired and stored into the bitmap table.
The consumption field data summarized on the present day (flag=1) and the consumption field data summarized on the day before the present day (flag=2) are integrated and used as the consumption field data currently summarized (flag=2) and inserted into the bitmap table, thereby accomplishing interpolation and updating of the bitmap table.
It is to be noted that flag=1 only represents the consumption field data updated on the present day and flag=2 represents all consumption field data existing currently. The latter comprises the consumption field data updated on the present day and all consumption field data of the present day and all previous days. It is known from the foregoing implementation that the present embodiment can continuously update the consumption field data in the bitmap table by accumulated integration instead of repeated computing for all historical data, and thus effectively reduce computing loads while ensuring a precise selection result.
[0067] Optionally, referring to FIG. 1, in the embodiment described previously, after the step of regularly interpolating and updating the field data in the bitmap table, so as to generate a bitmap table corresponding to the present time node, the method further comprises:
cleaning the bitmap table and eliminating irrelevant latitudinal field data.
cleaning the bitmap table and eliminating irrelevant latitudinal field data.
[0068] In practical implementations, this step is equivalent to creating a CUBE model. After the field data of irrelevant dimensions are eliminated from the bitmap table, the data cardinality after grouping can be decreased, which means the query efficiency can be Date Regue/Date Received 2022-07-18 increased and the selection can be accelerated.
[0069] In order to further accelerate the process of selection, in the present embodiment, before the step according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, outputting selection result, the method further comprises:
[0070] pre-setting plural query instruction, performing a bitwise operation on the cleaned bitmap table in advance so as to obtain pre-selection results matching the plural query instructions; and storing the pre-selection results into a temporary result list for user query.
[0071] In practical implementations, since the amount of the field data in the bitmap table is huge, real-time computing can cause delayed output of the selection result. To prevent this, in the present embodiment, commonly used query instructions are stored in advance, so that the system can perform a bitwise operation on the cleaned bitmap table in advance according to these query instructions and get corresponding pre-selection results. These pre-selection results are stored into a temporary result list so as to be directly called and used in response to a user query.
[0072] Further, in the embodiment described previously, the step of according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, outputting selection result comprises:
[0073] receiving a query instruction from a user, and determining whether the query instruction is a preset query instruction; if said determining has a positive result, directly outputting the matching pre-selection results from the temporary result list, and if said determining has a negative result, based on the bitmap table updated by interpolation, performing a logical operation on plural said latitudinal consumption fields through integer type identifiers and then outputting a selection result. It is thus clear that, with the setting of the foregoing two computing modes, a pre-selection result can be directly called from the temporary result list if the query instruction made by the user matches one of the pre-stored query instructions, thereby reducing waiting time required by computing. If the query instruction made by the user does not match any of the pre-stored query instructions, Date Regue/Date Received 2022-07-18 a selection result can be generated through a logical operation based on the bitmap table updated by interpolation according to the query instruction, which means that the selection result in this case is generated through real-time computing.
Thereby, the present invention supports an expanded range of user queries and supports user custom multi-dimension selection.
Thereby, the present invention supports an expanded range of user queries and supports user custom multi-dimension selection.
[0074] For easy understanding, the present embodiment is now explained with an example of selection of consumption data of new and existing members. As shown in FIG. 2, the member consumption data of 0826 and 0827 are acquired from the data warehouse and a data model is created accordingly. The data model comprises consumption fields of plural latitudes, such as member codes, purchase channels, purchase categories, purchase dates, etc. By calling the mapping relationship in the dictionary table, the member codes are converted into integer type identifiers that facilitate operations of a bitmap table. For easy understanding and distinction, buyers are hereinafter represented letters A, B, C, and D.
After the bitmap table is cleaned, the consumption data of new and existing members each day are summarized by purchase channels into a bitmap table A. The statistical criteria include purchase channels (online or offline) + flag (flag=1 or flag=2) + purchase date (0826 or 0827). The statistical data for each day include two entries.
One is the bitmap table set of the members of the present day (flag=1), and the other is the bitmap table set of the currently existing (flag=2) members. Then the consumption data of new and existing members are summarized under the criteria including purchase category +
purchase channel + flag (flag=1 or flag=2) + purchase date (0826 or 0827) into bitmap table B. Similarly, the statistical data for each day include two entries. The first is the bitmap table set of the members of the present day (flag=1), and the second is the bitmap table set of the currently existing (flag=2) members. Afterward, according to the query instruction func( ), selection is made from the following three scenarios:
After the bitmap table is cleaned, the consumption data of new and existing members each day are summarized by purchase channels into a bitmap table A. The statistical criteria include purchase channels (online or offline) + flag (flag=1 or flag=2) + purchase date (0826 or 0827). The statistical data for each day include two entries.
One is the bitmap table set of the members of the present day (flag=1), and the other is the bitmap table set of the currently existing (flag=2) members. Then the consumption data of new and existing members are summarized under the criteria including purchase category +
purchase channel + flag (flag=1 or flag=2) + purchase date (0826 or 0827) into bitmap table B. Similarly, the statistical data for each day include two entries. The first is the bitmap table set of the members of the present day (flag=1), and the second is the bitmap table set of the currently existing (flag=2) members. Afterward, according to the query instruction func( ), selection is made from the following three scenarios:
[0075] Scenario I: the bitmap table A of online new buyers of the purchase date 0827 is as shown in FIG. 3. The selection process is essentially about a rb andnot cardinality bitwise operation of the bitmap set {A, DI and the bitmap set {A, C}. The obtained selection result is {D}, and the summed number of new buyers is 1.
Date Regue/Date Received 2022-07-18
Date Regue/Date Received 2022-07-18
[0076] Scenario II: the bitmap table B of online A/C (air conditioner) new buyers of the purchase date 0827 is as shown in FIG. 4. The process of identifying the number of the online A/C new buyers of the purchase date 0827 is essentially about a rb andnot cardinality bitwise operation of the bitmap set {C, AI and the bitmap set {C}.
The obtained selection result is {A}, and the summed number of new buyers is 1.
Similarly, the process of identifying the number of the online R&W
(refrigerators and washing machines) new buyers of the purchase date 0827 is essentially about a rb andnot cardinality bitwise operation of the bitmap set {D} and the bitmap empty set.
The obtained selection result is {D}, and the summed number of new buyers is 1.
The obtained selection result is {A}, and the summed number of new buyers is 1.
Similarly, the process of identifying the number of the online R&W
(refrigerators and washing machines) new buyers of the purchase date 0827 is essentially about a rb andnot cardinality bitwise operation of the bitmap set {D} and the bitmap empty set.
The obtained selection result is {D}, and the summed number of new buyers is 1.
[0077] Scenario III: the process of identifying how many online air conditioner new buyers of the purchase date 0827 are online new buyers is essentially about a rb and cardinality operation of the bitmap table A and the bitmap table B. In other words, this is about a rb andnot cardinality bitwise operation of the bitmap set {A} and the bitmap set {A, C}.
The obtained selection result is an empty set, and the summed number of new buyers is 0. Similarly, the process of identifying how many of the online R&W new buyers of the purchase date 0827 are online new buyers is essentially about a rb andnot cardinality bitwise operation of the bitmap set {D} and the bitmap set {A, C}. The obtained selection result is {D}, and the summed number of new buyers is 1.
The obtained selection result is an empty set, and the summed number of new buyers is 0. Similarly, the process of identifying how many of the online R&W new buyers of the purchase date 0827 are online new buyers is essentially about a rb andnot cardinality bitwise operation of the bitmap set {D} and the bitmap set {A, C}. The obtained selection result is {D}, and the summed number of new buyers is 1.
[0078] Embodiment 2
[0079] Referring to FIG. 1 and FIG. 5, the present embodiment provides a system for selecting member data for e-commerce platforms, which comprises:
[0080] a data model creating unit 1, for synchronizing member consumption data from a data warehouse so as to create plural data models;
[0081] a dictionary table creating unit 2, for generating plural integer type identifiers that are different from each other based on the member codes in the data model, and storing mapping relationship between the member codes and the integer type identifiers into a dictionary table;
[0082] a bitmap table generating unit 3, for associating the integers type identifier with plural Date Regue/Date Received 2022-07-18 latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate a bitmap table; and
[0083] a query outputting unit 4, for according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, and outputting a selection result.
[0084] Preferably, the system further comprises a bitmap table updating unit 6 connected to the bitmap table generating unit 3. The bitmap table updating unit 6 is for regularly interpolating and updating the field data in the bitmap table, so as to generate a bitmap table corresponding to the present time node.
[0085] Preferably, the bitmap table updating unit 6 comprises:
[0086] a data acquiring module 51, for acquiring new member consumption data from the data warehouse based on the present time node, and synchronizing the data to the data models;
and
and
[0087] a bitmap table updating module 52, for according to the mapping relationship of the member codes in the dictionary table, regularly interpolating the field data corresponding to the new member consumption data into the bitmap table, thereby updating the bitmap table by means of interpolation.
[0088] Preferably, the system further comprises a data cleaning unit 6 arranged between the bitmap table generating unit 3 and the query outputting unit 4; and
[0089] a data cleaning unit 6, for cleaning the bitmap table, and eliminating irrelevant latitudinal field data.
[0090] Preferably, the system further comprises a pre-selection unit 7 and a storing unit 8. The pre-selection unit 7 has its input end connected to the output end of the data cleaning unit 6, and the storing unit 8 has its output end connected to the input end of the query outputting unit 4;
[0091] a pre-selection unit 7, for presetting plural query instructions, performing a bitwise operation on the cleaned bitmap table in advance so as to obtain pre-selection results matching the plural query instructions; and
[0092] a storing unit 8, for storing the pre-selection results into a temporary result list for user Date Regue/Date Received 2022-07-18 query.
[0093] Preferably, the query outputting unit 4 comprises:
[0094] a determining module 41, for receiving a query instruction from a user, and determining whether the query instruction is a preset query instruction; and
[0095] an outputting module 42, for, if said determining has a positive result, directly outputting the matching pre-selection results from the temporary result list, and if said determining has a negative result, based on the bitmap table updated by interpolation, performing a logical operation on plural said latitudinal consumption fields through integer type identifiers and then outputting a selection result.
[0096] As compared to the prior art, the disclosed system for selecting member data for e-commerce platforms provides beneficial effects that are similar to those provided by the disclosed method for selecting member data for e-commerce platforms as enumerated in Embodiment 1, and thus no repetitions are made herein.
[0097] As will be appreciated by people of ordinary skill in the art, implementation of all or a part of the steps of the method of the present invention as described previously may be realized by having a program instruct related hardware components. The program may be stored in a computer-readable storage medium, and the program is about performing the individual steps of the methods described in the foregoing embodiments.
The storage medium may be a ROM/RAM, a hard drive, an optical disk, a memory card or the like.
The storage medium may be a ROM/RAM, a hard drive, an optical disk, a memory card or the like.
[0098] The present invention has been described with reference to the preferred embodiments and it is understood that the embodiments are not intended to limit the scope of the present invention. Moreover, as the contents disclosed herein should be readily understood and can be implemented by a person skilled in the art, all equivalent changes or modifications which do not depart from the concept of the present invention should be encompassed by the appended claims. Hence, the scope of the present invention shall only be defined by the appended claims.
Date Regue/Date Received 2022-07-18
Date Regue/Date Received 2022-07-18
Claims (12)
1. A method for selecting member data for e-commerce platforms, comprising:
synchronizing member consumption data from a data warehouse so as to create plural data models;
based on member codes in the data models, generating plural integer type identifiers that are different from each other, and storing mapping relationship between the member codes and the integer type identifiers into a dictionary table;
associating the integer type identifiers with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate a bitmap table; and according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, and outputting a selection result.
synchronizing member consumption data from a data warehouse so as to create plural data models;
based on member codes in the data models, generating plural integer type identifiers that are different from each other, and storing mapping relationship between the member codes and the integer type identifiers into a dictionary table;
associating the integer type identifiers with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate a bitmap table; and according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, and outputting a selection result.
2. The method of claim 1, wherein after the step of associating the integer type identifiers with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate bitmap table, the method further comprises:
regularly updating field data in the bitmap table by means of interpolation, so as to generate a bitmap table corresponding to the present time node.
regularly updating field data in the bitmap table by means of interpolation, so as to generate a bitmap table corresponding to the present time node.
3. The method of claim 2, wherein the step of regularly updating field data in the bitmap table by means of interpolation, so as to generate a bitmap table corresponding to the present time node comprises:
based on the present time node, acquiring new member consumption data from the data warehouse, and synchronizing the data to the data models;
according to the mapping relationship of the member codes in the dictionary table, regularly interpolating the field data corresponding to the new member consumption data into the bitmap Date Regue/Date Received 2022-07-18 table, thereby updating the bitmap table by means of interpolation.
based on the present time node, acquiring new member consumption data from the data warehouse, and synchronizing the data to the data models;
according to the mapping relationship of the member codes in the dictionary table, regularly interpolating the field data corresponding to the new member consumption data into the bitmap Date Regue/Date Received 2022-07-18 table, thereby updating the bitmap table by means of interpolation.
4. The method of claim 2, wherein after the step of regularly updating field data in the bitmap table by means of interpolation, so as to generate a bitmap table corresponding to the present time node, the method further comprises:
cleaning the bitmap table, and eliminating irrelevant latitudinal field data.
cleaning the bitmap table, and eliminating irrelevant latitudinal field data.
5. The method of claim 4, wherein before the step of according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, outputting selection result, the method further comprises:
presetting plural query instructions, performing a bitwise operation on the cleaned bitmap table in advance so as to obtain pre-selection results matching the plural query instructions; and storing the pre-selection results into a temporary result list for user query.
presetting plural query instructions, performing a bitwise operation on the cleaned bitmap table in advance so as to obtain pre-selection results matching the plural query instructions; and storing the pre-selection results into a temporary result list for user query.
6. The method of claim 5, wherein the step of according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, outputting selection result comprises:
receiving a query instruction from the user, and determining whether the query instruction is a said preset query instruction or not; and if said determining has a positive result, directly outputting the matching pre-selection results from the temporary result list, and if said determining has a negative result, based on the bitmap table updated by interpolation, performing a logical operation on plural said latitudinal consumption fields through integer type identifiers and then outputting a selection result.
receiving a query instruction from the user, and determining whether the query instruction is a said preset query instruction or not; and if said determining has a positive result, directly outputting the matching pre-selection results from the temporary result list, and if said determining has a negative result, based on the bitmap table updated by interpolation, performing a logical operation on plural said latitudinal consumption fields through integer type identifiers and then outputting a selection result.
7. A system for selecting member data for e-commerce platforms, comprising:
a data model creating unit, for synchronizing member consumption data from a data warehouse so as to create plural data models;
a dictionary table creating unit, for generating plural integer type identifiers that are different from each other based on member codes in the data models, and storing mapping relationship Date Regue/Date Received 2022-07-18 between the member codes and the integer type identifiers into a dictionary table;
a bitmap table generating unit, for associating the integers type identifier with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate a bitmap table; and a query outputting unit, for according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, and outputting a selection result.
a data model creating unit, for synchronizing member consumption data from a data warehouse so as to create plural data models;
a dictionary table creating unit, for generating plural integer type identifiers that are different from each other based on member codes in the data models, and storing mapping relationship Date Regue/Date Received 2022-07-18 between the member codes and the integer type identifiers into a dictionary table;
a bitmap table generating unit, for associating the integers type identifier with plural latitudinal consumption fields of the member consumption data in a one-to-one manner, so as to generate a bitmap table; and a query outputting unit, for according to a query instruction from a user, performing bitwise operation on the plural latitudinal consumption fields in the bitmap table using the integer type identifiers, and outputting a selection result.
8. The system of claim 7, further comprising a bitmap table updating unit connected to the bitmap table generating unit;
in which the bitmap table updating unit is for regularly updating field data in the bitmap table by means of interpolation, so as to generate a bitmap table corresponding to the present time node.
in which the bitmap table updating unit is for regularly updating field data in the bitmap table by means of interpolation, so as to generate a bitmap table corresponding to the present time node.
9. The system of claim 8, wherein the bitmap table updating unit comprises:
data acquiring module, for acquiring new member consumption data from the data warehouse based on the present time node, and synchronizing the data to the data models;
bitmap table updating module, for according to the mapping relationship of the member codes in the dictionary table, regularly interpolating the field data corresponding to the new member consumption data into the bitmap table, thereby updating the bitmap table by means of interpolation.
data acquiring module, for acquiring new member consumption data from the data warehouse based on the present time node, and synchronizing the data to the data models;
bitmap table updating module, for according to the mapping relationship of the member codes in the dictionary table, regularly interpolating the field data corresponding to the new member consumption data into the bitmap table, thereby updating the bitmap table by means of interpolation.
10. The system of claim 8, further comprising a data cleaning unit arranged between the bitmap table generating unit and the query outputting unit;
in which the data cleaning unit is for cleaning the bitmap table, and eliminating irrelevant latitudinal field data.
in which the data cleaning unit is for cleaning the bitmap table, and eliminating irrelevant latitudinal field data.
11. The system of claim 10, further comprising a pre-selection unit and a storing unit, wherein the pre-selection unit has an input end connected to an output end of the data cleaning unit, and the storing unit has an output end connected to an input end of the query outputting unit;
Date Regue/Date Received 2022-07-18 the pre-selection unit, for presetting plural query instructions, performing a bitwise operation on the cleaned bitmap table in advance so as to obtain pre-selection results matching the plural query instructions; and the storing unit, for storing the plural pre-selection results into a temporary result list for user query.
Date Regue/Date Received 2022-07-18 the pre-selection unit, for presetting plural query instructions, performing a bitwise operation on the cleaned bitmap table in advance so as to obtain pre-selection results matching the plural query instructions; and the storing unit, for storing the plural pre-selection results into a temporary result list for user query.
12. The system of claim 11, wherein the query outputting unit comprises:
a determining module, for receiving a query instruction from the user, and determining whether the query instruction is a said preset query instruction or not; and an outputting module, for, when said determining has a positive result, directly finding and outputting a matching said pre-selection result from the temporary result list, and if said determining has a negative result, based on the bitrnap table updated by interpolation, performing a logical operation on plural said latitudinal consumption fields through integer type identifiers and then outputting a selection result.
Date Regue/Date Received 2022-07-18
a determining module, for receiving a query instruction from the user, and determining whether the query instruction is a said preset query instruction or not; and an outputting module, for, when said determining has a positive result, directly finding and outputting a matching said pre-selection result from the temporary result list, and if said determining has a negative result, based on the bitrnap table updated by interpolation, performing a logical operation on plural said latitudinal consumption fields through integer type identifiers and then outputting a selection result.
Date Regue/Date Received 2022-07-18
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910040702.5A CN111444165B (en) | 2019-01-16 | 2019-01-16 | Member data selection method and system for e-commerce platform |
| CN201910040702.5 | 2019-01-16 | ||
| PCT/CN2019/106863 WO2020147335A1 (en) | 2019-01-16 | 2019-09-20 | Method and system for clustering member data on electronic commerce platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA3168300A1 true CA3168300A1 (en) | 2020-07-23 |
Family
ID=71614009
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA3168300A Pending CA3168300A1 (en) | 2019-01-16 | 2019-09-20 | Method for selecting member data for e-commerce platforms and system thereof |
Country Status (3)
| Country | Link |
|---|---|
| CN (1) | CN111444165B (en) |
| CA (1) | CA3168300A1 (en) |
| WO (1) | WO2020147335A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112540972A (en) * | 2020-12-16 | 2021-03-23 | 中盈优创资讯科技有限公司 | Roaring bitmap-based massive user efficient selection method and device |
| CN115982206B (en) * | 2023-02-09 | 2023-08-29 | 中国证券登记结算有限责任公司 | Method and device for processing data |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663114B (en) * | 2012-04-17 | 2013-09-11 | 中国人民大学 | Database inquiry processing method facing concurrency OLAP (On Line Analytical Processing) |
| CN104715073B (en) * | 2015-04-03 | 2017-11-24 | 江苏物联网研究发展中心 | Based on the association rule mining system for improving Apriori algorithm |
| CN105260442B (en) * | 2015-10-08 | 2018-11-02 | 西安培华学院 | A kind of association rules mining algorithm based on bit arithmetic and inverted index |
| CN108269107B (en) * | 2016-12-30 | 2021-12-14 | 阿里巴巴集团控股有限公司 | User information processing method and device |
| CN106934636A (en) * | 2017-02-28 | 2017-07-07 | 杭州搜娱科技有限公司 | Integrated management approach and system |
| CN107291842B (en) * | 2017-06-01 | 2021-11-19 | 武汉理工大学 | Track query method based on track coding |
| CN107273483B (en) * | 2017-06-06 | 2019-11-05 | 贵州易鲸捷信息技术有限公司 | The access method and system of sparse data |
| US10009832B1 (en) * | 2017-08-11 | 2018-06-26 | At&T Intellectual Property I, L.P. | Facilitating compact signaling design for reserved resource configuration in wireless communication systems |
| CN108415978B (en) * | 2018-02-09 | 2021-04-09 | 北京腾云天下科技有限公司 | User tag storage method, user portrait calculation method and calculation equipment |
-
2019
- 2019-01-16 CN CN201910040702.5A patent/CN111444165B/en active Active
- 2019-09-20 WO PCT/CN2019/106863 patent/WO2020147335A1/en active Application Filing
- 2019-09-20 CA CA3168300A patent/CA3168300A1/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| CN111444165B (en) | 2022-12-02 |
| WO2020147335A1 (en) | 2020-07-23 |
| CN111444165A (en) | 2020-07-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108733681B (en) | Information processing method and device | |
| CN110941598A (en) | Data deduplication method, device, terminal and storage medium | |
| CN106599052B (en) | Apache Kylin-based data query system and method | |
| US20060288045A1 (en) | Method for aggregate operations on streaming data | |
| CN111400482B (en) | Method and device for establishing question-answer pairs | |
| US20180276252A1 (en) | Data Storage Method And Coordinator Node | |
| CN110399395B (en) | Pre-calculation-based accelerated query method and storage medium | |
| CN110990372A (en) | Dimensional data processing method and device and data query method and device | |
| CA3168300A1 (en) | Method for selecting member data for e-commerce platforms and system thereof | |
| CN112396462B (en) | Crowd circling method and device based on click house | |
| CN110765126A (en) | Data storage and query method, device and storage medium of distributed database | |
| CN104408192A (en) | Compression processing method and device of character string type column | |
| CN114218267A (en) | Query request asynchronous processing method and device, computer equipment and storage medium | |
| CN111737537B (en) | POI recommendation method, device and medium based on graph database | |
| CN106372121B (en) | Server and data processing method | |
| CN108536798A (en) | The restoration methods and system of the other database data of order level | |
| CN112632115A (en) | BI-based data query method and system | |
| CN111143316A (en) | Version management system and method for BIM forward design | |
| CN111143356A (en) | Report retrieval method and device | |
| CN112069177A (en) | Data query method and device, computer equipment and readable storage medium | |
| CN104573077A (en) | Method and device for displaying data | |
| US20190228089A1 (en) | Data structure with identifiers | |
| JPH1074156A (en) | Data processor | |
| CN110033098A (en) | Online GBDT model learning method and device | |
| EP3955138A1 (en) | Database search query enhancer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request |
Effective date: 20220718 |
|
| EEER | Examination request |
Effective date: 20220718 |
|
| EEER | Examination request |
Effective date: 20220718 |
|
| EEER | Examination request |
Effective date: 20220718 |
|
| EEER | Examination request |
Effective date: 20220718 |
|
| EEER | Examination request |
Effective date: 20220718 |
|
| EEER | Examination request |
Effective date: 20220718 |
|
| EEER | Examination request |
Effective date: 20220718 |