CN115564300A - E-commerce management system with big data analysis function - Google Patents
E-commerce management system with big data analysis function Download PDFInfo
- Publication number
- CN115564300A CN115564300A CN202211354072.7A CN202211354072A CN115564300A CN 115564300 A CN115564300 A CN 115564300A CN 202211354072 A CN202211354072 A CN 202211354072A CN 115564300 A CN115564300 A CN 115564300A
- Authority
- CN
- China
- Prior art keywords
- data
- crowd
- user
- module
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0637—Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Economics (AREA)
- Development Economics (AREA)
- Human Resources & Organizations (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Entrepreneurship & Innovation (AREA)
- Databases & Information Systems (AREA)
- Educational Administration (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Operations Research (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides an e-commerce management system with a big data analysis function, which comprises: the system comprises a data preprocessing module, a data service module, a brand insights module, a crowd management module, a model management module and an operation management module. The system can provide fusion of diversified data and can flexibly access data provided by a third party, good preconditions are provided for data analysis at the later stage, and diversified and comprehensive high-quality data can help an advertiser to perform more accurate analysis; in addition, the system can realize user portrait and consumer life cycle statistics, and display results on a page in a chart form, so that an advertiser can know consumers more conveniently through visualized data. The crowd mining function is realized, data are analyzed by combining a machine learning algorithm, differential pushing of advertisements can be achieved, and thousands of people and thousands of faces in marketing strategies are realized.
Description
Technical Field
The invention relates to the fields of information management, artificial intelligence, big data technology, e-commerce management system and computer, in particular to an e-commerce management system with big data analysis function.
Background
With the recent development of internet technology, data is generated in various channels. Not only desktop devices, but a wide variety of wearable devices, mobile devices, also generate a large amount of data. The increase of data makes people more and more concerned about the large data field. Through this field, we can view the world from another perspective. Through the fusion processing of a large amount of data, people can obtain meaningful data gold mine, and the method can create higher value for various industries in various fields.
In the E-commerce field, the data of the commodity transaction platform rapidly expands in a geometric situation, and the development and the change lead the domestic network advertisement industry chain to be continuously innovated. How to expand the market and reach new users becomes a matter of more concern for advertisers and third-party data service providers. The traditional marketing means emphasizes large-scale undifferentiated coverage, but with the arrival of the big data era, the defects generated by the marketing strategy are gradually highlighted, the marketing cost of advertisers is improved by large-scale advertisement coverage, the cost of non-directive marketing can be caused to be too much, good income is not obtained, and the situations that the market cannot be expanded, the sales volume is reduced, old users run off, new customers cannot be pulled and the like are caused. In addition, with the development of the e-commerce advertisers, higher requirements are provided, and in order to better meet the real requirements of users, the popularization of products needs to have rich creativity from the first pursuit of simple display to the present, and the efficient propagation and accurate delivery are concerned from the first merely obtaining clicks to the present. Advertisers and data service providers have higher requirements on data analysis of internet advertisement markets, which prompts people to analyze the internet advertisement markets by means of data mining to better understand users and judge markets so as to achieve the aim of promoting marketing strategy planning by data.
Therefore, the processing and application of big data is just the most worth of solving the problem and the most worth of deep ploughing. The advertiser needs a data analysis and management system to carry out integrated analysis on the consumer data, and also needs a system to complete secondary marketing of the data to provide marketing strategy support for the advertiser. It is urgent to apply big data technology and big data analysis technology to the management system in the e-commerce field to solve such problems.
Disclosure of Invention
In view of the above disadvantages of the prior art, the present invention provides an e-commerce management system based on big data analysis, which provides the most intuitive channel for merchants to know consumers, and the past advertisers may not be able to comprehensively obtain data generated by the e-commerce platform, nor be able to comprehensively know which users purchased the brand and characteristic information of the users. The E-commerce management system with the big data analysis function disclosed by the invention performs fusion processing on data in E-commerce, and the data is exposed to an advertiser for use in a multi-dimensional label form in consideration of the problem of user information safety, so that the information safety of consumers can be protected, and a large amount of data can be utilized to generate value. The invention aims to provide an e-commerce management system with a big data analysis function, which is used for solving the problems in the prior art.
In order to achieve the above and other related objects, the present invention provides an e-commerce management system with big data analysis function, comprising: the system comprises a data preprocessing module, a data service module, a brand insights module, a crowd packet management module, a model management module and an operation management module;
the data preprocessing module completes extraction of data from a data source at a data preprocessing part, cleans the data and stores the data into Hive, and improves the efficiency of extracting data at a business layer;
the data service module is a data processing method with a bottom layer abstracted by data service, generates a tool class for upper-layer services to use, can be collectively called as a data service tool, and performs uniform management on data, and the abstracted tool interface comprises file format conversion, file encryption and decryption, file circulation, file segmentation and file management;
the brand insights module comprises a 4A asset part and a crowd image part, wherein the 4A asset part comprises a 4A asset overview, a 4A full link and a 4A circulation analysis, a brand can be continuously tracked through the function of the 4A asset, and the crowd image part has rich functions and portrait labels, so that crowd data can be more accurately displayed, wherein dimensions comprise user basic attributes, user consumption habits and industry preferences;
the crowd pack management module comprises a crowd mining function, a data fusion function and a crowd application function, wherein a user carries out crowd data mining through the crowd mining function, integrates other data source data through the data fusion function, and realizes advertisement putting and crowd expansion through the crowd application function;
the model management module mainly realizes the functions of rapid data modeling, model training, model evaluation and estimation of advertisers and third-party data service providers, and in the module, a user can simplify the process by clicking, selecting and other operations by means of non-codes;
the operation management module is mainly used for managing the relation among the users and controlling the authority, and in the charging management, the operation management module is mainly used for counting and charging the data consumption and the machine resource consumption generated by data mining, model training, model evaluation and model estimation tasks and obtaining related data.
Optionally, the data preprocessing module cleans data stored in the Hive in a data conversion stage according to business process requirements, a user data conversion process is executed in a script form, the processed data generates a new data table to be stored, a user basic consumption information data table and a user multi-store consumption information data table are respectively generated, the user basic consumption information data table and the user multi-store consumption information data table are final results of the data conversion stage, data conversion provides a data base, in the data loading process, for example, a user selects a tag to perform crowd mining, the data table is queried and the results are packaged according to mining conditions transmitted from a front end, the results and related information of people are stored in a crowd mining table in a formatted JSON form so as to be displayed by foreground data, wherein the information conversion includes user basic consumption information conversion and user multi-store consumption information conversion.
Optionally, the data service module includes a core class sub-module, a flow sub-module and an interface sub-module; wherein the core sub-module comprises: the TransferManager class and the FileManager class; the transferManager class is a main function class for realizing file transfer, and a multi-platform file uploading and downloading method and a file inter-platform transfer method are packaged in the class; the FileManager class realizes the functions of encrypting and decrypting the file, segmenting the file, converting the file format and managing the file; the flow sub-module is the main function of the bottom data service tool, it is to operate the data file, the realization of the function is exposed to the upper business to use by the form of defining the interface, when the user uses the data analysis and management system to initiate some requests, the front end will obtain the account name of the user, operate the main body, the front end requests the file to register the interface, and obtain the file information, judge whether the file size accords with uploading the form requirement and size requirement, if not, prompt the user to upload the data form or file size abnormity, if meet the condition, will count the data volume of the file, obtain the file attribute and file name, and call the encryption algorithm to encrypt the file, upload the data to the market finally and generate the only file label to register the file, and store the file information in the database; the interface sub-module is realized by the function provided by the bottom data service in the form of an interface to the upper layer, the access paths for file format conversion of Hive and HDFS are/create/HDFS/Hive/task and/create/Hive/HDFS/task respectively, the references for file format conversion of HDFS to Hive are respectively a file path, a set Hive table name, a callback address and a header to be added, the references for file format conversion of HDFS to Hive are a Hive table name, a url to be stored in HDFS, a callback address, a list name and a selection condition.
Optionally, the brand insights module packages a summery info class, a LinkFlowSummary class and a contact distribution class, the summery info class implements information gathering, updating and querying functions of a 4A asset overview part, the querying methods are two, the query method implements information query according to brands and categories, the query method implements information query only through categories, the LinkFlowSummary class implements a link circulation function, queries the circulation number according to an incoming link circulation state and a 4A state, the queries are divided into queries with brands and with brands only, the contact distribution class provides information gathering, queries and other functions, the queries are divided into two types as the 4A asset overview, the query method in the summery class is called, queries related Hive tables, and stores query results in a database, the results include a total consumer amount, a day ring ratio, a week ring ratio, a latent customer/customer ratio, a perceived day ring ratio, a week ring ratio, a perceived day ring ratio, a purchase ring ratio, a trend of a day ring ratio, and a trend chart, and the query is implemented according to a trend chart.
Optionally, the crowd management module includes: the crowd mining method comprises three parts, namely crowd mining, data fusion and crowd application, wherein a user in the crowd mining function can set mining rules according to selected conditions, mining conditions can be set to be an intersection or a union, a target crowd can be obtained by the user through a crowd mining module, in addition, the number of the crowd in a crowd packet needs to be counted in the crowd mining process, and charging is carried out according to the mined data volume; the data fusion part mainly realizes the fusion of the data uploaded by the user and the internal data in the system, and the fusion can be carried out on the relevant information of the user according to the unique identification through a user account and a user mobile phone number; the crowd application part mainly realizes the functions of crowd expansion and crowd package delivery, the crowd expansion function mainly realizes the expansion of similar crowds based on the existing crowd package, the crowd delivery function is in butt joint with a plurality of delivery platforms, and the analyzed crowd package is pushed to the delivery platforms to be delivered with advertisements.
Optionally, the model management module is divided into three sub-function modules, which are respectively a ConfigureServer (CS), an Execute Server (ES), and a Monitor Server (MS); configuring a service in a CS (client side), receiving configuration parameters transmitted by a front end, verifying the parameters, writing executable tasks into a task queue, executing the tasks such as model training and the like by calling a relevant machine learning algorithm and allocating machine resources by the service in an ES (execution service), and monitoring the executed tasks in real time and updating the state in an MS (Mobile station) monitoring service; the executive service and the monitoring service can regularly visit the task queue to complete respective work, in addition, regular requests are realized for each service by regularly visiting a Supervisor (SV) module, three main function modules and the SV in the system have independent version numbers and are respectively provided for different service lines in a company for use, and the version numbers are composed of a main version number and a minor version number and are generally embodied in a configuration file in a form of a [ main version number ]. A [ minor version number ].
Optionally, the operation management module comprises two functions of account management and charging management, in the account management function, the information query method returns by acquiring user account information, the information modification method firstly performs non-null length check on the access parameter, updates the relevant information of the user table through the user account, verifies roles and authorities of user login and group administrator, acquires sub-accounts and ISV authorization conditions according to query conditions, queries all ISVs with an authorization book and effective time, verifies roles and authorities of the user login and the group administrator, verifies whether the authorization book and the authorization book time range exist, and verifies whether the authorization book does not exist or authorization in audit exists; the charging management part mainly counts two major tasks, one is a model task, when a user counts the number of data consumed by the task and consumed machine tables in the processes of model training, model evaluation and model estimation, the other task is a big data task which comprises crowd mining and crowd expansion, the class level of crowd generated by the main statistical task is counted, data in the data service module is uniformly registered and managed for counting, all statistical information is written into a charging information correlation table, a timing task scans once every ten minutes to write generated charges into a queue, and the data in the queue waits for counting and summarizing into daily and monthly bills.
Optionally, the data preprocessing module, the data service module, the brand insight module, the crowd management module, the model management module, and the operation management module are arranged in parallel and share a thread space on the operating system.
As described above, the present invention provides an e-commerce management system with big data analysis function, which has the following beneficial effects: the system can provide fusion of diversified data and flexible access to data provided by a third party, provides good prerequisite conditions for data analysis at the later stage, and the diversified and comprehensive high-quality data can help an advertiser to perform more accurate analysis. Secondly, in order to cope with the continuous business development and market development of the e-commerce industry, the system labels the user attributes, purchasing behaviors, advertising behaviors and searching behaviors of consumers, and creates a flexible label market. The label of multidimensionization can help the advertiser more accurate delineation crowd to this data expands and user draws operations such as new, thereby brings better advertisement putting effect. In addition, the system can realize user portrait and consumer life cycle statistics, and display results on a page in a chart form, so that an advertiser can know consumers more conveniently through visualized data. The crowd mining function is realized, data are analyzed by combining a machine learning algorithm, differential pushing of advertisements can be achieved, and thousands of people and thousands of faces in marketing strategies are realized. The big data analysis and management system provides decision basis and decision capability for accurate marketing and accurate putting of the advertising industry, can accurately and directly hit pain spots, improves the activity and retention of consumers, and achieves the purpose of utilizing data to drive decision.
Drawings
Fig. 1 is a schematic structural diagram of an e-commerce management system with big data analysis function according to an embodiment.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
Referring to fig. 1, the present invention provides an e-commerce management system with big data analysis function, which includes: the system comprises a data preprocessing module, a data service module, a brand insight module, a crowd pack management module, a model management module and an operation management module;
the data preprocessing module finishes the extraction of data from a data source in a data preprocessing part, the data is cleaned and stored in Hive, the efficiency of extracting the data in a business layer is improved, in the extracting process, the data used by a project is internal data of a company, the bottom data comprises user browsing information, user purchase adding information, consumption frequency and a main consumption channel, the internal data is provided by other departments, the data required to be displayed in the business layer is extracted from a large amount of data according to business requirements in the extracting stage, irrelevant data is filtered, in the converting process, the data is integrated according to a business module or a theme to form a theme table closely related to the business for the use of upper-layer business, the theme table comprises a user full-platform consumption information table, a user multi-shop consumption information table and a user browsing brand shop table, and in the data loading process, the data generated in the converting process is stored in MySQL according to the business request type for the display and use;
the data service module is a data processing method with a bottom layer abstracted by data service, generates tool classes for upper-layer services to use, and can be collectively called as data service tools to uniformly manage data, and an abstracted tool interface comprises file format conversion, file encryption and decryption, file circulation, file segmentation and file management;
the brand insights module comprises a 4A asset part and a crowd image part, wherein the 4A asset part comprises a 4A asset overview, a 4A full link and a 4A circulation analysis, a brand can be continuously tracked through the function of the 4A asset, and the crowd image part has rich functions and portrait labels, so that crowd data can be more accurately displayed, wherein dimensions comprise user basic attributes, user consumption habits and industry preferences;
the crowd pack management module comprises a crowd mining function, a data fusion function and a crowd application function, wherein a user carries out crowd data mining through the crowd mining function, integrates other data source data through the data fusion function, and realizes advertisement putting and crowd expansion through the crowd application function;
the model management module mainly realizes the functions of rapid data modeling, model training, model evaluation and estimation of advertisers and third-party data service providers, and in the module, a user can simplify the process by clicking, selecting and other operations by means of non-codes;
the operation management module is mainly used for managing the relation among users and controlling the authority, and in the charging management, the operation management module is mainly used for counting and charging the data consumption and the machine resource consumption generated by data mining, model training, model evaluation and model estimation tasks and obtaining related data.
In an exemplary embodiment, the data preprocessing module cleans data stored in the Hive in a data conversion stage according to business process requirements, a user data conversion process is executed in a script form, the processed data generates a new data table to be stored, a user basic consumption information data table and a user multi-store consumption information data table are respectively generated, the user basic consumption information data table and the user multi-store consumption information data table are final results of the data conversion stage, data conversion provides a data base, crowd mining is performed in a data loading process, for example, a user selects a tag, the data table is queried and packaged according to mining conditions transmitted by a front end, the results and related information of people are stored in the crowd mining table in a formatted JSON form so as to be displayed by foreground data, and the information conversion includes user basic consumption information conversion and user multi-store consumption information conversion. The data service module comprises a core class submodule, a flow submodule and an interface submodule; wherein the core sub-module comprises: the TransferManager class and the FileManager class; the transferManager class is a main function class for realizing file transfer, and a multi-platform file uploading and downloading method and a file inter-platform transfer method are packaged in the class; the FileManager class realizes the functions of encrypting and decrypting the file, segmenting the file, converting the file format and managing the file; the flow sub-module is the main function of the bottom data service tool, it is to operate the data file, the realization of the function is exposed to the upper business to use by the form of defining the interface, when the user uses the data analysis and management system to initiate some requests, the front end will obtain the account name of the user, operate the main body, the front end requests the file to register the interface, and obtain the file information, judge whether the file size accords with uploading the form requirement and size requirement, if not, prompt the user to upload the data form or file size abnormity, if meet the condition, will count the data volume of the file, obtain the file attribute and file name, and call the encryption algorithm to encrypt the file, upload the data to the market finally and generate the only file label to register the file, and store the file information in the database; the interface sub-module is realized by the function provided by the bottom data service in the form of an interface to the upper layer, the access paths for file format conversion of Hive and HDFS are/create/HDFS/Hive/task and/create/Hive/HDFS/task respectively, the references for file format conversion of HDFS to Hive are respectively a file path, a set Hive table name, a callback address and a header to be added, the references for file format conversion of HDFS to Hive are a Hive table name, a url to be stored in HDFS, a callback address, a list name and a selection condition. The brand insights module packages summaryInfo, linkFlowSummary and contact distribution, the summaryInfo realizes the information gathering, updating and inquiring functions of the 4A asset overview part, the inquiring methods are two, wherein the query method realizes the information inquiry according to the brand and the category, the query method realizes the information inquiry only through the category, the LinkFlowSummary realizes the link circulation function, the circulation number is inquired according to the incoming link circulation state and the 4A state, the inquiry is divided into the inquiry with the brand category and the inquiry with the brand only, the contact distribution type provides functions of information summarization, query and the like, the query is divided into two types as the 4A asset overview, a query method in the Summarylnfo type is called, a relevant Hive table is queried, query results are stored in a database, the results comprise the total consumer amount, the date-to-ring ratio, the week-to-ring ratio, the hidden customer/customer ratio, the deepening rate of the relation week, the perception date-to-ring ratio, the attraction date-to-ring ratio, the purchase date-to-ring ratio and the revere date-to-ring ratio, the query function of a trend graph is realized, and the query according to the 4A state and the query through indexes are respectively realized. The crowd pack management module comprises: the crowd mining method comprises three parts, namely crowd mining, data fusion and crowd application, wherein a user in the crowd mining function can set mining rules according to selected conditions, mining conditions can be set to be intersection or union, the user can obtain target crowds through a crowd mining module, in addition, the number of crowds in a crowd packet needs to be counted in the crowd mining process, and charging is carried out according to the mined data volume; the data fusion part mainly realizes the fusion of the data uploaded by the user and the internal data in the system, and the fusion can be carried out on the relevant information of the user according to the unique identification through the user account and the user mobile phone number; the crowd application part mainly achieves the functions of crowd expansion and crowd package delivery, the crowd expansion function mainly achieves the purpose of expanding similar crowds based on the existing crowd package, the crowd delivery function is in butt joint with a plurality of delivery platforms, and the analyzed crowd package is pushed to the delivery platforms to be delivered with advertisements. The model management module is divided into three sub-function modules, namely a Configuration Server (CS), an Execution Server (ES) and a Monitor Server (MS); the method comprises the steps that configuration parameters transmitted by a front end are received by a configuration service in a CS (client server), the parameters are verified, executable tasks are written into a task queue, the execution service in an ES (execution service) calls a relevant machine learning algorithm and allocates machine resources to execute tasks such as model training and the like, and the executed tasks are monitored in real time and the state is updated in an MS (Mobile station) monitoring service; the executive service and the monitoring service can regularly visit the task queue to complete respective work, in addition, regular requests are realized for each service through regularly visiting a Supervisor (SV) module, three main function modules and SVs in the system have independent version numbers and are respectively provided for different service lines in a company for use, and the version numbers are composed of a main version number and a minor version number and are generally embodied in a configuration file in a form of a [ main version number ]. A [ minor version number ]. The operation management module comprises two functions of account management and charging management, in the account management function, an information query method returns by acquiring user account information, an information modification method firstly performs non-null length check on access participation, updates related information of a user table through a user account, verifies roles and authorities of user login and a group administrator, acquires sub-accounts and ISV authorization conditions according to query conditions, queries all ISVs with an authorization book and effective time, verifies the roles and authorities of the user login and the group administrator, verifies whether the ISVs have an authorization book and an authorization book time range, and verifies whether the authorization book has non-verification or authorization in verification; the charging management part mainly counts two major tasks, one is a model task, when a user counts the number of data consumed by the task and consumed machine tables in the processes of model training, model evaluation and model estimation, the other task is a big data task which comprises crowd mining and crowd expansion, the class level of crowd generated by the main statistical task is counted, data in the data service module is uniformly registered and managed for counting, all statistical information is written into a charging information correlation table, a timing task scans once every ten minutes to write generated charges into a queue, and the data in the queue waits for counting and summarizing into daily and monthly bills. The data preprocessing module, the data service module, the brand insight module, the crowd management module, the model management module and the operation management module are arranged in parallel and share thread space on the operating system.
In summary, the invention provides an e-commerce management system with big data analysis function, which has the following beneficial effects: the system can provide fusion of diversified data and flexible access to data provided by a third party, provides good prerequisite conditions for data analysis at the later stage, and the diversified and comprehensive high-quality data can help an advertiser to perform more accurate analysis. Secondly, in order to cope with the continuous business development and market development of the e-commerce industry, the system labels the user attributes, purchasing behaviors, advertising behaviors and searching behaviors of consumers and creates a flexible label market. The label of multidimensionization can help the advertiser more accurate delineation crowd to this data expands and user draws operations such as new, thereby brings better advertisement putting effect. In addition, the system can realize the user portrait and the consumption life cycle statistics of the consumer, and the result is displayed on the page in a chart form, so that the advertiser can know the consumer more conveniently through the visualized data. The crowd mining function is realized, data are analyzed by combining a machine learning algorithm, differential pushing of advertisements can be achieved, and thousands of people and thousands of faces in marketing strategies are realized. The big data analysis and management system provides decision basis and decision capability for accurate marketing and accurate putting of the advertising industry, can accurately and directly hit pain spots, improves the activity and retention of consumers, and achieves the purpose of utilizing data to drive decision.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (8)
1. An E-commerce management system with big data analysis function comprises: the system comprises a data preprocessing module, a data service module, a brand insights module, a crowd packet management module, a model management module and an operation management module;
the data preprocessing module finishes the extraction of data from a data source in a data preprocessing part, the data is cleaned and stored in Hive, the efficiency of extracting the data in a business layer is improved, in the extracting process, the data used by a project is internal data of a company, the bottom data comprises user browsing information, user purchase adding information, consumption frequency and a main consumption channel, the internal data is provided by other departments, the data required to be displayed in the business layer is extracted from a large amount of data according to business requirements in the extracting stage, irrelevant data is filtered, in the converting process, the data is integrated according to a business module or a theme to form a theme table closely related to the business for the use of upper-layer business, the theme table comprises a user full-platform consumption information table, a user multi-shop consumption information table and a user browsing brand shop table, and in the data loading process, the data generated in the converting process is stored in MySQL according to the business request type for the display and use;
the data service module is a data processing method with a bottom layer abstracted by data service, generates a tool class for upper-layer services to use, can be collectively called as a data service tool, and performs uniform management on data, and the abstracted tool interface comprises file format conversion, file encryption and decryption, file circulation, file segmentation and file management;
the brand insights module comprises a 4A asset part and a crowd image part, wherein the 4A asset part comprises a 4A asset overview, a 4A full link and 4A circulation analysis, brands can be continuously tracked through the functions of the 4A assets, and the crowd image part has rich functions of portrait labels and can more accurately display crowd data, wherein dimensions comprise user basic attributes, user consumption habits and industry preferences;
the crowd pack management module comprises a crowd mining function, a data fusion function and a crowd application function, a user carries out crowd data mining through the crowd mining function, integrates other data source data through the data fusion function, and realizes advertisement putting and crowd expansion through the crowd application function;
the model management module mainly realizes the functions of rapid data modeling, model training, model evaluation and estimation of advertisers and third-party data service providers, and in the module, a user can simplify the process by clicking, selecting and other operations by means of non-codes;
the operation management module is mainly used for managing the relation among users and controlling the authority, and in the charging management, the operation management module is mainly used for counting and charging the data consumption and the machine resource consumption generated by data mining, model training, model evaluation and model estimation tasks and obtaining related data.
2. The e-commerce management system with big data analysis function as claimed in claim 1, wherein the data preprocessing module cleans the data stored in the Hive in a data conversion stage according to business process requirements, the user data conversion process is executed in a script form, the processed data generates a new data table for storage, a user basic consumption information data table and a user multi-store consumption information data table are generated respectively, the user basic consumption information data table and the user multi-store consumption data table are final results of the data conversion stage, data conversion provides a data base for data loading, in the data loading process, for example, a user selects a tag for crowd mining, the data table is queried and the results are packaged according to mining conditions transmitted from a front end, the results and related information of people are stored in the crowd mining table in a formatted JSON form for foreground data presentation, and the information conversion includes user basic consumption information conversion and user multi-store consumption information conversion.
3. The e-commerce management system with the big data analysis function of claim 1, wherein the data service module comprises a core class sub-module, a process sub-module and an interface sub-module; wherein the core sub-module comprises: the TransferManager class and the FileManager class; the transferManager class is a main function class for realizing file transfer, and a multi-platform file uploading and downloading method and a file inter-platform transfer method are packaged in the class; the FileManager realizes the functions of encryption and decryption, file segmentation, file format conversion and file management of the file; the flow sub-module is the main function of the bottom data service tool, it is to operate the data file, the realization of the function is exposed to the upper business to use by the form of defining the interface, when the user uses the data analysis and management system to initiate some requests, the front end will obtain the account name of the user, operate the main body, the front end requests the file to register the interface, and obtain the file information, judge whether the file size accords with uploading the form requirement and size requirement, if not, prompt the user to upload the data form or file size abnormity, if meet the condition, will count the data volume of the file, obtain the file attribute and file name, and call the encryption algorithm to encrypt the file, upload the data to the market finally and generate the only file label to register the file, and store the file information in the database; the interface sub-module is realized by the function provided by the bottom data service in the form of an interface to the upper layer, the access paths for file format conversion of Hive and HDFS are/create/HDFS/Hive/task and/create/Hive/HDFS/task respectively, the references for file format conversion of HDFS to Hive are respectively a file path, a set Hive table name, a callback address and a header to be added, the references for file format conversion of HDFS to Hive are a Hive table name, a url to be stored in HDFS, a callback address, a list name and a selection condition.
4. The e-commerce management system with big data analysis function of claim 1, wherein the brand insight module encapsulates SummaryInfo class, linkFlowSummary class and contact distribution class, the SummaryInfo class realizes the information gathering, updating and inquiring functions of 4A asset overview part, and there are two inquiring methods, wherein the inquiring method realizes the information inquiry according to brand and category, the inquiring method realizes the information inquiry only through category, the LinkFlowSummary class realizes the link circulation function, inquires the number of people who circulate according to the incoming link circulation state and 4A state, the query is divided into a query with a brand category and a query with only a brand, a contact distribution type provides functions of information summarization, query and the like, the query is divided into two types as same as a 4A asset overview, a query method in a Summarylnfo type is called, a related Hive table is queried, query results are stored in a database, the results comprise a total consumer amount, a day-to-ring ratio, a week-to-ring ratio, a hidden customer/customer ratio, a relationship week deepening rate, a perception day-to-ring ratio, an attraction day-to-ring ratio, a purchase day-to-ring ratio and a defending day-to-ring ratio, the query function of a trend graph is realized, and the query according to a 4A state and the query through indexes are respectively realized.
5. The e-commerce management system with big data analysis function of claim 1, wherein the crowd management module comprises: the crowd mining method comprises three parts, namely crowd mining, data fusion and crowd application, wherein a user in the crowd mining function can set mining rules according to selected conditions, mining conditions can be set to be an intersection or a union, a target crowd can be obtained by the user through a crowd mining module, in addition, the number of the crowd in a crowd packet needs to be counted in the crowd mining process, and charging is carried out according to the mined data volume; the data fusion part mainly realizes the fusion of the data uploaded by the user and the internal data in the system, and the fusion can be carried out on the relevant information of the user according to the unique identification through a user account and a user mobile phone number; the crowd application part mainly achieves the functions of crowd expansion and crowd package delivery, the crowd expansion function mainly achieves the purpose of expanding similar crowds based on the existing crowd package, the crowd delivery function is in butt joint with a plurality of delivery platforms, and the analyzed crowd package is pushed to the delivery platforms to be delivered with advertisements.
6. The e-commerce management system with big data analysis function of claim 1, wherein the model management module is divided into three sub-function modules, which are respectively a Configuration Server (CS), an Execution Server (ES), and a Monitor Server (MS); the method comprises the steps that configuration parameters transmitted by a front end are received by a configuration service in a CS (client server), the parameters are verified, executable tasks are written into a task queue, the execution service in an ES (execution service) calls a relevant machine learning algorithm and allocates machine resources to execute tasks such as model training and the like, and the executed tasks are monitored in real time and the state is updated in an MS (Mobile station) monitoring service; the executive service and the monitoring service can regularly visit the task queue to complete respective work, in addition, regular requests are realized for each service by regularly visiting a Supervisor (SV) module, three main function modules and the SV in the system have independent version numbers and are respectively provided for different service lines in a company for use, and the version numbers are composed of a main version number and a minor version number and are generally embodied in a configuration file in a form of a [ main version number ]. A [ minor version number ].
7. The e-commerce management system with the big data analysis function of claim 1 is characterized in that the operation management module comprises two functions of account management and charging management, in the account management function, the information query method returns by acquiring user account information, the information modification method firstly performs non-null sum length check on the access participant, updates related information of a user table through a user account, checks roles and permissions of a user login and a group administrator, acquires sub-accounts and ISV authorization conditions according to query conditions, queries all ISVs with authorization and effective time, checks roles and permissions of the user login and the group administrator, checks whether an authorization book and an authorization book time range exist, and checks whether the authorization book exists or whether authorization in auditing exists; the charging management part mainly counts two major tasks, one is a model task, when a user counts the number of data consumed by the task and consumed machine tables in the processes of model training, model evaluation and model estimation, the other task is a big data task which comprises crowd mining and crowd expansion, the class level of crowd generated by the main statistical task is counted, data in the data service module is uniformly registered and managed for counting, all statistical information is written into a charging information correlation table, a timing task scans once every ten minutes to write generated charges into a queue, and the data in the queue waits for counting and summarizing into daily and monthly bills.
8. The e-commerce management system with big data analysis function of claim 1, wherein the data preprocessing module, the data service module, the brand insights module, the crowd management module, the model management module and the operation management module are collocated and share thread space on the operating system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211354072.7A CN115564300A (en) | 2022-10-30 | 2022-10-30 | E-commerce management system with big data analysis function |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211354072.7A CN115564300A (en) | 2022-10-30 | 2022-10-30 | E-commerce management system with big data analysis function |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115564300A true CN115564300A (en) | 2023-01-03 |
Family
ID=84768267
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211354072.7A Withdrawn CN115564300A (en) | 2022-10-30 | 2022-10-30 | E-commerce management system with big data analysis function |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115564300A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116451056A (en) * | 2023-06-13 | 2023-07-18 | 支付宝(杭州)信息技术有限公司 | Terminal feature insight method, device and equipment |
| CN116797266A (en) * | 2023-08-22 | 2023-09-22 | 深圳市百慧文化发展有限公司 | Ticketing system and account management method thereof |
-
2022
- 2022-10-30 CN CN202211354072.7A patent/CN115564300A/en not_active Withdrawn
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116451056A (en) * | 2023-06-13 | 2023-07-18 | 支付宝(杭州)信息技术有限公司 | Terminal feature insight method, device and equipment |
| CN116451056B (en) * | 2023-06-13 | 2023-09-29 | 支付宝(杭州)信息技术有限公司 | Terminal feature insight method, device and equipment |
| CN116797266A (en) * | 2023-08-22 | 2023-09-22 | 深圳市百慧文化发展有限公司 | Ticketing system and account management method thereof |
| CN116797266B (en) * | 2023-08-22 | 2023-11-21 | 深圳市百慧文化发展有限公司 | Ticketing system and account management method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4981146B2 (en) | Cross-channel optimization system and method | |
| CN115564300A (en) | E-commerce management system with big data analysis function | |
| US8819121B2 (en) | Unaffiliated web domain hosting service based on service pools with flexible resource | |
| CN101937545B (en) | Touchpoint customization system | |
| JP6362602B2 (en) | System and method for processing digital traffic metrics | |
| US20090319365A1 (en) | System and method for assessing marketing data | |
| US10373194B2 (en) | System and method for measuring advertising effectiveness | |
| US20020128910A1 (en) | Business supporting system and business supporting method | |
| US20070233565A1 (en) | Online Advertising System and Method | |
| US20060143086A1 (en) | System and method for tracking web campaign effectiveness | |
| US11494800B2 (en) | System and method for capability packages offering based on analysis of edited websites and their use | |
| US9111292B2 (en) | Dynamic messaging in a personal data protection suite | |
| CN101730890A (en) | Advertisement service system, equipment and wherein employed method | |
| EP3091494A1 (en) | Systems and methods for targeted content presentation based on search query analysis | |
| US10769711B2 (en) | User task focus and guidance for recurring revenue asset management | |
| Ayyagari | A framework for analytical CRM assessments challenges and recommendations | |
| US20090271272A1 (en) | Electronic Marketing System | |
| US10657558B1 (en) | System and method for using a plurality of different data sources to control displayed content | |
| Burke et al. | Rethinking marketing research in the digital world | |
| Farruh | Consumer life cycle and profiling: A data mining perspective | |
| CN114663105A (en) | Enterprise customer relationship management system and method | |
| Liao et al. | Mining customer knowledge for channel and product segmentation | |
| Karunanithy et al. | An evolution of customer relationship management: a conceptual approach | |
| WO2009097625A1 (en) | Electronic marketing system | |
| Mishra et al. | CRM system implementation in a multinational enterprise |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20230103 |
|
| WW01 | Invention patent application withdrawn after publication |