CN109871380B - Crowd pack application method and system based on Redis - Google Patents
Crowd pack application method and system based on Redis Download PDFInfo
- Publication number
- CN109871380B CN109871380B CN201910031226.0A CN201910031226A CN109871380B CN 109871380 B CN109871380 B CN 109871380B CN 201910031226 A CN201910031226 A CN 201910031226A CN 109871380 B CN109871380 B CN 109871380B
- Authority
- CN
- China
- Prior art keywords
- crowd
- packet
- redis
- platform
- pack
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Mobile Radio Communication Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a crowd packet application method and a crowd packet application system based on Redis.
Description
Technical Field
The invention relates to the technical field of mobile internet, in particular to a crowd packet application method and system based on Redis.
Background
The DMP (Data Management Platform) Data Management Platform is a Platform that integrates and incorporates dispersed multi-party Data into a unified technical Platform, standardizes and subdivides the Data, and allows users to push the subdivided results to the existing interactive marketing environment.
The DMP has the functions of quickly inquiring, feeding back and quickly presenting results, helping guests enter a market period, promoting cooperation between enterprise users and partners, deeply predicting, analyzing and doing, bringing competitive advantages in all aspects, and reducing information acquisition and labor cost.
The crowd packet is a set of equipment IDs of a batch of crowd identifications obtained by performing analysis and calculation based on a keyword or a relevancy algorithm on the basis of accurate orientation as a DMP.
In order to realize quick query and feedback, the crowd bag needs to be stored through structured data, and most of the current storage modes are relational data, such as Mysql, sqlserver, oracle and the like. The relational data storage has the advantages of simple storage structure, support for a large amount of data, storage to a disk, and simple updating, deletion and maintenance, and has the defect that the query speed is often not enough to support a large amount of concurrency.
Disclosure of Invention
The present invention aims to provide a crowd packet application method and system based on Redis to solve the problems proposed in the background art.
In order to achieve the purpose, the invention provides the following technical scheme:
a crowd pack application method based on Redis comprises the following steps:
step 01: the user data storage system processes the data logs through a large amount of data log accumulation, performs the lookelike calculation, and the crowd packet calculation platform pulls the log to calculate the crowd packet;
step 02: after the crowd packet computing platform completes the computation, the crowd packet computing platform informs the crowd packet management service platform, and the crowd packet management service platform performs preliminary inspection and file verification on data;
step 03: the crowd packet management service platform synchronizes the corresponding import, update and deletion requirements to a crowd packet import module, a crowd packet update module and a crowd packet deletion module in the crowd packet management service platform;
step 04: the crowd packet importing module, the crowd packet updating module and the crowd packet deleting module preprocess the crowd packet file data;
step 05: storing the crowd packet into a Redis cluster, using a hash structure, and configuring a compression parameter of the hash, specifically comprising the following steps:
step 51: the advertisement demand side platform DSP or the advertisement transaction platform ADX reads the advertisement request parameter or the user equipment number, the first 6 or 10 bits are taken as keys according to the rule, and the rest are taken as fields;
step 52: reading a list of crowd packet IDs corresponding to the equipment numbers from the hash in the Redis cluster;
step 53: and processing the related service of the crowd packet on the basis of the obtained crowd packet ID list.
A Redis-based crowd pack application system, comprising: the system comprises a user data storage system, a crowd packet computing platform, a crowd packet management service platform and a Redis cluster, wherein the user data storage system is connected with the crowd packet computing platform and used for accumulating data logs and carrying out lookelike computation to generate crowd packet files and crowd packet IDs; crowd wraps computing platform and crowd and wraps management service platform and be connected, crowd wraps management service platform and Redis cluster connection, and crowd wraps the pipeline service platform and includes: the crowd packet updating module is used for updating the crowd packet and deleting the crowd packet.
As a further scheme of the invention: the data log at least comprises a user equipment number and a keyword.
As a further scheme of the invention: in step 01, the data log processing method comprises the following steps: mapping between the user equipment number and the corresponding keywords, then binding a large number of keywords to the user equipment number, and generating a crowd package file and a corresponding crowd package ID.
As a further scheme of the invention: the pretreatment method in the step 04 comprises the following steps: the first 6 or 10 bits are taken as keys according to the rule, the rest are taken as fields, the value is the crowd packet ID, and a plurality of crowd IDs are separated by commas.
As a further scheme of the invention: the crowd package application system based on Redis further comprises a crowd package using party, the crowd package using platform is connected with the Redis cluster, and the crowd package using platform is an advertisement demand party platform DSP or an advertisement transaction party platform ADX.
Compared with the prior art, the invention has the beneficial effects that:
the invention relates to a crowd packet application method and a system based on Redis, which can map a group or a batch of keywords to crowd packet ID through the calculation of the keywords in the stage of forming the crowd packet, save a large amount of memory through the storage of Redis and the optimization of a hash data structure, save a large amount of equipment ID storage, and provide the functions of high concurrency, transverse expansion and automatic disaster tolerance at the same time, and have the following advantages:
(1) In the aspect of an advertiser, the crowd bag function can be used more flexibly, and comprises a plurality of keywords, the combination, exclusion, inclusion, orientation and the like of a plurality of crowd bags, so that more flexible and accurate advertisement orientation is provided for the advertiser, and the advertisement cost is saved;
(2) In the aspect of service performance of the crowd pack, as the used memory database Redis is optimized and stored based on the hash, a large amount of memory is saved, and simultaneously, a high-availability and high-concurrency solution is provided;
(3) The storage of a memory database Redis is used, the performance and the expandability of the Redis are fully utilized, and a more flexible solution for data synchronization and transverse expansion is provided;
(4) The high availability and capacity expansion scheme of Redis can facilitate disaster recovery of operation and maintenance and service faults and upgrade of servers.
Drawings
Fig. 1 is a flowchart of a method of the crowd sourcing application method based on Redis according to the present invention.
Fig. 2 is a flowchart of a method for configuring hash compression parameters in a crowd packet application method based on Redis according to the present invention.
Fig. 3 is a schematic block diagram of a structure of a people group application system based on Redis according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
Referring to fig. 1 and 2, in an embodiment of the present invention, a crowd sourcing application method based on Redis includes the following steps:
step 01: the user data storage system 100 is used for calculating a crowd package by accumulating a large amount of data logs, wherein the data logs comprise information such as user equipment numbers, keywords and the like, mapping between the user equipment numbers and corresponding keywords, and performing lookelike calculation, then binding a large amount of keywords to the user equipment numbers, generating crowd package files and corresponding crowd package IDs, and pulling the log by the crowd package calculation platform 200 to calculate the crowd package;
step 02: after the crowd packet computing platform 200 completes computing, the crowd packet management service platform 300 is informed, and the crowd packet management service platform 300 performs preliminary inspection and file verification on data;
step 03: the crowd packet management service platform 300 synchronizes the corresponding import, update and deletion requirements to a crowd packet import module 301, a crowd packet update module 302 and a crowd packet deletion module 303 in the crowd packet management service platform 300;
step 04: the crowd packet importing module 301, the crowd packet updating module 302 and the crowd packet deleting module 303 preprocess the crowd packet file data, and the preprocessing method comprises the following steps: according to the rule, the first 6 bits or 10 bits are taken as keys, the rest are taken as fields, the values are crowd packet IDs, and a plurality of crowd IDs are separated by commas;
step 05: storing the crowd packet into the Redis cluster 400, using a hash structure, and configuring a compression parameter of the hash, specifically comprising the following steps:
step 51: reading an advertisement request parameter or a user equipment number by an advertisement demander platform DSP or an advertisement transaction platform ADX, taking the first 6 or 10 bits as keys according to a rule, and taking the rest as field;
step 52: reading a list of crowd packet IDs corresponding to device numbers from a hash in the Redis cluster 400;
step 53: and processing the related services of the crowd packet on the basis of the obtained crowd packet ID list, such as: the method stores different equipment numbers into a hash structure in a certain equal (hash fragmentation) proportion mode as much as possible, and prevents the problem of memory explosion after the hash response data volume is too large.
Referring to fig. 2, a system for applying a Redis-based crowd pack, includes: the system comprises a user data storage system 100, a crowd package computing platform 200, a crowd package management service platform 300, a Redis cluster 400 and a crowd package user 500, wherein the user data storage system 100 is connected with the crowd package computing platform 200 and used for accumulating data logs and carrying out lookelike computing to generate crowd package files and crowd package IDs; crowd package computing platform 200 is connected with crowd package management service platform 300, crowd package management service platform 300 is connected with Redis cluster 400, and crowd package pipeline service platform 300 includes: crowd package import module 301, crowd package update module 302 and crowd package delete module 303, crowd package service platform 500 is connected with Redis cluster 400, and crowd package service platform 500 is advertisement demand side platform DSP or advertisement transaction side platform ADX.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (6)
1. A crowd packet application method based on Redis is characterized in that: the method comprises the following steps:
step 01: the user data storage system (100) processes the data logs through a large amount of data log accumulation, performs lookup alike calculation, and the crowd packet calculation platform (200) pulls the logs to calculate the crowd packet;
step 02: after the crowd packet computing platform (200) completes computing, the crowd packet management service platform (300) is informed, and the crowd packet management service platform (300) performs preliminary inspection and file verification on data;
step 03: the crowd pack management service platform (300) synchronizes the corresponding import, update and deletion requirements to a crowd pack import module (301), a crowd pack update module (302) and a crowd pack deletion module (303) in the crowd pack management service platform (300);
step 04: the crowd packet importing module (301), the crowd packet updating module (302) and the crowd packet deleting module (303) preprocess the crowd packet file data;
step 05: storing the crowd packet into a Redis cluster (400), using a hash structure and configuring the compression parameters of the hash, and specifically comprising the following steps:
step 51: the advertisement demand side platform DSP or the advertisement transaction platform ADX reads the advertisement request parameter or the user equipment number, the first 6 or 10 bits are taken as keys according to the rule, and the rest are taken as fields;
step 52: reading a list of crowd packet IDs corresponding to device numbers from a hash in a Redis cluster (400);
step 53: and processing the related service of the crowd packet on the basis of the obtained crowd packet ID list.
2. The Redis-based crowd pack application method according to claim 1, wherein: the data log at least comprises a user equipment number and keywords.
3. The crowd package application method based on Redis of claim 2, wherein: in step 01, the data log processing method comprises the following steps: mapping the user equipment number and the corresponding keywords, then binding a large number of keywords to the user equipment number, and generating a crowd packet file and a corresponding crowd packet ID.
4. The Redis-based crowd pack application method according to claim 1, wherein: the pretreatment method in the step 04 comprises the following steps: the first 6 or 10 bits are taken as keys according to the rule, the rest are field, the value is the crowd packet ID, and a plurality of crowd IDs are separated by commas.
5. A crowd package application system based on Redis, characterized in that: the method comprises the following steps: the system comprises a user data storage system (100), a crowd pack computing platform (200), a crowd pack management service platform (300) and a Redis cluster (400), wherein the user data storage system (100) is connected with the crowd pack computing platform (200) and used for accumulating data logs and carrying out lookelike computing to generate crowd pack files and crowd pack IDs; the crowd package computing platform (200) is connected with a crowd package management service platform (300), the crowd package management service platform (300) is connected with a Redis cluster (400), and the crowd package pipeline service platform (300) comprises: the system comprises a crowd packet importing module (301), a crowd packet updating module (302) and a crowd packet deleting module (303).
6. The Redis-based crowd pack application system according to claim 5, wherein: still include crowd package service platform (500), crowd package service platform (500) is connected with Redis cluster (400), and crowd package service platform (500) is advertisement demand side platform DSP or advertisement transaction side platform ADX.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910031226.0A CN109871380B (en) | 2019-01-14 | 2019-01-14 | Crowd pack application method and system based on Redis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910031226.0A CN109871380B (en) | 2019-01-14 | 2019-01-14 | Crowd pack application method and system based on Redis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109871380A CN109871380A (en) | 2019-06-11 |
| CN109871380B true CN109871380B (en) | 2022-11-11 |
Family
ID=66917623
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910031226.0A Active CN109871380B (en) | 2019-01-14 | 2019-01-14 | Crowd pack application method and system based on Redis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109871380B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113139824A (en) * | 2020-01-20 | 2021-07-20 | 上海哔哩哔哩科技有限公司 | Crowd packet construction method and system based on negative feedback |
| CN111292137A (en) * | 2020-03-10 | 2020-06-16 | 联通沃音乐文化有限公司 | Method for accurately delivering internet advertisements |
| CN112559857A (en) * | 2020-12-10 | 2021-03-26 | 上海明略人工智能(集团)有限公司 | Redis-based crowd pack application method and system, electronic device and storage medium |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20050118386A (en) * | 2004-06-14 | 2005-12-19 | 주식회사 에듀파이 | Method of processing data and digital assistant thereof |
| CN102779308A (en) * | 2012-01-09 | 2012-11-14 | 合一网络技术(北京)有限公司 | Advertisement release method and system |
| CN103324645A (en) * | 2012-03-23 | 2013-09-25 | 腾讯科技（深圳）有限公司 | Method and device for recommending webpage |
| CN104680396A (en) * | 2015-02-13 | 2015-06-03 | 北京集奥聚合科技有限公司 | Method and system for monitoring broadcast of accurate exposure of advertisements in real time |
| WO2016108471A1 (en) * | 2015-01-02 | 2016-07-07 | 에스케이플래닛 주식회사 | Apparatus, method and system for advertisement targeting |
| CN106126673A (en) * | 2016-06-29 | 2016-11-16 | 上海浦东发展银行股份有限公司信用卡中心 | A kind of based on Redis and HBase point of locking method |
| CN106599104A (en) * | 2016-11-29 | 2017-04-26 | 北京锐安科技有限公司 | Mass data association method based on redis cluster |
| CN106919682A (en) * | 2017-03-01 | 2017-07-04 | 北京再塑宝科技有限公司 | A kind of search associational word implementation method based on redis technologies |
| CN107273384A (en) * | 2016-04-08 | 2017-10-20 | 百度在线网络技术（北京）有限公司 | The determination method and apparatus of crowd's attribute |
| CN107479829A (en) * | 2017-08-03 | 2017-12-15 | 杭州铭师堂教育科技发展有限公司 | A kind of Redis cluster mass datas based on message queue quickly clear up system and method |
| CN107861859A (en) * | 2017-11-22 | 2018-03-30 | 北京汇通金财信息科技有限公司 | A kind of blog management method and system based on micro services framework |
| CN107992398A (en) * | 2017-12-22 | 2018-05-04 | 宜人恒业科技发展（北京）有限公司 | The monitoring method and monitoring system of a kind of operation system |
| CN108197966A (en) * | 2017-11-01 | 2018-06-22 | 上海新数网络科技股份有限公司 | A kind of accurate advertisement analysis method based on crowd's label data |
| CN108197289A (en) * | 2018-01-18 | 2018-06-22 | 吉浦斯信息咨询（深圳）有限公司 | A kind of data store organisation, data store query method, terminal and medium |
| CN108648003A (en) * | 2018-04-28 | 2018-10-12 | 北京聚通达科技股份有限公司 | Precision launches advertisement and launches data statistics, effect tracking |
| CN108874773A (en) * | 2018-05-31 | 2018-11-23 | 平安医疗科技有限公司 | Keyword increases method, apparatus, computer equipment and storage medium newly |
| CN109101635A (en) * | 2018-08-16 | 2018-12-28 | 广州小鹏汽车科技有限公司 | A kind of data processing method and device based on Redis Hash structure |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012150602A1 (en) * | 2011-05-03 | 2012-11-08 | Yogesh Chunilal Rathod | A system and method for dynamically monitoring, recording, processing, attaching dynamic, contextual & accessible active links & presenting of physical or digital activities, actions, locations, logs, life stream, behavior & status |
| US9319374B2 (en) * | 2013-08-29 | 2016-04-19 | Intuit Inc. | Personalized bookmarking of textsite applications via a text message |
| US10536541B2 (en) * | 2015-12-18 | 2020-01-14 | Bitly, Inc. | Systems and methods for analyzing traffic across multiple media channels via encoded links |
| US10296538B2 (en) * | 2016-02-23 | 2019-05-21 | Baidu Usa Llc | Method for matching images with content based on representations of keywords associated with the content in response to a search query |
| US10872061B2 (en) * | 2016-12-08 | 2020-12-22 | Exterro, Inc. | Systems and methods for document search and aggregation with reduced bandwidth and storage demand |
-
2019
- 2019-01-14 CN CN201910031226.0A patent/CN109871380B/en active Active
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20050118386A (en) * | 2004-06-14 | 2005-12-19 | 주식회사 에듀파이 | Method of processing data and digital assistant thereof |
| CN102779308A (en) * | 2012-01-09 | 2012-11-14 | 合一网络技术(北京)有限公司 | Advertisement release method and system |
| CN103324645A (en) * | 2012-03-23 | 2013-09-25 | 腾讯科技（深圳）有限公司 | Method and device for recommending webpage |
| WO2016108471A1 (en) * | 2015-01-02 | 2016-07-07 | 에스케이플래닛 주식회사 | Apparatus, method and system for advertisement targeting |
| CN104680396A (en) * | 2015-02-13 | 2015-06-03 | 北京集奥聚合科技有限公司 | Method and system for monitoring broadcast of accurate exposure of advertisements in real time |
| CN107273384A (en) * | 2016-04-08 | 2017-10-20 | 百度在线网络技术（北京）有限公司 | The determination method and apparatus of crowd's attribute |
| CN106126673A (en) * | 2016-06-29 | 2016-11-16 | 上海浦东发展银行股份有限公司信用卡中心 | A kind of based on Redis and HBase point of locking method |
| CN106599104A (en) * | 2016-11-29 | 2017-04-26 | 北京锐安科技有限公司 | Mass data association method based on redis cluster |
| CN106919682A (en) * | 2017-03-01 | 2017-07-04 | 北京再塑宝科技有限公司 | A kind of search associational word implementation method based on redis technologies |
| CN107479829A (en) * | 2017-08-03 | 2017-12-15 | 杭州铭师堂教育科技发展有限公司 | A kind of Redis cluster mass datas based on message queue quickly clear up system and method |
| CN108197966A (en) * | 2017-11-01 | 2018-06-22 | 上海新数网络科技股份有限公司 | A kind of accurate advertisement analysis method based on crowd's label data |
| CN107861859A (en) * | 2017-11-22 | 2018-03-30 | 北京汇通金财信息科技有限公司 | A kind of blog management method and system based on micro services framework |
| CN107992398A (en) * | 2017-12-22 | 2018-05-04 | 宜人恒业科技发展（北京）有限公司 | The monitoring method and monitoring system of a kind of operation system |
| CN108197289A (en) * | 2018-01-18 | 2018-06-22 | 吉浦斯信息咨询（深圳）有限公司 | A kind of data store organisation, data store query method, terminal and medium |
| CN108648003A (en) * | 2018-04-28 | 2018-10-12 | 北京聚通达科技股份有限公司 | Precision launches advertisement and launches data statistics, effect tracking |
| CN108874773A (en) * | 2018-05-31 | 2018-11-23 | 平安医疗科技有限公司 | Keyword increases method, apparatus, computer equipment and storage medium newly |
| CN109101635A (en) * | 2018-08-16 | 2018-12-28 | 广州小鹏汽车科技有限公司 | A kind of data processing method and device based on Redis Hash structure |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109871380A (en) | 2019-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109871380B (en) | Crowd pack application method and system based on Redis | |
| CN102682059B (en) | Method and system for distributing users to clusters | |
| US10169385B2 (en) | Managing replicated data | |
| CN101639831B (en) | Search method, search device and search system | |
| CN105303456A (en) | Method for processing monitoring data of electric power transmission equipment | |
| US20130218837A1 (en) | Cloud data synchronization with local data | |
| CN105956123A (en) | Local updating software-based data processing method and apparatus | |
| CN102193917A (en) | Method and device for processing and querying data | |
| EP2705419A1 (en) | Database system and method | |
| CN102236672A (en) | Method and device for importing data | |
| CN103631924B (en) | A kind of application process and system of distributive database platform | |
| CN105530272A (en) | Method and device for application data synchronization | |
| CN104239377A (en) | Platform-crossing data retrieval method and device | |
| CN103793493A (en) | Method and system for processing car-mounted terminal mass data | |
| CN110858194A (en) | Method and device for expanding database | |
| CN115422169B (en) | Data warehouse construction method and device based on commercial advertisement scene | |
| CN109150964B (en) | Migratable data management method and service migration method | |
| CN110781036A (en) | Data recovery method and device, computer equipment and storage medium | |
| CN106649602A (en) | Way, device and server of processing business object data | |
| CN107169110A (en) | A kind of big data collection method and system based on cloud service | |
| CN115858488A (en) | Parallel migration method and device based on data governance and readable medium | |
| CN107169003B (en) | Data association method and device | |
| CN106920057A (en) | Material management method and system | |
| CN116401271A (en) | Database table query method, computer device and computer storage medium | |
| CN114757805A (en) | Block chain-based convenient government affair service system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |