CN104636477A - Push list duplicate removal method before information push - Google Patents
Push list duplicate removal method before information push Download PDFInfo
- Publication number
- CN104636477A CN104636477A CN201510081194.7A CN201510081194A CN104636477A CN 104636477 A CN104636477 A CN 104636477A CN 201510081194 A CN201510081194 A CN 201510081194A CN 104636477 A CN104636477 A CN 104636477A
- Authority
- CN
- China
- Prior art keywords
- information
- list
- sent
- push
- storage space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24556—Aggregation; Duplicate elimination
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Storage Device Security (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a push list duplicate removal method before information push, and belongs to the technical field of mobile communication. The push list duplicate removal method includes: step a, by an operator, selecting a piece of to-be-sent information and determining a column related to the to-be-sent information; step b, acquiring client information of clients of the selected column; step c, performing primary duplicate removal to delete repeated client information and generate a push list; step d, judging whether the push list is empty or not by a system; step e, inquiring according to fingerprints, executing the step f if same information does not exist, or otherwise, performing the step g; step f, taking information fingerprints of the to-be-sent information and the corresponding push list as a record which is stored into a storage space; step g, performing secondary duplicate removal to generate a new push list. The push list duplicate removal method has the advantages that the situation that the same information is sent to the same client repeatedly is avoided, so that sending cost is reduced greatly, and sending efficiency is improved.
Description
Technical field
Push a De-weight method for list before information pushing, belong to mobile communication technology field.
Background technology
At present, people in productive life to the increase day by day of various information demand, simultaneously also more and more higher to the requirement of information timeliness, and utilize mobile phone terminal to receive various types of messages to become irreplaceable a kind of mode when people receive information with its ageing and portability.Service provider sends information to client and sends mainly through the form of note before, but note generally presses bar charging, costly.Along with the development of mobile communication technology and Software Industry, the application software being installed to mobile phone terminal has generally possessed the function of note propelling movement, service provider directly pushes information in the client of Client handset by wireless network, this kind of mode, to the information sender formula of current comparatively main flow, has the advantage that expense is lower.
But present stage, article one, the quantity of information of information is often larger, article one, information likely relates to two even multiple fields (or special column), if client has customized the service in the multiple fields (or special column) involved by this information simultaneously, many will be received and relate to field (or special column), but the same information that content is identical, no matter service provider is the transmission carrying out information with note or the form of network push all can run into same problem.For client, if receive many identical information simultaneously, the obstruction of file in inbox can be caused, while being inconvenient to check, be also unfavorable for arrangement and the classification of information.For service provider, many information repeat to send, and first greatly can increase the transmission cost of information, secondly can reduce whole efficiency, cause server stress excessive.
In the prior art, although also there are some data duplicate removal method, existing De-weight method not designs for information pushing.In existing data duplicate removal method, because data volume is comparatively large, involved data, list etc. are all present in hard disk, therefore carry out repeating data to search hourly velocity slower, and increasingly increasing along with data, its seek rate can be more and more slower, and efficiency is very low.
Summary of the invention
The technical problem to be solved in the present invention is: overcome the deficiencies in the prior art, provide a kind of avoid same information pointer to same client repeat send, greatly reduce sending cost, before the information pushing simultaneously improving transmission efficiency, push the De-weight method of list.
The technical solution adopted for the present invention to solve the technical problems is: the De-weight method pushing list before this information pushing, is characterized in that: comprise the steps:
Step a, operating personnel select an information to be sent, and determine this column involved by information to be sent;
Step b, column that system is selected according to operating personnel, obtains respectively to have customized and is eachly selected the customer information of column client;
Step c, the customer information of system to all columns obtained gathers, and carries out first time duplicate removal, the customer information repeated is deleted, and generates this propelling movement list sent of information to be sent;
Steps d, system judges that whether push list is empty, if be sky, returns step a, if be not empty, then this information to be sent is encrypted to the information fingerprint generating this information to be sent;
Step e, the information fingerprint generated in steps d is inquired about by system in storage space, judges whether that identical information fingerprint is present in storage space, if there is not identical information fingerprint in storage space, then performs step f; If had identical information fingerprint in storage space, then order has performed step g;
Step f, records the propelling movement list of the information fingerprint of information to be sent and correspondence stored in storage space as one;
Step g, system reads and has been present in family list corresponding to storage space internal information fingerprint, and the propelling movement list generated in this user list and step c is compared, carry out the operation of second time duplicate removal, push being present in list and be not present in record in the list of family as a supplement list fill in former propelling movement list and become new propelling movement list.
Preferably, the information fingerprint of the information to be sent described in step f and the propelling movement list of correspondence with the relation of key/value stored in described storage space.
Preferably, described storage space is system cache.
Preferably, the encryption method be encrypted information to be sent described in steps d is md5 encryption.
Compared with prior art, the beneficial effect that the present invention has is:
1, push the De-weight method of list before this information pushing, by the duplicate removal operation pushing list, avoid same information pointer to same client repeat send, greatly reduce sending cost, facilitate the finish message of client simultaneously.
2, the De-weight method of list is pushed before this information pushing, information to be sent be encrypted in the mode of MD5, corresponding generation unique information fingerprint of 32, when therefore system carries out searching of finger-print cipher in the buffer, 32 search need be carried out at most only, improve seek rate.
3, the information fingerprint of all information and the user list of correspondence are all stored in buffer memory, and therefore seek rate is better than the seek rate in database greatly, further reduces and searches the required time, improve work efficiency.
4, redis software is utilized to operate buffer memory, expired time setting is carried out to the finger print information in buffer memory simultaneously, every day, timing automatic deleted the information in buffer memory, therefore can not cause the overcrowding of buffer memory internal information amount, further increase the speed of carrying out searching in buffer memory.
Accompanying drawing explanation
Fig. 1 is the De-weight method process flow diagram pushing list before information pushing.
Embodiment
Fig. 1 is most preferred embodiment of the present invention, and below in conjunction with accompanying drawing 1, the present invention will be further described.
As shown in Figure 1, push the De-weight method of list before a kind of information pushing, comprise the steps:
Step 1001: select information to be pushed;
Operating personnel select this information to be pushed sent in information bank;
Step 1002, the column of setting also involved by select tape pushed information;
Operating personnel set and select the column involved by information to be pushed;
Step 1003, obtains the customer information of each column of customization respectively;
Column that system is selected according to operating personnel, obtains respectively to have customized and is eachly selected the customer information of column client;
Step 1004, first time duplicate removal operation, generates and pushes list;
System gathers customer information and carries out first time duplicate removal after obtaining and having customized the customer information of each column, the customer information repeated is deleted, and generates this propelling movement list sent of information to be sent;
Customize the situation of two or more column owing to there is same client simultaneously, therefore needed to carry out a duplicate removal operation when generating and pushing list, avoid pushing the customer information that there is repetition in list, thus cause and repeat to send.
Step 1005, whether push list is empty;
System judges whether the propelling movement list generated is empty, if be empty, then returns step 1001, if be not empty, then performs step 1006;
Step 1006, generates the information fingerprint of information to be sent;
System carries out md5 encryption to information to be sent, generates the information fingerprint of information to be sent;
Step 1007, whether the information fingerprint of information to be sent is present in buffer memory;
The information fingerprint of the information to be sent generated in step 1006 is searched by system in the buffer, judge whether that identical information fingerprint has been present in buffer memory, if there is identical information fingerprint in the buffer, then represent that this information is in non-first time transmission on the same day, perform step 1008, if do not find identical information fingerprint in the buffer, then represent that this information to be sent is sent first time on the same day, perform step 1010;
After md5 encryption is carried out to information to be sent, corresponding generation unique information fingerprint of 32, therefore when system carries out searching of finger-print cipher in the buffer, 32 search need be carried out at most only, simultaneously because the information fingerprint of all information is stored in buffer memory, therefore seek rate is better than the seek rate in database greatly, decreases and searches the required time, improve work efficiency.Push before this information pushing in the De-weight method of list, redis software is utilized to operate buffer memory, expired time setting is carried out to the finger print information in buffer memory simultaneously, be arranged on clock two o'clock in the morning every day (can sets itself) and delete information in buffer memory voluntarily, therefore information only retains the time of one day in buffer memory, therefore can not cause the overcrowding of buffer memory internal information amount, further increase the speed of carrying out searching in buffer memory.
Step 1008, reads the user list corresponding with information fingerprint;
System reads user list corresponding with in the finger print information identical recordings of information to be sent in buffer memory;
Step 1009, carries out duplicate removal operation to user list, generates new user list,
After system reads already present in buffer memory and that information identical information fingerprint to be sent is corresponding user list, compare with the propelling movement list generated in step 1004, carry out second time duplicate removal, the customer information be present in user list in propelling movement list is deleted, by push in list be not present in customer information in user list as a supplement list add in user list, in buffer memory, form the new user list corresponding with information fingerprint;
Step 1010, closes the record of the information generated fingerprint/user list relation in buffer memory that ties up to key/value;
Information fingerprint/user list is generated in the buffer the original records of information to be sent with the relation of key/value.
After the record of information generated fingerprint/propelling movement list in the buffer, represent that this information to be sent sent on the same day, namely the propelling movement list generated in step 1004 becomes the user list described in step 1008.
If when searching buffer memory, do not find the information fingerprint identical with information to be sent, then represent that this information to be sent is sent first time on the same day, now push according to propelling movement list, ensure that customized relevant programs client all can and receive an information to be sent only, avoid information repeat send.If when to cache lookup, find the information fingerprint identical with information to be sent, then represent that this information sent on the same day, and the use in the user list corresponding with this finger print information receives per family and only receives this information to be sent, now, system sends according to supplementary list.
The above is only preferred embodiment of the present invention, and be not restriction the present invention being made to other form, any those skilled in the art may utilize the technology contents of above-mentioned announcement to be changed or be modified as the Equivalent embodiments of equivalent variations.But everyly do not depart from technical solution of the present invention content, any simple modification, equivalent variations and the remodeling done above embodiment according to technical spirit of the present invention, still belong to the protection domain of technical solution of the present invention.
Claims (4)
1. push a De-weight method for list before information pushing, it is characterized in that: comprise the steps:
Step a, operating personnel select an information to be sent, and determine this column involved by information to be sent;
Step b, column that system is selected according to operating personnel, obtains respectively to have customized and is eachly selected the customer information of column client;
Step c, the customer information of system to all columns obtained gathers, and carries out first time duplicate removal, the customer information repeated is deleted, and generates this propelling movement list sent of information to be sent;
Steps d, system judges that whether push list is empty, if be sky, returns step a, if be not empty, then this information to be sent is encrypted to the information fingerprint generating this information to be sent;
Step e, the information fingerprint generated in steps d is inquired about by system in storage space, judges whether that identical information fingerprint is present in storage space, if there is not identical information fingerprint in storage space, then performs step f; If had identical information fingerprint in storage space, then order has performed step g;
Step f, records the propelling movement list of the information fingerprint of information to be sent and correspondence stored in storage space as one;
Step g, system reads and has been present in family list corresponding to storage space internal information fingerprint, and the propelling movement list generated in this user list and step c is compared, carry out the operation of second time duplicate removal, push being present in list and be not present in record in the list of family as a supplement list fill in former propelling movement list and become new propelling movement list.
2. push the De-weight method of list before information pushing according to claim 1, it is characterized in that: the information fingerprint of the information to be sent described in step f and the propelling movement list of correspondence with the relation of key/value stored in described storage space.
3. push the De-weight method of list before information pushing according to claim 1 and 2, it is characterized in that: described storage space is system cache.
4. push the De-weight method of list before information pushing according to claim 1, it is characterized in that: the encryption method be encrypted information to be sent described in steps d is md5 encryption.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510081194.7A CN104636477B (en) | 2015-02-15 | 2015-02-15 | The De-weight method of push list before a kind of information push |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510081194.7A CN104636477B (en) | 2015-02-15 | 2015-02-15 | The De-weight method of push list before a kind of information push |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104636477A true CN104636477A (en) | 2015-05-20 |
| CN104636477B CN104636477B (en) | 2017-11-24 |
Family
ID=53215223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510081194.7A Active CN104636477B (en) | 2015-02-15 | 2015-02-15 | The De-weight method of push list before a kind of information push |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104636477B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105227662A (en) * | 2015-09-30 | 2016-01-06 | 努比亚技术有限公司 | Message treatment method, device and system |
| CN106649646A (en) * | 2016-12-09 | 2017-05-10 | 北京锐安科技有限公司 | Method and device for deleting duplicated data |
| CN107665225A (en) * | 2016-07-29 | 2018-02-06 | 北京京东尚科信息技术有限公司 | Information-pushing method and device |
| CN107832406A (en) * | 2017-11-03 | 2018-03-23 | 北京锐安科技有限公司 | Duplicate removal storage method, device, equipment and the storage medium of massive logs data |
| CN109246213A (en) * | 2018-09-06 | 2019-01-18 | 郑州云海信息技术有限公司 | A kind of target zone information transmission system and method based on GPS positioning |
| CN111245706A (en) * | 2020-01-03 | 2020-06-05 | 湖南省梦网科技发展有限公司 | Information processing method, device, server and medium |
| CN113434301A (en) * | 2021-07-19 | 2021-09-24 | 深圳市链融科技股份有限公司 | Information pushing method and device, computer equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102456059A (en) * | 2010-10-21 | 2012-05-16 | 英业达股份有限公司 | Data deduplication processing system |
| WO2012109056A1 (en) * | 2011-02-11 | 2012-08-16 | Symantec Corporation | Processes and methods for client-side fingerprint caching to improve deduplication system backup performance |
| CN102810107A (en) * | 2011-06-01 | 2012-12-05 | 英业达股份有限公司 | Processing method for repeating data |
| CN103685420A (en) * | 2012-09-24 | 2014-03-26 | 华为技术有限公司 | Method, server and system for media file duplication removal |
-
2015
- 2015-02-15 CN CN201510081194.7A patent/CN104636477B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102456059A (en) * | 2010-10-21 | 2012-05-16 | 英业达股份有限公司 | Data deduplication processing system |
| WO2012109056A1 (en) * | 2011-02-11 | 2012-08-16 | Symantec Corporation | Processes and methods for client-side fingerprint caching to improve deduplication system backup performance |
| CN102810107A (en) * | 2011-06-01 | 2012-12-05 | 英业达股份有限公司 | Processing method for repeating data |
| CN103685420A (en) * | 2012-09-24 | 2014-03-26 | 华为技术有限公司 | Method, server and system for media file duplication removal |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105227662A (en) * | 2015-09-30 | 2016-01-06 | 努比亚技术有限公司 | Message treatment method, device and system |
| CN107665225A (en) * | 2016-07-29 | 2018-02-06 | 北京京东尚科信息技术有限公司 | Information-pushing method and device |
| US11038975B2 (en) | 2016-07-29 | 2021-06-15 | Beijing Jingdong Shangke Information Technology Co., Ltd. | Information pushing method and device |
| CN107665225B (en) * | 2016-07-29 | 2022-01-28 | 北京京东尚科信息技术有限公司 | Information pushing method and device |
| CN106649646A (en) * | 2016-12-09 | 2017-05-10 | 北京锐安科技有限公司 | Method and device for deleting duplicated data |
| CN107832406A (en) * | 2017-11-03 | 2018-03-23 | 北京锐安科技有限公司 | Duplicate removal storage method, device, equipment and the storage medium of massive logs data |
| CN107832406B (en) * | 2017-11-03 | 2020-09-11 | 北京锐安科技有限公司 | Method, device, equipment and storage medium for removing duplicate entries of mass log data |
| CN109246213A (en) * | 2018-09-06 | 2019-01-18 | 郑州云海信息技术有限公司 | A kind of target zone information transmission system and method based on GPS positioning |
| CN111245706A (en) * | 2020-01-03 | 2020-06-05 | 湖南省梦网科技发展有限公司 | Information processing method, device, server and medium |
| CN113434301A (en) * | 2021-07-19 | 2021-09-24 | 深圳市链融科技股份有限公司 | Information pushing method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104636477B (en) | 2017-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104636477A (en) | Push list duplicate removal method before information push | |
| US9792340B2 (en) | Identifying data items | |
| CN102323923B (en) | Method for processing historical record and equipment | |
| CN102479223A (en) | Data query method and system | |
| CN102769640B (en) | The update method of user profile, server and system | |
| CN101645086A (en) | Retrieval method | |
| CN107231485B (en) | Method and device for establishing event reminding | |
| CN103167171B (en) | Selection method and mobile terminal of contact mode | |
| CN109271449A (en) | A kind of distributed storage inquiry system file-based and querying method | |
| CN100574340C (en) | A kind of method of searching SMS | |
| CN109241031B (en) | Model generation method, model using method, device, system and storage medium | |
| CN114398520A (en) | Data retrieval method, system, device, electronic equipment and storage medium | |
| US9398434B2 (en) | Method and system for zone analysis in a charging system | |
| CN104579920A (en) | Mail sending method and device | |
| CN102970401A (en) | Method and device for recoding contact information | |
| CN105095224A (en) | Method, apparatus and system for carrying out OLAP analysis in mobile communication network | |
| CN101384050A (en) | Mobile terminal, method and system for resource management | |
| CN101217385B (en) | A method and system for the temporary storage and treatment of charging bill | |
| CN112448880A (en) | Method and device for sending RCS service message, client and server | |
| CN112887925B (en) | Short message pushing method, edge server node and service server node | |
| CN107977381B (en) | Data configuration method, index management method, related device and computing equipment | |
| CN102646136B (en) | Method and system for efficiently storing and inquiring data | |
| WO2017167101A1 (en) | Multimedia message attachment management method and device, communication system and computer storage medium | |
| CN109308229A (en) | A method of restoring wechat chat record | |
| CN103218445A (en) | Mobile terminal information pushing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 255100, 6, Yifan Road, Zhangdian District, Shandong, Zibo Applicant after: Shandong zhuochuang Touchplus information Corp Address before: 255400, 2678, Xin Cheng Road, Linzi District, Shandong, Zibo Applicant before: Shandong Zhuo Chuan information Group Co., Ltd |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |