CN107463588A - Data submit judging method - Google Patents

Data submit judging method Download PDF

Info

Publication number
CN107463588A
CN107463588A CN201610533721.8A CN201610533721A CN107463588A CN 107463588 A CN107463588 A CN 107463588A CN 201610533721 A CN201610533721 A CN 201610533721A CN 107463588 A CN107463588 A CN 107463588A
Authority
CN
China
Prior art keywords
page
value
tag value
data
tag
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610533721.8A
Other languages
Chinese (zh)
Inventor
梁仕强
韩军涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Publication of CN107463588A publication Critical patent/CN107463588A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0633Lists, e.g. purchase orders, compilation or processing

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • Computing Systems (AREA)
  • Development Economics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A kind of data submit judging method, including:Page-tag value generation step (S1, S2), page feature value is extracted from the page of client, and the unique page-tag value of the page universe is generated based on the page feature value, and be sent to service end;Counter generation step (S3), in the service end, the page-tag value is extracted, and the page-tag value is generated into Redis counters as Key values;And counter determination step (S4), in the service end, judge the return value of Redis counters, when the return value is 1, it is newly to ask to be judged as current request, and the current request is forwarded in background service process and continued with, when the return value is more than 1, it is judged as current request to repeat the request submitted, and directly refuses.

Description

Data submit judging method
Technical field
The present invention relates to a kind of data for preventing Data duplication from submitting to submit judging method, more particularly in distributed system Prevent the data that Data duplication is submitted from submitting judging method.
Background technology
With Internet development and its extensive use, have become Internet service using the electric business system of network Key player.In Internet service, prevent Data duplication from submitting, ensure the idempotence of interface function has for operation system Very important meaning.Particularly in electric business system, how to prevent from submitting repeating order, how to prevent from repeating to submit reimbursement please Ask, how to prevent malice from placing an order, prevent malice from calling the meaning of the work such as service interface particularly important.We unite these behaviors Referred to as the repetition of data is submitted.
Submitted to tackle the repetition of data, existing solution there are following 3 kinds.
Scheme 1:After data are submitted, limited using front end page script, the method for forbidding user to submit again.
But the program 1 is unable to prohibiting page and refreshes or retract and then submit, and can not prevent from directly invoking connecing The malicious act of mouth, thus it is substantially invalid during this scheme present electric business business demand of reply.
Scheme 2:Imply token in the data of submission, in service end by sentencing using session and token simultaneously Whether disconnected user has been filed on data.
But present large scale business application service all uses distributed deployment, i.e., it is deployed in multiple servers (physics simultaneously Machine or virtual machine) on, user access request can be fallen on wherein on a server at random, so, this scheme 2 needs institute is useful The session at family is synchronized on each server.The session information of all in store all users of each server, this The substantial amounts of memory source of expense is not only needed, while synchronous session information is also required to take substantial amounts of network bandwidth in itself.Institute With, this scheme 2 can not adaptation server distributed deployment large scale business application.
Scheme 3:After data are submitted, specific record is inserted in database, for accessing every time, by reading data Storehouse is recorded to judge whether to repeat.
But in reality system, read-write database is often the performance bottleneck of whole system, in the electricity that high concurrent accesses More it is to try to avoid the read-write of database in business's application system.Thus, this actual efficiency of scheme 3 is extremely low, and in reality Application is little in electric business system.
As can be seen here, scheme (above 1,2,3 kinds of schemes) can not meet existing large-scale distributed application system at this stage The demand that middle detection Data duplication is submitted.
The content of the invention
The present invention be in view of prior art it is above-mentioned so the problem of and make, it is a kind of based on spy its object is to provide Sign extraction of values and the ingenious combination of caching sentence weight to carry out data submission, and can be well by memory cost and time complexity expense The horizontal data in 0 (N) are controlled to submit judging method.The data of one aspect of the present invention submit judging method, including:
Page-tag value generation step, page feature value is extracted from the page of client, and be based on the page feature value The unique page-tag value of the page universe is generated, and is sent to service end;
Counter generation step, in the service end, extract the page-tag value, and using the page-tag value as Key values and generate Redis counters;And
Counter determination step, in the service end, judge the return value of Redis counters, when the return value is 1, It is newly to ask to be judged as current request, and the current request is forwarded in background service process and continued with, in the return value During more than 1, it is judged as current request to repeat the request submitted, and directly refuses.
Data according to an aspect of the present invention submit judging method, in the page-tag value generation step, institute The extraction for stating page feature value is realized by java-script.
Data according to an aspect of the present invention submit judging method, and the page feature value is crucial in the page And unduplicated data.
Data according to an aspect of the present invention submit judging method, when the page is submits the order page, institute State the mark Id and quantity that page feature value is each single product.
Data according to an aspect of the present invention submit judging method, the page-tag value generation step bag Include:From the template of the page, the step of the domain as characteristic value is marked;Collect all in the page mark The value in the domain come, and the step of these values are assembled into an overall character string according to the page sequencing;And The step of according to the overall character string being assembled into generate the page-tag value.
Data according to an aspect of the present invention submit judging method, and the template includes:velocity、jsp.
Data according to an aspect of the present invention submit judging method, in the page-tag value generation step, When assembling the overall character, also user's login PIN value is assembled into the overall character as prefix.
Data according to an aspect of the present invention submit judging method, in the page-tag value generation step, When generating the page-tag value, the overall character string is compressed using fixed bit length algorithm, obtains the page mark Label value.
Data according to an aspect of the present invention submit judging method, are being judged as that current request is newly to ask and continue In the case of background process processing, after background service process has handled the current request, the Redis counters are deleted.
In summary, the above-mentioned technical proposal of judging method is submitted according to the data of the present invention, realizes a kind of effectively inspection Survey and prevent from distribution application system Data duplication to submit a question and reduce detecting Data duplication in distribution application system and carrying The time complexity and the data of space complexity needed during friendship submits judging method.
Brief description of the drawings
Fig. 1 is that the data of the present invention submit the schematic diagram of judging method.
Fig. 2 is that the data of the present invention submit the flow chart of judging method.
Fig. 3 is the flow chart that the data of the present invention submit the step S1 of judging method.
Embodiment
For the object, technical solutions and advantages of the present invention are more clearly understood, below in conjunction with specific embodiment, and reference Accompanying drawing, the present invention is described in detail.
It should be noted that the embodiments described herein is served only for for example, being not limited to the disclosure.It is described below In, in order to provide the thorough understanding to the disclosure, elaborate a large amount of specific details.However, for those of ordinary skill in the art It is evident that:The disclosure need not be carried out using these specific details.
Throughout the specification, meaning is referred to " one embodiment ", " embodiment ", " example " or " example " :It is comprised in reference to special characteristic, structure or the characteristic that the embodiment or example describe at least one embodiment of the disclosure. Therefore, each local phrase " in one embodiment " occurred in entire disclosure, " in embodiment ", " example " Or " example " is not necessarily all referring to the same embodiment or example.Furthermore, it is possible to will be specific with any appropriate combination and/or sub-portfolio Feature, structure or property combination in one or more embodiments or example.In addition, those of ordinary skill in the art should manage Solution, accompanying drawing is provided to the purpose of explanation provided herein, and accompanying drawing is not necessarily drawn to scale.Identical accompanying drawing mark Note instruction identical key element.Term "and/or" used herein includes any and institute for the project that one or more correlations are listed There is combination.
First, above-mentioned time complexity and above-mentioned space complexity are briefly described.
Time complexity refers to perform the amount of calculation required for algorithm.Common time complexity has:Constant order 0 (1), (log of logarithm rank 02N), linear rank 0 (n), (nlog of linear-logarithmic rank 02N), square rank 0 (n^2), cube rank 0 (n^3) etc., N represents the number or scale of event execution.In the case of wherein constant order and linear order algorithm complete same task, the time spends Time it is minimum.
Space complexity refers to perform the memory headroom required for this algorithm.Common space complexity has:Constant order 0 (1), (log of logarithm rank 02N), linear rank 0 (n), linear-logarithmic rank 0 (nlog2n), square rank 0 (n^2), cube rank 0 (n^3) Deng the number or scale that n expression events perform.It is required in the case of wherein constant order and linear order algorithm complete same task The space wanted is minimum.
Below, judging method is submitted with reference to accompanying drawing to illustrate the data of the present invention.
Fig. 1 is that the data of the present invention submit the schematic diagram of judging method.
As shown in figure 1, the data of the present invention submit judging method, first, pass through js (java- from the page of client Script page feature value) is extracted, and the unique page-tag value of universe of the page is generated based on page feature value.
Then, the page-tag value of generation is sent to service end by Hidden field in client.For example, it is sent to service The back-end server (for example, Nginx background servers etc.) at end.
Then, page-tag value is extracted in the back-end server of service end, and is being taken using page-tag value as Key values It is engaged in generating Redis counters in the caching at end.
Then, in the caching of service end, the return value of Redis counters is judged, when counter return value is 1, table Bright current request is the new request of first arrival, then forwards this request in background service process and continue with;Counting When device return value is more than 1, show that current request belongs to the request of repetition submission, and directly refuse, processing terminates.
In addition, after background service process has handled corresponding request, deleted in Redis servers with the page-tag It is worth the Redis counters for Key, processing terminates.
Below, the flow that the data of the present invention submit judging method is illustrated using Fig. 2.
Fig. 2 is that the data of the present invention submit the flow chart of judging method.
As shown in Fig. 2 the data of the present invention submit judging method, in step sl, pass through js from the page P of client (java-script) page feature value is extracted, and the unique page-tag value of page P universes is generated based on page feature value.
Then, in step s 2, the page-tag value of generation is sent to service end S by client by Hidden field.
Then, in step s3, service end S extracts page-tag value, and is generated using page-tag value as Key values Redis counters.
Then, in step s 4, judge the return value of Redis counters, when counter return value is 1, show current Request is the new request of first arrival, then forwards this request in background service process and continue with, be transferred to step S5; When counter return value is more than 1, show that current request belongs to the request of repetition submission, and directly refuse, processing terminates.
In step s 5, after background service process has handled corresponding request, deleted in Redis servers with the page Face label value is Key Redis counters, and processing terminates.
So, judging method is submitted according to the data of the present invention, there is atomicity due to creating Redis counters, so No matter how many platform background server, unique counter can be generated in Redis servers using label value as key, And the time complexity for generating counter is 0 (N), i.e., directly proportional with client request quantity.In addition, in Redis servers The number of the counter of middle generation is also only directly proportional to concurrent visit capacity, i.e. space complexity shared by method of the invention For 0 (N).
Hereinafter, the data for the present invention submit the above-mentioned steps S1 of judging method to be further described.
Fig. 3 is the flow chart that the data of the present invention submit the step S1 of judging method.
As shown in figure 3, in step S1-1, in the page P of client template (for example, velocity, jsp etc.), The domain as characteristic value will be needed to be marked, for example,<Div class=" gather-trait "/>, class titles at this Do not fix.The object for needing to mark is crucial in page P and unduplicated data.Such as:It is excellent if submitting the order page Choosing is by the skuId (mark Id) and quantity of each single product labeled as the value for needing to extract.
Then, in step S1-2, all in page P be marked is collected by js (java-script) codes Domain value, and these values are assembled into an overall character string according to page sequencing, then using user log in PIN value as Prefix is assembled into the character.
Then, in step S1-3, " fixed bit length algorithm " is used to enter the character string being assembled into above-mentioned steps S1-2 Row compression, obtains the page P unique page-tag value of universe.
Thus, data of the invention submit the above-mentioned technical proposal of judging method, pass through feature based extraction of values and caching Ingenious combination sentences weight to carry out data submission, and the Data duplication that can efficiently solve distribution application system submits a question, and together When had both efficient time complexity 0 (N) and outstanding space complexity 0 (N), can be in big flow, the electric business of high concurrent And generally used in large-scale distributed application system.
More than, for the technical scheme of the task method for pushing of the present invention, it is described in detail by taking embodiment as an example.Though So it is illustrated by way of software flow in the above-described embodiment, but for those skilled in the art, these Part or all of technical scheme can be realized that this is self-evident by the combination of hardware, software or hardware and software 's.
Detailed description above has elaborated sensitive word search method by using block diagram, flow chart and/or example Embodiment.In the case where this block diagram, flow chart and/or example include one or more functions and/or operation, ability Field technique personnel should be understood that each function in this block diagram, flow chart or example and/or operation can be by various hard Part, software, firmware or they substantial any combination to realize individually and/or jointly.In one embodiment, the disclosure If the stem portion of theme described in embodiment can pass through application specific integrated circuit (ASIC), field programmable gate array (FPGA), number Word signal processor (DSP) or other integrated forms are realized.It is however, those skilled in the art will appreciate that disclosed herein The some aspects of embodiment can equally realize in integrated circuits on the whole or partly, be embodied as at one or more The one or more computer programs run on platform computer are (for example, be embodied as running in one or more computer system One or more programs), the one or more programs for being embodied as running on the one or more processors are (for example, be embodied as The one or more programs run in one or more microprocessors), it is embodied as firmware, or be substantially embodied as above-mentioned side Any combination of formula, and those skilled in the art will be provided with designing circuit and/or write software and/or consolidate according to the disclosure The ability of part code.In addition, it would be recognized by those skilled in the art that the mechanism of theme described in the disclosure can be used as diversified forms Program product be distributed, and regardless of the actual particular type for being used for performing the signal bearing medium of distribution, this public affairs The exemplary embodiment for opening the theme is applicable.The example of signal bearing medium includes but is not limited to:Recordable-type media, such as Floppy disk, hard disk drive, compact-disc (CD), digital universal disc (DVD), digital magnetic tape, computer storage etc.;And mode transmission Medium, such as numeral and/or analogue communication medium (for example, optical fiber cable, waveguide, wired communications links, wireless communication link).
Although with reference to exemplary embodiment describing the disclosure, it is to be understood that, term used be explanation and it is exemplary, And nonrestrictive term.Because the disclosure can be embodied without departing from disclosed spirit or essence, institute in a variety of forms Should be appreciated that above-described embodiment is not limited to any foregoing details, and should be in the spirit and model that appended claims are limited Widely explained in enclosing, therefore the whole changes fallen into claim or its equivalent scope and remodeling all should be right of enclosing and wants Ask and covered.

Claims (9)

1. a kind of data submit judging method, including:
Page-tag value generation step, page feature value is extracted from the page of client, and generated based on the page feature value The unique page-tag value of the page universe, and it is sent to service end;
Counter generation step, in the service end, the page-tag value is extracted, and using the page-tag value as Key It is worth and generates Redis counters;And
Counter determination step, in the service end, judge the return value of Redis counters, when the return value is 1, judge It is newly to ask for current request, and the current request is forwarded in background service process and continued with, is more than 1 in the return value When, it is judged as current request to repeat the request submitted, and directly refuse.
2. data according to claim 1 submit judging method, it is characterised in that
In the page-tag value generation step, the extraction of the page feature value is realized by java-script.
3. web color method according to claim 1, it is characterised in that
The page feature value is crucial in the page and unduplicated data.
4. web color method according to claim 1, it is characterised in that
When the page is submits the order page, the page feature value is the mark Id and quantity of each single product.
5. web color method according to claim 1, it is characterised in that
The page-tag value generation step includes:
From the template of the page, the step of the domain as characteristic value is marked;
The value in all domains being marked in the page is collected, and these values are assembled according to the page sequencing The step of into an overall character string;And
The step of according to the overall character string being assembled into generate the page-tag value.
6. web color method according to claim 5, it is characterised in that
The template includes:velocity、jsp.
7. web color method according to claim 5, it is characterised in that
In the page-tag value generation step, when assembling the overall character, user is also logged in into PIN value as prefix It is assembled into the overall character.
8. web color method according to claim 5, it is characterised in that
In the page-tag value generation step, when generating the page-tag value, using fixed bit length algorithm by described in Overall character string is compressed, and obtains the page-tag value.
9. web color method according to claim 1, it is characterised in that
Be judged as current request be newly ask and continue background process processing in the case of, handled this in background service process After current request, the Redis counters are deleted.
CN201610533721.8A 2016-06-03 2016-07-07 Data submit judging method Pending CN107463588A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN2016103901437 2016-06-03
CN201610390143 2016-06-03

Publications (1)

Publication Number Publication Date
CN107463588A true CN107463588A (en) 2017-12-12

Family

ID=60545126

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610533721.8A Pending CN107463588A (en) 2016-06-03 2016-07-07 Data submit judging method

Country Status (1)

Country Link
CN (1) CN107463588A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109325196A (en) * 2018-07-27 2019-02-12 四川长虹电器股份有限公司 The method for preventing interface repetitive requests based on redis
CN112039913A (en) * 2020-09-07 2020-12-04 上海浦东发展银行股份有限公司 Server API calling method, device and storage medium
CN112988881A (en) * 2019-12-13 2021-06-18 Oppo广东移动通信有限公司 Data synchronization method and device and computer readable storage medium
CN113452713A (en) * 2021-06-29 2021-09-28 成都九洲电子信息系统股份有限公司 Network data carrier duplication judgment method based on double sampling under special line environment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103530201A (en) * 2013-07-17 2014-01-22 华中科技大学 Safety data repetition removing method and system applicable to backup system
CN103841111A (en) * 2014-03-17 2014-06-04 北京京东尚科信息技术有限公司 Method for preventing data from being submitted repeatedly and server
CN104572263A (en) * 2014-12-30 2015-04-29 腾讯科技(深圳)有限公司 Page data interaction method, related device and system
CN105095462A (en) * 2015-07-30 2015-11-25 北京京东尚科信息技术有限公司 Method and system for processing webpage repeated requests
CN105608207A (en) * 2015-12-25 2016-05-25 广州华多网络科技有限公司 Data statistics system based on Redis database and statistics method of data statistics system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103530201A (en) * 2013-07-17 2014-01-22 华中科技大学 Safety data repetition removing method and system applicable to backup system
CN103841111A (en) * 2014-03-17 2014-06-04 北京京东尚科信息技术有限公司 Method for preventing data from being submitted repeatedly and server
CN104572263A (en) * 2014-12-30 2015-04-29 腾讯科技(深圳)有限公司 Page data interaction method, related device and system
CN105095462A (en) * 2015-07-30 2015-11-25 北京京东尚科信息技术有限公司 Method and system for processing webpage repeated requests
CN105608207A (en) * 2015-12-25 2016-05-25 广州华多网络科技有限公司 Data statistics system based on Redis database and statistics method of data statistics system

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109325196A (en) * 2018-07-27 2019-02-12 四川长虹电器股份有限公司 The method for preventing interface repetitive requests based on redis
CN112988881A (en) * 2019-12-13 2021-06-18 Oppo广东移动通信有限公司 Data synchronization method and device and computer readable storage medium
CN112039913A (en) * 2020-09-07 2020-12-04 上海浦东发展银行股份有限公司 Server API calling method, device and storage medium
CN113452713A (en) * 2021-06-29 2021-09-28 成都九洲电子信息系统股份有限公司 Network data carrier duplication judgment method based on double sampling under special line environment
CN113452713B (en) * 2021-06-29 2022-04-19 成都九洲电子信息系统股份有限公司 Network data carrier duplication judgment method based on double sampling under special line environment

Similar Documents

Publication Publication Date Title
US20140019979A1 (en) Automated web task procedures based on an analysis of actions in web browsing history logs
CN108365971A (en) Daily record analytic method, equipment and computer-readable medium
US20100094860A1 (en) Indexing online advertisements
Bennett et al. Malstone: towards a benchmark for analytics on large data clouds
CN107463588A (en) Data submit judging method
CN103890709A (en) Cache based key-value store mapping and replication
CN108595517A (en) A kind of extensive document similarity detection method
CN103248677B (en) The Internet behavioural analysis system and method for work thereof
WO2019017997A1 (en) Distributed graph database writes
US20060288272A1 (en) Computer-implemented method, system, and program product for developing a content annotation lexicon
WO2011051208A2 (en) Apparatus and method for providing page navigation in multirole-enabled network application
US10394868B2 (en) Generating important values from a variety of server log files
US10305845B1 (en) Accurate user alignment across online social media platforms
Motoyama et al. Measuring online service availability using twitter
Subraya Integrated Approach to Web Performance Testing: A Practitioner's Guide: A Practitioner's Guide
Franceschet The large‐scale structure of journal citation networks
CN106547646A (en) A kind of data backup and restoration methods, data backup and recovery device
US20210264297A1 (en) Content recommendation based upon continuity and grouping information of attributes
JP6700554B2 (en) Distributed processing management method, distributed processing management program, and distributed processing management device
CN1987775A (en) Method and apparatus for resolving events
CN107357810A (en) A kind of method and system for being used to export special topic landing page
McDonough 'Knee-Deep in the Data': Practical Problems in Applying the OAIS Reference Model to the Preservation of Computer Games
Gruppi et al. Tell me who your friends are: Using content sharing behavior for news source veracity detection
He et al. How were science mapping tools applied? the application of science mapping tools in LIS and non‐LIS domains
CN109522211A (en) Interface parameters transmission method, device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20171212

RJ01 Rejection of invention patent application after publication