CN105827608B - Distributed API service abnormal user identifying and analyzing method and reverse proxy gateway - Google Patents
Distributed API service abnormal user identifying and analyzing method and reverse proxy gateway Download PDFInfo
- Publication number
- CN105827608B CN105827608B CN201610195972.XA CN201610195972A CN105827608B CN 105827608 B CN105827608 B CN 105827608B CN 201610195972 A CN201610195972 A CN 201610195972A CN 105827608 B CN105827608 B CN 105827608B
- Authority
- CN
- China
- Prior art keywords
- user
- control centre
- api request
- suspicious
- log
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
- H04L63/101—Access control lists [ACL]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
- H04L63/1466—Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks
Abstract
The present invention provides a kind of distributed API service abnormal user identifying and analyzing method and reverse proxy gateway, this method comprises: obtaining the API request of user, and access log is recorded in the API request of the user；Real-time statistic analysis is carried out to the described request, judges whether the user is abnormal user；If so, injecting cognizance code when the user sends API request again；If receiving the feedback information that the user is directed to the injection cognizance code within the regulation time limit, sends user and normally notify control centre；Otherwise it sends user and notifies control centre extremely；The process instruction that control centre sends is obtained, reason instruction according to this performs corresponding processing the subsequent API request of the user；Because using script embedded mode, recognition rule script flexibly heat can be updated by control centre interface, manually can also dynamically set shielding rules；Because cooperating real-time calculation and analysis user request by reverse proxy, the frequency of abnormal user can be made to identify more efficient more complete.
Description
Technical field
The present invention relates to technical field of network security more particularly to a kind of distributed API service abnormal user discriminance analysis
Method and reverse proxy gateway.
Background technique
When one website externally provides SAAS service, the robot of various malice can be usually encountered in order to obtain data
Or influence website experience and API (Application Programming Interface, the application programming to us
Interface) high-frequency data crawl is carried out, many websites want problems faced to be, how to distinguish common user and unconventional use
Family.Normal users " are accidentally injured " in order to prevent, and " default " this evil in the case where not influencing it and externally servicing is taken in many websites
The crawl behavior of meaning.
To the normal operation of our systems, common needs these illegal customer impacts for this kind of attack in order to prevent
It wants, counts, identifies, shielding three steps, during which also will use and some manually carry out arrangement setting.
In realizing process of the present invention, at least there are the following problems in the prior art for inventor's discovery:
Receive others' crawl behavior, but QPS (IP being pressed, by UID (User ID)) is in this way in restricted unit time
Mode guarantee the access of normal users, limit the request amount of improper user.But generally speaking often result is inaccurate by IP,
Some companies or area are caused to use limited.It even results in and unfriendly to search engine spider includes the problems such as slow.
Discovery user request is abnormally frequent in unit time, unique identifier is injected to the request of user, if client does not have
There is feedback designated code automatic shield then to this user's marking.All API require to be implanted into this identification.Such mode is only fitted
Internal platform is closed using external service interface once disclose this it is regular if this mechanism be easy to be hacked and cause in vain.
Real-time big data analysis mode, such mode is currently most flexible, but cost is larger to need distributed type assemblies
It calculates in real time and passing data reference comparative analysis goes out user characteristics the problem of wherein, feature is even so still required and goes to advise
Keep away different users.Support that the cost of real-time big data analysis is not that general company is able to bear at present.
It is unable to artificial observation and specific characteristic shielding or white list or is manually specified be difficult to regularly update.
It does not support heat to update rule, is not available assemblage characteristic in addition to big data analysis mode.
Single-point calculated performance problem is mostly single-point service at present
It is all that each company oneself stand-alone development is needed to complete such function at present, such function needs mating distributed day
Will, customized reverse proxy.Conventional medium-sized and small enterprises temporarily do similar functions without this kind of strength and time.
It can seldom accomplish complicated real-time identification.
To sum up, be all completed by identification feature it is stifled to chasing after for illegal act.But at present there is no a kind of method or
System is capable of providing efficient and complete support.
Summary of the invention
The embodiment of the present invention provides a kind of distributed API service abnormal user frequency recognition methods and reverse proxy service network
It closes, reverse proxy cooperates real-time calculation and analysis user request, so that the frequency identification of abnormal user is more efficient more complete.
On the one hand, the embodiment of the invention provides a kind of distributed API service abnormal user frequency recognition methods, comprising:
The API request of user is obtained, and access log is recorded in the API request of the user；
Real-time statistic analysis is carried out according to API request of the access log to the user, confirms that the user is suspicious use
When family, control centre is notified；Control centre is obtained to be ordered according to the user identity identification that the notice that the user is suspicious user generates
It enables, and injection cognizance code is generated according to the order, when the suspicious user sends API request again, the injection is identified
Code returns to the suspicious user and obtains the user identity identification that control centre generates according to the notice that the user is suspicious user
Order；
If receiving the feedback information that the suspicious user is directed to the injection cognizance code within the regulation time limit, user is being sent just
Normal open knows control centre；If not receiving the feedback information that the suspicious user is directed to the injection cognizance code within the regulation time limit,
It then sends user and notifies control centre extremely；
Obtain that control centre normally notify according to the user or what user's notice extremely generated is directed to the suspicious user
Process instruction, by the user information and the storage of corresponding process instruction, to processing log, and according to this, reason is instructed to the user
Subsequent API request perform corresponding processing.
Preferably, before the API request progress real-time statistic analysis to the user, comprising:
The process instruction of caching is searched, confirms and the user information is not present in process instruction.
It is preferably, described that real-time statistic analysis is carried out according to API request of the access log to the user, comprising:
The access log is pushed to off-line analysis service node according to IP consistency Hash hash；The off-line analysis service
It include historical operation log in node；The historical operation log includes the record of the various operations carried out to user's API request；
The historical operation log of the off-line analysis service node is transferred by script engine, and according to the historical operation log
And this access log carries out real-time statistic analysis.
Preferably, real-time statistic analysis is carried out by script engine embedded mode and carries out interacting message with control centre.
Preferably, the acquisition control centre normally notifies according to the user or user's being directed to for notice generation extremely is somebody's turn to do
The process instruction of suspicious user, comprising:
The process instruction of the artificial dynamic setting of control centre's transmission is directly acquired by the script engine embedded mode.
On the other hand, the present invention provides a kind of distributed API service abnormal user frequency identification reverse proxy gateway,
Include:
Access log is recorded for obtaining the API request of user, and by the API request of the user in request unit；
Real-time statistic analysis unit, for carrying out real-time statistics point according to API request of the access log to the user
Analysis when confirming that the user is suspicious user, notifies control centre；
User identification unit, the user identity generated for obtaining control centre according to the notice that the user is suspicious user
Recognition command, and injection cognizance code is generated according to the order, when the suspicious user sends API request again, by the note
Enter cognizance code and returns to the suspicious user；
Feedback analysis unit, for working as the feedback for receiving the suspicious user within the regulation time limit and being directed to the injection cognizance code
When information, sends user and normally notify control centre；Know when not receiving the suspicious user within the regulation time limit for the injection
When the feedback information of other code, sends user and notify control centre extremely；
Requesting processing, normally notifies according to the user for obtaining control centre or user's notice extremely generates
For the process instruction of the suspicious user, by the user information and the storage of corresponding process instruction to processing log, and according to this
It is suspicious user that process instruction, which performs corresponding processing the subsequent API request of the user and obtains control centre according to the user,
Notify the user identity identification order generated.
Preferably, further includes:
Pretreatment unit confirms and the user information is not present in process instruction for searching the process instruction of caching.
Preferably, the real-time statistic analysis unit includes:
Log pushes subelement, for the access log to be pushed to off-line analysis service according to IP consistency Hash hash
Node；It include historical operation log in the off-line analysis service node；The historical operation log include to user's API request into
The record of capable various operations；
Subelement is statisticallyd analyze, for transferring the historical operation log of the off-line analysis service node by script engine,
And real-time statistic analysis is carried out according to the historical operation log and this access log.
Preferably, which carries out real-time statistic analysis and and control centre by script engine embedded mode
Carry out interacting message.
Preferably, the requesting processing directly acquires control centre's transmission by the script engine embedded mode
The process instruction of artificial dynamic setting.
Above-mentioned technical proposal has the following beneficial effects: by using the API request for receiving user；Identify the user's
API request, and generate access log；The access log is pushed to corresponding Analysis Service section according to IP consistency Hash hash
Point, with for statistical analysis, generation frequency statistics data；Identification splits the keyword of the API request of the user；According to described
Keyword and the frequency statistics data carry out frequency analysis to the API request of the user, and analysis result is pushed to
Control centre, or process instruction is directly generated according to the analysis result, and the process instruction is sent to control centre；Because passing through
Reverse proxy cooperates real-time calculation and analysis user request, the frequency of abnormal user can be made to identify more efficient more complete.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of flow chart of distributed API service abnormal user frequency recognition methods of the present invention；
Fig. 2 is that a kind of structure of distributed API service abnormal user frequency identification reverse proxy gateway of the present invention is shown
It is intended to；
Fig. 3 is a kind of flow chart of distributed API service abnormal user frequency recognition methods of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Illegal user influences the normal operation of system in order to prevent, for the attack of API high-frequency data crawl, needs to unite
Three meter, identification and shielding steps, during which can also use some manual sorting settings.
Wherein, it is counted for user's API request, comprising:
Identification judgement is carried out according to passing user's request amount, if yesterday asks section simultaneously, last week, same week period, was gone
Period on the same day in year.
Frequency and number are counted in a period of time, and if low-frequency degree sees cumulative number, frequent pulsed sees that QPS is (every
Second query rate), whether the requesting interval time is regular.
It is identified according to IP sections, frequently than other IP sections such as certain IP sections of access frequency.
It is identified according to request header feature.As having the request of XXX keyword many in user agent UserAgent, no
It is proportional.
It is simplest: not do any differentiation, counted just for the request amount of current service.
Identification includes:
Frequency identification: it is identified according to the frequency of access and number, such as ordinary user
Keyword recognition: being identified according to the feature critical word of robot, and Curl can be taken individually by such as grabbing robot
Keywords such as (for constructing http: the order of // request).
Feature combination identification: according to it is requested come information judge syntagmatic keyword and corresponding statistical information
Judge whether be suspicious user.Such as the red fox browser client access exception of certain IP user
Injection readjustment: it is implanted into JS (javascript description language, a kind of computer script language) by force and passes through to the page
Readjustment judges user's exception, and discovery has individual user's visitation frequency exception, passes through js returning to injection JS in request page content
Readjustment observation user.Identification field then is added in result if it is interface to return for client.
Cookie also can be injected if it is the page: updating user local cookie, judge whether return value changes.
Shielding includes:
It is counted using sliding window and provides specific characteristic user largest request amount per second, be more than refusal.
As required whether with, feedback judges that (such as certain IP is logged in user behavior with the non-of red fox browser to detection type injection script
User needs speed limit).
Improper user is refused completely fall and be arranged its how long after can normally access.
Known abnormal request is focused on a slow server.So that services is normal
It is marked when reverse proxy is done in high frequency time user request, system is allowed to carry out service degradation to it, such as only returned
Cache (caching) interior data are returned, latest data is not returned.
Specified IP addition Iptable is thoroughly shielded.
Shielding Cookie is injected to user, requests automatic shield every time.
The present invention is carried out into elaborating below with reference to attached drawing:
Embodiment one
Fig. 1 be the embodiment of the invention provides a kind of flow chart of distributed API service abnormal user frequency recognition methods,
As shown in the figure, comprising:
Step 101, the API request of user is obtained, and access log is recorded in the API request of the user；
Step 102, real-time statistic analysis is carried out according to API request of the access log to the user, confirms the user
When for suspicious user, control centre is notified；
If the user is normal, the API request of the user is transparent to Service Process Server, and historical operation is recorded
Log.Meanwhile notifying control centre.
Optionally, receive can be without any processing after user normally notifies for control centre, or sends transparent transmission instruction to each
A reverse proxy gateway, the subsequent similar legitimate request of transparent transmission.
Step 103, the user identity identification order that control centre generates according to the notice that the user is suspicious user is obtained,
And injection cognizance code is generated according to the order, when the suspicious user sends API request again, by the injection cognizance code
It returns to the suspicious user and obtains the user identity identification order that control centre generates according to the notice that the user is suspicious user；
When finding suspicious user, reverse proxy gateway can be automatically injected cognizance code into returned content, if
The client for not using official to provide can ignore injection data, to realize unauthorized client end shield function, cognizance code
It is divided into more set continuous updatings.
Step 104, it if receiving the feedback information that the suspicious user is directed to the injection cognizance code within the regulation time limit, sends out
User is sent normally to notify control centre；If not receiving the feedback that the suspicious user is directed to the injection cognizance code in the regulation time limit
Information then sends user and notifies control centre extremely；
Such as:
Identify that the user is suspicious user；User identity identification code is injected to interface return value；If interface returns to user
Identity and information then record log and exclude suspicion；Suspect to be that unofficial provide services user if interface is without subsequent operation；
Step 105, acquisition control centre normally notifies according to the user or what user's notice extremely generated is directed to the use
The process instruction at family, and by the user information and the storage of corresponding process instruction to processing log, and reason instruction pair according to this
The subsequent API request of the user performs corresponding processing.
Above step 103-105 is that the preferred embodiment of the present embodiment will pass through if specified protocol is not used in user
User IP and its all features that can be got are screened.That is, different requests sends terminal, corresponding different identification side
Case.
Preferably, before the API request progress real-time statistic analysis to the user, comprising:
The process instruction of caching is searched, confirms and the user information is not present in process instruction.
If it exists, then by the user information and the storage of corresponding process instruction to processing log, and reason instruction according to this
The subsequent API request of the user is performed corresponding processing.
That is, the request similar with passing statistical value will be ignored, identification function will not be triggered.
It is preferably, described that real-time statistic analysis is carried out according to API request of the access log to the user, comprising:
The access log is pushed to off-line analysis service node according to IP consistency Hash hash；The off-line analysis service
It include historical operation log in node；The historical operation log includes the record of the various operations carried out to user's API request；
Analysis Service node analyzes log, splits out feature, generates frequency statistics data for feature.
Simultaneously frequency statistics data also can be periodically saved in Distributed Storage server, for subsequent request
With reference to.
The progress time divides bucket, adds up request number of times, requesting interval and highest QPS in time bucket.
The present embodiment is preferred, is used as a point of bucket within every 5 minutes, carries out to all user characteristics keywords that can be got
Statistics.
The present embodiment is preferred, carries out off-line operation by big data Hadoop+Hive, calculates historical time bucket range
Interior frequency statistics data, and the internal buffering method that provides inquires foundation；Or it will statistics when current service counts current results
As a result frequency statistics data are generated by merging after dump to local.
The historical operation log of the off-line analysis service node is transferred by script engine, and according to the historical operation log
And this access log carries out real-time statistic analysis.
Preferably, script engine uses Lua script in this implementation.
The present embodiment is optional, and individual interfaces can judge the illegal situation of user by interface return value.As certain user is crazy
Mad to log in repeatedly (as being more than twice), each interface returns to refusal login prompt, and status code is that (value is about to agreement -100000
Definite value, or other values) it can then cause special statistics to be paid close attention to.
Lua script also passes through built-in imbedding function, obtain the storage of Distributed Storage server memory with the user API
Request corresponding frequency statistics data.
Preferably, real-time statistic analysis is carried out by script engine embedded mode and carries out interacting message with control centre.
Lua script can send push to control centre by Embedding function or directly return to judging result；
Preferably, in step 105, acquisition control centre normally notifies according to the user or user notifies generation extremely
For the process instruction of the suspicious user, specifically include:
The process instruction of the artificial dynamic setting of control centre's transmission is directly acquired by the script engine embedded mode.
The present embodiment uses script engine embedded mode, flexibly heat can update user identity knowledge by control centre interface
Other command script, manually can also dynamically set shielding rules.
The process instruction includes shielding rules, is sent to each reverse proxy gateway by control centre；The present embodiment is excellent
Choosing, process instruction is sent by way of broadcast.Shielding rules include shielding, speed limit or transparent transmission；
Wherein, the mode of shielding includes:
Nominal key shielding；
Composite key shielding: such as IP, User-agent, Agent IP, UID, specified network address；
Sending out notice mode notifies administrator, by administrator's decision；
Permanent shielding, stipulated time inner shield；
It forwards the request to low with server cluster；
Additional request service degradation mark is to rear end.
Shielding rank and mode are determined by Lua script.
The present embodiment is preferred, and each reverse proxy gateway shields the class of subsequent abnormal user according to the shielding rules
The similar illegal request of the abnormal user for needing further to judge like illegal request or speed limit or transparent transmission normal users it is legal
Request.
Abnormality code can also be returned by interface as statistics according to identification illegal user.
The present embodiment uses script engine embedded mode, thus flexibly heat can update identification rule by control centre interface
Then script manually can also dynamically set shielding rules.The present embodiment cooperates real-time calculation and analysis user to ask by reverse proxy
It asks, the frequency of abnormal user can be made to identify more efficient more complete.
Embodiment two
Fig. 2 is that the embodiment of the invention provides a kind of distributed API service abnormal user frequencys to identify reverse proxy service
The structure chart of gateway, as shown in the figure, comprising:
Access day is recorded for obtaining the API request of user, and by the API request of the user in request unit 201
Will；
Real-time statistic analysis unit 202, for carrying out real-time statistics according to API request of the access log to the user
Analysis when confirming that the user is suspicious user, notifies control centre；
If the user is normal, the API request of the user is transparent to Service Process Server, and historical operation is recorded
Log.Meanwhile notifying control centre.
Optionally, receive can be without any processing after user normally notifies for control centre, or sends transparent transmission instruction to each
A reverse proxy gateway, the subsequent similar legitimate request of transparent transmission.
User identification unit 203, the user generated for obtaining control centre according to the notice that the user is suspicious user
Identification order, and injection cognizance code is generated according to the order, when the suspicious user sends API request again, by institute
It states injection cognizance code and returns to the use that suspicious user acquisition control centre generates according to the notice that the user is suspicious user
Family identification order；
When finding suspicious user, reverse proxy gateway can be automatically injected cognizance code into returned content, if do not had
There is the client provided using official that can ignore injection data, thus realize unauthorized client end shield function, cognizance code point
To cover continuous updating more.
Feedback analysis unit 204 receives the suspicious user for the injection cognizance code for working as within the regulation time limit
When feedback information, sends user and normally notify control centre；The injection is directed to when not receiving the suspicious user in the regulation time limit
When the feedback information of cognizance code, sends user and notify control centre extremely；
Such as:
Identify that the user is suspicious user；User identity identification code is injected to interface return value；If interface returns to user
Identity and information then record log and exclude suspicion；Suspect to be that unofficial provide services user if interface is without subsequent operation；
Requesting processing 205, normally notifies according to the user for obtaining control centre or user's notice life extremely
At the process instruction for the user, by the user information and the storage of corresponding process instruction to processing log, and according to this
Process instruction performs corresponding processing the subsequent API request of the user.
Preferably, further includes:
Pretreatment unit confirms and the user information is not present in process instruction for searching the process instruction of caching.
, and if it exists, respective handling is then carried out according to corresponding process instruction, no longer progress subsequent analysis identification step.
That is, the request similar with passing statistical value will be ignored, identification function will not be triggered.
Preferably, the real-time statistic analysis unit 202 includes:
Log pushes subelement, for the access log to be pushed to off-line analysis service according to IP consistency Hash hash
Node；It include historical operation log in the off-line analysis service node；The historical operation log include to user's API request into
The record of capable various operations；
Analysis Service node analyzes log, splits out feature, generates frequency statistics data for feature.
Simultaneously frequency statistics data also can be periodically saved in Distributed Storage server, for subsequent request
With reference to.
The progress time divides bucket, adds up request number of times, requesting interval and highest QPS in time bucket.
The present embodiment is preferred, is used as a point of bucket within every 5 minutes, carries out to all user characteristics keywords that can be got
Statistics.
The present embodiment is preferred, carries out off-line operation by big data Hadoop+Hive, calculates historical time bucket range
Interior frequency statistics data, and the internal buffering method that provides inquires foundation；Or it will statistics when current service counts current results
As a result frequency statistics data are generated by merging after dump to local.
Subelement is statisticallyd analyze, for transferring the historical operation log of the off-line analysis service node by script engine,
And real-time statistic analysis is carried out according to the historical operation log and this access log.
Preferably, the present embodiment script engine uses Lua script.
The present embodiment is optional, and individual interfaces can judge the illegal situation of user by interface return value.As certain user is crazy
Mad to log in repeatedly (as being more than twice), each interface returns to refusal login prompt, and status code is that (value is about to agreement -100000
Definite value, or other values) it can then cause special statistics to be paid close attention to.
Preferably, which carries out real-time statistic analysis and and control centre by script engine embedded mode
Carry out interacting message.
Lua script can send push to control centre by Embedding function or directly return to judging result；
Preferably, the requesting processing 205 directly acquires control centre's hair by the script engine embedded mode
The process instruction for the artificial dynamic setting sent.
The present embodiment uses script engine embedded mode, flexibly heat can update user identity knowledge by control centre interface
Other command script, manually can also dynamically set shielding rules.
The process instruction includes shielding rules, is sent to each reverse proxy gateway by control centre；The present embodiment is excellent
Choosing, process instruction is sent by way of broadcast.Shielding rules include shielding, speed limit or transparent transmission；
Wherein, the mode of shielding includes:
Nominal key shielding；
Composite key shielding: such as IP, User-agent, Agent IP, UID, specified network address；
Sending out notice mode notifies administrator, by administrator's decision；
Permanent shielding, stipulated time inner shield；
It forwards the request to low with server cluster；
Additional request service degradation mark is to rear end.
Shielding rank and mode are determined by Lua script.
The present embodiment is preferred, and each reverse proxy gateway shields the class of subsequent abnormal user according to the shielding rules
The similar illegal request of the abnormal user for needing further to judge like illegal request or speed limit or transparent transmission normal users it is legal
Request.
Abnormality code can also be returned by interface as statistics according to identification illegal user.
The present embodiment uses script embedded mode, thus flexibly heat can update recognition rule foot by control centre interface
This, manually can also dynamically set shielding rules.The present embodiment cooperates real-time calculation and analysis user request by reverse proxy, can
Identify the frequency of abnormal user more efficient more complete.
Embodiment three
Fig. 3 is a kind of flow chart of distributed API service abnormal user frequency recognition methods of the present embodiment, as shown,
The following steps are included:
Step 301, reverse proxy gateway receives the API request of user, and access day is recorded in user's API request information
Will；
Step 302, access log is pushed to off-line analysis service node；
Step 303, distributed statistical server carries out data according to the access log in off-line analysis service node real
When analyze；
The present embodiment is preferred, which is offline service device, is asked by calculating previous normal users
Frequency and accumulative request number of times is asked to analyze a few class user reference datas, such as enterprise customer's frequency, ordinary user's frequency, active stage
Between frequency etc..It can analyze out certain rule additionally by analyzing passing processing result and analyzing access log, can also increase
Auxiliary machinery mathematical model of learning carries out off-line learning analysis.
The present embodiment is optional, and individual interfaces can judge the illegal situation of user by interface return value.As certain user is crazy
Mad to log in repeatedly (as being more than twice), each interface returns to refusal login prompt, and status code is that (value is about to agreement -100000
Definite value, or other values) it can then cause special statistics to be paid close attention to.
Step 304, find that the user is suspicious user in real-time analytic process；
The API request of the user is different with historical statistical data frequency.
Step 305, user's API request is sent to control centre；
Step 306, identification user identity order is assigned to each reverse proxy gateway by control centre；
Step 307, each reverse proxy gateway is injected back into cognizance code to the API request of this user again；
The present embodiment is preferred, this user of speed limit while identification instruction issues.
Step 308, customer feedback identification detection information is received, which is added white list.
Here white list identity is equipped with validity period, and the present embodiment is preferably 30 seconds.
The priority level highest of white list, can unconditional transparent transmission, thus effective time must be set.
If the client for not using official to provide can ignore injection data, to realize unauthorized client end shield function
Can, cognizance code is divided into more set continuous updatings.
Step 309, processing log is recorded simultaneously in the normal information of the user, and notifies control centre.
Step 310, control centre's broadcast transparent transmission instruction.
The present embodiment is optional, and control centre can analyze whether user is normal users according to these data again.
The present embodiment uses script embedded mode, thus flexibly heat can update recognition rule foot by control centre interface
This, manually can also dynamically set shielding rules.The present embodiment cooperates real-time calculation and analysis user request by reverse proxy, can
Identify the frequency of abnormal user more efficient more complete.
Those skilled in the art will also be appreciated that the various illustrative components, blocks that the embodiment of the present invention is listed
(illustrative logical block), unit and step can by electronic hardware, computer software, or both knot
Conjunction is realized.For the replaceability (interchangeability) for clearly showing that hardware and software, above-mentioned various explanations
Property component (illustrative components), unit and step universally describe their function.Such function
It can be that the design requirement for depending on specific application and whole system is realized by hardware or software.Those skilled in the art
Can be can be used by various methods and realize the function, but this realization is understood not to for every kind of specific application
Range beyond protection of the embodiment of the present invention.
Various illustrative logical blocks or unit described in the embodiment of the present invention can by general processor,
Digital signal processor, specific integrated circuit (ASIC), field programmable gate array or other programmable logic devices, discrete gate
Or transistor logic, discrete hardware components or above-mentioned any combination of design carry out implementation or operation described function.General place
Managing device can be microprocessor, and optionally, which may be any traditional processor, controller, microcontroller
Device or state machine.Processor can also be realized by the combination of computing device, such as digital signal processor and microprocessor,
Multi-microprocessor, one or more microprocessors combine a digital signal processor core or any other like configuration
To realize.
The step of method described in the embodiment of the present invention or algorithm can be directly embedded into hardware, processor execute it is soft
The combination of part module or the two.Software module can store in RAM memory, flash memory, ROM memory, EPROM storage
Other any form of storaging mediums in device, eeprom memory, register, hard disk, moveable magnetic disc, CD-ROM or this field
In.Illustratively, storaging medium can be connect with processor, so that processor can read information from storaging medium, and
It can be to storaging medium stored and written information.Optionally, storaging medium can also be integrated into the processor.Processor and storaging medium can
To be set in asic, ASIC be can be set in user terminal.Optionally, processor and storaging medium also can be set in
In different components in the terminal of family.
In one or more exemplary designs, above-mentioned function described in the embodiment of the present invention can be in hardware, soft
Part, firmware or any combination of this three are realized.If realized in software, these functions be can store and computer-readable
On medium, or it is transferred on a computer readable medium in the form of one or more instructions or code forms.Computer readable medium includes electricity
Brain storaging medium and convenient for so that computer program is allowed to be transferred to from a place telecommunication media in other places.Storaging medium can be with
It is that any general or special computer can be with the useable medium of access.For example, such computer readable media may include but
It is not limited to RAM, ROM, EEPROM, CD-ROM or other optical disc storages, disk storage or other magnetic storage devices or other
What can be used for carry or store with instruct or data structure and it is other can be by general or special computer or general or specially treated
The medium of the program code of device reading form.In addition, any connection can be properly termed computer readable medium, example
Such as, if software is to pass through a coaxial cable, fiber optic cables, double from a web-site, server or other remote resources
Twisted wire, Digital Subscriber Line (DSL) are defined with being also contained in for the wireless way for transmitting such as example infrared, wireless and microwave
In computer readable medium.The disk (disk) and disk (disc) includes compress disk, radium-shine disk, CD, DVD, floppy disk
And Blu-ray Disc, disk is usually with magnetic replicate data, and disk usually carries out optically replicated data with laser.Combinations of the above
Also it may be embodied in computer readable medium.
Above-described specific embodiment has carried out further the purpose of the present invention, technical scheme and beneficial effects
It is described in detail, it should be understood that being not intended to limit the present invention the foregoing is merely a specific embodiment of the invention
Protection scope, all within the spirits and principles of the present invention, any modification, equivalent substitution, improvement and etc. done should all include
Within protection scope of the present invention.
Claims (10)
1. a kind of distribution API service abnormal user frequency recognition methods characterized by comprising
The API request of user is obtained, and access log is recorded in the API request of the user；
Real-time statistic analysis is carried out according to API request of the access log to the user, when confirming that the user is suspicious user,
Notify control centre；
The user identity identification order that control centre generates according to the notice that the user is suspicious user is obtained, and according to the order
It generates injection cognizance code and it is suspicious that the injection cognizance code is returned to this when the suspicious user sends API request again
User；
If receiving the feedback information that the suspicious user is directed to the injection cognizance code within the regulation time limit, reverse proxy service network
It closes transmission user and normally notifies control centre；If not receiving the suspicious user within the regulation time limit for the injection cognizance code
Feedback information, then reverse proxy gateway sends user and notifies control centre extremely；
Obtain the processing for the suspicious user that control centre normally notifies according to the user or user's notice extremely generates
After instruction stores the user information and corresponding process instruction to processing log, and reason instruction is to the user according to this
Continuous API request performs corresponding processing.
2. the method according to claim 1, wherein the API request to the user carries out real-time statistics
Before analysis, comprising:
The process instruction of caching is searched, confirms and the user information is not present in process instruction.
3. the method according to claim 1, wherein described ask according to API of the access log to the user
Seek carry out real-time statistic analysis, comprising:
The access log is pushed to off-line analysis service node according to IP consistency Hash hash；The off-line analysis service node
In include historical operation log；The historical operation log includes the record of the various operations carried out to user's API request；
The historical operation log of the off-line analysis service node is transferred by script engine, and according to the historical operation log and originally
Secondary access log carries out real-time statistic analysis.
4. according to the method described in claim 3, it is characterized in that, carrying out real-time statistic analysis by script engine embedded mode
And interacting message is carried out with control centre.
5. according to the method described in claim 4, it is characterized in that, the acquisition control centre normally notifies according to the user
Or the process instruction for the suspicious user that user's notice extremely generates, comprising:
The process instruction of the artificial dynamic setting of control centre's transmission is directly acquired by the script engine embedded mode.
6. a kind of distribution API service abnormal user frequency identifies reverse proxy gateway characterized by comprising
Access log is recorded for obtaining the API request of user, and by the API request of the user in request unit；
Real-time statistic analysis unit, for carrying out real-time statistic analysis according to API request of the access log to the user, really
Recognize the user be suspicious user when, notify control centre；
User identification unit, the user identity identification generated for obtaining control centre according to the notice that the user is suspicious user
Order, and injection cognizance code is generated according to the order, when the suspicious user sends API request again, the injection is known
Other code returns to the suspicious user；
Feedback analysis unit, for working as the feedback information for receiving the suspicious user within the regulation time limit and being directed to the injection cognizance code
When, it sends user and normally notifies control centre；Generation is identified for the injection when not receiving the suspicious user within the regulation time limit
When the feedback information of code, sends user and notify control centre extremely；
Requesting processing, normally notifies according to the user for obtaining control centre or what user's notice extremely generated is directed to
The process instruction of the suspicious user by the user information and the storage of corresponding process instruction to processing log, and is managed according to this
Instruction performs corresponding processing the subsequent API request of the user.
7. reverse proxy gateway according to claim 6, which is characterized in that further include:
Pretreatment unit confirms and the user information is not present in process instruction for searching the process instruction of caching.
8. reverse proxy gateway according to claim 6, which is characterized in that the real-time statistic analysis unit packet
It includes:
Log pushes subelement, for the access log to be pushed to off-line analysis service section according to IP consistency Hash hash
Point；It include historical operation log in the off-line analysis service node；The historical operation log includes carrying out to user's API request
Various operations record；
Subelement is statisticallyd analyze, for transferring the historical operation log of the off-line analysis service node, and root by script engine
Real-time statistic analysis is carried out according to the historical operation log and this access log.
9. reverse proxy gateway according to claim 8, which is characterized in that the agency service gateway is drawn by script
Embedded mode is held up to carry out real-time statistic analysis and carry out interacting message with control centre.
10. reverse proxy gateway according to claim 9, which is characterized in that the requesting processing passes through institute
State the process instruction that script engine embedded mode directly acquires the artificial dynamic setting of control centre's transmission.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610195972.XA CN105827608B (en) | 2016-03-31 | 2016-03-31 | Distributed API service abnormal user identifying and analyzing method and reverse proxy gateway |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610195972.XA CN105827608B (en) | 2016-03-31 | 2016-03-31 | Distributed API service abnormal user identifying and analyzing method and reverse proxy gateway |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105827608A CN105827608A (en) | 2016-08-03 |
| CN105827608B true CN105827608B (en) | 2019-02-12 |
Family
ID=56525470
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610195972.XA Active CN105827608B (en) | 2016-03-31 | 2016-03-31 | Distributed API service abnormal user identifying and analyzing method and reverse proxy gateway |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105827608B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845208B (en) * | 2017-02-13 | 2020-04-24 | 北京安云世纪科技有限公司 | Abnormal application control method and device and terminal equipment |
| CN107493302A (en) * | 2017-09-28 | 2017-12-19 | 北京云衢科技有限公司 | A kind of user information acquiring method and device |
| CN108197444A (en) | 2018-01-23 | 2018-06-22 | 北京百度网讯科技有限公司 | Right management method, device and server under a kind of distributed environment |
| CN109067728B (en) * | 2018-07-25 | 2021-08-27 | 苏州科达科技股份有限公司 | Access control method and device for application program interface, server and storage medium |
| CN109325691B (en) * | 2018-09-27 | 2020-10-16 | 上海观安信息技术股份有限公司 | Abnormal behavior analysis method, electronic device and computer program product |
| CN111162932A (en) * | 2019-12-12 | 2020-05-15 | 苏州博纳讯动软件有限公司 | API gateway monitoring method based on log analysis |
| CN111212038B (en) * | 2019-12-23 | 2021-02-09 | 江苏国泰新点软件有限公司 | Open data API gateway system based on big data artificial intelligence |
| CN111930700A (en) * | 2020-07-13 | 2020-11-13 | 车智互联（北京）科技有限公司 | Distributed log processing method, server, system and computing equipment |
| CN113079176B (en) * | 2021-04-14 | 2022-04-05 | 西安交通大学 | High-speed network flow abnormity detection system suitable for mass data |
| US20230102292A1 (en) * | 2021-09-29 | 2023-03-30 | Moesif, Inc. | Secure management of application programming interface (api) request information |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102946320A (en) * | 2012-10-10 | 2013-02-27 | 北京邮电大学 | Distributed supervision method and system for user behavior log forecasting network |
| CN103475637A (en) * | 2013-04-24 | 2013-12-25 | 携程计算机技术（上海）有限公司 | Network access control method and system based on IP access behaviors |
| CN104065657A (en) * | 2014-06-26 | 2014-09-24 | 北京思特奇信息技术股份有限公司 | Method for dynamically controlling user behavior based on IP access and system thereof |
| US9060012B2 (en) * | 2007-09-26 | 2015-06-16 | The 41St Parameter, Inc. | Methods and apparatus for detecting fraud with time based computer tags |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101620630A (en) * | 2009-06-29 | 2010-01-06 | 北京黑米天成科技有限公司 | WEB action data collecting model based on JS script |
| CN103778052B (en) * | 2012-10-17 | 2017-12-19 | 百度在线网络技术（北京）有限公司 | A kind of method of testing and device of front end javascript codes |
-
2016
- 2016-03-31 CN CN201610195972.XA patent/CN105827608B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9060012B2 (en) * | 2007-09-26 | 2015-06-16 | The 41St Parameter, Inc. | Methods and apparatus for detecting fraud with time based computer tags |
| CN102946320A (en) * | 2012-10-10 | 2013-02-27 | 北京邮电大学 | Distributed supervision method and system for user behavior log forecasting network |
| CN103475637A (en) * | 2013-04-24 | 2013-12-25 | 携程计算机技术（上海）有限公司 | Network access control method and system based on IP access behaviors |
| CN104065657A (en) * | 2014-06-26 | 2014-09-24 | 北京思特奇信息技术股份有限公司 | Method for dynamically controlling user behavior based on IP access and system thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105827608A (en) | 2016-08-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105827608B (en) | Distributed API service abnormal user identifying and analyzing method and reverse proxy gateway | |
| US11494460B2 (en) | Internet-based proxy service to modify internet responses | |
| US10855798B2 (en) | Internet-based proxy service for responding to server offline errors | |
| CN107690640B (en) | Method and computing device for managing website tracking | |
| CN105306465B (en) | Web portal security accesses implementation method and device | |
| US20090055929A1 (en) | Local Domain Name Service System and Method for Providing Service Using Domain Name Service System |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |