CN109039817A - A kind of information processing method and device for traffic monitoring - Google Patents
A kind of information processing method and device for traffic monitoring Download PDFInfo
- Publication number
- CN109039817A CN109039817A CN201810878320.5A CN201810878320A CN109039817A CN 109039817 A CN109039817 A CN 109039817A CN 201810878320 A CN201810878320 A CN 201810878320A CN 109039817 A CN109039817 A CN 109039817A
- Authority
- CN
- China
- Prior art keywords
- information
- recalls
- recalls information
- amount
- service interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
- H04L43/106—Active monitoring, e.g. heartbeat, ping or trace-route using time related information in packets, e.g. by adding timestamps
Abstract
The invention discloses a kind of information processing methods and device for traffic monitoring, are related to field of computer technology.One specific embodiment of this method includes: the call request for receiving multiple called sides to service interface, counts each called side in current period to the calling amount of service interface, to generate recalls information corresponding with each called side；Processing is grouped to all recalls informations in current period, is stored in recalls information to the storage side after asynchronous transmission grouping.The embodiment summarizes by the periodicity to recalls information and is grouped Asynchronous Transfer Mode, solves efficiency when flow is higher in large-scale cluster, compared with existing way, the utilization rate of hardware and Internet resources can be greatly reduced, while ensure that higher real-time.
Description
Technical field
The present invention relates to field of computer technology more particularly to a kind of information processing methods and dress for traffic monitoring
It sets.
Background technique
In large-scale cluster application, how to monitor and manage traffic requests is often a critically important task.When
The preceding monitoring for flow mainly carries out burying an acquisition by the service provider in flow, records recalls information each time,
And be stored in the form of log on every machine, center processor is transferred to by asynchronous mode and is summarized.
But in the implementation of the present invention, at least there are the following problems for inventor's discovery prior art:
Existing way can preferably handle medium scale cluster, but the expansion exponential with internet scale, huge
For big cluster often up to hundreds of millions grades of flow, this method called and transmitted each time that records can greatly waste server
And Internet resources, while the timeliness monitored also cannot be guaranteed, and often have the delay of minute grade.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of information processing method and device for traffic monitoring, at least can
Enough solve the problem of that hardware in the prior art and the excessively high traffic monitoring timeliness of network resource utilization cannot be guaranteed.
To achieve the above object, according to an aspect of an embodiment of the present invention, a kind of letter for traffic monitoring is provided
Cease processing method, comprising:
Call request of multiple called sides to service interface is received, each called side is to the service in statistics current period
The calling amount of interface, to generate recalls information corresponding with each called side；
Processing is grouped to all recalls informations in the current period, the recalls information after asynchronous transmission grouping is extremely
It is stored in storage side.
Optionally, the recalls information includes at least called side, service interface, calling amount and current period affiliated time
The mapping relations of stamp；
The method also includes:
According to predetermined value conversion regime, convert respectively the called side in the recalls information, the service interface,
The calling amount and the timestamp are corresponding numerical information, and generate corresponding numerical value recalls information；Or
According to predetermined hash function conversion regime, the called side in the recalls information, the service are converted respectively
Interface, the call request amount and the timestamp are corresponding Hash functional value, and generate corresponding hash function and call
Information.
Optionally, all recalls informations in the current period are grouped processing, comprising:
According to the predetermined information amount in the quantity of all recalls informations and every group, the grouping that is created needed for determining with
And the recalls information amount in every group；Or
According to the quantity and predetermined packet amount of all recalls informations, the recalls information amount in every group is determined.
Optionally, after being stored in recalls information to the storage side after the asynchronous transmission grouping, further includes:
According at least to the called side, the service interface, history recalls information corresponding with the recalls information is determined；
And
It is stabbed according to the affiliated timestamp of the current period and the historical time of the history recalls information, statistics is predetermined to be gone through
History recalls information in history duration, to generate corresponding calling total information.
To achieve the above object, according to another aspect of an embodiment of the present invention, a kind of letter for traffic monitoring is provided
Cease processing unit, comprising:
Acquisition module counts each tune in current period for receiving call request of multiple called sides to service interface
With side to the calling amount of the service interface, to generate recalls information corresponding with each called side；
Transmission module, for being grouped processing, asynchronous transmission grouping to all recalls informations in the current period
It is stored in recalls information afterwards to storage side.
Optionally, the recalls information includes at least called side, service interface, calling amount and current period affiliated time
The mapping relations of stamp；
Described device further includes info conversion module, is used for:
According to predetermined value conversion regime, convert respectively the called side in the recalls information, the service interface,
The calling amount and the timestamp are corresponding numerical information, and generate corresponding numerical value recalls information；Or
According to predetermined hash function conversion regime, the called side in the recalls information, the service are converted respectively
Interface, the call request amount and the timestamp are corresponding Hash functional value, and generate corresponding hash function and call
Information.
Optionally, the transmission module, is used for:
According to the predetermined information amount in the quantity of all recalls informations and every group, the grouping that is created needed for determining with
And the recalls information amount in every group；Or
According to the quantity and predetermined packet amount of all recalls informations, the recalls information amount in every group is determined.
Optionally, further include Information Statistics module, be used for:
According at least to the called side, the service interface, history recalls information corresponding with the recalls information is determined；
And
It is stabbed according to the affiliated timestamp of the current period and the historical time of the history recalls information, statistics is predetermined to be gone through
History recalls information in history duration, to generate corresponding calling total information.
To achieve the above object, according to an embodiment of the present invention in another aspect, providing a kind of letter for traffic monitoring
Breath processing electronic equipment.
The electronic equipment of the embodiment of the present invention includes: one or more processors；Storage device, for storing one or more
A program, when one or more of programs are executed by one or more of processors, so that one or more of processing
Device realizes any of the above-described information processing method for traffic monitoring.
To achieve the above object, according to an embodiment of the present invention in another aspect, provide a kind of computer-readable medium,
On be stored with computer program, any of the above-described information for traffic monitoring is realized when described program is executed by processor
Processing method.
The scheme of the offer according to the present invention, one embodiment in foregoing invention have the following advantages that or beneficial to effects
Fruit: summarizing by the periodicity to recalls information and be grouped Asynchronous Transfer Mode, and it is higher to solve flow in large-scale cluster
When efficiency can greatly reduce the utilization rate of hardware and Internet resources compared with traditional approach, while ensure that higher
Real-time.
Further effect possessed by above-mentioned non-usual optional way adds hereinafter in conjunction with specific embodiment
With explanation.
Detailed description of the invention
Attached drawing for a better understanding of the present invention, does not constitute an undue limitation on the present invention.Wherein:
Fig. 1 is a kind of main flow signal of information processing method for traffic monitoring according to an embodiment of the present invention
Figure；
Fig. 2 is different recalls information form schematic diagrames；
Fig. 3 is four terminal schematic diagrames according to the present invention；
Fig. 4 is a kind of main modular signal of information processing unit for traffic monitoring according to an embodiment of the present invention
Figure；
Fig. 5 is that the embodiment of the present invention can be applied to exemplary system architecture figure therein；
Fig. 6 is adapted for the structural representation for realizing the mobile device of the embodiment of the present invention or the computer system of server
Figure.
Specific embodiment
Below in conjunction with attached drawing, an exemplary embodiment of the present invention will be described, including the various of the embodiment of the present invention
Details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize
It arrives, it can be with various changes and modifications are made to the embodiments described herein, without departing from scope and spirit of the present invention.Together
Sample, for clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.
It should be noted that present invention can be suitably applied to all kinds of back-end software systems such as electric business, social networks, finance, with right
Interface service request flow therein is monitored.
The service provider told about is the service provider in application system, and application and development both faces towards serviceization at present,
I.e. internal is all service interface one by one.Therefore service monitoring of the invention refers to being monitored each service interface,
Provider is also not movement, connection, telecommunications network provider.
For example, the transaction department of electric business platform interior provides order placement service, order placement service includes to place an order and inquire order
Etc. functions, the APP (Application, application software) of the called side such as platform or website, by calling the service interface i.e.
The operation such as order can be placed an order and be inquired, transaction department here is service provider.
The data storage end of the invention, including but is not limited to is Redis cluster, can also be any other distributed collection
Group, the embodiment of the present invention are illustrated by taking Redis cluster as an example.The flow that the present invention is told about, can be the volume of the flow of passengers, data flow
Amount etc..In fact, traffic monitoring is can to monitor each traffic source situation of each service interface.
The present invention uses distributed processing mode, when data scale increases, will not generate hot spot bottleneck, therefore expansible
It is monitored to hundreds of millions grades of flows, present invention is generally directed to hundreds of millions grades of flows to be monitored analysis processing.
Word according to the present invention is explained as follows:
It buries a little: by the specific position executed in program, obtaining current operational information actively to acquire information.
Filter: by public process code, the program function handled before and after specific process.
A kind of Redis cluster: the key-value data-base cluster in advanced memory.
Pipeline: multiple requests can be merged into once-through operation by a kind of batch processing method.
Redis Incrby order: by the number stored in key plus specified incremental value.If key is not present,
The value of key can first be initialized to 0, then execute Incrby order again.If type or character string type of the value comprising mistake
Value cannot be expressed as number, then return a mistake.
Referring to Fig. 1, thus it is shown that a kind of master of the information processing method for traffic monitoring provided in an embodiment of the present invention
Flow chart is wanted, is included the following steps:
S101: call request of multiple called sides to service interface is received, each called side is to institute in statistics current period
The calling amount of service interface is stated, to generate recalls information corresponding with each called side；
S102: processing is grouped to all recalls informations in the current period, the calling after asynchronous transmission grouping
It is stored in information to storage side.
In above embodiment, for step S101, the server-side that the present invention is told about, be electric business platform, social networks,
The application module of the service systems such as the website the APP/ such as finance passes through POM (Project Object Model, item objects model)
Dependence is integrated in all service side's applications.POM is the content in JAVA system maven component, can be convenient service using POM
End is integrated into collection module.
The flow call request that called side is initiated must include some subsequent key messages for needing to use, for example, clothes
Business information (such as interface name, method name), called side application message and timestamp.
For the processing of flow call request, can be temporarily stored in by the filter program configured in service side
In local memory.Here filter is a kind of general design method, is considered as acquiring each service interface and is received
Call request inlet module, for adding new function to service system, what the present invention added is acquisition function, but not
Influence former processing function.Flow call request can also be recorded, simply to do interim storage for counter in memory.
For the called side in request, it is different from service side, specifically can be some APP/ web station system, red packet sends system
System etc..
In usual distributed system, source IP (the Internet Protocol of called side only can be simply got
Address), but IP acquired in this kind of mode be it is non-intuitive, can not also determine it is which called side is applied, it is even more impossible to will
The same source application gathers, it is therefore desirable to it is converted by the function of internal " IP is transformed into Apply Names ", with
Obtain specific called side application.
It should be noted that the determination for calling source, i.e. called side application is a critical function of the invention,
And it is different from a distinctive points of other current monitor modes.When encountering a large amount of flows, it can clearly know that flow calls
Which calls source from, and then takes reply current limiting measures or exchanges and processed with source, for example, increasing blacklist；
Determination for calling amount can automatically create a monitoring main thread when service side's application starts, by fixed
When device periodically (for example, 4s) executive plan task, that is, count the flow call request amount that each called side is initiated.It is counted
Numerical value be probably in hundreds of millions grades of systems million numerical value.
For example, an atomic counters are created in service side, whenever receiving what called side sent the service interface
When measuring call request, counter increment operator is carried out.
For recalls information, the corresponding relationship containing service interface, called side, calling amount and timestamp can wrap.Example
Such as, a concurrent Map of static state is created in the application example of each service side, by each system name-interface name-method name-
Called side application name-timestamp is the calling amount counted as key, respective value value, generates the tune of key-value format
With information, the structure is more convenient simple.
Wherein, monitoring service can be supplied to many service systems and use, and each service system has the system of oneself
Name, can not interfere with each other when recording information in this way；Such as transaction system, marketing system, ad system of enterprises etc. is all
It is system name；
Method name refers to the classification of interface next stage, is considered as sub-interface or service；For example, transaction system-order
Service-creation order is system name, interface name and method name respectively.
For step S102, for the transmission of gained key-value format recalls information, for Redis cluster, actually
It is Incrby operation one by one, i.e., by the numerical value stored in key plus specified incremental value, for example, every 4 seconds primary calling
Total amount.In addition, operation is atomicity for Redis Incrby, when a large amount of request amounts carry out this operation, also can guarantee
The numerical value added up is correctly, not have conflict or loss.
But the above operation will cause a large amount of Redis and concurrently be written, for example, can from entire Redis cluster monitoring
To the primary very high TPS wave crest of 4s.For the situation, to reduce resource occupation to the greatest extent and reducing computational complexity, the present invention
The existing mode for selecting local log recording to retransmit has been abandoned, will periodically have been united directly in the asynchronous thread of consumption queue
The information of meter is sent in distributed type assemblies；Wherein, consumption queue is in the consumption terminal of asynchronous queue in system, i.e. reading queue
Processing.
It, can be according to the execution periodic transfer of same time length for gained recalls information in each period.For example, when reaching
When stabbing (for example, 4s) to execution cycle time, the key-value recalls information of statistics all in current period is transferred to storage
In device.
It should be noted that needing to empty the numerical value in atomic counters, otherwise after sending recalls information and finishing
The numerical value of the current period counted will be accumulated in the next period, and timer is not no timestamp, be to execute
The sending time in period is stabbed.
For the timeliness for guaranteeing multiplexed transport, the timestamp of duty cycle needs constantly adjustment deviation, for example, transmission cycle
For 4s, need to strictly maintain per minute 0 second, 4 seconds, timestamp executes as 56 seconds 8 seconds ....
Transmission for key-value recalls information can be sent in asynchronous queue, be passed in an asynchronous manner
Defeated, compared with the mode that is locally stored, which is highly efficient, stablizes.
It is executed for example, something sorts according to 1,2,3 sequence, if but at this time between 4 insertions 1 and 2, then 2/3 just
Need to postpone and wait, use asynchronous system at this time, then original 1,2,3 can still operate normally, 4 by an independent space at
Reason.
In same period, possible gained recalls information amount is more, and for this kind of situation, the present invention abandons history and passes one by one
Defeated mode is carried out bulk transfer to it, is accelerated using pipeline mechanism to it:
1) number of packet is fixed, for example, service side has received 1100 key-value pairs in current period, if 5 groups in total,
It so needs to carry out this 1100 key-value pairs Homogenization Treatments, has 1100/5=220 key-value pair in every group；
But if 1100 key-value pairs, that is, indicate 1 surplus, which can arbitrarily distribute to one of them i.e.
It can；
When number of packet is only 1, that is, indicates that all key-value pairs are all distributed into a group and carry out a bulk transfer.
2) information content in every group is fixed, for example, being equally 1100 key-value pairs, if the key-value pair in every group is fixed as
200, then available 5 groups and 1 group comprising 100 key-value pairs comprising 200 key-value pairs；
For 100 key-value pairs in a remaining group, before indicating that next period arrives, and not new information is raw
At, but be no more than group in information content the case where；
When the information content upper limit is infinitary value in group, equally indicates that all key-value pairs are all distributed and carry out one into a group
Secondary bulk transfer.
It should be noted that the delay of this grouping and batch transmission is usually second two-stage, system tolerance interval it
It is interior, and delay can be further reduced by configuration parameter, but delay is inevitable.
By asynchronous queue, the mode of bulk transfer recalls information need to will be collected by queue task and a batch for accumulating is appointed
Business is sent, we are the poll and time-out detection by queue, and the primary transmission of task batch of team's time-out out will not occur.
Service side's performance and fault-tolerance can be improved in Asynchronous Transfer Mode.Wherein, performance raising refers to asynchronous bulk transfer
Speed faster；Fault-tolerance improve refer to it is asynchronous after do not influence the function of former interface service, its will not be made slack-off or malfunctioned.
And operating each time for existing Redis can all undergo the calling of multiple I/O systems to switch, but using pipeline into
Row batch operation, can greatly reduce internal I/O system call total degree, save every time individually call when RTT (network come
Return delay).For example, the information for originally needing to transmit one by one, now disposable transmission.It is whole after bulk transfer it has been confirmed by experiments that
Body performance improves 10 times or so.
Other than filter, starting Bean can also be configured at provider, and (entity object class can be grasped database
It is abstracted into the operation to class), summarize and send to complete above- mentioned information amount, can be functional with bean.
The acquisition and statistic processes of the above-mentioned call request for called side, time consuming nature task is less, such as:
1) in service side counter increase, only simple addition cumulative calculation；
2) periodically send and empty the numerical value that counter adds up；
These operations are not related to a large amount of calculating/hard disk/network operations, and internal memory operation is second grade, to guarantee
It precisely completes statistic processes and does not influence former service processes.
For the data storage end, compared with the existing thought being stored in inside service side, the present invention individually extracts it
Processing invades each service system to reduce, keeps the independence and stability of each service system.For distributed type assemblies,
Performance can be improved by more split blade types, i.e., the whole cluster being made of the fragment of multiple redis, such performance is more preferably also more
Safety and stability.
Above-mentioned key-value is logic direct design method the simplest, but most wastes memory space, because of key
It can be very long.
Therefore, it when storing recalls information to the data storage end, or after generating recalls information, before storage, needs
Recalls information is optimized:
1) total interface name, method name etc. are converted into digital ID, such key is more brief；For example, transaction system 1- is ordered
Single service 1- creates order 1, then is converted to 1-1-1, is all Serial No., and every level-one is from increasing order-assigned according to independent
Can, for example, subsequent 1-2-3.
But the numerical value conversion mode increases complexity, because number conversion is not only needed when information stores, subsequent
It is also needed when inquiry；And when needing to increase, modify or delete information, it is also easy error；There are also the realities of number
Border occupied space is still larger.
Accordingly, it is desirable to provide the method more optimized:
2) optimal method is exactly to be stored using this kind of data structure of hash, as included system name-interface in key
Brief timestamp is stored in name-method name-called side, filed, count number is stored in corresponding value, and which itself can
To greatly reduce memory space.
For example, when there is 1,000,000 key-value, key and value have 1,000,000 parts, if using hash structure, only
There is the sub-fraction (timestamp) in value and key to need to deposit 1,000,000 parts, identical prefix only deposits 1 part in key, therefore can save
Save space.
It is demonstrated experimentally that similarly generating 100 acquisition data within the scope of 1 day, the third hash structure is stored than the first
Key-value plain mode reduces the memory space of more than half, referring specifically to shown in Fig. 2.
Management end of the invention, for the application independently disposed.After carrying out structured storage for the flow monitored,
It is subsequent to carry out visualization processing by forms such as charts, or the abnormal flow monitored is reported by timed task
Alert notice, the configuration functions such as degradation current limliting and authorization.
It is subsequent, during reading and generating monitoring chart, it is frequently encountered very big time range, if also making at this time
With second grade data, it will lead to the phenomenon that inquired fruiting quantities are very more, and data can be more than chart pixel capacity.
For this kind of situation, timestamp can summarize recalls information in storage hour hands, such as by the predetermined time
Second grade monitoring data in section carries out minute grade and summarizes, and such larger range can directly inquire minute grade data when inquiring, and should
Process is to operate redis in back-stage management to complete.
Optionally, it if the inquiry of inquiry higher level, carries out summarizing displaying by inquiring all next stage data.It converges
The data granularity of the General Logistics Department is thicker, and occupied space is less.In addition, batch reading can also be carried out with pipeline by reading batch data
Accelerate, principle is similar to batch and stores.
Method provided by the embodiment of the present invention carries out weight by acquisition, transmission and the storing process to institute's monitoring information
New design, it is ingenious be utilized distributed treatment, it is asynchronous the modes such as summarize, monitoring information such as is summarized at the operations and is distributed in every service
It is executed in machine, then the information bulk transfer after summarizing summarizes into distributed type assemblies.In addition, being directed to transmitted letter
Breath carries out compression processing, it is possible to reduce hardware and network resource utilization, while ensure that the higher real-time of overall plan.
The present invention relates to four terminals, called side, server-side, the data storage end and management ends, referring specifically to Fig. 3 institute
Show:
1) called side: for transmitted traffic call request to service side/service interface；
2) traffic service side: call request transmitted by called side is received；
The call request amount received from the called side in predetermined period is counted, recalls information is generated；
Processing is grouped to the recalls information in the period, once or according to grouping, bulk transfer to the data storage end；
3) data storage side: mainly distributed type assemblies, for example, Redis cluster, is used for structured storage data；
4) manager: control is alarmed and managed using monitoring data.
Referring to fig. 4, a kind of master of information processing unit 400 for traffic monitoring provided in an embodiment of the present invention is shown
Want module diagram, comprising:
Acquisition module 401 counts each in current period for receiving call request of multiple called sides to service interface
Called side is to the calling amount of the service interface, to generate recalls information corresponding with each called side；
Transmission module 402, for being grouped processing, asynchronous transmission point to all recalls informations in the current period
It is stored in recalls information to storage side after group.
In implementing device of the present invention, the recalls information includes at least called side, service interface, calling amount and current week
The mapping relations of phase affiliated timestamp；
Described device further includes info conversion module 403 (not marking in figure), is used for:
According to predetermined value conversion regime, convert respectively the called side in the recalls information, the service interface,
The calling amount and the timestamp are corresponding numerical information, and generate corresponding numerical value recalls information；Or
According to predetermined hash function conversion regime, the called side in the recalls information, the service are converted respectively
Interface, the call request amount and the timestamp are corresponding Hash functional value, and generate corresponding hash function and call
Information.
In implementing device of the present invention, the transmission module 402 is used for:
According to the predetermined information amount in the quantity of all recalls informations and every group, the grouping that is created needed for determining with
And the recalls information amount in every group；Or
According to the quantity and predetermined packet amount of all recalls informations, the recalls information amount in every group is determined.
In implementing device of the present invention, further includes Information Statistics module 404 (not marked in figure), is used for:
According at least to the called side, the service interface, history recalls information corresponding with the recalls information is determined；
And
It is stabbed according to the affiliated timestamp of the current period and the historical time of the history recalls information, statistics is predetermined to be gone through
History recalls information in history duration, to generate corresponding calling total information.
In addition, the specific implementation content of the information processing unit for traffic monitoring described in embodiments of the present invention,
It has been described in detail in information processing method described above for traffic monitoring, therefore has no longer illustrated in this duplicate contents.
Fig. 5, which is shown, can supervise using the embodiment of the present invention for the information processing method of traffic monitoring or for flow
The exemplary system architecture 500 of the information processing unit of control.
As shown in figure 5, system architecture 500 may include terminal device 501,502,503, network 504 and server 505
(only example).Network 504 between terminal device 501,502,503 and server 505 to provide Jie of communication link
Matter.Network 504 may include various connection types, such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 501,502,503 and be interacted by network 504 with server 505, to receive or send out
Send message etc..Various Communications service end applications, such as the application of shopping class, net can be installed on terminal device 501,502,503
(merely illustrative) such as the application of page browsing device, searching class application, instant messaging tools, mailbox service end, social platform softwares.
Terminal device 501,502,503 can be the various electronic equipments with display screen and supported web page browsing, packet
Include but be not limited to smart phone, tablet computer, pocket computer on knee and desktop computer etc..
Server 505 can be to provide the server of various services, such as utilize terminal device 501,502,503 to user
The shopping class website browsed provides the back-stage management server (merely illustrative) supported.Back-stage management server can be to reception
To the data such as information query request analyze etc. processing, and by processing result (such as target push information, product letter
Breath -- merely illustrative) feed back to terminal device.
It should be noted that the information processing method provided by the embodiment of the present invention for traffic monitoring is generally by servicing
Device 505 executes, and correspondingly, the information processing unit for traffic monitoring is generally positioned in server 505.
It should be understood that the number of terminal device, network and server in Fig. 5 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.
Below with reference to Fig. 6, it illustrates the computer systems 600 for the terminal device for being suitable for being used to realize the embodiment of the present invention
Structural schematic diagram.Terminal device shown in Fig. 6 is only an example, function to the embodiment of the present invention and should not use model
Shroud carrys out any restrictions.
As shown in fig. 6, computer system 600 includes central processing unit (CPU) 601, it can be read-only according to being stored in
Program in memory (ROM) 602 or be loaded into the program in random access storage device (RAM) 603 from storage section 608 and
Execute various movements appropriate and processing.In RAM 603, also it is stored with system 600 and operates required various programs and data.
CPU 601, ROM 602 and RAM 603 are connected with each other by bus 604.Input/output (I/O) interface 605 is also connected to always
Line 604.
I/O interface 605 is connected to lower component: the importation 606 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 607 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 608 including hard disk etc.；
And the communications portion 609 of the network interface card including LAN card, modem etc..Communications portion 609 via such as because
The network of spy's net executes communication process.Driver 610 is also connected to I/O interface 605 as needed.Detachable media 611, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 610, in order to read from thereon
Computer program be mounted into storage section 608 as needed.
Particularly, disclosed embodiment, the process described above with reference to flow chart may be implemented as counting according to the present invention
Calculation machine software program.For example, embodiment disclosed by the invention includes a kind of computer program product comprising be carried on computer
Computer program on readable medium, the computer program include the program code for method shown in execution flow chart.?
In such embodiment, which can be downloaded and installed from network by communications portion 609, and/or from can
Medium 611 is dismantled to be mounted.When the computer program is executed by central processing unit (CPU) 601, system of the invention is executed
The above-mentioned function of middle restriction.
It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the present invention, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In invention, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc. are above-mentioned
Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of various embodiments of the invention, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
Being described in module involved in the embodiment of the present invention can be realized by way of software, can also be by hard
The mode of part is realized.Described module also can be set in the processor, for example, can be described as: a kind of processor packet
It includes acquisition module, transmission module, obtain module.Wherein, the title of these modules is not constituted under certain conditions to the module
The restriction of itself, for example, acquisition module is also described as " the recalls information acquisition module in the period ".
As on the other hand, the present invention also provides a kind of computer-readable medium, which be can be
Included in equipment described in above-described embodiment；It is also possible to individualism, and without in the supplying equipment.Above-mentioned calculating
Machine readable medium carries one or more program, when said one or multiple programs are executed by the equipment, makes
Obtaining the equipment includes:
Call request of multiple called sides to service interface is received, each called side is to the service in statistics current period
The calling amount of interface, to generate recalls information corresponding with each called side；
Processing is grouped to all recalls informations in the current period, the recalls information after asynchronous transmission grouping is extremely
It is stored in storage side.
Technical solution according to an embodiment of the present invention is carried out by acquisition, transmission and the storing process to institute's monitoring information
Redesign, it is ingenious be utilized distributed treatment, it is asynchronous the modes such as summarize, monitoring information such as is summarized at the operations and is distributed in every clothes
It is executed in business machine, then the information bulk transfer after summarizing summarizes into distributed type assemblies.In addition, for being transmitted
Information carries out compression processing, it is possible to reduce hardware and network resource utilization, while ensure that overall plan is higher in real time
Property.
Above-mentioned specific embodiment, does not constitute a limitation on the scope of protection of the present invention.Those skilled in the art should be bright
It is white, design requirement and other factors are depended on, various modifications, combination, sub-portfolio and substitution can occur.It is any
Made modifications, equivalent substitutions and improvements etc. within the spirit and principles in the present invention, should be included in the scope of the present invention
Within.
Claims (10)
1. a kind of information processing method for traffic monitoring characterized by comprising
Call request of multiple called sides to service interface is received, each called side is to the service interface in statistics current period
Calling amount, to generate corresponding with each called side recalls information；
Processing is grouped to all recalls informations in the current period, the recalls information after asynchronous transmission grouping to storage
It is stored in side.
2. the method according to claim 1, wherein the recalls information include at least called side, service interface,
The mapping relations of calling amount and the affiliated timestamp of current period；
The method also includes:
According to predetermined value conversion regime, the called side in the recalls information, service interface, described is converted respectively
Calling amount and the timestamp are corresponding numerical information, and generate corresponding numerical value recalls information；Or
According to predetermined hash function conversion regime, convert respectively the called side in the recalls information, the service interface,
The call request amount and the timestamp are corresponding Hash functional value, and generate corresponding hash function recalls information.
3. the method according to claim 1, wherein all recalls informations in the current period into
Row packet transaction, comprising:
According to the predetermined information amount in the quantity of all recalls informations and every group, the grouping that is created needed for determining and every
Recalls information amount in group；Or
According to the quantity and predetermined packet amount of all recalls informations, the recalls information amount in every group is determined.
4. the method according to claim 1, wherein the recalls information after asynchronous transmission grouping is to the side of storage
In stored after, further includes:
According at least to the called side, the service interface, history recalls information corresponding with the recalls information is determined；And
It is stabbed according to the affiliated timestamp of the current period and the historical time of the history recalls information, when counting order history
History recalls information in length, to generate corresponding calling total information.
5. a kind of information processing unit for traffic monitoring characterized by comprising
Acquisition module counts each called side in current period for receiving call request of multiple called sides to service interface
To the calling amount of the service interface, to generate recalls information corresponding with each called side；
Transmission module, for being grouped processing to all recalls informations in the current period, after asynchronous transmission grouping
It is stored in recalls information to storage side.
6. device according to claim 5, which is characterized in that the recalls information include at least called side, service interface,
The mapping relations of calling amount and the affiliated timestamp of current period；
Described device further includes info conversion module, is used for:
According to predetermined value conversion regime, the called side in the recalls information, service interface, described is converted respectively
Calling amount and the timestamp are corresponding numerical information, and generate corresponding numerical value recalls information；Or
According to predetermined hash function conversion regime, convert respectively the called side in the recalls information, the service interface,
The call request amount and the timestamp are corresponding Hash functional value, and generate corresponding hash function recalls information.
7. device according to claim 5, which is characterized in that the transmission module is used for:
According to the predetermined information amount in the quantity of all recalls informations and every group, the grouping that is created needed for determining and every
Recalls information amount in group；Or
According to the quantity and predetermined packet amount of all recalls informations, the recalls information amount in every group is determined.
8. device according to claim 5, which is characterized in that further include Information Statistics module, be used for:
According at least to the called side, the service interface, history recalls information corresponding with the recalls information is determined；And
It is stabbed according to the affiliated timestamp of the current period and the historical time of the history recalls information, when counting order history
History recalls information in length, to generate corresponding calling total information.
9. a kind of electronic equipment characterized by comprising
One or more processors；
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
The now method as described in any in claim 1-4.
10. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that described program is held by processor
The method as described in any in claim 1-4 is realized when row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810878320.5A CN109039817B (en) | 2018-08-03 | 2018-08-03 | Information processing method, device, equipment and medium for flow monitoring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810878320.5A CN109039817B (en) | 2018-08-03 | 2018-08-03 | Information processing method, device, equipment and medium for flow monitoring |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109039817A true CN109039817A (en) | 2018-12-18 |
| CN109039817B CN109039817B (en) | 2020-09-01 |
Family
ID=64648437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810878320.5A Active CN109039817B (en) | 2018-08-03 | 2018-08-03 | Information processing method, device, equipment and medium for flow monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109039817B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110399067A (en) * | 2019-07-31 | 2019-11-01 | 江苏满运软件科技有限公司 | Point methods, device, electronic equipment, storage medium are buried in visualization |
| CN111782259A (en) * | 2020-06-24 | 2020-10-16 | 北京计算机技术及应用研究所 | Micro-service management method based on reverse proxy |
| CN112069021A (en) * | 2020-08-21 | 2020-12-11 | 北京五八信息技术有限公司 | Flow data storage method and device, electronic equipment and storage medium |
| CN113268518A (en) * | 2020-02-17 | 2021-08-17 | 百度在线网络技术（北京）有限公司 | Flow statistical method and device and distributed flow statistical system |
| CN113542245A (en) * | 2021-07-02 | 2021-10-22 | 广州华多网络科技有限公司 | Data flow monitoring method and device, computer equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103684898A (en) * | 2012-09-14 | 2014-03-26 | 阿里巴巴集团控股有限公司 | Method and device for monitoring operation of user request in distributed system |
| US20140095133A1 (en) * | 2012-09-28 | 2014-04-03 | General Electric Company | Systems and methods for managing mechanical systems and components |
| CN106354765A (en) * | 2016-08-19 | 2017-01-25 | 广东亿迅科技有限公司 | Log analysis system and method based on distributed collection |
| CN106411547A (en) * | 2015-07-28 | 2017-02-15 | 北京京东尚科信息技术有限公司 | Redis call monitoring method and device |
| CN106487596A (en) * | 2016-10-26 | 2017-03-08 | 宜人恒业科技发展（北京）有限公司 | Distributed Services follow the tracks of implementation method |
| WO2017051420A1 (en) * | 2015-09-21 | 2017-03-30 | Yissum Research Development Company Of The Hebrew University Of Jerusalem Ltd. | Advanced computer implementation for crawling and/or detecting related electronically catalogued data using improved metadata processing |
| CN107919980A (en) * | 2017-10-30 | 2018-04-17 | 贵州白山云科技有限公司 | A kind of evaluating method and device of clustering system |
-
2018
- 2018-08-03 CN CN201810878320.5A patent/CN109039817B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103684898A (en) * | 2012-09-14 | 2014-03-26 | 阿里巴巴集团控股有限公司 | Method and device for monitoring operation of user request in distributed system |
| US20140095133A1 (en) * | 2012-09-28 | 2014-04-03 | General Electric Company | Systems and methods for managing mechanical systems and components |
| CN106411547A (en) * | 2015-07-28 | 2017-02-15 | 北京京东尚科信息技术有限公司 | Redis call monitoring method and device |
| WO2017051420A1 (en) * | 2015-09-21 | 2017-03-30 | Yissum Research Development Company Of The Hebrew University Of Jerusalem Ltd. | Advanced computer implementation for crawling and/or detecting related electronically catalogued data using improved metadata processing |
| CN106354765A (en) * | 2016-08-19 | 2017-01-25 | 广东亿迅科技有限公司 | Log analysis system and method based on distributed collection |
| CN106487596A (en) * | 2016-10-26 | 2017-03-08 | 宜人恒业科技发展（北京）有限公司 | Distributed Services follow the tracks of implementation method |
| CN107919980A (en) * | 2017-10-30 | 2018-04-17 | 贵州白山云科技有限公司 | A kind of evaluating method and device of clustering system |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110399067A (en) * | 2019-07-31 | 2019-11-01 | 江苏满运软件科技有限公司 | Point methods, device, electronic equipment, storage medium are buried in visualization |
| CN113268518A (en) * | 2020-02-17 | 2021-08-17 | 百度在线网络技术（北京）有限公司 | Flow statistical method and device and distributed flow statistical system |
| CN113268518B (en) * | 2020-02-17 | 2024-03-29 | 百度在线网络技术（北京）有限公司 | Flow statistics method and device and distributed flow statistics system |
| CN111782259A (en) * | 2020-06-24 | 2020-10-16 | 北京计算机技术及应用研究所 | Micro-service management method based on reverse proxy |
| CN111782259B (en) * | 2020-06-24 | 2023-11-10 | 北京计算机技术及应用研究所 | Reverse proxy-based microservice treatment method |
| CN112069021A (en) * | 2020-08-21 | 2020-12-11 | 北京五八信息技术有限公司 | Flow data storage method and device, electronic equipment and storage medium |
| CN112069021B (en) * | 2020-08-21 | 2024-02-20 | 北京五八信息技术有限公司 | Flow data storage method and device, electronic equipment and storage medium |
| CN113542245A (en) * | 2021-07-02 | 2021-10-22 | 广州华多网络科技有限公司 | Data flow monitoring method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109039817B (en) | 2020-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109039817A (en) | A kind of information processing method and device for traffic monitoring | |
| US10447772B2 (en) | Managed function execution for processing data streams in real time | |
| CN110019350A (en) | Data query method and apparatus based on configuration information | |
| CN109189835A (en) | The method and apparatus of the wide table of data are generated in real time | |
| CN103942287B (en) | Mass data processing method, database server and application server | |
| Tsuchiya et al. | Big data processing in cloud environments | |
| CN111666490A (en) | Information pushing method, device, equipment and storage medium based on kafka | |
| CN107103064B (en) | Data statistical method and device | |
| CN111339073A (en) | Real-time data processing method and device, electronic equipment and readable storage medium | |
| CN103106585A (en) | Real-time duplication eliminating method and device of product information | |
| CN109783562B (en) | Service processing method and device | |
| CN114090366A (en) | Method, device and system for monitoring data | |
| WO2019076001A1 (en) | Information updating method and device | |
| CN110300084A (en) | A kind of IP address-based portrait method and apparatus | |
| CN110019258A (en) | The method and apparatus for handling order data | |
| CN110928934A (en) | Data processing method and device for business analysis | |
| CN111814024A (en) | Distributed data acquisition method, system and storage medium | |
| CN103729385A (en) | Method and device for automatically updating reports | |
| CN110110196A (en) | A kind of information push method and device | |
| CN113190558A (en) | Data processing method and system | |
| CN109299089A (en) | The calculating and storage method and calculating of a kind of label data of drawing a portrait and storage system | |
| CN110928594A (en) | Service development method and platform | |
| CN108985805A (en) | A kind of method and apparatus that selectivity executes push task | |
| CN113127225A (en) | Method, device and system for scheduling data processing tasks | |
| CN111597033A (en) | Task scheduling method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 101111 Room 221, 2nd Floor, Block C, 18 Kechuang 11th Street, Beijing Economic and Technological Development Zone Applicant after: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. Address before: 101111 Room 221, 2nd Floor, Block C, 18 Kechuang 11th Street, Beijing Economic and Technological Development Zone Applicant before: BEIJING JINGDONG FINANCIAL TECHNOLOGY HOLDING Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 101111 Room 221, 2nd Floor, Block C, 18 Kechuang 11th Street, Beijing Economic and Technological Development Zone Patentee after: Jingdong Technology Holding Co.,Ltd. Address before: 101111 Room 221, 2nd Floor, Block C, 18 Kechuang 11th Street, Beijing Economic and Technological Development Zone Patentee before: Jingdong Digital Technology Holding Co.,Ltd. Address after: 101111 Room 221, 2nd Floor, Block C, 18 Kechuang 11th Street, Beijing Economic and Technological Development Zone Patentee after: Jingdong Digital Technology Holding Co.,Ltd. Address before: 101111 Room 221, 2nd Floor, Block C, 18 Kechuang 11th Street, Beijing Economic and Technological Development Zone Patentee before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. |