CN109101440A - Method for processing concurrent request of trace data based on JVM (Java virtual machine) cache - Google Patents
Method for processing concurrent request of trace data based on JVM (Java virtual machine) cache Download PDFInfo
- Publication number
- CN109101440A CN109101440A CN201810863056.8A CN201810863056A CN109101440A CN 109101440 A CN109101440 A CN 109101440A CN 201810863056 A CN201810863056 A CN 201810863056A CN 109101440 A CN109101440 A CN 109101440A
- Authority
- CN
- China
- Prior art keywords
- trace back
- data
- back data
- request
- concurrent request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000012545 processing Methods 0.000 title claims abstract description 21
- 239000000872 buffer Substances 0.000 claims description 25
- 238000012795 verification Methods 0.000 claims description 11
- 230000002688 persistence Effects 0.000 claims description 9
- 230000003139 buffering effect Effects 0.000 claims description 7
- 238000007726 management method Methods 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 claims description 4
- 238000013500 data storage Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 2
- 238000005194 fractionation Methods 0.000 claims description 2
- 238000004080 punching Methods 0.000 claims 1
- 230000003068 static effect Effects 0.000 claims 1
- 238000003860 storage Methods 0.000 abstract description 3
- 230000000694 effects Effects 0.000 description 4
- 238000013480 data collection Methods 0.000 description 2
- 235000013372 meat Nutrition 0.000 description 2
- 238000001514 detection method Methods 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003472 neutralizing effect Effects 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0806—Multiuser, multiprocessor or multiprocessing cache systems
- G06F12/084—Multiuser, multiprocessor or multiprocessing cache systems with a shared cache
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0806—Multiuser, multiprocessor or multiprocessing cache systems
- G06F12/0842—Multiuser, multiprocessor or multiprocessing cache systems for multiprocessing or multitasking
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for processing a trace data concurrence request based on JVM cache, and belongs to the technical field of application software in the Internet industry. The method for processing the concurrent request of the trace data based on the JVM cache comprises the steps of setting a uniform interface protocol standard and realizing internal processing. The method for processing the concurrent request of the trace data based on the JVM cache can provide a convenient and fast interface protocol for the outside, provides stable and efficient concurrent storage processing capability for the inside, and has good popularization and application values.
Description
Technical field
The present invention relates to internet industry application software technology fields, specifically provide a kind of based on the retrospect of JVM caching process
The method of Data Concurrent request.
Background technique
Former meat dish traces the data collection interface service of urban platform by the way of WebService+XML,
WebService agreement is heavier, and XML readability is not high, and resolution rules are complicated, and efficiency of transmission is lower.
The data collection interface service of former meat dish retrospect urban platform, starts to process from request is received, simultaneously to data storage
Return response result, by the way of same thread sequential processes, processing speed is slow, and concurrent capability is not high, can not load height simultaneously
Hair data pour in.
Summary of the invention
Technical assignment of the invention is that in view of the above problems, providing one kind can externally provide conveniently
Interface protocol is internally provided and stablizes and efficiently concurrently store concurrently being asked based on JVM caching process trace back data for processing capacity
The method asked.
To achieve the above object, the present invention provides the following technical scheme that
A method of based on JVM caching process trace back data concurrent request, this method includes the unified interface association of setting
Standard of assessing a bid for tender and realization inter-process.
Conveniently interface protocol is externally provided by this method, is internally provided at stable and efficient concurrently storage
Reason ability.
Preferably, the unified interface protocol standard of the setting specifically includes setting transmission protocol, data format, word
Accord with collection, digest algorithm, date format, server time, picture format, request packet header, response packet header and signature.
The transport protocol is HTTP POST.
Data format is application/json.
Character set is UTF-8.
Digest algorithm is SHA-256.
Date format is yyyy-MM-dd HH:mm:ss.
Server time is Beijing time, is made within 24 hours.With dock the method, system time allow 30 minutes within error.
Picture format is BASE64 coded string.
Requesting packet header includes title, meaning, type, required item and explanation.
Responding packet header includes title, meaning, type, required item and explanation.
Signature algorithm is SHA256 (json+key), and key is provided by authenticating party.
Preferably, the signature signs to complete json character string using SHA-256, signature result is attached
It is sent together in the tail portion of json character string.
Preferably, the realization inter-process includes connection inlet, interface processing, trace back data assembling and writes out, line
Cheng Chi management, data buffering and batch data persistence.
Preferably, the connection inlet is realized by CollectServlet class, including acquisition request information, fractionation
Json body is encoded with signature section, parsing json body, verification request, verification certification and signature, verification request time, calling connect
Mouth processing class executes interface and handles and obtain processing result, writes out accordingly result.
Preferably, the interface is handled according to interface type, calling interface implementation method, interface realizing method includes
Service parameter verifying generates traceability code, writes out trace back data and return response result.
Preferably, calling rule correction verification module carries out service parameter verifying, calls traceability code raw in interface realizing method
Generate according to traceability code create-rule that traceability code, calling trace back data to write out realization, class is asynchronous writes out trace back data at module.
Preferably, thread pool manager is that each data reports unit to distribute a line in the thread pool management
Cheng Chi, be isolated it is different report unit, make that the concurrent request between unit is respectively reported to be independent of each other.
Preferably, Data buffer is public to report the concurrent data storage of unit to provide in the data buffering
Buffer pool, dissolves wave crest, and batch is written.
Preferably, the batch data persistence is realized by class TraceDB, TraceDB is received slow by building method
Pond data set is rushed, and task is write out in the thread pool submission for the single thread held by internal stationary, realizes that number is written in trace back data
According to library.
Compared with prior art, the method for the invention based on JVM caching process trace back data concurrent request has following
It is outstanding the utility model has the advantages that the method based on JVM caching process trace back data concurrent request can be effectively solved important production
Product trace the request of platform high concurrent and the problem of big data store, for the collection of important product back-tracing platform data provide it is stable and
Efficient support, while conveniently interface protocol being externally provided, it internally provides and stablizes and efficient concurrently storage is handled
Ability has good application value.
Specific embodiment
Below in conjunction with embodiment, to the method for the invention based on JVM caching process trace back data concurrent request make into
One step is described in detail.
Embodiment
Method based on JVM caching process trace back data concurrent request of the invention, including setting unified interface protocol
Standard and realization inter-process.
(1) it sets unified interface protocol standard and specifically includes setting transmission protocol, data format, character set, abstract
Algorithm, date format, server time, picture format, request packet header, response packet header and signature.
Wherein transport protocol is HTTP POST.
Data format is application/json.
Character set is UTF-8.
Digest algorithm is SHA-256.
Date format is yyyy-MM-dd HH:mm:ss.
Server time is Beijing time, is made within 24 hours.With dock the method, system time allow 30 minutes within error.
Picture format is BASE64 coded string.
Requesting packet header includes title, meaning, type, required item and explanation, is specifically shown in Table 1.
Table 1 requests packet header
Interface example:
Responding packet header includes title, meaning, type, required item and explanation, is specifically shown in Table 2.Table 2 responds packet header
| Title | Meaning | Type | It must fill out | Explanation |
| BIZ_TYPE | Type of service | varchar(32) | It is | Such as " traceDataResponse " |
| RESP_TIME | Response time | char(19) | It is | yyyy-MM-dd HH24:MI:SS |
| RESP_ID | Response coding | varchar(19) | It is | The unique encodings that responder generates |
| REQ_ID | Request coding | varchar(32) | It is | The unique encodings that requesting party generates |
| RESULT | Response data parameter | clob | It is | Request data parameter |
Interface example:
Signature signs to complete json character string using SHA-256, and signature result is invested json character string
Tail portion is sent together.Signature algorithm is SHA256 (json+key), and wherein key is provided by authenticating party.
(2) realize that inter-process specifically includes connection inlet, interface processing, trace back data assembling and writes out, thread pool
Management, data buffering and batch data persistence.
(1) connection inlet
Connection inlet is realized that basic logic step includes: acquisition request message, splits by CollectServlet class
Json body is encoded with signature section, parsing json body, verification request, verification certification and signature, verification request time, calling connect
Mouth processing class executes interface and handles and obtain processing result, writes out response results.
(2) interface is handled
According to interface type BIZ_TYPE, obtain interface processing class TraceDataCollect's from interface processing plant
Example, and calling interface implementation method process, the basic logic step of interface realizing method include: calling rule calibration mode
Block carries out necessary service parameter verifying, traceability code generation module is called to generate traceability code according to traceability code create-rule, call
Trace back data writes out realization, and class is asynchronous writes out trace back data, return response result.
(3) trace back data writes out
Trace back data writes out calling code: new
TraceData((String)reqMap.get("REQ_ID"),
(String)reqMap.get("AUTH_ID"),paramMap,traceIdList,
traceStatus).write()；
Building method TraceData receives necessary service parameter.
Write method obtains corresponding thread pool according to certification coding from thread pool manager, and by this object reference
The asynchronous of thread pool writes out:
public void write(){
writeManager.getExecutorService(authId).execute(this)；
}
Service parameter tissue is chasing after of being stored in run method by the run method of this object of thread pool asynchronous call
Trace back data object, and data object is added in buffer pool TraceBuffer.
(4) thread pool management
Thread pool manager is that each data reports unit to distribute a thread pool in thread pool management, plays isolation
The effect for reporting unit makes that the concurrent request between unit is respectively reported to be independent of each other, carries out simultaneously；And it same reports in unit
Concurrent request, can also be carried out simultaneously when being less than the maximum concurrent thread number of thread pool, when being more than maximum concurrent thread
It is pending then to need to enter task queue etc. when number, prevents from influencing system because concurrently excessively thread being caused to run on a bank and stablizing.
The core code of thread pool manager is as follows:
(5) data buffering
Data buffer provides public buffer pool for all concurrent data storages for reporting unit in data buffering, rises
To neutralizing wave crest, the effect of batch write-in.
After thering is data object to be added to buffer pool, when judging whether buffer pool has expired, or reaching the buffering of restriction
Between, if meeting one of two conditions, give buffer pool data set to data persistence class TraceDB, carries out batch and write out
To database.Buffer time is refreshed in clearing buffers pond simultaneously.
Hold a timer inside Data buffer, whether reaches limit every data in fixed frequency detection buffer pool
Fixed buffer time gives buffer pool data set to data persistence class TraceDB if meeting condition, carries out batch and writes out
To database.Buffer time is refreshed in clearing buffers pond simultaneously.
The effect of timer is to solve not arrive for a long time because of follow-up data, and buffer pool is discontented, causes in buffer pool
Data the problem of database cannot be written for a long time.
The core code of Data buffer is as follows:
(6) batch data persistence
Batch data persistence is realized by class TraceDB.Call code: new
TraceDB(bufferList).write()；
TraceDB receives buffer pool data set, and the thread for the single thread held by internal stationary by building method
Task is write out in pond, submission, and asynchronous call run method realizes that database is written in trace back data.
TraceDB asynchronous operation is asynchronous as the second level in process, effect be solve the data set write time it is longer and
Cause buffer pool that cannot terminate to call in time, so that the problem of new data enters buffer pool cannot be received.
Embodiment described above, the only present invention more preferably specific embodiment, those skilled in the art is at this
The usual variations and alternatives carried out within the scope of inventive technique scheme should be all included within the scope of the present invention.
Claims (10)
1. a kind of method based on JVM caching process trace back data concurrent request, it is characterised in that: this method includes that setting is unified
Interface protocol standard and realize inter-process.
2. the method according to claim 1 based on JVM caching process trace back data concurrent request, it is characterised in that: institute
It states the unified interface protocol standard of setting and specifically includes setting transmission protocol, data format, character set, digest algorithm, date lattice
Formula, server time, picture format, request packet header, response packet header and signature.
3. the method according to claim 2 based on JVM caching process trace back data concurrent request, it is characterised in that: institute
It states signature to sign to complete json character string using SHA-256, signature result is invested to the tail portion one of json character string
And it sends.
4. the method according to claim 1,2 or 3 based on JVM caching process trace back data concurrent request, feature exist
In: the realization inter-process include connection inlet, interface processing, trace back data assembling and write out, thread pool management, data it is slow
Punching and batch data persistence.
5. the method according to claim 4 based on JVM caching process trace back data concurrent request, it is characterised in that: institute
It states connection inlet to be realized by CollectServlet class, including acquisition request information, fractionation json body and signature section, parsing
Json body, verification request coding, verification certification and signature, verification request time, calling interface processing class execute interface processing simultaneously
It obtains processing result, write out accordingly result.
6. the method according to claim 5 based on JVM caching process trace back data concurrent request, it is characterised in that: institute
Interface processing is stated according to interface type, calling interface implementation method, interface realizing method includes service parameter verifying, generation retrospect
Code writes out trace back data and return response result.
7. the method according to claim 6 based on JVM caching process trace back data concurrent request, it is characterised in that: connect
Calling rule correction verification module carries out service parameter verifying, traceability code generation module is called to generate according to traceability code in mouth implementation method
Rule generates that traceability code, calling trace back data to write out realization, class is asynchronous writes out trace back data.
8. the method according to claim 7 based on JVM caching process trace back data concurrent request, it is characterised in that: institute
It states in thread pool management, thread pool manager is that each data reports unit to distribute a thread pool, and different report is isolated
Unit makes that the concurrent request between unit is respectively reported to be independent of each other.
9. the method according to claim 8 based on JVM caching process trace back data concurrent request, it is characterised in that: institute
It states in data buffering, Data buffer dissolves wave crest, batch to report the concurrent data storage of unit to provide public buffer pool
Write-in.
10. the method according to claim 9 based on JVM caching process trace back data concurrent request, it is characterised in that: institute
It states batch data persistence to be realized by class TraceDB, TraceDB receives buffer pool data set by building method, and by interior
Task is write out in the thread pool submission for the single thread that portion's static state is held, and realizes that database is written in trace back data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810863056.8A CN109101440B (en) | 2018-08-01 | 2018-08-01 | Method for processing trace data concurrent request based on JVM (Java virtual machine) cache |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810863056.8A CN109101440B (en) | 2018-08-01 | 2018-08-01 | Method for processing trace data concurrent request based on JVM (Java virtual machine) cache |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109101440A true CN109101440A (en) | 2018-12-28 |
| CN109101440B CN109101440B (en) | 2023-08-04 |
Family
ID=64848135
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810863056.8A Active CN109101440B (en) | 2018-08-01 | 2018-08-01 | Method for processing trace data concurrent request based on JVM (Java virtual machine) cache |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109101440B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6766349B1 (en) * | 1999-09-24 | 2004-07-20 | Sun Microsystems, Inc. | Mechanism for obtaining a thread from, and returning a thread to, a thread pool without attaching and detaching |
| JP2005044205A (en) * | 2003-07-24 | 2005-02-17 | Sony Corp | Electronic equipment, format method and format instructing method |
| CN1822545A (en) * | 2005-02-14 | 2006-08-23 | 耶德托存取公司 | Method of controlling communication between a head-end system and a plurality of client systems |
| CN102025606A (en) * | 2009-09-23 | 2011-04-20 | 中兴通讯股份有限公司 | Data transmission method and system |
| CN103366471A (en) * | 2013-06-26 | 2013-10-23 | 福建联迪商用设备有限公司 | Online service concurrent processing method, system and server |
| CN105635298A (en) * | 2016-01-12 | 2016-06-01 | 安徽科力信息产业有限责任公司 | Service isolation principle based data acquisition equipment unified access system |
| WO2016082371A1 (en) * | 2014-11-25 | 2016-06-02 | 中国科学院声学研究所 | Ssh protocol-based session parsing method and system |
| CN105868412A (en) * | 2016-04-28 | 2016-08-17 | 焦点科技股份有限公司 | Multithread data grabbing method based on B2B (business to business) platform |
| CN106790763A (en) * | 2017-01-18 | 2017-05-31 | 北京鑫通运科信息技术有限公司 | A kind of Internet of Things unifies the implementation and system of trace platform |
| CN106991196A (en) * | 2017-05-02 | 2017-07-28 | 山东浪潮通软信息科技有限公司 | A kind of general data traceability method and system |
| CN107360035A (en) * | 2017-07-27 | 2017-11-17 | 北京市天元网络技术股份有限公司 | A kind of data processing method and system |
-
2018
- 2018-08-01 CN CN201810863056.8A patent/CN109101440B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6766349B1 (en) * | 1999-09-24 | 2004-07-20 | Sun Microsystems, Inc. | Mechanism for obtaining a thread from, and returning a thread to, a thread pool without attaching and detaching |
| JP2005044205A (en) * | 2003-07-24 | 2005-02-17 | Sony Corp | Electronic equipment, format method and format instructing method |
| CN1822545A (en) * | 2005-02-14 | 2006-08-23 | 耶德托存取公司 | Method of controlling communication between a head-end system and a plurality of client systems |
| CN102025606A (en) * | 2009-09-23 | 2011-04-20 | 中兴通讯股份有限公司 | Data transmission method and system |
| CN103366471A (en) * | 2013-06-26 | 2013-10-23 | 福建联迪商用设备有限公司 | Online service concurrent processing method, system and server |
| WO2016082371A1 (en) * | 2014-11-25 | 2016-06-02 | 中国科学院声学研究所 | Ssh protocol-based session parsing method and system |
| CN105635298A (en) * | 2016-01-12 | 2016-06-01 | 安徽科力信息产业有限责任公司 | Service isolation principle based data acquisition equipment unified access system |
| CN105868412A (en) * | 2016-04-28 | 2016-08-17 | 焦点科技股份有限公司 | Multithread data grabbing method based on B2B (business to business) platform |
| CN106790763A (en) * | 2017-01-18 | 2017-05-31 | 北京鑫通运科信息技术有限公司 | A kind of Internet of Things unifies the implementation and system of trace platform |
| CN106991196A (en) * | 2017-05-02 | 2017-07-28 | 山东浪潮通软信息科技有限公司 | A kind of general data traceability method and system |
| CN107360035A (en) * | 2017-07-27 | 2017-11-17 | 北京市天元网络技术股份有限公司 | A kind of data processing method and system |
Non-Patent Citations (2)
| Title |
|---|
| 李永进;田丰;倪兆阳;: "面向高并发复杂民航业务的服务器架构设计", 计算机应用与软件, no. 05 * |
| 毛林;成维莉;程涛;金梦凯;钟泽;蔡乐;: "基于产品电子代码(EPC)编码的农资追溯系统应用研究", 浙江农业科学, no. 07 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109101440B (en) | 2023-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021244211A1 (en) | Blockchain message processing method and apparatus, computer and readable storage medium | |
| US7415444B2 (en) | Determining compliance rates for probabilistic requests | |
| CN107657532A (en) | The processing method and system of a kind of operation flow | |
| CN110781082A (en) | Method, device, medium and equipment for generating test case of interface | |
| WO2022142781A1 (en) | Asynchronous bookkeeping method and apparatus for blockchain, medium, and electronic device | |
| ATE472774T1 (en) | TRANSPARENT EDGE OF NETWORK DATA CACHE | |
| WO2021151312A1 (en) | Method for determining inter-service dependency, and related apparatus | |
| CN110704484A (en) | Method and system for processing mass real-time data stream | |
| WO2020259017A1 (en) | Time sequence data processing method, apparatus and system, and server and readable storage medium | |
| CN112631806B (en) | Asynchronous message arrangement and scheduling method and device, electronic equipment and storage medium | |
| CN104679493B (en) | A kind of improved method of the event processing mechanism of procedure | |
| CN108549592A (en) | A kind of monitoring method and monitoring device, application server of database connection pool | |
| WO2022247201A1 (en) | Data query method and apparatus | |
| CN103577469B (en) | Database connection multiplexing method and apparatus | |
| CN109299193A (en) | Method of data synchronization and relevant device | |
| CN105373563B (en) | Database switching method and device | |
| CN110543783A (en) | Voting system and implementation method, equipment and storage medium thereof | |
| CN109101440A (en) | Method for processing concurrent request of trace data based on JVM (Java virtual machine) cache | |
| CN115269709B (en) | Inquiry and playback method, device, equipment and medium based on market data | |
| CN1200382C (en) | Data updating method and system | |
| CN112099957B (en) | Resource allocation method, system, device and electronic equipment | |
| CN115460265A (en) | Interface calling method, device, equipment and medium | |
| CN104572316A (en) | Game program and running platform interaction method and system | |
| CN115174698B (en) | Market data decoding method, device, equipment and medium based on table entry index | |
| CN114389988B (en) | Remote procedure call method, device, equipment and medium based on network architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |