CN207939557U - A kind of statistics of data acquisition hardware platform - Google Patents
A kind of statistics of data acquisition hardware platform Download PDFInfo
- Publication number
- CN207939557U CN207939557U CN201820305063.1U CN201820305063U CN207939557U CN 207939557 U CN207939557 U CN 207939557U CN 201820305063 U CN201820305063 U CN 201820305063U CN 207939557 U CN207939557 U CN 207939557U
- Authority
- CN
- China
- Prior art keywords
- server
- database
- application server
- web
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Abstract
The utility model provides a kind of statistics of data acquisition hardware platform, including load equalizer, Web static servers, Web Application Server, Report form application server, message-oriented middleware server, data processing server, database write server and database read server.Wherein, load equalizer is connected with Web static servers, Web Application Server, Report form application server respectively；Message-oriented middleware server is connected with data processing server；Data processing server is connected with database write server；Database write server reads server with database and is connected；Database reads server and is connected respectively with Web Application Server and Report form application server；Database write server is connected with Web Application Server.The problems such as performance that the statistics of data acquisition hardware platform solves existing Features of Railway Logistics warehouse receipt management systems face is insufficient, availability is low, poor expandability.
Description
Technical field
The utility model is related to railway information technology field more particularly to a kind of data for Features of Railway Logistics warehouse receipt management
Acquisition statistics hardware platform.
Background technology
Features of Railway Logistics informationization passes through with modern information technology means, improvement flow and sides such as computer, networks
Formula improves working efficiency, management level and the service level of Features of Railway Logistics.Railway Logistics Enterprises are carrying out the logistics such as current assets mortgage
When financial business, the warehouse receipt information in each goods yard, warehouse on the one hand in time, is accurately obtained, it on the other hand will be to logistics client
History, for statistical analysis in fortune warehouse receipt, to pledge risk assessment provide decision support.
The existing statistics of data acquisition hardware platform for Features of Railway Logistics warehouse receipt management, mostly by an application server and
One database server is constituted.Application server provides data acquisition and application access for client computer connected reference
Service；Database server is connected with application server, is responsible for storage data.This hardware plan access client compared with
It is few, the little fashion energy meet demand of data volume of acquisition, but when warehouse receipt data volume is big, concurrent intensive, platform will appear response
The case where slow even delay machine.
The reason of this problem occur is, existing hardware platform only makees application server and one layer of database server is drawn
Point, cause server to be performed simultaneously a variety of different types of tasks, performance is caused to decline.By taking application server as an example, separate unit is answered
Data acquisition and two kinds of data statistics request are handled simultaneously with server, and both requests have the characteristics that different, correspond to
Demand with the computing unit of server, storage unit is also different.Data harvesting request amount is big, frequency is high, and single request occupies
CPU, memory source are few and discharge quickly；Data statistics request amount is relatively smaller, but to occupy CPU, memory time long for single request.Two
Class request is simultaneously in an application server processes, and the long connection of data statistics request occupies resource and do not discharge, and data acquisition is asked
It acquires less than timely responding to, causes request queue constantly to increase, throughput degradation.Similarly, separate unit data database is located simultaneously
It manages gathered data write request (belonging to OLTP types) and statistical data reads request (belonging to OLAP types), also will appear similar
The problem of.In addition, the single structure that separate unit application server is connected with separate unit database server so that go out there are one component
Existing problem, entire platform all can be unavailable.It should be noted that the simple application server that increases builds application cluster, can make
Application access situation is alleviated, but the effectiveness of performance of single application server node is still very low；And database server by
It, can not the dilatation by way of directly increasing server in the requirement and limitation of read-write data consistency.
Utility model content
In view of the deficiencies of the prior art, the utility model provides a kind of new statistics of data acquisition hardware platform.This is flat
Platform has done further subdivision to server, each server is made to hold on the basis of original application is detached with database server
Carry on a shoulder pole that type is identical, the similar task of hardware resource requirements, to improve the effectiveness of performance of individual server node；Between server
Connection relation be optimized, Single Point of Faliure is avoided, to promote the availability of platform entirety.
To achieve the above object, the technical solution adopted in the utility model is as follows：
A kind of statistics of data acquisition hardware platform, including load equalizer, Web static servers, Web Application Server,
Report form application server, message-oriented middleware server, data processing server, database write server and database read service
Device.Wherein, load equalizer is connected with Web static servers, Web Application Server, Report form application server respectively；Message
Middleware server is connected with data processing server；Data processing server is connected with database write server；Data
Library writing server reads server with database and is connected；Database read server respectively with Web Application Server and Report form application
Server is connected；Database write server is connected with Web Application Server.
Load equalizer is a kind of hardware device network request being distributed to different server.Load equalizer is unified
It receives, the client request of forwarding platform, the request for accessing static resource is distributed to Web static servers, dynamic will be accessed
The request of resource is distributed to Web Application Server, the request for accessing report is distributed to Report form application server, then by these
Server handles corresponding request.
Web static servers are responsible for response static resource access request.Static resource refer mainly to HTML, JavaScript,
Picture etc. does not include the file of dynamic content.Web static servers do not execute using logic, without connecting with database server
It connects.Web static servers can efficiently handle static state using lightweight, high-performance, the design of multi-process, these features
Request.
Web Application Server is responsible for processing dynamic data request, executes and apply logic, needs to connect database server.
The speed that Web Application Server handles static resource is slow, so giving static resource access request to processing static resource
Faster Web static servers processing, Web Application Server only handle dynamic requests.Since dynamic requests are asked relative to static state
Ask accounting very little, the structure of this sound separation that can efficiently use two kinds of respective advantages of server, promote platform globality
Energy.
Report form application server is responsible for processing report data request.Report data request is a kind of certain types of dynamic
Request of data needs the inquiry for carrying out macrooperation amount, statistical to database server, asks the resources such as the CPU, the memory that occupy
The more and time is long.Report data is asked to separate with other dynamic requests, by individual Report form application server process, Neng Goujin
One step separating load increases handling capacity.
Message-oriented middleware server, responsible reception carrys out the data harvesting request of self terminal, and is transmitted to data processing service
Device.Static requests above-mentioned, dynamic requests, report request are the access requests to platform application, based on pulling data, and are counted
It is then based on propelling data according to acquisition request.Data harvesting request light weight, high concurrent, it is high to response speed requirement, in message
Between the message queue of part, Asynchronous Invocation can be good at meeting these demands, so platform uses among independent message
The acquisition request of part server response data.
Data processing server cleans the acquired original data of reception, is converted, is standardized, is loaded into database
Server.In the structure of this platform, the receptions of acquired original data and processing carry out on a different server, both ensured from
Scattered data harvesting request obtains quick response, and the data to gather together is made to obtain batch integration processing.
Database server, be responsible for tissue, storage, management platform data, ensure the consistency and reliability of data, it is real
The centralized control of existing data and access.Web Application Server, Report form application server, data processing server are all to database
Server has a large amount of reading and writing to operate, wherein：Based on Web Application Server is read and write with small data quantity, data connection is more, single
Affairs are short, require response speed high；Based on Report form application server is read with big data quantity, data connection is few, individually connects thing
Business is long, it is more to occupy resource, requires corresponding speed relatively not high；The small data quantity of data processing server then existing high frequency writes behaviour
Make, also there is the big data quantity read-write operation of low frequency.If the above different types of data manipulation is by separate unit database server
Situations such as managing, leading to cache invalidation, resource contention occurs, to make database server become the performance bottle of entire platform
Neck.
This platform uses the structure of data base read-write separation, is read using mutually independent database write server and database
Server handles different types of data manipulation respectively.Wherein, database write server is mainly responsible for all numbers of processing platform
According to the read operation in library write operation and same transaction context, data processing server, Web Application Server and database write
Server is connected；Database reads server and is only responsible for response database read operation request, Web Application Server, Report form application
Server reads server with database and is connected.According to the difference of read request type, independent database can also be used to read clothes
Business device responds high frequency small data quantity read request and low frequency big data quantity read request, further separating load respectively.Database write takes
Business device reads server sync data by its included synchronization mechanism to database, ensures data consistency.
The technologies such as the utility model integrated use network, server, database, middleware, in conjunction with Web application sound point
It is made that new design from thoughts, the structures between the role of server and server such as the separation of, data base read-write, it is proposed that
A kind of statistics of data acquisition hardware platform suitable for Features of Railway Logistics warehouse receipt management.Compared with prior art, advantage is：
(1) performance and handling capacity are improved.By detaching different type, feature asks summation process, and to specific role
Server targetedly optimized, make individual server execute same type task, reduce conflict, hardware resource obtains
It efficiently uses.
(2) availability is improved.Data acquisition, Web applications, Report form application, database are in different server clusters
The risk of upper operation, Single Point of Faliure substantially reduces.In most cases, even if a server goes wrong, will not cause to put down
Platform is integrally unavailable.
(3) scalability is enhanced.In the case where not changing platform structure, by increasing server into a cluster
Dilatation can be realized in node, and is hardly impacted to other clusters.
It should be noted that provided by the utility model is only a kind of infrastructure scheme of statistics of data acquisition platform,
Do not include that Features of Railway Logistics warehouse receipt manages relevant specific business and data processing software.The utility model is by improving server
Positioning and server between structure solve the problems, such as platform in performance, availability, scalability etc., the technical solution
What is be related to is all known hardware and software, does not include the improvement of computer program, therefore referred to as a kind of statistics of data acquisition hardware
Platform.
Description of the drawings
It, below will be to needed in embodiment description in order to illustrate more clearly of the technical solution of the utility model
Attached drawing is briefly described, it should be apparent that, the accompanying drawings in the following description is only one embodiment of the utility model, for
For those of ordinary skill in the art, without creative efforts, other are can also be obtained according to these attached drawings
Attached drawing.
Fig. 1 is a kind of network topology logical architecture figure of the specific implementation mode of statistics of data acquisition hardware platform.
Specific implementation mode
With reference to the attached drawing in the utility model embodiment, to this implement it is novel in technical solution carry out it is clear, complete
Site preparation describes, it is clear that the described embodiments are only a part of the embodiments of the utility model, instead of all the embodiments.
Based on the embodiments of the present invention, those of ordinary skill in the art are obtained under the premise of not making creative work
Every other embodiment, shall fall within the protection scope of the present invention.
As shown in Figure 1, a kind of statistics of data acquisition hardware platform for Features of Railway Logistics warehouse receipt management, including load balancing
Device, Web static servers, Web Application Server, Report form application server, message-oriented middleware server, data processing service
Device, database write server and database read server.Wherein, load equalizer is applied with Web static servers, Web respectively
Server, Report form application server are connected；Message-oriented middleware server is connected with data processing server；Data processing takes
Business device is connected with database write server；Database write server reads server with database and is connected；Database reads service
Device is connected with Web Application Server and Report form application server respectively；Database write server and Web Application Server phase
Even.The above server centered is deployed in the internal network of enterprise, it is contemplated that the needs of information security, Intranet can be drawn
Divide Intranet and outer net.Outer net is the areas DMZ of Intranet, accesses internet, places load equalizer, Web services
Device, Web Application Server, Report form application server；Intranet places message-oriented middleware server, data processing server, data
Library writing server, database read server.Intranet realizes logic isolation with outer net by fire wall, intrusion prevention system.
Load-balanced server uses the special load balancing hardware device of model F5BIG-IP LTM 1600.Using
F5 two-node cluster hot backup deployment way, can further increase network flow and availability.
Web static servers, using the x86 framework PC servers of model association System x3650, processor is 1
8 core Intel Xeon E5, memory 16GB；HTTP and reverse proxy software Nginx is installed.More structure Web static state clothes can be disposed
Business device cluster.
Web Application Server, using the x86 framework PC servers of model association System x3650, processor is 1
8 core Intel Xeon E5, memory 32GB；Java EE application service softwares Tomcat 8.0 is installed.More structure Web can be disposed
Application server cluster.
Report form application server uses the x86 framework PC servers of model association System x3650, processor 1
8 core Intel Xeon E5, memory 64GB；Pentaho Report report query softwares are installed, the report data provided is slow
The mechanism of depositing can avoid the repetition to identical statistical data from inquiring；Pentaho Dashboard report visual softwares, branch are installed
Hold a variety of report display forms such as chart, instrument board.More structure Report form application server clusters can be disposed.
Message-oriented middleware server server uses the x86 framework PC servers of model association System x3650, place
It is 18 core Intel Xeon E5, memory 32GB to manage device；Installation message middleware software Apache ActiveMQ 5.14.Profit
High concurrent is eliminated with the message queue mechanism of ActiveMQ and accesses peak, mitigates the pressure of data processing server, accelerates platform
Response speed.
Data processing server uses the x86 framework PC servers of model association System x3850, processor 4
8 core Intel Xeon E7, memory 64GB；Kettle 6.0 is installed and is used as ETL tools.Kettle supports various mainstream datas
Library, convenient for docking for data processing server and database server.
Database write server uses the x86 framework PC servers of model association System x3850, processor 4
8 core Intel Xeon E7, memory 128GB；Installation relation type database MySQL 5.7.
Database reads server, uses the x86 framework PC servers of model association System x3850, processor 4
8 core Intel Xeon E7, memory 128GB；Installation relation type database MySQL 5.7.Utilize the leader follower replication of MySQL
(Master-Slave) mechanism configures a database writing server and Duo Tai databases and reads server, realize read and write abruption and
Data synchronize.
As shown in Figure 1, this example demonstrates the modes that a kind of terminal connects the statistics of data acquisition hardware platform.Eventually
End is divided into application access terminal and data collection station.
Application access terminal can be, but not limited to desktop PC, mobile PC, smart mobile phone.The present embodiment uses B/S structures,
The server that application access terminal browser passes through internet access platform outer net.
Data collection station can be, but not limited to hand-held code reader, desktop scanner, smart mobile phone.In the present embodiment, it is
Ensure the safety of data transmission, the server that these terminal devices pass through access via telephone line platform Intranet.
Described above is only specific embodiment of the present utility model, but the scope of protection of the utility model not office
It is limited to this, any one skilled in the art within the technical scope disclosed by the utility model, can readily occur in change
Change or modification, should be covered within the scope of the utility model.
Claims (3)
1. a kind of statistics of data acquisition hardware platform, it is characterised in that：The statistics of data acquisition hardware platform includes that load is equal
Weighing apparatus, Web static servers, Web Application Server, Report form application server, message-oriented middleware server, data processing service
Device, database write server and database read server；Wherein,
The load equalizer is connected with the Web static servers, Web Application Server, Report form application server respectively；
The message-oriented middleware server is connected with the data processing server；
The data processing server is connected with the database write server；
The database write server reads server with the database and is connected；
The database reads server and is connected respectively with the Web Application Server and Report form application server；
The database write server is connected with the Web Application Server.
2. statistics of data acquisition hardware platform as described in claim 1, it is characterised in that：
The Web static servers are multiple servers；
The Web Application Server is multiple servers；
The Report form application server is multiple servers；
It is multiple servers that the database, which reads server,.
3. statistics of data acquisition hardware platform as described in claim 1, it is characterised in that：
The statistics of data acquisition hardware platform is divided into two parts of outer net and Intranet；
The outer net includes the load equalizer, Web static servers, Web Application Server and Report form application server；
The Intranet includes that the message-oriented middleware server, data processing server, database write server and database are read
Server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201820305063.1U CN207939557U (en) | 2018-03-06 | 2018-03-06 | A kind of statistics of data acquisition hardware platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201820305063.1U CN207939557U (en) | 2018-03-06 | 2018-03-06 | A kind of statistics of data acquisition hardware platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN207939557U true CN207939557U (en) | 2018-10-02 |
Family
ID=63653824
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201820305063.1U Expired - Fee Related CN207939557U (en) | 2018-03-06 | 2018-03-06 | A kind of statistics of data acquisition hardware platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN207939557U (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111049933A (en) * | 2019-12-28 | 2020-04-21 | 杭州电力设备制造有限公司 | Monitoring system for temperature of key power node |
| CN113177039A (en) * | 2021-04-27 | 2021-07-27 | 中通服咨询设计研究院有限公司 | Data center data cleaning system based on data fusion |
-
2018
- 2018-03-06 CN CN201820305063.1U patent/CN207939557U/en not_active Expired - Fee Related
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111049933A (en) * | 2019-12-28 | 2020-04-21 | 杭州电力设备制造有限公司 | Monitoring system for temperature of key power node |
| CN113177039A (en) * | 2021-04-27 | 2021-07-27 | 中通服咨询设计研究院有限公司 | Data center data cleaning system based on data fusion |
| CN113177039B (en) * | 2021-04-27 | 2024-02-27 | 中通服咨询设计研究院有限公司 | Data center data cleaning system based on data fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109492040B (en) | System suitable for processing mass short message data in data center | |
| CN103024014B (en) | By the method and system of the mass data distribution processor of message queue | |
| CN101493826B (en) | Database system based on WEB application and data management method thereof | |
| CN100568193C (en) | The system and method that is used for performance management in the multilayer computing environment | |
| US20200099606A1 (en) | Distrubuted testing service | |
| CN108365971A (en) | Daily record analytic method, equipment and computer-readable medium | |
| CN102831156A (en) | Distributed transaction processing method on cloud computing platform | |
| CN103092698A (en) | System and method of cloud computing application automatic deployment | |
| CN103995807B (en) | Magnanimity data query and the method for after-treatment under a kind of framework based on Web | |
| CN102262680A (en) | Distributed database proxy system based on massive data access requirement | |
| CN103793485A (en) | Method for inquiring network data by client terminal based on cached data | |
| US10158709B1 (en) | Identifying data store requests for asynchronous processing | |
| CN103399894A (en) | Distributed transaction processing method on basis of shared storage pool | |
| EP4216061A1 (en) | Transaction processing method, system, apparatus, device, storage medium, and program product | |
| CN113010818A (en) | Access current limiting method and device, electronic equipment and storage medium | |
| CN101146127A (en) | A client buffer update method and device in distributed system | |
| CN106713391A (en) | Session information sharing method and sharing system | |
| CN113301590B (en) | Virtual resource management and control system facing 5G access network | |
| CN207939557U (en) | A kind of statistics of data acquisition hardware platform | |
| CN113468221A (en) | System integration method based on kafka message data bus | |
| CN112199427A (en) | Data processing method and system | |
| CN112163001A (en) | High-concurrency query method, intelligent terminal and storage medium | |
| CN111984505A (en) | Operation and maintenance data acquisition engine and acquisition method | |
| US6850957B2 (en) | Information system and data access method | |
| Ma et al. | Load-balancing mechanism for the RFID middleware applications over grid networking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20181002 Termination date: 20210306 |