CN107018180B - Intelligent transportation big data platform based on JSEDA distributes shared system - Google Patents
Intelligent transportation big data platform based on JSEDA distributes shared system Download PDFInfo
- Publication number
- CN107018180B CN107018180B CN201710172901.2A CN201710172901A CN107018180B CN 107018180 B CN107018180 B CN 107018180B CN 201710172901 A CN201710172901 A CN 201710172901A CN 107018180 B CN107018180 B CN 107018180B
- Authority
- CN
- China
- Prior art keywords
- thread
- data
- event
- distribution
- listening
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/567—Integrating service provisioning from a plurality of service providers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
Abstract
The invention discloses a kind of, and the intelligent transportation big data platform based on JSEDA distributes shared system.The system includes: the adaptive distribution subsystem of high performance network, by the design pattern for improving SEDA framework, using asynchronous event-driven mechanism, application flows are decomposed into the stage connected by event queue, it forms high performance network and adaptively distributes framework JSEDA, for active user, need Push Service and provide data distribution shared service by the user that interface accesses；The tradition distribution subsystem, to provide the related service of data downloading by browsing Web site and by the user of FTP；The configuration and management subsystem, configure system resource and are managed；The transmission statistics sub system carries out programming count to the distribution situation of data and is stored in database table.The a variety of ways of distribution of present system are simultaneously deposited, and the different demands of different user are met, and are provided stable support for data sharing, are provided convenience for user.
Description
Technical field
The present invention relates to data distribution technology of sharing field, especially a kind of intelligent transportation big data based on JSEDA is flat
Platform distributes shared system.
Background technique
With the continuous fast development accelerated and urban motorization is gone on a journey of urbanization process, one side urban road demand
It increases rapidly, urban transportation data increasingly increase, data class and quantity rapid growth；Another aspect citizen it is intensive trip with
And logistics pressure makes urban transportation be faced with stern challenge.Therefore, resolving urban transport problems and having become city can hold
One important content of supervention exhibition.
To alleviate huge urban traffic pressure, improve the solution mode of traffic congestion by increasing infrastructure investment
Because being limited by land resource, it is unfavorable between this three of transport development, development of urban space and earth development
Integration.In order to adapt to the development of urban transportation, big data platform is come into being.Big data is applied excellent in intelligent transportation system
Gesture is mainly manifested in the following aspects: first, traffic circulation efficiency can be obviously improved.The real-time that big data is possessed
Feature can allow idle data to be fully utilized, by intellectualized technology come to traffic dynamic carry out real time monitoring with
Prediction, to be obviously improved traffic circulation efficiency.Second, traffic safety level can be obviously improved.Third, big data it is real-time
Property is able to ascend the data-handling capacity of intelligent transportation.
Since traffic big data platform user of service type is different, kind of different types of crowd for traffic data
Class and the timeliness for providing data require also different.Currently, traffic big data distribution shared system cannot support a variety of distributions altogether
Mode is enjoyed, and the received situation of data can not be monitored, often has and lacks the case where receiving.
Summary of the invention
The purpose of the present invention is to provide a kind of, and the intelligent transportation big data platform based on JSEDA distributes shared system, with
Guarantee data and data product passing to user in the correct time with correct transmission mode.
The technical solution for realizing the aim of the invention is as follows: a kind of intelligent transportation big data platform distribution based on JSEDA
Shared system, including the adaptive distribution subsystem of high performance network, traditional distribution subsystem, configuration and management subsystem and transmission
Statistics sub system, in which:
The adaptive distribution subsystem of high performance network, by improving the design pattern of SEDA framework, using asynchronous
Application flows are decomposed into the stage connected by event queue by event-driven mechanism, and it is adaptive to form high performance network
Distribute framework JSEDA, for active user, needs Push Service and provide data distribution shared clothes by the user that interface accesses
Business；
The tradition distribution subsystem, to provide the related of data downloading to the user by FTP by browsing Web site
Service；
The configuration and management subsystem configure system resource according to current network state, system load situation
And management；
The transmission statistics sub system carries out programming count to the distribution situation of data, and these statistical informations is stored
In database table.
Further, the adaptive distribution subsystem of the high performance network uses SEDA thought stage by stage, and different are answered
With different stages, and the asynchronous event-driven mechanism of use is divided into as desired, complicated application flows are decomposed
For the stage connected by event queue；The event that all stages generate uniformly is scheduled by scheduler, according to loading condition,
Event is added to the event handling queue of each thread, the execution of each phased mission is driven by event；In processing I/0 request
Stage is supported by listening thread, and EPOLL technology is used at listening thread.
Further, the transmission statistics sub system includes three submodules, is respectively: range statistics module, traffic number
According to detection site statistical module and traffic data information data product statistical module, in which:
Range statistics module realizes the automatic system of seclected time period, the traffic data distribution situation of selection area file-level
Meter；
Traffic data detection site statistical module realizes the traffic data of seclected time period, selected traffic data detection site
Distribute the programming count of situation；
The friendship of traffic data information data product statistical module realization seclected time period, selection area file data classification grade
The programming count of logical data distribution situation；
Finally above-mentioned statistical information is stored in database table.
Further, the adaptive distribution subsystem of the high performance network includes 4 modules, specifically:
Stage: the execution of each phased mission is by event-driven, after a certain phased mission is finished, will generate one
Event；
Listening thread pond: being initially a thread, and listening thread and physical thread are bound, when listening thread load reaches
Increase listening thread collaboration processing request after the upper limit, reduces listening-in line number of passes after listening thread load is reduced to lower limit, together
When listening thread pond number of threads be equipped with maximum value, when reaching maximum value, whole system oepration at full load；
Mission thread pond: mission thread pond possesses multiple threads, and each mission thread has a corresponding event queue, appoints
The quantity of business thread is generated based on hardware, and consistent with physical thread quantity, and mission thread and physical thread are bound；
Scheduler: considering the loading condition of mission thread and listening thread, and event is added to the event queue of mission thread,
Keep whole system load balancing.
Compared with prior art, the present invention its remarkable advantage are as follows: (1) upper layer of the adaptive distribution subsystem of high performance network
Using being associated with stage by stage by event queue, extended convenient for Development of Modular and system；(2) high performance network adaptively distributes son
System is based on multicore and carries out performance optimization, greatly reduces cross-thread switching bring expense, has played the property of multi-core processor
It can advantage；(3) the adaptive distribution subsystem of high performance network is supported a large amount of concurrent, and has good load balancing；(4) it sends out
Send statistics sub system can the distribution situation to data count, effectively prevent data lack receive the case where.
Detailed description of the invention
Fig. 1 is that the present invention is based on the intelligent transportation big data platforms of JSEDA to distribute shared system framework map.
Fig. 2 is the frame construction drawing of the adaptive distribution subsystem of high performance network.
Fig. 3 is to send statistics sub system framework map.
Specific embodiment
The present invention is based on the intelligent transportation big data platform of JSEDA distribution shared system, including high performance network are adaptive
Distribution subsystem, traditional distribution subsystem, configuration and management subsystem and transmission statistics sub system, in which:
The adaptive distribution subsystem of high performance network, by improving the design pattern of SEDA framework, using asynchronous
Complicated application flows are decomposed into the stage connected by event queue by event-driven mechanism, form high performance network
Adaptive distribution framework (JSEDA) for active user, needs Push Service and provides data distribution by the user that interface accesses
Shared service；
The tradition distribution subsystem is related to provide data downloading etc. to the user by FTP by browsing Web site
Service；
The configuration and management subsystem match system resource according to current network state, system load situation etc.
It sets and manages；
The transmission statistics sub system carries out programming count to the distribution situation of data, and these statistical informations is stored
In database table.
Further, the adaptive distribution subsystem of the high performance network uses SEDA thought stage by stage, and being will be different
Using different stages, and the asynchronous event-driven mechanism of use is divided into as desired, by complicated application flows point
Solution is the stage connected by event queue；The event that all stages generate uniformly is scheduled by scheduler, according to load feelings
Event is added to the event handling queue of each thread, the execution of each phased mission is driven by event by condition.In processing I/0
Request stage is supported by listening thread, and EPOLL technology is used at listening thread, improves I/0 treatment effeciency, and to asking
Ask several limitations.
Further, the transmission statistics sub system is made of three submodules, is respectively: range statistics module is handed over
Logical data detection site statistical module and traffic data information data product statistical module, in which:
Seclected time period may be implemented in range statistics module, the traffic data of selection area file-level distributes the automatic of situation
Statistics；
The traffic of seclected time period, selected traffic data detection site may be implemented in traffic data detection site statistical module
The programming count of data distribution situation；
Seclected time period, selection area file data classification grade may be implemented in traffic data information data product statistical module
Traffic data distribution situation programming count.
These statistical informations can be stored in database table by final system.
Embodiment 1
Technical solution of the present invention is described in further detail with reference to the accompanying drawing:
In conjunction with Fig. 1, the present invention is based on the intelligent transportation big data platforms of JSEDA to distribute shared system, including high-performance net
The adaptive distribution subsystem of network, traditional distribution subsystem, configuration and management subsystem and transmission statistics sub system, in which:
The adaptive distribution subsystem of high performance network passes through the design pattern for improving SEDA framework, using asynchronous thing
Complicated application flows are decomposed into the stage connected by event queue by part driving mechanism, form high performance network certainly
Distribution framework (JSEDA) is adapted to, for active user, needs Push Service and provide data distribution by the user that interface accesses to be total to
The service of enjoying；
The tradition distribution subsystem is related to provide data downloading etc. to the user by FTP by browsing Web site
Service；
The configuration and management subsystem match system resource according to current network state, system load situation etc.
It sets and manages；
The transmission statistics sub system carries out programming count to the distribution situation of data, and these statistical informations is stored
In database table.
As a kind of specific example, the adaptive distribution subsystem of high performance network include 4 modules, such as Fig. 2, specifically
Are as follows:
Stage: the execution of each phased mission is by event-driven, after a certain phased mission is finished, will generate one
Event；
Listening thread pond: being initially a thread, considers to bind listening thread and physical thread.When listening thread loads
Increase listening thread collaboration processing request after reaching the upper limit, reduces listening thread after listening thread load is reduced to lower limit
Number is equipped with maximum value while listening for the number of threads of thread pool, is determined by hardware, and when reaching maximum value, whole system is completely negative
Lotus operation；
Mission thread pond: mission thread pond possesses multiple threads, and each mission thread has a corresponding event queue, appoints
The quantity of business thread is generated based on hardware, and consistent with physical thread quantity, considers to bind mission thread and physical thread；
Scheduler: considering the loading condition of mission thread and listening thread, and event is added to the event queue of mission thread,
Whole system load balancing is kept as far as possible.
As a kind of specific example, the transmission statistics sub system, including 3 submodules, such as Fig. 3, specifically:
Seclected time period may be implemented in range statistics module, the traffic data of selection area file-level distributes the automatic of situation
Statistics.
The traffic of seclected time period, selected traffic data detection site may be implemented in traffic data detection site statistical module
The programming count of data distribution situation.
Seclected time period, selection area file data classification grade may be implemented in traffic data information data product statistical module
Traffic data distribution situation programming count, and these statistical informations are stored in database table.
Above system mainly provides following functions: (1) being retrieved, downloaded to required data；(2) user according to oneself
Demand customizes data order, and supplier can be according to the requirement of user's order, the data-pushing of customization at user；(3)
User carries out the acquisition of data by client programming interface；(4) supplier can provide newest real-time number to active user
According to.
Claims (2)
1. a kind of intelligent transportation big data platform based on JSEDA distributes shared system, which is characterized in that including high performance network
Adaptive distribution subsystem, traditional distribution subsystem, configuration and management subsystem and transmission statistics sub system, in which:
The adaptive distribution subsystem of high performance network, by improving the design pattern of SEDA framework, using asynchronous event
Application flows are decomposed into the stage connected by event queue by driving mechanism, are formed high performance network and are adaptively distributed
Framework JSEDA for active user, needs Push Service and provides data distribution shared service by the user that interface accesses；
The tradition distribution subsystem, to provide the related clothes of data downloading to the user by FTP by browsing Web site
Business；
The configuration and management subsystem are configured and are managed to system resource according to current network state, system load situation
Reason；
The transmission statistics sub system carries out programming count to the distribution situation of data, and these statistical informations is stored in number
According in the table of library；
The adaptive distribution subsystem of high performance network uses SEDA thought stage by stage, and different applications is divided as desired
At the different stages, and asynchronous event-driven mechanism is used, complicated application flows are decomposed into and pass through event queue
The stage of connection；The event that all stages generate uniformly is scheduled by scheduler, and according to loading condition, event is added to respectively
The event handling queue of thread, the execution of each phased mission is driven by event；In processing I/0 request stage, by listening-in line
Journey support, uses EPOLL technology at listening thread；
The transmission statistics sub system includes three submodules, is respectively: range statistics module, traffic data detection site statistics
Module and traffic data information data product statistical module, in which:
Range statistics module realizes the programming count of seclected time period, the traffic data distribution situation of selection area file-level；
Traffic data detection site statistical module realizes the traffic data distribution of seclected time period, selected traffic data detection site
The programming count of situation；
Traffic data information data product statistical module realizes the traffic number of seclected time period, selection area file data classification grade
According to the programming count of distribution situation；
Finally above-mentioned statistical information is stored in database table.
2. the intelligent transportation big data platform according to claim 1 based on JSEDA distributes shared system, feature exists
In, the adaptive distribution subsystem of high performance network includes 4 modules, specifically:
Stage: the execution of each phased mission is by event-driven, after a certain phased mission is finished, will generate a thing
Part；
Listening thread pond: being initially a thread, and listening thread and physical thread are bound, when listening thread load reaches the upper limit
After increase a listening thread collaboration processing request, when listening thread load be reduced to lower limit after reduce listening-in line number of passes, supervise simultaneously
The number of threads of thread pool is listened to be equipped with maximum value, when reaching maximum value, whole system oepration at full load；
Mission thread pond: mission thread pond possesses multiple threads, and each mission thread has a corresponding event queue, task line
The quantity of journey is generated based on hardware, and consistent with physical thread quantity, and mission thread and physical thread are bound；
Scheduler: considering the loading condition of mission thread and listening thread, and event is added to the event queue of mission thread, keeps
Whole system load balancing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710172901.2A CN107018180B (en) | 2017-03-22 | 2017-03-22 | Intelligent transportation big data platform based on JSEDA distributes shared system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710172901.2A CN107018180B (en) | 2017-03-22 | 2017-03-22 | Intelligent transportation big data platform based on JSEDA distributes shared system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107018180A CN107018180A (en) | 2017-08-04 |
| CN107018180B true CN107018180B (en) | 2019-11-15 |
Family
ID=59440760
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710172901.2A Active CN107018180B (en) | 2017-03-22 | 2017-03-22 | Intelligent transportation big data platform based on JSEDA distributes shared system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107018180B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107729137B (en) * | 2017-09-04 | 2021-06-22 | 深圳壹账通智能科技有限公司 | Server, block chain signature verification decryption method and storage medium |
| US11070639B2 (en) * | 2017-09-28 | 2021-07-20 | Electronics And Telecommunications Research Institute | Network infrastructure system and method for data processing and data sharing using the same |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101448027A (en) * | 2008-12-29 | 2009-06-03 | 中国科学院计算技术研究所 | System of a pipeline Web server and operating method thereof |
| CN103036927A (en) * | 2011-09-29 | 2013-04-10 | 中国电信股份有限公司 | Method, device and system of intelligent traffic control |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6978206B1 (en) * | 2002-06-21 | 2005-12-20 | Infogation Corporation | Distributed navigation system |
-
2017
- 2017-03-22 CN CN201710172901.2A patent/CN107018180B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101448027A (en) * | 2008-12-29 | 2009-06-03 | 中国科学院计算技术研究所 | System of a pipeline Web server and operating method thereof |
| CN103036927A (en) * | 2011-09-29 | 2013-04-10 | 中国电信股份有限公司 | Method, device and system of intelligent traffic control |
Non-Patent Citations (1)
| Title |
|---|
| 动态服务资源分发网络的若干关键技术研究;刘廉如;《中国博士学位论文全文数据库》;20130115;第5.3节 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107018180A (en) | 2017-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Alresheedi et al. | Improved multiobjective salp swarm optimization for virtual machine placement in cloud computing | |
| CN103426072B (en) | The order processing system of a kind of high concurrent competition stock and disposal route thereof | |
| CN104461740A (en) | Cross-domain colony computing resource gathering and distributing method | |
| CN102469126B (en) | Application scheduling system, method thereof and related device | |
| CN104778080A (en) | Job scheduling processing method and device based on coprocessor | |
| CN105094751B (en) | A kind of EMS memory management process for stream data parallel processing | |
| CN102299959A (en) | Load balance realizing method of database cluster system and device | |
| CN106027596A (en) | Task distributing method and device | |
| CN103473365A (en) | File storage method and device based on HDFS (Hadoop Distributed File System) and distributed file system | |
| CN107018180B (en) | Intelligent transportation big data platform based on JSEDA distributes shared system | |
| CN105491150A (en) | Load balance processing method based on time sequence and system | |
| CN103002043B (en) | Method and system used for resource management of cloud environment | |
| CN102394903A (en) | Active reconstruction calculating system constructing system | |
| CN108920274B (en) | Performance optimization and device for image processing server side | |
| CN106131227A (en) | Balancing method of loads, meta data server system and load balance system | |
| CN108681481A (en) | The processing method and processing device of service request | |
| DE112012003307T5 (en) | System for saving energy at company data centers | |
| CN103473848B (en) | Network invoice checking framework and method based on high concurrency | |
| CN102779183A (en) | Data inquiry method, equipment and system | |
| CN103297511B (en) | The dispatching method of the client/server under high dynamic environments and system | |
| Zhou et al. | Deep reinforcement learning-based algorithms selectors for the resource scheduling in hierarchical cloud computing | |
| CN106874108A (en) | Thin cloud is minimized in mobile cloud computing use number technology | |
| CN103268261A (en) | Hierarchical computing resource management method suitable for large-scale high-performance computer | |
| Lin et al. | A workload-driven approach to dynamic data balancing in MongoDB | |
| CN112968962B (en) | Cloud platform storage resource scheduling method based on distributed computer cluster architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |