CN107610468A - Speed density Analysis System and method based on recorder management - Google Patents
Speed density Analysis System and method based on recorder management Download PDFInfo
- Publication number
- CN107610468A CN107610468A CN201710898046.3A CN201710898046A CN107610468A CN 107610468 A CN107610468 A CN 107610468A CN 201710898046 A CN201710898046 A CN 201710898046A CN 107610468 A CN107610468 A CN 107610468A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- data processing
- coordinate data
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses the speed density Analysis System and method based on recorder management, the speed density Analysis System based on recorder management includes：Node management unit, to the pending coordinate data of circulation reading from node database, according to the positional information of pending coordinate data, pending coordinate data is sent to corresponding data processing node；Data processing unit, connecting node administrative unit, data processing unit includes plurality of data processing node, each corresponding positional information of data processing node, data processing node exports the combined data after merging the pending coordinate data received is cleaned, changes, arranges and merged；Data storage cell, data processing unit is connected, combined data is stored in management database.The present invention can fast and efficiently carry out big data computing；Based on the speed density analysis of the data such as car speed section, operation period, section, carrier, the speed density data analyzed.
Description
Technical field
The invention belongs to field of traffic control, more particularly to a kind of speed density Analysis System based on recorder management and
Method.
Background technology
Current 10,000 vehicles that have more than daily use Space Science and Technology recorder management platform, the distribution of these vehicles online
Throughout the country, to be passed to daily to platform nearly 2,000 ten thousand position, speed, running status, the coordinate data of alarm condition etc..
Coordinate data adds up to get off, and will have 70~8,000,000,000 coordinate datas to be stored in platform every year.These data are really reacted
Each bar highway, provinces and cities' highway, the road conditions of city road, weather, the vehicle in the whole nation are assembled the information such as ground, and indirectly reacted
The information such as effluent stream hot zones, tourist hot spot area.But current Space Science and Technology recorder management platform is mainly used in car
Distribution statistics, the data volume of operation is big, and speed is slow.
The content of the invention
The present invention is big for the data volume of the existing operation of current Space Science and Technology recorder management platform, slow-footed to ask
Topic, there is provided a kind of fast performance of overall operation speed high speed density Analysis System and method based on recorder management.
A kind of speed density Analysis System based on recorder management of the present invention, including：
Node management unit, to the pending coordinate data of circulation reading from node database, according to described pending
The positional information of coordinate data, the pending coordinate data is sent to corresponding data processing node；
Data processing unit, connects the node management unit, and the data processing unit includes a plurality of data
Node is handled, each corresponding positional information of the data processing node, the data processing node is to the institute received
State pending coordinate data to be cleaned, changed, arranged and merged, the combined data after merging is exported；
Data storage cell, the data processing unit is connected, the combined data is stored in into management database
In.
Preferably, the node management unit includes：
Read module, the pending coordinate data is read temporally to stab the circulation from the node database；
First message passage, the read module is connected, to cache the pending seat that the read module is read
Mark data；
Distribution module, the first message passage is connected, to preset precision for every pending coordinate data generation
Coordinate Major key, and the Hash codes of the coordinate Major key, i.e.,：Positional information, each corresponding data processing of the Hash codes
Node, the pending coordinate data is sent to corresponding data processing node according to the Hash codes.
Preferably, the read module is in the pending coordinate data in reading the node database, if reading
The timestamp taken goes to preset time, then stops reading the pending coordinate data in the node database, meanwhile, to
The first message passage sends end, is each data after the distribution module receives the end
Handle node and send the end, the node management unit work finishes；The data processing node receives described
By data summarization after end, the combined data is obtained, and the end is sent to the data storage cell；When
After the data storage cell receives the end that all data processing nodes are sent, terminate storage behaviour
Make.
Preferably, the data processing node includes：
Second message channel, to cache the pending coordinate data that the node management unit is sent, and export；
Mapping block, second message channel is connected, to filter out no-fix in second message channel, and/or
The pending coordinate data of time anomaly, to obtain effective coordinate data, by the time of the effectively coordinate data monthly
Part, period carry out conversion and obtain month, period information, and the speed of the effectively coordinate data is segmented according to threshold speed
Acquisition speed section is changed, coordinate Major key, month, period information, the velocity shooting of the effectively coordinate data are mapped as a number
According to processing Major key；
Detection module, connect the modular converter, to according to the data processing Major key by effective number of coordinates
According to be stored in it is default conversion list in, also to detect it is described effectively coordinate data whether two month of span；
Abbreviation module, the detection module is connected, when effectively coordinate data two month of span, the abbreviation mould
Block by the effectively coordinate data of the current moon to carry out Reduction, to effective number of coordinates in the conversion list
According to computing is merged, add up out all data processing Major key identical coordinates sums, to obtain combined data and defeated
Go out, empty the conversion list, the detection module by effective coordinate data of the lower January being stored in default turn
Change in list.
Preferably, the data storage cell includes：
Receiving module, to receive the combined data of data processing node transmission；
3rd message channel, the receiving module is connected, to cache the combined data；
Preserving module, the 3rd message channel is connected, to by the combined data in the 3rd message channel
Batch is preserved to the management database.
Present invention also offers a kind of speed density analysis method based on recorder management, comprise the steps：
S1. pending coordinate data is read in circulation from node database, according to the position of the pending coordinate data
Information, the pending coordinate data is sent to corresponding data processing node, each data processing node corresponding one
Positional information；
S2. the data processing node is cleaned, is changed, arranged and closed to the pending coordinate data received
And the combined data after merging is exported；
S3. the combined data is stored in management database.
Preferably, the detailed process of the step S1 is：
S11. temporally stab the circulation from the node database and read the pending coordinate data；
S12. the pending coordinate data of reading is cached；
S13. it is the coordinate Major key of every pending default precision of coordinate data generation, and the Kazakhstan of the coordinate Major key
Uncommon code is positional information, each corresponding data processing node of the Hash codes, according to the Hash codes by the pending seat
Mark data are sent to corresponding data processing node.
Preferably, in the step S11 when reading the pending coordinate data in the node database, if
The timestamp of reading goes to preset time, then stops reading the pending coordinate data in the node database, meanwhile,
End is sent to first message passage, after receiving the end, institute is sent for each data processing node
State end；The data processing node is received data summarization after the end, obtains the combined data, when
After receiving the end that all data processing nodes are sent, terminate storage operation.
Preferably, the detailed process of the step S2 is：
S21. the pending coordinate data is cached, and is exported；
S22. no-fix in the second message channel, and/or the pending coordinate data of time anomaly are filtered out, to obtain
Effective coordinate data, the time of the effectively coordinate data is subjected to conversion by month, period and obtains month, period information, will
The speed of the effectively coordinate data carries out segmentation conversion acquisition speed section according to threshold speed, by the effectively coordinate data
Coordinate Major key, month, period information, velocity shooting are mapped as a data processing Major key；
S23. also used in the default conversion list of the effectively coordinate data deposit according to the data processing Major key
With detect it is described effectively coordinate data whether two month of span；
S24. when effectively coordinate data two month of span, by the effectively coordinate data of the current moon
Letter operation, computing is merged to the effectively coordinate data in the conversion list, add up out all data processings
Major key identical coordinate sum, to obtain combined data and export, empties the conversion list, by having for the lower January
Imitate in the default conversion list of coordinate data deposit.
Preferably, the detailed process of the S3 is：
S31. the combined data that data processing node is sent is received；
S32. the combined data is cached；
S33. the combined data batch in the 3rd message channel is preserved to the management database.
Above-mentioned technical characteristic can in any suitable manner be combined or substituted by equivalent technical characteristic, as long as can reach
To the purpose of the present invention.
The beneficial effects of the present invention are can fast and efficiently carry out big data computing；Based on car speed section, fortune
The speed density analysis of the data such as row period, section, carrier, the speed density data analyzed, in order to be looked forward to for transport
The safe driving of industry provides authentic and valid data and supported.
Brief description of the drawings
Fig. 1 is that a kind of module of embodiment of the speed density Analysis System of the present invention based on recorder management is shown
It is intended to；
Fig. 2 is a kind of method stream of embodiment of the speed density analysis method of the present invention based on recorder management
Cheng Tu.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art obtained on the premise of creative work is not made it is all its
His embodiment, belongs to the scope of protection of the invention.
It should be noted that in the case where not conflicting, the feature in embodiment and embodiment in the present invention can phase
Mutually combination.
The invention will be further described with specific embodiment below in conjunction with the accompanying drawings, but not as limiting to the invention.
As shown in figure 1, a kind of speed density Analysis System based on recorder management, including：Node management unit
(NodeManager) 2, data processing unit 3 and data storage cell (DataSaver) 4；
Node management unit 2, to the pending coordinate data of circulation reading from node database 1, according to pending seat
The positional information of data is marked, pending coordinate data is sent to corresponding data processing node (HandleNode) 31, each
The corresponding positional information of data processing node 31；
Data processing unit 3, connecting node administrative unit 2, data processing unit 3 include plurality of data processing node
31, data processing node 31 will merge the pending coordinate data received is cleaned, changes, arranges and merged
Combined data output afterwards；
Data storage cell 4, data processing unit 3 is connected, combined data is stored in management database 5.
In the present embodiment, by node management unit 2 from node database 1 reading mass data, and by magnanimity number
According to each data processing node 31 is distributed to, wait pending.After data processing node 31 receives data, data are carried out clear
The computing such as wash, change, arranging, merging, and result is sent to data storage cell 4.Data storage cell 4 receives respectively
The result that individual data processing node 31 is sent, it is saved into batches in management database 5, completes big data processing.It is based on
The speed density Analysis System of recorder management can fast and efficiently carry out big data computing；Based on car speed section, operation
The speed density analysis of the data such as period, section, carrier, the speed density data analyzed, in order to be transport enterprise
Safe driving authentic and valid data be provided supported.The speed density data that the network analysis goes out, can on map each bar
Highway generates the VELOCITY DISTRIBUTION thermodynamic chart of different periods, and authentic and valid data support is provided for the safe driving of transport enterprise.
In a preferred embodiment, node management unit 2 includes：Read module (reader) 22, first message passage
(message channel) 21 and distribution module (sender) 23；
Read module 22, pending coordinate data is read temporally to stab the circulation from node database 1；
First message passage 21, read module 22 is connected, to cache the pending coordinate data of the reading of read module 22；
Distribution module 23, connection first message passage 21, to preset precision for every pending coordinate data generation
Coordinate major key (KEY) value, and the Hash codes (hashCode) of coordinate Major key, i.e.,：Positional information, the corresponding number of each Hash codes
According to processing node 31, pending coordinate data is sent to corresponding data processing node 31 according to Hash codes.
In the present embodiment, read module 22 temporally stabs, and circulates and pending number of coordinates is read from node database 1
According to, and transmit data in first message passage 21 and cached.Distribution module 23 receives the seat in first message passage 21
Data are marked, are that every coordinate data generates the coordinate KEY values of 100 meters of precision and the hashCode of the KEY values.Distribution module 23
Coordinate is distributed to corresponding data processing node 31 according to hashCode, to ensure position identical vehicle, in same number
Data processing is carried out according to handling in node 31.
Be applied to LnglatMsg classes in the present embodiment, such be in system the coordinate data in need handled
Message base class, due to the message need carry out network transmission, so need realize serializing.Such can be generated with specified
The Key values of precision.Such hashCode is to calculate gained according to Key values, so, when sender is according to such hashCode
When carrying out route data, the coordinate in same accuracy rating can be ensured, must be routed in same HandleNode.Such
Type of message attribute is provided simultaneously, when property value is " MT_END ", it is end to represent the object.LnglatMsg classes have
The implementation of body is as follows：
The object of Sender classes instantiation is also applied in the present embodiment, the be responsible in receiving node administrative unit 2
Message in one message channel 21, and the message is routed in the data processing node 31 of correlation.The object is responsible for reception two
Class message, it is necessary to handle the message when by common message, to obtain the hashCode values of the message, grasp herein
The coordinate in the range of given accuracy is must assure that, there is identical hashCode values (i.e.：Car in the range of 100 meters of circumference
, its hashCode value is identical).Therefore, Sender classes just be can guarantee that, the coordinate in the same area is sent to same
Data processing is carried out in individual data processing node 31.Sender class concrete implementation modes are as follows：
In a preferred embodiment, data processing node 31 includes：Second message channel 311, mapping block (Map) 312,
Detection module (Shuffle) 313 and abbreviation module (Reduce) 314；
Second message channel 311, the pending coordinate data sent to cache node administrative unit 2, and export；
Mapping block 312, the second message channel 311 is connected, to filter out no-fix in the second message channel 311, and/or
The pending coordinate data of time anomaly, to obtain effective coordinate data, the time of effective coordinate data is entered by month, period
Row conversion obtains month, period information, and the speed of effective coordinate data is carried out into segmentation conversion acquisition speed according to threshold speed
Section, a data processing major key is mapped as by coordinate Major key, month, period information, the velocity shooting of effective coordinate data
(HandleKEY) value；
Detection module 313, modular converter is connected, it is default to be stored in effective coordinate data according to data processing Major key
Conversion list in, also to detect effective coordinate data whether two month of span；
Abbreviation module 314, connection detection module 313, when effective two month of coordinate data span, abbreviation module 314
Effective coordinate data of the current moon is carried out into Reduction, fortune is merged to effective coordinate data in conversion list
Calculate, add up out all data processing Major key identical coordinate sums, to obtain combined data and export, empties conversion list,
Detection module 313 is to by the default conversion list of effective coordinate data of lower January deposit.
In the present embodiment, after data processing node 31 receives the incoming message of node management unit 2, cache to the
In two message channels 311, after mapping block 312 receives the data in the second message channel 311, no-fix, time are washed
Abnormal etc. invalid data, meanwhile, by the time of valid data according to month, the period (such as：1 hour is 1 section, i.e.,：When 0~1
For 1 section, for 2 sections, the rest may be inferred when 1~2, until 24 sections) changed, speed is subjected to segmentation conversion (i.e. according to 10：1~10
For 10 sections, 10~20 be 20 sections, and the rest may be inferred), finally, coordinate KEY values, carrier, velocity shooting, month, period etc. are believed
Breath, a new KEY value is synthesized, is designated as HandleKEY, the computing for subsequent step.Detection module 313 is connected to mapping block
After 312 incoming data, according to the HandleKEY values of data, internal memory Map is deposited into<Object,List>In table, ultimately form
Shaped like<lnglat(handlekey1),[1,1,1,1]>,<lnglat(handlekey2),[1,1]>One group of data.Meanwhile
Detection module 313 is also responsible for checking whether coordinate data strides into next month, when coordinate enters next month, then triggeringization
Simple module 314 performs Reduction, and the data of this month are carried out into Reduction, meanwhile, empty Map<Object,List>Table, it is right
The data in next month carry out detection operation.After Reduction is triggered by shuffle, to Map<Object,List>Number in table
According to computing is merged, add up out all HandleKEY values identical coordinates sum, and by summarized results, sends to data and deposit
Storage unit 4.
Map classes i.e. mapping class is applied in the present embodiment, refers to by attribute is identical or approximate object, be mapped to attribute
Value identical one group objects (such as：Under same viaduct, between 4 points to 5 points, vehicle that speed travels between 20 to 30,
The all properties of its coordinates object being mapped to are all identical, and equals computings are carried out between the group objects, and its result is also
true).Map class concrete implementation modes are as follows：
Also apply Shuffle classes in the present embodiment, Shuffle first task, be the message that will be received again
Wash in handleMap (data processing mapping table) memory table, because LnglatMsg classes have rewritten equal () function,
HandleMap can find with given message attribute value identical element, and to the element value increase by 1 quantity.Shuffle
Second task, be when scheduling Reduce can work, and the content for needing work be transmitted to Reduce.
Shuffle class concrete implementation modes are as follows：
Reduce classes are also applied in the present embodiment, and Reduce classes are responsible for two pieces thing, Shuffle is passed to first
HandleMap data are calculated, and obtain the number that each Major key occurs, i.e. carrier, velocity shooting, period, location phase
The number that same coordinate occurs.Second task, the result after calculating is transmitted to data storage cell 4.Reduce classes are specific
Implementation is as follows：
In a preferred embodiment, data storage cell 4 includes：The message channel of receiving module (Accepter) the 42, the 3rd
41 and preserving module (Saver) 43；
Receiving module 42, to receive the combined data of the transmission of data processing node 31；
3rd message channel 41, receiving module 42 is connected, to cache combined data；
Preserving module 43, the 3rd message channel 41 is connected, the combined data batch in the 3rd message channel 41 to be protected
Deposit to management database 5.
In the present embodiment, data storage cell 4 receives the incoming summarized results of all data processing nodes 31, caching
Into the 3rd message channel 41, and by preserving module 43 by the data in the 3rd message channel 41, batch is preserved to management data
In storehouse 5.
In a preferred embodiment, read module 22 is in the pending coordinate data in reading node database 1, if reading
The timestamp taken goes to preset time, then stops reading the pending coordinate data in node database 1, meanwhile, disappear to first
Cease passage 21 and send end (end message), be each data processing section after distribution module 23 receives end
Point 31 sends end, and the work of node management unit 2 finishes；Data processing node 31 converges data after receiving end
Always, combined data is obtained, and end is sent to data storage cell 4；When data storage cell 4 receives all data
After handling the end that node 31 is sent, terminate storage operation.
In the present embodiment, end is a particular message, is initiated by the read module 22 of node management unit 2,
Preserving module 43 to data storage cell 4 terminates.Whole process is not involved in collecting computing, is only each several part of computing, brings data
The mark that computing finishes.In node management unit 2, when read module 22 reads data from node database 1
Between after stamp goes to the time specified, will no longer read data from database, while send end to first message passage 21 and disappear
Breath.After distribution module 23 receives end, end, data processing node 31 are sent for each data processing node 31
Work finishes.In data processing node 31, mapping block 312 receives that the message is without any processing (can not to clean, change
Deng), it is directly transmitted to detection module 313.After detection module 313 is connected to the message, abbreviation module 314 is triggered at once and performs abbreviation behaviour
Make.Abbreviation module 314 is disposed after the packet of end, and end is sent to data storage cell 4.Deposited in data
In storage unit 4, receiving module 42 receives and records the end of each data processing node 31 transmission, when all data
Processing node 31 all have sent end, and receiving module 42 sends one article of end to the 3rd message channel 41.Preserve mould
After block 43 receives the message, terminate data storage cell 4, the data processing of whole system finishes.
As shown in Fig. 2 present invention also offers a kind of speed density analysis method based on recorder management, including it is following
Step：
S1. pending coordinate data is read in circulation from node database, according to the positional information of pending coordinate data,
Pending coordinate data is sent to corresponding data processing node, the corresponding positional information of each data processing node；
S2. data processing node is cleaned, is changed, arranged and merged to the pending coordinate data received, will be closed
Combined data output after and；
S3. combined data is stored in management database.
In the present embodiment, the speed density analysis method based on recorder management can be counted fast and efficiently greatly
According to computing；Based on the speed density analysis of the data such as car speed section, operation period, section, carrier, the speed analyzed
Density data, supported in order to provide authentic and valid data for the safe driving of transport enterprise.
In a preferred embodiment, step S1 detailed process is：
S11. temporally stab the circulation from node database and read pending coordinate data；
S12. the pending coordinate data of reading is cached；
S13. it is the coordinate Major key of every pending default precision of coordinate data generation, and the Hash codes of coordinate Major key
That is positional information, the corresponding data processing node of each Hash codes, sends pending coordinate data to corresponding according to Hash codes
Data processing node.
In the present embodiment, temporally stab, circulate and pending coordinate data is read from node database, and by data
It is sent in first message passage and is cached.The coordinate KEY values and the KEY of 100 meters of precision are generated for every coordinate data
The hashCode of value.Coordinate is distributed to corresponding data processing node according to hashCode, to ensure position identical vehicle,
Data processing is carried out in same data processing node.
In a preferred embodiment, in step s 11 when reading the pending coordinate data in node database, if reading
The timestamp taken goes to preset time, then stops reading the pending coordinate data in node database, meanwhile, to first message
Passage sends end, and after receiving end, end is sent for each data processing node；Data processing node
Receive data summarization after end, obtain combined data, when the end for receiving all data processing nodes and sending
After message, terminate storage operation.
In a preferred embodiment, step S2 detailed process is：
S21. pending coordinate data is cached, and is exported；
S22. no-fix in the second message channel, and/or the pending coordinate data of time anomaly are filtered out, it is effective to obtain
Coordinate data, the time of effective coordinate data is subjected to conversion by month, period and obtains month, period information, by effective coordinate
The speed of data according to threshold speed carry out segmentation conversion acquisition speed section, by the coordinate Major key of effective coordinate data, month,
Period information, velocity shooting are mapped as a data processing Major key；
S23. according to data processing Major key by the default conversion list of effective coordinate data deposit, also having detected
Imitate coordinate data whether two month of span；
S24. when effective two month of coordinate data span, effective coordinate data of the current moon is subjected to Reduction,
Computing is merged to effective coordinate data in conversion list, all data processing Major key identical coordinates that add up out are total
Number, to obtain combined data and export, empties conversion list, and effective coordinate data of lower January is stored in into default conversion table
Dan Zhong.
In the present embodiment, after data processing node receives incoming message, cache into the second message channel, connect
After receiving the data in the second message channel, the invalid data of no-fix, time anomaly etc. is washed, meanwhile, by valid data
Time according to month, the period (such as：1 hour is 1 section, i.e.,：It it is 1 section when 0~1, for 2 sections, the rest may be inferred when 1~2, until 24
Section) changed, speed is subjected to segmentation conversion (i.e. according to 10：1~10 is 10 sections, and 10~20 be 20 sections, and the rest may be inferred), most
Afterwards, by information such as coordinate KEY values, carrier, velocity shooting, month, periods, a new KEY value is synthesized, is designated as HandleKEY,
Computing for subsequent step.After being connected to incoming data, according to the HandleKEY values of data, internal memory Map is deposited into<
Object,List>In table, ultimately form shaped like<lnglat(handlekey1),[1,1,1,1]>,<lnglat
(handlekey2),[1,1]>One group of data.Meanwhile also check for whether coordinate data strides into next month, when coordinate enters
Entering next month, then triggering performs Reduction, and the data of this month are carried out into Reduction, meanwhile, empty Map<Object,
List>Table, detection operation is carried out to the data in next month.After Reduction is triggered by shuffle, to Map<Object,
List>Data in table merge computing, and add up out all HandleKEY values identical coordinates sum, and by summarized results,
Send to data storage cell.
In a preferred embodiment, S3 detailed process is：
S31. the combined data that data processing node is sent is received；
S32. combined data is cached；
S33. the combined data batch in the 3rd message channel is preserved to management database.
In the present embodiment, the incoming summarized results of all data processing nodes is received, is cached to the 3rd message channel
In, and by the data in the 3rd message channel, batch is preserved into management database.
The present invention applies spring Http Invoker frameworks, and spring Http Invoker are in spring frameworks
One far call model, perform the far call based on HTTP (HTTP).The present invention can with cross-platform deployment,
Due to using spring HttpInvoker to carry out rpc (Remote Procedure Call Protocol, remote process
Invocation protocol) communication so that each subsystem can carry out data communication between different platforms.Using spring
Integration is using solid in framework as the error-prone part by mistake such as system framework, multithreading, timer, message channel
The mature and stable module having, use is called by xml (extensible markup language) configuration file, which is by research staff
Freed from the technical problem of complexity, concentrate one's energy to pay close attention to the realization of specific service logic, substantially increase research and development speed
The stability of degree and whole system.The major calculations of system, caching part, are concentrated mainly in HandleNode, the node can
According to task amount, flexibly to be expanded, so as to the integral operation performance of lifting system.HandleNode, which is provided, to have ready conditions
Reduce strategies, in whole big data processing procedure, change according to condition, ceaselessly trigger Reduce tasks, output processing
As a result, so as to effectively reduce EMS memory occupation situation, the overall operational performance of lifting system.
Although the present invention is described herein with reference to specific embodiment, it should be understood that, these realities
Apply the example that example is only principles and applications.It should therefore be understood that exemplary embodiment can be carried out
Many modifications, and other arrangements are can be designed that, the spirit of the invention limited without departing from appended claims
And scope.It should be understood that different appurtenances can be combined by way of different from described by original claim
Profit requires and feature specifically described herein.It will also be appreciated that the feature with reference to described by separate embodiments can use
In other described embodiments.
Claims (10)
- A kind of 1. speed density Analysis System based on recorder management, it is characterised in that including：Node management unit, to the pending coordinate data of circulation reading from node database, according to the pending coordinate The positional information of data, the pending coordinate data is sent to corresponding data processing node；Data processing unit, connects the node management unit, and the data processing unit includes a plurality of data processings Node, each corresponding positional information of the data processing node, the data processing node is to treating described in receiving Processing coordinate data is cleaned, is changed, arranged and merged, and the combined data after merging is exported；Data storage cell, the data processing unit is connected, the combined data is stored in management database.
- 2. the speed density Analysis System according to claim 1 based on recorder management, it is characterised in that the node Administrative unit includes：Read module, the pending coordinate data is read temporally to stab the circulation from the node database；First message passage, the read module is connected, to cache the pending number of coordinates that the read module is read According to；Distribution module, the first message passage is connected, to the coordinate for every pending default precision of coordinate data generation Major key, and the Hash codes of the coordinate Major key, i.e.,：Positional information, each corresponding data processing node of the Hash codes, The pending coordinate data is sent to corresponding data processing node according to the Hash codes.
- 3. the speed density Analysis System according to claim 2 based on recorder management, it is characterised in that the reading Module is in the pending coordinate data in reading the node database, if the timestamp read goes to preset time, Then stop reading the pending coordinate data in the node database, meanwhile, send and tie to the first message passage Beam message, after the distribution module receives the end, send the end for each data processing node and disappear Breath, the node management unit work finish；The data processing node is received data summarization after the end, is obtained The combined data is taken, and the end is sent to the data storage cell；When the data storage cell receives After the end that all data processing nodes are sent, terminate storage operation.
- 4. the speed density Analysis System according to claim 1 based on recorder management, it is characterised in that the data Processing node includes：Second message channel, to cache the pending coordinate data that the node management unit is sent, and export；Mapping block, second message channel is connected, to filter out no-fix in second message channel, and/or time The abnormal pending coordinate data, to obtain effective coordinate data, by the time of the effectively coordinate data by month, when Duan Jinhang conversions obtain month, period information, and the speed of the effectively coordinate data is carried out into segmentation conversion according to threshold speed Acquisition speed section, coordinate Major key, month, period information, the velocity shooting of the effectively coordinate data are mapped as at a data Manage Major key；Detection module, the modular converter is connected, the effectively coordinate data is deposited according to the data processing Major key Enter it is default conversion list in, also to detect it is described effectively coordinate data whether two month of span；Abbreviation module, the detection module is connected, when effectively coordinate data two month of span, the abbreviation module is used So that the effectively coordinate data of the current moon is carried out into Reduction, the effectively coordinate data in the conversion list is entered Row union operation, add up out all data processing Major key identical coordinate sums, to obtain combined data and export, clearly The empty conversion list, the detection module by effective coordinate data of the lower January being stored in default conversion list In.
- 5. the speed density Analysis System according to claim 1 based on recorder management, it is characterised in that the data Memory cell includes：Receiving module, to receive the combined data of data processing node transmission；3rd message channel, the receiving module is connected, to cache the combined data；Preserving module, the 3rd message channel is connected, to by the combined data batch in the 3rd message channel Preserve to the management database.
- 6. a kind of speed density analysis method based on recorder management, it is characterised in that comprise the steps：S1. pending coordinate data is read in circulation from node database, according to the positional information of the pending coordinate data, The pending coordinate data is sent to corresponding data processing node, each corresponding position letter of the data processing node Breath；S2. the data processing node is cleaned, is changed, arranged and merged to the pending coordinate data received, Combined data after merging is exported；S3. the combined data is stored in management database.
- 7. the speed density analysis method according to claim 6 based on recorder management, it is characterised in that the step S1 detailed process is：S11. temporally stab the circulation from the node database and read the pending coordinate data；S12. the pending coordinate data of reading is cached；S13. it is the coordinate Major key of every pending default precision of coordinate data generation, and the Hash codes of the coordinate Major key That is positional information, each corresponding data processing node of the Hash codes, according to the Hash codes by the pending number of coordinates According to transmission to corresponding data processing node.
- 8. the speed density analysis method according to claim 7 based on recorder management, it is characterised in that in the step In rapid S11 when reading the pending coordinate data in the node database, if the timestamp read goes to default Between, then stop reading the pending coordinate data in the node database, meanwhile, send and terminate to first message passage Message, after receiving the end, the end is sent for each data processing node；The data processing Node is received data summarization after the end, obtains the combined data, when receiving at all data After managing the end that node is sent, terminate storage operation.
- 9. the speed density analysis method according to claim 6 based on recorder management, it is characterised in that the step S2 detailed process is：S21. the pending coordinate data is cached, and is exported；S22. no-fix in the second message channel, and/or the pending coordinate data of time anomaly are filtered out, it is effective to obtain Coordinate data, the time of the effectively coordinate data is subjected to conversion by month, period and obtains month, period information, by described in The speed of effective coordinate data carries out segmentation conversion acquisition speed section according to threshold speed, by the coordinate of the effectively coordinate data Major key, month, period information, velocity shooting are mapped as a data processing Major key；S23. according to the data processing Major key by the default conversion list of the effectively coordinate data deposit, also examining Survey it is described effectively coordinate data whether two month of span；S24. when effectively coordinate data two month of span, the effectively coordinate data of the current moon is subjected to abbreviation behaviour Make, computing is merged to the effectively coordinate data in the conversion list, add up out all data processing major keys It is worth identical coordinate sum, to obtain combined data and export, the conversion list is emptied, by effective seat of the lower January Mark in the default conversion list of data deposit.
- 10. the speed density analysis method according to claim 6 based on recorder management, it is characterised in that the S3 Detailed process be：S31. the combined data that data processing node is sent is received；S32. the combined data is cached；S33. the combined data batch in the 3rd message channel is preserved to the management database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710898046.3A CN107610468A (en) | 2017-09-28 | 2017-09-28 | Speed density Analysis System and method based on recorder management |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710898046.3A CN107610468A (en) | 2017-09-28 | 2017-09-28 | Speed density Analysis System and method based on recorder management |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107610468A true CN107610468A (en) | 2018-01-19 |
Family
ID=61059237
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710898046.3A Pending CN107610468A (en) | 2017-09-28 | 2017-09-28 | Speed density Analysis System and method based on recorder management |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107610468A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509646A (en) * | 2018-04-13 | 2018-09-07 | 航天科技控股集团股份有限公司 | A kind of method and system of real-time judgment vehicle drift applied to recorder management platform |
| CN108520071A (en) * | 2018-04-13 | 2018-09-11 | 航天科技控股集团股份有限公司 | A kind of log searching system and method based on recorder platform |
| CN108616523A (en) * | 2018-04-13 | 2018-10-02 | 航天科技控股集团股份有限公司 | A kind of message handling system and method based on intelligent management platform |
| CN110136436A (en) * | 2019-05-13 | 2019-08-16 | 特斯联（北京）科技有限公司 | A kind of road conditions sharing method and equipment based on information database |
| CN117171701A (en) * | 2023-08-14 | 2023-12-05 | 陕西天行健车联网信息技术有限公司 | Vehicle running data processing method, device, equipment and medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6333703B1 (en) * | 1998-11-24 | 2001-12-25 | International Business Machines Corporation | Automated traffic mapping using sampling and analysis |
| CN100463407C (en) * | 2006-12-25 | 2009-02-18 | 北京世纪高通科技有限公司 | Method and system for real-time dynamic traffic information collecting, handling, and issuing |
| US7949463B2 (en) * | 2003-12-15 | 2011-05-24 | Gary Ignatin | Information filtering and processing in a roadway travel data exchange network |
| CN103903436A (en) * | 2012-12-28 | 2014-07-02 | 上海优途信息科技有限公司 | Expressway traffic jam detecting method and system based on floating car |
| CN104424220A (en) * | 2013-08-23 | 2015-03-18 | 阿里巴巴集团控股有限公司 | Data processing method and equipment |
| CN105354896A (en) * | 2015-11-13 | 2016-02-24 | 江苏南亿迪纳数字科技发展有限公司 | Automobile driving recorder locus segmentation method |
| CN105894610A (en) * | 2015-11-11 | 2016-08-24 | 乐卡汽车智能科技（北京）有限公司 | Data processing method, device and system |
| CN106558221A (en) * | 2016-11-29 | 2017-04-05 | 北京掌行通信息技术有限公司 | Real-time distributed traffic information processing system |
-
2017
- 2017-09-28 CN CN201710898046.3A patent/CN107610468A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6333703B1 (en) * | 1998-11-24 | 2001-12-25 | International Business Machines Corporation | Automated traffic mapping using sampling and analysis |
| US7949463B2 (en) * | 2003-12-15 | 2011-05-24 | Gary Ignatin | Information filtering and processing in a roadway travel data exchange network |
| CN100463407C (en) * | 2006-12-25 | 2009-02-18 | 北京世纪高通科技有限公司 | Method and system for real-time dynamic traffic information collecting, handling, and issuing |
| CN103903436A (en) * | 2012-12-28 | 2014-07-02 | 上海优途信息科技有限公司 | Expressway traffic jam detecting method and system based on floating car |
| CN104424220A (en) * | 2013-08-23 | 2015-03-18 | 阿里巴巴集团控股有限公司 | Data processing method and equipment |
| CN105894610A (en) * | 2015-11-11 | 2016-08-24 | 乐卡汽车智能科技（北京）有限公司 | Data processing method, device and system |
| CN105354896A (en) * | 2015-11-13 | 2016-02-24 | 江苏南亿迪纳数字科技发展有限公司 | Automobile driving recorder locus segmentation method |
| CN106558221A (en) * | 2016-11-29 | 2017-04-05 | 北京掌行通信息技术有限公司 | Real-time distributed traffic information processing system |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509646A (en) * | 2018-04-13 | 2018-09-07 | 航天科技控股集团股份有限公司 | A kind of method and system of real-time judgment vehicle drift applied to recorder management platform |
| CN108520071A (en) * | 2018-04-13 | 2018-09-11 | 航天科技控股集团股份有限公司 | A kind of log searching system and method based on recorder platform |
| CN108616523A (en) * | 2018-04-13 | 2018-10-02 | 航天科技控股集团股份有限公司 | A kind of message handling system and method based on intelligent management platform |
| CN110136436A (en) * | 2019-05-13 | 2019-08-16 | 特斯联（北京）科技有限公司 | A kind of road conditions sharing method and equipment based on information database |
| CN110136436B (en) * | 2019-05-13 | 2020-04-24 | 特斯联（北京）科技有限公司 | Information database-based road condition sharing method and equipment |
| CN117171701A (en) * | 2023-08-14 | 2023-12-05 | 陕西天行健车联网信息技术有限公司 | Vehicle running data processing method, device, equipment and medium |
| CN117171701B (en) * | 2023-08-14 | 2024-05-14 | 陕西天行健车联网信息技术有限公司 | Vehicle running data processing method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107610468A (en) | Speed density Analysis System and method based on recorder management | |
| Clemente et al. | The vehicle relocation problem in car sharing systems: Modeling and simulation in a petri net framework | |
| CN100484148C (en) | Information terminals, information sharing method and p2p system and point system using the same | |
| CN103854472B (en) | Taxi cloud intelligent dispatching method and system | |
| CN102968494B (en) | The system and method for transport information is gathered by microblogging | |
| CN105868861A (en) | Bus passenger flow evolution analysis method based on time-space data fusion | |
| Tan et al. | New path size formulation in path size logit for route choice modeling in public transport networks | |
| CN105761492A (en) | Method for dynamic flow distribution of wide-range high-speed road network based on network flows | |
| CN106528720A (en) | Transfer station recommendation method and transfer station recommendation system | |
| CN107730877A (en) | Towards the reservation trip management system and method for the quick trunk road network of overhead road of city | |
| CN106097060A (en) | A kind of university students is left unused Cycle Hire software screening method system and its implementation | |
| CN111625604B (en) | Credible evaluation system and method for travel service quality based on block chain | |
| CN110737665B (en) | Data processing method and device | |
| Bouziyane et al. | Solving dynamic vehicle routing problem with soft time windows basing on the static problem resolution by a hybrid approach | |
| CN111353759B (en) | Dynamic calculation method, device, equipment and storage medium for line transportation cost | |
| CN111739180B (en) | Path segmentation method based on ETC portal frame | |
| CN101276442A (en) | System for clearing and classifying track traffic tickets as well as method using the same | |
| EP3570569B1 (en) | Measuring and analyzing the movement of mobile devices with the help of a wireless telecommunication network | |
| CN105976091A (en) | Individual activity steady state training method under large data environment | |
| CN112418535A (en) | School bus scheduling planning method based on binary tree | |
| Leong | On real-time distributed geographical database systems | |
| CN107633682B (en) | Vehicle region temperature analysis system and method based on recorder management platform | |
| CN109727391B (en) | Vehicle leasing system and method based on feature search | |
| CN109426433A (en) | Read-write Catrol, car networking data processing method and service background, car networking system | |
| CN109040290A (en) | A method of host node is elected based on node diagnostic |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180119 |
|
| RJ01 | Rejection of invention patent application after publication |