CN106485915B - A kind of information processing method, device and MapReduce platform - Google Patents
A kind of information processing method, device and MapReduce platform Download PDFInfo
- Publication number
- CN106485915B CN106485915B CN201510531824.6A CN201510531824A CN106485915B CN 106485915 B CN106485915 B CN 106485915B CN 201510531824 A CN201510531824 A CN 201510531824A CN 106485915 B CN106485915 B CN 106485915B
- Authority
- CN
- China
- Prior art keywords
- grid
- gps data
- node
- processing node
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Traffic Control Systems (AREA)
- Position Fixing By Use Of Radio Waves (AREA)
Abstract
This application discloses a kind of information processing method, device and MapReduce platforms, and wherein method comprises determining that at least one corresponding grid of GPS data of Floating Car passback, the grid are to obtain after carrying out grid dividing to electronic map in advance；Determine processing node corresponding with the grid；The GPS data is distributed to the processing node, so that the processing node determines the traffic behavior of road in the grid.The present processes are by being divided into multiple grids for electronic map, each processing node can be merely responsible for the corresponding GPS data of several grids, it is handled by the way that a large amount of GPS data is distributed to multiple processing nodes, so that overall treatment efficiency is greatly promoted.
Description
Technical field
This application involves technical field of data processing, more specifically to a kind of information processing method, device and
MapReduce platform.
Background technique
Traffic information processing technique has had the development of many years, and current main map web site and electronic map client side are all
Real-time Traffic Information service has been accessed, user can be allowed to see entire city or urban parts region Nei Dao on the electronic map
The traffic condition (jogging, congestion, unimpeded) on road.
Existing traffic information processing is that the GPS data returned in real time based on Floating Car is analyzed.Since Floating Car returns
Data volume is larger, and individual node (node) is difficult to calculate the traffic information of national all areas.Therefore, existing traffic information processing
Service is usually the GPS data work in the same city or same road set (such as whole nation high speed) that are returned with Floating Car
It is handled for a unit in the same node, that is, the GPS data of each unit is handled by a node.
But the city or a kind of data volume of the GPS data of road set due to Floating Car passback are also very huge
Greatly, there are the limit for the processing capacity and storage capacity of individual node, and therefore, there are still processing for existing traffic information processing scheme
Low efficiency, the problem of taking considerable time.
Summary of the invention
In view of this, this application provides a kind of information processing method, device and MapReduce platforms, for solving
The problem of certainly existing traffic information processing mode can not quickly be handled a large amount of GPS datas.
To achieve the goals above, it is proposed that scheme it is as follows:
A kind of information processing method, comprising:
Determine Floating Car passback at least one corresponding grid of GPS data, the grid be in advance to electronic map into
It is obtained after row grid dividing；
Determine processing node corresponding with the grid；
The GPS data is distributed to the processing node, so that the processing node determines road in the grid
Traffic behavior.
Preferably, at least one corresponding grid of GPS data of the determining Floating Car passback, comprising:
The boundary of grid is translated into extension outward, obtains the corresponding Expanding grid of the grid；
Determine at least one Expanding grid belonging to positioning coordinate that the GPS data includes；
The determining corresponding grid of Expanding grid is determined as the corresponding grid of the GPS data.
Preferably, determination processing node corresponding with the grid, comprising:
Hash processing is carried out to the mark of the grid, obtains the cryptographic Hash of the mark；
With the cryptographic Hash of the mark divided by the total number n of processing node, remainder is obtained；
Serial number processing node identical with the remainder is determined as the corresponding processing node of the grid, wherein each
The serial number of node is handled by 0 to n-1, uniquely and is not repeated.
A kind of information processing method is applied in processing node, this method comprises:
Receive GPS data；
It determines at least one corresponding grid of the GPS data, and judges whether determining grid belongs to present treatment node
Corresponding grid；
If belonging to, traffic behavior calculating is carried out using the GPS data received, obtains the traffic of road in the grid
State.
Preferably, before the corresponding grid of the determination GPS data, further includes:
It detects with the presence or absence of identical GPS data in received GPS data, if so, retaining in identical GPS data
One GPS data deletes other duplicate GPS datas.
It is preferably, described to judge whether determining grid belongs to the corresponding grid of present treatment node, comprising:
Hash processing is carried out to the mark of determining grid, obtains the cryptographic Hash of the mark；
With the cryptographic Hash of the mark divided by the total number n of processing node, remainder is obtained；
Judge whether the remainder is identical as the serial number of present treatment node, wherein the serial number of each processing node is by 0 to n-
1, uniquely and do not repeat；
If so, determining that the grid belongs to the corresponding grid of present treatment node.
A kind of traffic information processing apparatus, comprising:
Grid determination unit, at least one the corresponding grid of GPS data returned for determining Floating Car, the grid are
It is obtained after carrying out grid dividing to electronic map in advance；
Node determination unit, for determining processing node corresponding with the grid；
File distributing unit, for the GPS data to be distributed to the processing node, so that the processing node determines
The traffic behavior of road in the grid.
Preferably, the grid determination unit includes:
First grid determines subelement, for the boundary of grid to be translated extension outward, obtains the corresponding expansion of the grid
Open up grid；
Second grid determines subelement, for determining at least one expansion belonging to positioning coordinate that the GPS data includes
Open up grid；
Third grid determines subelement, for the corresponding grid of the Expanding grid determined to be determined as the GPS data pair
The grid answered.
Preferably, the node determination unit includes:
First node determines subelement, carries out Hash processing for the mark to the grid, obtains the Kazakhstan of the mark
Uncommon value；
Second node determines subelement, for the cryptographic Hash using the mark divided by the total number n of processing node, obtains
Remainder；
Third node determines subelement, for serial number processing node identical with the remainder to be determined as the grid pair
The processing node answered, wherein the serial number of each processing node uniquely and is not repeated by 0 to n-1.
A kind of traffic information processing apparatus is applied in processing node, which includes:
Data receipt unit, for receiving GPS data；
Grid judging unit for determining at least one corresponding grid of the GPS data, and judges that determining grid is
It is no to belong to the corresponding grid of present treatment node；
Traffic behavior computing unit, for utilizing the GPS received when the grid judging unit judging result, which is, is
Data carry out traffic behavior calculating, obtain the traffic behavior of road in the grid.
It is a kind of for handling the MapReduce platform of traffic information, comprising: data distributing node and processing node；
Data distributing node, for determining at least one corresponding grid of GPS data of Floating Car passback, it is determining with it is described
The GPS data is distributed to the processing node by the corresponding processing node of grid, wherein the grid is in advance to electronically
Figure obtains after carrying out grid dividing；
It handles node and determines at least one corresponding grid of the GPS data, and judge for receiving the GPS data
Whether determining grid belongs to the corresponding grid of present treatment node, for belonging to the grid of the corresponding grid of present treatment node, benefit
Traffic behavior calculating is carried out with the GPS data received, obtains the traffic behavior of road in the grid.
It can be seen from the above technical scheme that information processing method provided by the embodiments of the present application is in advance to electronics
Map carries out grid dividing, after the GPS data for receiving Floating Car passback, determines at least one corresponding grid of GPS data,
Then it determines processing node corresponding with the grid, and then the GPS data is distributed to processing node, node for processing
Determine the traffic behavior of road in the grid.The present processes are by being divided into multiple grids, each place for electronic map
Reason node can be merely responsible for the corresponding GPS data of several grids, by the way that a large amount of GPS data is distributed to multiple processing nodes
It is handled, so that overall treatment efficiency is greatly promoted.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of application for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of information processing method flow chart disclosed in the embodiment of the present application；
Fig. 2 is another kind information processing method flow chart disclosed in the embodiment of the present application；
Fig. 3 a is electronic map schematic diagram after a kind of grid dividing disclosed in the embodiment of the present application；
Fig. 3 b is electronic map schematic diagram after a kind of net boundary extension disclosed in the embodiment of the present application；
Fig. 4 is another information processing method flow chart disclosed in the embodiment of the present application；
Fig. 5 is another information processing method flow chart disclosed in the embodiment of the present application；
Fig. 6 is a kind of method stream for judging grid and whether belonging to the corresponding grid of processing node disclosed in the embodiment of the present application
Cheng Tu；
Fig. 7 is another information processing method flow chart disclosed in the embodiment of the present application；
Fig. 8 shows for a kind of traffic information processing apparatus structure applied to data distributing node disclosed in the embodiment of the present application
It is intended to；
Fig. 9 is a kind of traffic information processing apparatus structural representation applied to processing node disclosed in the embodiment of the present application
Figure；
Figure 10 is a kind of MapReduce platform structural schematic diagram disclosed in the embodiment of the present application.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of embodiments of the present application, instead of all the embodiments.It is based on
Embodiment in the application, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall in the protection scope of this application.
It is a large amount of to be mainly used for solving individual node processing in the prior art for information processing method disclosed in the present application
The problem of inefficiency caused by GPS data.Traffic information processing is combined by the application with MapReduce platform, by
There is numerous processing nodes in MapReduce platform, multiple processing nodes can work at the same time, therefore by by the GPS of magnanimity
Data distribution is handled simultaneously to different processing nodes, solves the problems, such as the treatment effeciency of mass GPS data.
Wherein, processing node can be understood as the server for carrying out data processing, and a processing node is one corresponding
Server, such as a computer.
The present processes are based on MapReduce platform, and MapReduce platform includes data distributing node and processing section
Point, wherein data distributing node is mainly used for carrying out the distribution work of GPS data, and handles node and be then used to receive GPS data,
And the calculating work of traffic behavior is carried out using GPS data.
It is the application referring to Fig. 1, Fig. 1 next, application scheme is introduced from data distributing node side first
A kind of information processing method flow chart disclosed in embodiment.
As shown in Figure 1, this method comprises:
Step S100, at least one corresponding grid of GPS data of Floating Car passback is determined；
Specifically, the GPS data of Floating Car passback needs to be uploaded in MapReduce platform, and MapReduce platform provides
The function of data warehouse, the GPS data of magnanimity can be uploaded in the file system of MapReduce, can be according to GPS data
Affiliated grid saves GPS data subnetting lattice.
In the embodiment of the present application, the GPS data of the Floating Car passback can be Floating Car and be back in real time
The data of MapReduce platform are also possible to be transferred to the GPS data of MapReduce platform after other platform real-time receptions.
The present embodiment carries out grid dividing to electronic map according to certain level in advance, obtains several grids, wherein
The size of each grid is all the same.By adjusting level size, the size of grid can control.It is understood that by electronics
After map carries out grid dividing, the corresponding determining geographical coverage area of a grid.
It optionally, can be using compress techniques such as gzip or lzo to sea in order to save memory space and save bandwidth
Amount GPS data is uploaded to MapReduce platform after being compressed again.
Step S110, determination processing node corresponding with the grid；
The application can preset grid and handle the corresponding relationship between node, or the correspondence algorithm that setting is certain,
After determining grid, the corresponding processing node of the grid can be determined according to corresponding algorithm.
It should be noted that a grid is only corresponding with a processing node, and a processing node may handle it is multiple
The GPS data of grid.
Illustrate such as: in the corresponding relationship of pre-set grid and processing node, grid 1 is corresponding with processing node 1,
Grid 2 is corresponding with processing node 2, if GPS data 1 falls into grid 1, GPS data 2 falls into grid 2, then GPS1 data need
It is distributed to processing node 1, the needs of GPS data 2 are distributed to processing node 2.
Step S120, the GPS data is distributed to the processing node, so that the processing node determines the grid
The traffic behavior of interior road.
Specifically, after determining the processing node that GPS data is distributed, it is distributed to corresponding processing node, is located
Reason node can use the traffic behavior that GPS data calculates road in grid.
Information processing method provided by the embodiments of the present application carries out grid dividing to electronic map in advance, is receiving
After the GPS data of Floating Car passback, at least one corresponding grid of GPS data is determined, then determination is corresponding with the grid
Node is handled, and then the GPS data is distributed to the processing node, so that the processing node determines road in the grid
The traffic behavior on road.By the way that electronic map is divided into multiple grids, each processing node can be merely responsible for the present processes
The corresponding GPS data of several grids is handled by the way that a large amount of GPS data is distributed to multiple processing nodes, so that whole
Body treatment effeciency is greatly promoted.
In another embodiment of the application, another information processing method is described, referring to fig. 2.
As shown in Fig. 2, this method comprises:
Step S200, the boundary of grid is translated into extension outward, obtains the corresponding Expanding grid of the grid；
As shown in figure 3, Fig. 3 a be to electronic map carry out grid dividing after obtain as a result, wherein each grid of A-I it
Between mutually close to not overlapping.Fig. 3 b is the result after net boundary extends.Wherein, for the ease of reading, only to net
Lattice E and grid F has carried out border extension, and the result after grid E extension is grid R1, and the result after grid F extension is grid R2.
Illustrate by taking grid R1 as an example, for ease of description, 9 small grids for being included by grid R1 in Fig. 3 b, according to from left to right, from
Sequence under, the R19 that is successively positioned as R11, R12 ....
It is understood that wherein the status R11, R13, R17, R19 is identical, by taking R11 as an example, belong to grid A, B, C, D
The shared region of corresponding Expanding grid, so if the positioning coordinate that a GPS data includes is fallen in grid R11, then generation
The table GPS data is corresponding with grid A, B, C, D, therefore the GPS data is distributed to and is saved with the corresponding processing of A, B, C, D
Point.
Further, the status R12, R14, R16, R18 is also identical, by taking R12 as an example, belongs to grid B and the corresponding expansion of grid E
The shared region of exhibition grid then represents the GPS number so if the positioning coordinate that a GPS data includes is fallen in grid R12
According to corresponding with grid B and grid E, therefore the GPS data is distributed to and the corresponding processing node of B and E.
Further, for R15, the corresponding Expanding grid of grid E is only belonged to, so if a GPS data includes
Positioning coordinate fall in grid R15, then it is corresponding with grid E to represent the GPS data, therefore the GPS data is distributed to and E
Corresponding processing node.
Step S210, it determines at least one Expanding grid belonging to positioning coordinate that the GPS data includes, and will determine
The corresponding grid of Expanding grid be determined as target gridding；
Since each GPS data includes unique positioning coordinate, according to positioning coordinate in the electricity being made of Expanding grid
Position in sub- map can determine grid corresponding to GPS data.
One positioning coordinate is possible to only belong to an Expanding grid, it is also possible to belong to two or more expansion
Open up grid.If the positioning coordinate of a GPS data is fallen in two or more Expanding grid, will be where it
Grid corresponding to Expanding grid is determined as target gridding.
Step S220, the corresponding processing node of the target gridding is determined；
Step S230, the GPS data is distributed to the processing node, so that the processing node determines the grid
The traffic behavior of interior road.
Specifically, it is assumed that a GPS data corresponding two and more than two target griddings are determined in step S210, then
The GPS data is distributed in processing node corresponding to these target griddings.
In the present embodiment, a kind of GPS data specific embodiment corresponding with grid is described.To net in the present embodiment
The grid in electronic map after lattice division has carried out border extension, and the purpose of extension mainly considers traffic information processing simultaneously
Non-grid is isolated, when calculating the traffic behavior of certain section of road, needs the context data by user trajectory, therefore
When calculating the traffic behavior of grid edge road, in order to improve accuracy in computation, it is desirable to provide adjacent with the grid edge
The part GPS data of grid, and the present embodiment is exactly based on and carries out border extension to grid, hands over so that calculating a certain grid road
The part GPS data that its adjacent mesh can be used when logical state, so that computational accuracy is higher.
It is understood that the size of Expanding grid corresponding with grid can be set by user, according to extension net
The number of the difference of lattice size, grid corresponding to GPS data may also be different.
Certainly, the corresponded manner of above-mentioned only a kind of optional GPS data and grid, if not considering grid edge road
The precision of the traffic behavior on road, can directly set each GPS data only with it includes positioning coordinate belonging to grid it is opposite
Answer namely a GPS data in positioning coordinate fall in which grid, then the GPS data is corresponding with the grid.
In another embodiment of the application, another information processing method is described, referring to fig. 4.
As shown in figure 4, this method comprises:
Step S400, at least one corresponding grid of GPS data of Floating Car passback is determined；
Step S410, Hash processing is carried out to the mark of the grid, obtains the cryptographic Hash of the mark；
Step S420, remainder is obtained divided by the total number of processing node using the cryptographic Hash of the mark；
Step S430, serial number processing node identical with the remainder is determined as the corresponding processing node of the grid；
Wherein, the serial number of each processing node uniquely and is not repeated by 0 to n-1, and n is the total number for handling node.
Step S440, the GPS data is distributed to the processing node, so that the processing node determines the grid
The traffic behavior of interior road.
It is to be understood that the mark of grid not necessarily number format, thus can be handled by Hash, by grid
Mark is converted to the cryptographic Hash of number format.By cryptographic Hash divided by processing node total number, remainder, serial number and the remainder phase are obtained
Same processing node is the distribution object of corresponding GPS data.
It lifts for example, a total of 10 processing node, serial number is respectively 0-9.Determining grid corresponding with GPS1 data is
Grid B and grid C.Hash processing is carried out by the mark to grid B and grid C, obtaining the corresponding cryptographic Hash of grid B is 15,
The corresponding cryptographic Hash of grid C is 27.Obviously, 15 be 5,27 divided by 10 remainder divided by 10 remainder be 7.Then by GPS1 data point
It is sent to processing node 5 and processing node 7.
It is understood that the case where grid B and grid C corresponds to different disposal node is merely illustrated in above-mentioned example,
It is understood that grid B and grid C is by that may correspond to same processing node, such as the cryptographic Hash of grid B in some cases
It is 15, the cryptographic Hash of grid C is 25, then the two is the equal alignment processing section of 5 namely grid B and grid C divided by the remainder after 10
Point 5.
When corresponding at least two grid of a certain GPS data with same processing node to it is corresponding when for example above-mentioned and GPS1
Data corresponding grid B and grid C, corresponding with processing node 5), according to above-mentioned ways of distribution, need identical two
GPS data is distributed to same processing node, namely sends out two identical GPS datas to same processing node.And for place
It manages for node, two identical GPS datas have no any effect to traffic information calculating, can occupy depositing for processing node on the contrary
Store up resource.Therefore, in the present embodiment, there are identical GPS datas in detecting the GPS data for being sent to same processing node
When, only any one GPS data in identical GPS data can be sent, remaining identical GPS data is without hair
It send.Certainly, in addition to this it is possible to carry out the deletion work of identical GPS data by processing node, to this application without
It limits.
It is understood that when judging whether two GPS datas are identical, the user name that can include according to GPS data
Judged with uplink time, if the user name of two GPS datas and uplink time are all the same, theoretically for this two
A GPS data is identical, because the same user same time can not be at two different positioning coordinates.And if examined
Consider the special circumstances such as system mistake, it can be in user name, uplink time and the positioning complete phase of coordinate for determining two GPS datas
Meanwhile just thinking that two GPS datas are identical.
Certainly, above-described embodiment merely illustrates a kind of determination mode for handling node corresponding with grid, in addition to this
In practical applications, those skilled in the art can also determine grid and corresponding grid using other modes, such as be arranged quiet
State mapping table, the table are used to record Marking the cell and handle the static corresponding relationship between node ID.In this way, passing through inquiry
Static relation table can directly determine the serial number of processing node corresponding with grid.In addition, above-mentioned hash algorithm is only for example,
It is not construed as using the application the limitation of algorithm, any algorithm that can make to handle node and grid generation corresponding relationship
As the application specific embodiment.
Above-described embodiment is described scheme with the angle of data distributing node, and following the present embodiment is to handle section
The scheme of the application is introduced in the angle of point again.
It is another kind information processing method flow chart disclosed in the embodiment of the present application referring to Fig. 5, Fig. 5.
As shown in figure 5, to handle the angle analysis of node, this method comprises:
Step S500, GPS data is received；
Specifically, GPS data, which can be, is distributed by processing node.
Step S510, it determines at least one grid corresponding to the GPS data, and judges whether determining grid belongs to
The corresponding grid of present treatment node；
Specifically, it for each received GPS data, determines at least one corresponding grid, and judges each net
Whether lattice belong to the corresponding grid of present treatment node.Specific judgment mode can there are many, such as in advance in processing node
Storage handles list, and the mark of grid that present treatment node should be handled, etc. other way is labelled in the processing list.
Step S520, for belonging to the grid of the corresponding grid of present treatment node, the GPS data is utilized to carry out traffic shape
State calculates, and obtains the traffic behavior of road in the grid.
It lifts for example, grid handled by present treatment node is only grid 1, then owns using the corresponding with grid 1 of upload
The calculating of GPS data progress 1 traffic behavior of grid.Certainly, if grid handled by processing node have it is multiple, for each
A grid carries out aforesaid operations.
Information processing method provided in this embodiment receives multiple GPS datas that data distributing node is sent, and really
Grid corresponding to fixed each GPS data, and then determine whether each grid belongs to the process range of present treatment node, if so,
The calculating that traffic behavior is then carried out using corresponding GPS data, obtains the traffic state information of road in grid.This implementation
In example, the mass GPS data distribution that user uploads carries out parallel processing to multiple processing nodes, and each processing node only needs to locate
GPS data corresponding with the grid within the scope of oneself is managed, thus the pressure of single processing node greatly reduces, and disposed of in its entirety is imitated
Rate is also greatly promoted.
Optionally, after receiving GPS data, GPS data can be successively ranked up according to uplink time.This
Sample allows and quickly handles several months GPS datas of multiple grids as according to time playback when handling traffic information.
Further, it is also possible to classification and ordination be carried out to GPS data according to user name, at the GPS data concentration of same user name
Reason, can be improved treatment effeciency.
A kind of reality for judging grid and whether belonging to the process range of present treatment node is merely illustrated in a upper embodiment
Apply mode.Strategy when GPS data distribution is carried out referring to context data distribution node side, the present embodiment additionally provides another
Judgment mode.As shown in fig. 6, this method comprises:
Step S600, Hash processing is carried out to the mark of grid, obtains the cryptographic Hash of the mark；
Step S610, remainder is obtained divided by the total number of processing node using the cryptographic Hash of the mark；
Step S620, judge whether the remainder is identical as the serial number of present treatment node, wherein the sequence of each processing node
It number by 0 to n-1, uniquely and does not repeat, n is the total number for handling node；If so, step S630 is executed, if it is not, executing step
S640；
Step S630, determine that the grid belongs to the corresponding grid of present treatment node；
Step S640, determine that the grid is not belonging to the corresponding grid of present treatment node.
In another embodiment of the application, another information processing method flow chart is described.
As shown in fig. 7, this method comprises:
Step S700, GPS data is received；
Specifically, GPS data can be by data distributing node distribution.
Step S710, detecting whether there is identical GPS data in received GPS data, delete behaviour if so, executing
Make, retains a GPS data in identical GPS data；
Step S720, determination at least one grid corresponding with the GPS data, and judge whether determining grid belongs to
The corresponding grid of present treatment node；
Step S730, for belonging to the grid of the corresponding grid of present treatment node, the GPS data is utilized to carry out traffic shape
State calculates, and obtains the traffic behavior of road in the grid.
In the embodiment of the method that context data distribution node side is introduced it has been mentioned that, when some GPS data is corresponding
At least two grids with same processing node to it is corresponding when identical two GPS datas can be distributed to same place
Node is managed, duplicate GPS data can occupy the storage resource of processing node at this time, can delete some identical GPS numbers thus
According to only retaining a GPS data in identical GPS data.
Traffic information processing apparatus provided by the embodiments of the present application is described below, at traffic information described below
Reference can be corresponded to each other with above-described information processing method by managing device.
Firstly, introducing the traffic information processing apparatus for being applied to data distributing node.It is the application implementation referring to Fig. 8, Fig. 8
A kind of traffic information processing apparatus structural schematic diagram applied to data distributing node disclosed in example.
As shown in figure 8, the device includes:
Grid determination unit 81, for determining at least one corresponding grid of GPS data of Floating Car passback, the grid
For in advance to electronic map carry out grid dividing after obtain；
Node determination unit 82, for determining processing node corresponding with the grid；
File distributing unit 83, for the GPS data to be distributed to the processing node, so that the processing node is true
The traffic behavior of road in the fixed grid.
Traffic information processing apparatus provided by the embodiments of the present application determines after the GPS data for receiving Floating Car passback
At least one corresponding grid of GPS data, then determines corresponding with grid processing node, and then by the GPS data
It is distributed to the processing node, so that the processing node determines the traffic behavior of road in the grid.The application pass through by
Electronic map is divided into multiple grids, and each processing node can be merely responsible for the corresponding GPS data of several grids, by will be big
The GPS data of amount is distributed to multiple processing nodes and is handled, so that overall treatment efficiency is greatly promoted.
Optionally, above-mentioned grid determination unit may include:
First grid determines subelement, for the boundary of grid to be translated extension outward, obtains the corresponding expansion of the grid
Open up grid；
Second grid determines subelement, for determining at least one expansion belonging to positioning coordinate that the GPS data includes
Open up grid；
Third grid determines subelement, for the corresponding grid of the Expanding grid determined to be determined as the GPS data pair
The grid answered.
Certainly, the corresponded manner of above-mentioned only a kind of optional GPS data and grid, if not considering grid edge road
The precision of the traffic behavior on road, can directly set each GPS data only with it includes positioning coordinate belonging to grid it is opposite
Answer namely a GPS data in positioning coordinate fall in which grid, then the GPS data is corresponding with the grid.
Optionally, above-mentioned node determination unit may include:
First node determines subelement, carries out Hash processing for the mark to the grid, obtains the Kazakhstan of the mark
Uncommon value；
Second node determines subelement, for the cryptographic Hash using the mark divided by the total number n of processing node, obtains
Remainder；
Third node determines subelement, for serial number processing node identical with the remainder to be determined as the grid pair
The processing node answered, wherein the serial number of each processing node uniquely and is not repeated by 0 to n-1.
Certainly, above-described embodiment merely illustrates a kind of determination mode for handling node corresponding with grid, in addition to this
There may also be the corresponding relationships between other grids and processing node, such as static mapping table is arranged, wherein recording net
Case marker knows the static corresponding relationship between processing node ID.In this way, by queries static relation table, can directly determine and net
The serial number of the corresponding processing node of lattice.
Next, the present embodiment introduction is applied to the traffic information processing apparatus of processing node one end.It is referring to Fig. 9, Fig. 9
A kind of traffic information processing apparatus structural schematic diagram applied to processing node disclosed in the embodiment of the present application.
As shown in figure 9, the device includes:
Data receipt unit 91, for receiving GPS data；
Grid judging unit 92 for determining at least one corresponding grid of the GPS data, and judges determining grid
Whether present treatment node corresponding grid is belonged to；
Traffic behavior computing unit 93, for utilizing what is received when the grid judging unit judging result, which is, is
GPS data carries out traffic behavior calculating, obtains the traffic behavior of road in the grid.
Traffic information processing apparatus provided in this embodiment receives multiple GPS datas that data distributing node is sent, and really
Grid corresponding to fixed each GPS data, and then determine whether each grid belongs to the corresponding grid of present treatment node, if
It is then to carry out the calculating of traffic behavior using corresponding GPS data, obtain the traffic state information of road in grid.This
In embodiment, the mass GPS data distribution that user uploads carries out parallel processing to multiple processing nodes, and each processing node only needs
GPS data corresponding with the grid within the scope of oneself is handled, thus the pressure of single processing node greatly reduces, and at entirety
Reason efficiency is also greatly promoted.
Optionally, above-mentioned traffic information processing apparatus can also include:
Data sorting unit arranges several received GPS datas for the sequence successive according to uplink time
Sequence.
Optionally, above-mentioned traffic information processing apparatus can also include:
Deduplication data cell, for detecting with the presence or absence of identical GPS data in the GPS data received, if so,
Delete operation is executed, a GPS data in identical GPS data is only retained.
Optionally, above-mentioned grid judging unit may include:
First grid judgment sub-unit carries out Hash processing for the mark for each grid, obtains the mark and correspond to
Cryptographic Hash；
Second grid judgment sub-unit, for the total number using the corresponding cryptographic Hash of the mark divided by processing node,
Obtain corresponding remainder；
Third grid judgment sub-unit, for judging whether each remainder is identical as the serial number of present treatment node, if so, really
The fixed corresponding grid of the remainder belongs to the corresponding grid of present treatment node, if not, it is determined that the corresponding grid of the remainder is not belonging to
The corresponding grid of present treatment node, wherein the serial number of each processing node is by 0 to n-1, different, n is the total of processing node
Number.
The embodiment of the present application also provides the MapReduce platforms that a kind of user handles traffic information, as shown in Figure 10, should
Platform includes:
Data distributing node 1 and several processing nodes 2, wherein
The data distributing node 1 be used for determine Floating Car passback at least one corresponding grid of GPS data, determine with
The corresponding processing node 2 of the grid, is distributed to the processing node 2 for the GPS data, wherein the grid is right in advance
Electronic map obtains after carrying out grid dividing；
The processing node 2 is for receiving the GPS data, at least one net corresponding to the determining and GPS data
Lattice, and judge whether each grid belongs to the corresponding grid of present treatment node, if so, carrying out traffic behavior using the GPS data
It calculates, obtains the traffic behavior of road in the grid.
The MapReduce platform of the embodiment of the present application realizes the distribution of GPS data by data distributing node, by magnanimity
GPS data be distributed to different processing nodes, the GPS data of certain grid is handled by each processing node respectively, so that whole
Body treatment effeciency is greatly promoted.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that the process, method, article or equipment for including a series of elements not only includes that
A little elements, but also including other elements that are not explicitly listed, or further include for this process, method, article or
The intrinsic element of equipment.In the absence of more restrictions, the element limited by sentence "including a ...", is not arranged
Except there is also other identical elements in the process, method, article or apparatus that includes the element.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with other
The difference of embodiment, the same or similar parts in each embodiment may refer to each other.
The foregoing description of the disclosed embodiments makes professional and technical personnel in the field can be realized or use the application.
Various modifications to these embodiments will be readily apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the application.Therefore, the application
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest scope of cause.
Claims (11)
1. a kind of information processing method characterized by comprising
Determine that at least one corresponding grid of GPS data of Floating Car passback, the grid are to carry out net to electronic map in advance
Lattice obtain after dividing；
The corresponding processing node of determination grid corresponding with the GPS data that the Floating Car returns；
The GPS data that the Floating Car returns is distributed to the corresponding place of the corresponding grid of GPS data of the Floating Car passback
Node is managed, so that the processing node determines the traffic behavior of road in the grid.
2. the method according to claim 1, wherein the GPS data of the determining Floating Car passback is corresponding extremely
A few grid, comprising:
The boundary of grid is translated into extension outward, obtains the corresponding Expanding grid of the grid；
Determine at least one Expanding grid belonging to positioning coordinate that the GPS data includes；
The determining corresponding grid of Expanding grid is determined as the corresponding grid of the GPS data.
3. the method according to claim 1, wherein determination processing node corresponding with the grid, packet
It includes:
Hash processing is carried out to the mark of the grid, obtains the cryptographic Hash of the mark；
With the cryptographic Hash of the mark divided by the total number n of processing node, remainder is obtained；
Serial number processing node identical with the remainder is determined as the corresponding processing node of the grid, wherein each processing
The serial number of node uniquely and is not repeated by 0 to n-1.
4. a kind of information processing method, which is characterized in that be applied in processing node, the processing node and Floating Car return
The corresponding grid of GPS data it is corresponding, the grid is to obtain after carrying out grid dividing to electronic map in advance, this method packet
It includes:
Receive GPS data；
Determine at least one corresponding grid of the GPS data, and it is corresponding to judge whether determining grid belongs to present treatment node
Grid；
If belonging to, traffic behavior calculating is carried out using the GPS data received, obtains the traffic shape of road in the grid
State.
5. according to the method described in claim 4, it is characterized in that, before the corresponding grid of the determination GPS data,
Further include:
It detects with the presence or absence of identical GPS data in received GPS data, if so, retaining one in identical GPS data
GPS data deletes other duplicate GPS datas.
6. according to the method described in claim 4, it is characterized in that, described judge whether determining grid belongs to present treatment node
Corresponding grid, comprising:
Hash processing is carried out to the mark of determining grid, obtains the cryptographic Hash of the mark；
With the cryptographic Hash of the mark divided by the total number n of processing node, remainder is obtained；
Judge whether the remainder is identical as the serial number of present treatment node, wherein the serial number of each processing node is by 0 to n-1, only
It one and does not repeat；
If so, determining that the grid belongs to the corresponding grid of present treatment node.
7. a kind of traffic information processing apparatus characterized by comprising
Grid determination unit, for determining at least one corresponding grid of GPS data of Floating Car passback, the grid is preparatory
It is obtained after carrying out grid dividing to electronic map；
Node determination unit, for the corresponding processing section of determination grid corresponding with the GPS data that the Floating Car returns
Point；
File distributing unit, the GPS data for returning the Floating Car are distributed to the GPS data pair of the Floating Car passback
The corresponding processing node of the grid answered, so that the processing node determines the traffic behavior of road in the grid.
8. device according to claim 7, which is characterized in that the grid determination unit includes:
First grid determines subelement, for the boundary of grid to be translated extension outward, obtains the corresponding extension net of the grid
Lattice；
Second grid determines subelement, for determining at least one extension net belonging to positioning coordinate that the GPS data includes
Lattice；
Third grid determines subelement, corresponding for the corresponding grid of the Expanding grid determined to be determined as the GPS data
Grid.
9. device according to claim 7, which is characterized in that the node determination unit includes:
First node determines subelement, carries out Hash processing for the mark to the grid, obtains the cryptographic Hash of the mark；
Second node determines subelement, for the cryptographic Hash using the mark divided by the total number n of processing node, obtains remaining
Number；
Third node determines subelement, corresponding for serial number processing node identical with the remainder to be determined as the grid
Handle node, wherein the serial number of each processing node uniquely and is not repeated by 0 to n-1.
10. a kind of traffic information processing apparatus, which is characterized in that be applied in processing node, the processing node and Floating Car are returned
The corresponding grid of the GPS data of biography is corresponding, and the grid is to obtain after carrying out grid dividing to electronic map in advance, the device packet
It includes:
Data receipt unit, for receiving GPS data；
Grid judging unit for determining at least one corresponding grid of the GPS data, and judges whether determining grid belongs to
In the corresponding grid of present treatment node；
Traffic behavior computing unit, for utilizing the GPS data received when the grid judging unit judging result, which is, is
Traffic behavior calculating is carried out, the traffic behavior of road in the grid is obtained.
11. a kind of for handling the MapReduce platform of traffic information characterized by comprising data distributing node and processing
Node；
Data distributing node, for determining at least one corresponding grid of GPS data of Floating Car passback, the determining and floating
The corresponding processing node of the corresponding grid of GPS data of vehicle passback, is distributed to institute for the GPS data that the Floating Car returns
State Floating Car passback the corresponding processing node of the corresponding grid of GPS data, wherein the grid be in advance to electronic map into
It is obtained after row grid dividing；
It handles node and determines at least one corresponding grid of the GPS data, and judge to determine for receiving the GPS data
Grid whether belong to the corresponding grid of present treatment node, for belonging to the grid of the corresponding grid of present treatment node, using connecing
The GPS data received carries out traffic behavior calculating, obtains the traffic behavior of road in the grid.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510531824.6A CN106485915B (en) | 2015-08-26 | 2015-08-26 | A kind of information processing method, device and MapReduce platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510531824.6A CN106485915B (en) | 2015-08-26 | 2015-08-26 | A kind of information processing method, device and MapReduce platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106485915A CN106485915A (en) | 2017-03-08 |
| CN106485915B true CN106485915B (en) | 2019-12-03 |
Family
ID=58233905
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510531824.6A Expired - Fee Related CN106485915B (en) | 2015-08-26 | 2015-08-26 | A kind of information processing method, device and MapReduce platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106485915B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111275961A (en) * | 2019-01-18 | 2020-06-12 | 北京交通大学 | Urban traffic running state feature calculation method based on floating car data |
| CN110728317A (en) * | 2019-09-30 | 2020-01-24 | 腾讯科技（深圳）有限公司 | Training method and system of decision tree model, storage medium and prediction method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101270997A (en) * | 2007-03-21 | 2008-09-24 | 北京交通发展研究中心 | Floating car dynamic real-time traffic information processing method based on GPS data |
| CN102445205A (en) * | 2011-10-12 | 2012-05-09 | 北京世纪高通科技有限公司 | Massive map data matching method and device |
| CN103235974A (en) * | 2013-04-25 | 2013-08-07 | 中国科学院地理科学与资源研究所 | Method for improving processing efficiency of massive spatial data |
| CN104182453A (en) * | 2014-06-20 | 2014-12-03 | 银江股份有限公司 | Distributed map matching method for massive historical floating car data |
| CN104408918A (en) * | 2014-11-20 | 2015-03-11 | 江苏物联网研究发展中心 | Floating car data and road matching method based on MapReduce calculation frame |
-
2015
- 2015-08-26 CN CN201510531824.6A patent/CN106485915B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101270997A (en) * | 2007-03-21 | 2008-09-24 | 北京交通发展研究中心 | Floating car dynamic real-time traffic information processing method based on GPS data |
| CN102445205A (en) * | 2011-10-12 | 2012-05-09 | 北京世纪高通科技有限公司 | Massive map data matching method and device |
| CN103235974A (en) * | 2013-04-25 | 2013-08-07 | 中国科学院地理科学与资源研究所 | Method for improving processing efficiency of massive spatial data |
| CN104182453A (en) * | 2014-06-20 | 2014-12-03 | 银江股份有限公司 | Distributed map matching method for massive historical floating car data |
| CN104408918A (en) * | 2014-11-20 | 2015-03-11 | 江苏物联网研究发展中心 | Floating car data and road matching method based on MapReduce calculation frame |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106485915A (en) | 2017-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Cheng et al. | Reliable diversity-based spatial crowdsourcing by moving workers | |
| CN106096023B (en) | Method for reading data, method for writing data and data server | |
| CN104376053B (en) | A kind of storage and retrieval method based on magnanimity meteorological data | |
| JP5027985B2 (en) | Method and system for forming, updating and using a geographic database | |
| Bertermann et al. | A pan-European planning basis for estimating the very shallow geothermal energy potentials | |
| WO2016095749A1 (en) | Method and device for querying spatial object and establishing spatial index of electronic map | |
| CN106528597A (en) | POI (Point Of Interest) labeling method and device | |
| CN111128398B (en) | Epidemic disease infected person number estimation method based on population migration big data | |
| Gao et al. | Geometric algorithms for sensor networks | |
| CN104298541A (en) | Data distribution algorithm and data distribution device for cloud storage system | |
| CN110263117B (en) | Method and device for determining POI (Point of interest) data | |
| CN104252489A (en) | Method for fast obtaining position character description information according to latitude and longitude data | |
| Aloui et al. | A new itinerary planning approach among multiple mobile agents in wireless sensor networks (WSN) to reduce energy consumption | |
| CN104102637B (en) | A kind of method and device of generation hot spot region | |
| CN103955511A (en) | Cloud platform data organization and retrieval method for 3D (three-dimensional) urban building data | |
| CN104679864A (en) | Intelligent tracking method and intelligent tracking device for suspected target based on GIS (Geographic Information System) | |
| CN106503196A (en) | The structure and querying method of extensible storage index structure in cloud environment | |
| CN101944096B (en) | Method and system for providing bus routes | |
| CN106485915B (en) | A kind of information processing method, device and MapReduce platform | |
| CN105592123A (en) | Storage management system, management apparatus and method | |
| CN104021223B (en) | The access method and device of a kind of Cluster Database measuring point | |
| CN105913347A (en) | House coding method orienting house information acquisition | |
| CN105117451A (en) | Storage method for map tile file | |
| JP2004354395A5 (en) | ||
| CN106980540B (en) | Distributed multi-dimensional discrete data calculation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20200511 Address after: 310052 room 508, floor 5, building 4, No. 699, Wangshang Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Patentee after: Alibaba (China) Co.,Ltd. Address before: 100080 Beijing City, Haidian District Suzhou Street No. 3 floor 16 room 2 Patentee before: AUTONAVI INFORMATION TECHNOLOGY Co.,Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20191203 Termination date: 20200826 |