CN103440278A - Data mining system and method - Google Patents
Data mining system and method Download PDFInfo
- Publication number
- CN103440278A CN103440278A CN2013103500673A CN201310350067A CN103440278A CN 103440278 A CN103440278 A CN 103440278A CN 2013103500673 A CN2013103500673 A CN 2013103500673A CN 201310350067 A CN201310350067 A CN 201310350067A CN 103440278 A CN103440278 A CN 103440278A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- unit
- user
- data mining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a data mining system. The data mining system comprises a data acquisition unit, a geographical information unit, a data analysis unit and a data mining unit, wherein the data acquisition unit is used for collecting position data of a user; the geographical information unit is used for storing related information of geographical data in advance; the data analysis unit is used for analyzing the position data of the user on the basis of the information of the geographical information unit, so as to obtain individual analysis information related to the user; the data mining unit is used for carrying out data mining on the individual analysis information of all users to obtain target information according to data mining demands. The invention also provides a corresponding data mining method. Compared with traditional investigation and statistical patterns, the data mining system and the data mining method are used for carrying out data analysis and mining on the same type of individual behaviors to obtain statistical results from the macroscopic perspective, and have the advantages that the amount of processed information is huge, time and labor are saved, the accuracy is high and the like.
Description
Technical field
The present invention relates to a kind of data digging system and method.
Background technology
The demonstration of Chinese mugwort matchmaker market consultation integrated data, European Location based service market is 3.20 hundred million dollars the operation revenue of 2009, and, by 2015, operation revenue will reach 6.1 hundred million dollars, annual compound growth rate will reach 12%.At home, Location based service also starts to become the standard configuration of large-scale website.It is introduced, the current domestic running fix service of registering has had 30 general-purpose families.Although due to causes such as the smart mobile phone application are extensive not enough, LBS (Location Based Service, the position-based service) business is not higher with huge cellphone subscriber's quantity, but along with smart mobile phone is more and more universal, utilize the user of mobile phone mobile Internet access also more and more, the development prospect of LBS business is very tempting.
Running fix is commonly referred to as the mobile phone location, namely by the cooperation of wireless terminal (mobile phone) and wireless network, determine mobile subscriber's actual position information (latitude and longitude coordinates data, comprise three-dimensional data), and issue the user or certain value-added service is provided based on this by SMS (the short service of disappearing), MMS (MMS), voice.According to the mode that realizes running fix, can divide two classes substantially, a class is the architecture that relies on common carrier; Another kind of is the GPS client machine location that relies on Global Positioning System (GPS) (Global PositioningSystem).Location Based Service mainly is applied to the geography information support now: conveniently inquire about required geography target (Perimeter) and route guidance function, automobile rescue, the aspects such as medical first aid are provided.
For the location that relies on common carrier (such as UNICOM, movement etc.), usually need consumption data flow (such as GPRS etc.), the armrest machine is realized location with the communication of communicating by letter between control tower, the positioning result error is no more than 600 meters, is applicable to common walking or goes window-shopping, the most of depended software of the realization of this class positioning function, and these softwares are substantially bound route and are searched plain function, more convenient, the time is short, and reaction is fast.Typical application realizes having: Google Maps, Baidu's map, pattra mobile base station etc.
But above-mentioned Mobile Location Technology, are all the individual services for single individuality, can't count on a macro scale same type crowd's behavioural characteristic, also can't realize the data mining of movement-based location.
Summary of the invention
The present invention is directed to the problems referred to above, proposed a kind of data digging system and method, be intended to realize the data mining of movement-based location, analyze valuable information, for the decision-making of government department or incorporated business provides foundation.
In one aspect, the invention provides a kind of data digging system, it is characterized in that, comprising:
Data acquisition unit, for gathering user's position data;
The geography information unit, the information that its pre-save geodata is relevant;
Data analysis unit, for the information based on the geography information unit, the position data of analysis user, to obtain the ontoanalysis information about described user; And
The data mining unit, for according to the data mining demand, carry out data mining to whole users' ontoanalysis information, to obtain target information.
In yet another aspect, the invention provides a kind of data digging method, it is characterized in that, comprise step:
Gather user's position data;
Based on geography information, the position data of analysis user, to obtain the ontoanalysis information about described user; And
According to the data mining demand, whole users' ontoanalysis information is carried out to data mining, to obtain target information.
The present invention is not for the locating information of single individuality, but carry out data analysis and excavation for the personal behavior of same type, thereby can, from macroscopic perspective, draw statistics, all there are to reference value in government department and commercial company, contribute to make the very important decision of taking the whole situation into account and plan accordingly.Than traditional investigation statistics mode of using questionnaire, interview etc., it is huge that the present invention has the quantity of information of processing, labour-saving characteristics during joint, and improved the accuracy of investigation statistics.
The accompanying drawing explanation
Specific embodiments of the invention are described below with reference to accompanying drawings, wherein:
Fig. 1 illustrates the entire block diagram of data digging system of the present invention;
Fig. 2 illustrates the workflow diagram of data digging system of the present invention;
Fig. 3 illustrates the schematic diagram of the geographical information grid storage mode of geographic position marker in data digging system of the present invention; And
Fig. 4 illustrates data digging method of the present invention.
Embodiment
In order to make technical scheme of the present invention and advantage clearer, below in conjunction with accompanying drawing, exemplary embodiment of the present invention is described in more detail, obviously, described embodiment is only a part of embodiment of the present invention, rather than all embodiment's is exhaustive.
As shown in Figure 1, when the holder of mobile terminal device 101 opens mobile positioning function, mobile terminal device 101 is according to gps satellite positioning system 102 or mobile device base station 103, obtain the positional information of self, again the positional information of self is sent to switch 104 by wireless network, final switch 104 sends to data digging system 2 by these data.In data digging system 2, will carry out Storage and Processing to these positional informations, in order to draw the net result of expectation.
Particularly, data digging system 2 comprises data acquisition unit 202, geography information unit 203, data analysis unit 204 and data mining unit 205.The same set of data-base cluster of the common use in these unit, public database 201.In the whole flow process of processing in data, these unit 202-205 plays the part of different roles, bears different tasks.They cooperatively interact, and are bringing into play separately the effect of oneself.
Below, describe in conjunction with Fig. 2 the task that unit 202-205 bears in detail.Data acquisition unit 202 is responsible for gathering user's position data, and position data is sent to the basic database 2011 of public database 201 and is stored in this.In the present embodiment, the position data of the mobile terminal device 101 of user's position data by obtaining the user obtains.That is to say, the position data that data acquisition unit 202 gathers mobile terminal device 101 is usingd as user's data.Geography information unit 203 pre-saves the relevant information of geodata, these information need to obtain by mapping by means of instrument of surveying and mapping 301 usually.These information comprise, such as: the Business Information on cartographic information and map.And, because geography information is all changing always, so need to carry out regular or irregular renewal.Data analysis unit 204 is in conjunction with the data in data acquisition unit 202 and geography information unit 203, according to different data mining task needs, extract corresponding data, and the mode that is beneficial to mining task stores in the analytical database 2012 of public database 201 into.Data analysis unit 204 is actually is cleaned, is filtered or is optimized data, only extracts valuable data.Data mining unit 205, according to user 304 needs, carries out data mining to the data in analytical database 2012, by excavation to the information with value present to user 304, as the reference of user's decision-making.
Below, for each unit, elaborate respectively.
By the location technology of mobile phone, gather a certain mobile phone account sometime the section in mobile message.Such as: certain mobile phone account moves to Shangdi Zhongguancun Software Park to 2012/9/24 8:55:00 from the residential block, Tiantong Yuan at 2012/9/24 8:40:00.When the displacement of this mobile phone account reaches 10 meters, time point and the positional information of 10 meters front and back of record move, and the time and the position that stop mobile two state switching points
The database table design is as follows:
For the redundancy that reduces data and the Comprehensible of data.To, to a bit of many continuous data messages, by data, store.The following mode of concrete employing operates: by many continuous data segments, only get starting position and the start time of article one, the end position of the last item and concluding time, displacement and traveling time using the displacement of whole segments and traveling time addition as a new record.Such as: certain mobile phone account is taken subway to the Shangdi Zhongguancun Software Park from the residential block, Tiantong Yuan, during will move more than 10 kilometer, through more than 10 subway stations, needing the data of record is only the data that move to successively next subway station.If 10 meters of every movements just generate a record, will produce a large amount of data recording, and these records are much all useless.Only record position and temporal information from last subway station to next subway station, and, in the residence time of this subway station, just greatly reduced useless data message.
In geography information unit 203, storage is the information that geographic position is relevant, such as weather condition, climatic condition, map datum (comprising the data of road, city, shop, buildings etc.).These data change at any time, still, map datum can not be carried out to real-time update, normally regular or irregular renewal.
The geography information of main each geographic position marker of storage in geography information unit 203, and the geographic position marker of part is used to gridding information storage mode (i.e. a polygonal region).Each geographic position marker has taken certain map space on map, and this map space must have many summits.Choose adjacent 2 of can be linked to be straight-line segment and as this polygonal limit, whole consecutive point are all coupled together, just formed a polygon, the space taken in this marker place, geographic position map, just mean with this polygon.Finally, the geographic position marker of part is with regard to the Adoption Network information storage means.Such as, in Fig. 3, the polygonal region that the coordinate of the Tian'anmen Square is A, B, C, D, these some compositions of E, F.The polygonal region that the coordinate of the Forbidden City is A, B, J, I, these some compositions of H, G.And classified in these zones.The database table design is as follows:
Data analysis unit 204
Data analysis unit 204, mainly according to the data of data acquisition unit 202 and geography information unit 203, forms the habits and customs of this account within this period, forms and is convenient to the data of analyzing.Must be pointed out, this unit is only to the data analysis of single account, and the data of formation are the personal data of this account.
Such as, the positional information of certain mobile phone account, from proxima luce (prox. luc) evening until next day 8 point, always in the polygon scope zone in the residential block, Tiantong Yuan, illustrate that this account lived in Tiantong Yuan the evening before yesterday.Then, take Subway Line 5 and turn the polygonal region that Line 1 arrives the Tian'anmen Square, stop in the polygonal region scope that arrives Palace Museum after 1 hour.After 2 hours, leave the Forbidden City and go to the cuisine variety street, Wangfujing again.The main activities that this account this morning is described so is to visit then to have a dinner.The database table design is as follows:
Data mining unit 205
Data mining unit 205 is mainly the data of analyzing in data analysis unit 204, excavates utilizable valuable information.Data analysis unit 204 is analyzed single individuality, and data mining unit 205 is that the data of whole individualities are carried out to statistical study, excavates these individual general character.Such as, to utilize data mining unit 205 can count How many people and leave the residential block, Tiantong Yuan morning, return here evening again, and the number of leaving/returning of each time period.Certainly, such analysis can have a variety of, below only enumerates some typical values:
(1) urban traffic conditions excavates.Data that can be based on above, analyze utilization factor and the congestion state of each road of city.To city morning evening peak traffic behavior can quantize to concrete numerical value, rather than the description of several ambiguities only.
For urban traffic conditions, excavate, data mining unit 205 can adopt K-to close on the data mining algorithm that search pattern is sent out in classification most.Close on most classification based on analogical learning, n dimension value attribute description for training sample, each sample represents a point of n-dimensional space, like this, all training samples all leave in the n-dimensional space pattern.A given unknown sample, this algorithm will be found out K the training sample that approaches unknown sample most, and this K training sample is K neighbour of unknown sample.Proximity defines with Euclidean distance, wherein two some X=(x1, x2, x3 ...) and Y=(y1, y2, y3 ...) Euclidean distance be
In urban transportation, adopt this algorithm to calculate the present flow rate of certain road, we can obtain apart from the positional information of whole individualities of certain road axis certain distance (because individuality extensively is distributed in whole map, only having those individual flow rate calculation that just participates in this road apart from the road certain distance).To just obtain flow Q and the unimpeded index V of this road at current slot apart from the present speed x of whole individualities of road axis certain distance addition or be averaging respectively,
The computing formula of flow Q is
The computing formula of unimpeded index V is
Thereby calculate flow Q and the unimpeded index V of each road, for traffic department's reference.
(2) city planning is excavated.May at weekend, urban population be more prone to suburb or market, still, is which zone and which market in suburb on earth? utilize data mining of the present invention unit 205, can calculate the main aggregation zone of each time period urban population.Such as, calculating 21 point～6 is that aggregation zone more than 6 hours is residential block in a certain zone residence time, 9 point～18 are that accumulation area more than 6 hours is workplace in a certain zone residence time, the zone stopped more than 2 hours is shopping center, other mechanisms such as tourist attractions or hospital.Like this, respect fully the civic wish in suitable construction leisure paradise, region, Technology Park, residential block, rather than the drawback that adopts government to force zoning to be built.
For city planning, excavate, data mining unit 205 can adopt the data mining algorithm of cluster algorithm.Object is according to the similarity maximized in class, and the principle that minimizes the similarity between class is carried out cluster or grouping.Cluster analysis mainly concentrates on the cluster analysis of distance-based.The distance of distinctiveness ratio between object based between object calculated.
In city planning, because all individuality is random each corner that is distributed in city.Need to calculate by aggregation algorithms their aggregation zone.Computing method need following several steps:
(i). calculate these body positions at map Zhong center (being the mean value of position) m
f, computing formula is as follows:
(ii). calculate average absolute deviation S
f, computing formula is as follows:
(iii). the metric Z of normalized
f, calculate a standardized metric for each is individual, when this value of certain individuality is less than the threshold values of setting, this individuality belongs to this zone.
Computing formula is as follows:
Z
f＝(x
i-m
f)/S
f
Like this, according to the zone of calculating, government planning department can make detailed program plan.
(3) habits and customs are excavated.Utilize data mining of the present invention unit 205, can also add up away the size of population of tourist attractions, the square that goes shopping the size of population, the size of population of going to seek medical advice, go size of population that other places goes on business etc.Can also at key crossing, warning sign be set according to their traffic path, billboard.If, introducing cell-phone number real name system, can count all age group, the habits and customs of each sex population.Wherein any one statistics all can produce huge commercial value.
In data mining of the present invention unit 205, can count each individual current movable type of being engaged in.As following table:
There are the Activity Type that current individuality is engaged in and the time spent in this table, are respectively field " social activities type " and " spended time ".Like this, just can calculate whole individualities ratio of every social activities of being engaged in of a period of time in the past.Be engaged in a certain movable ratio R used
x, computing formula is as follows:
Wherein, x
i=(x1, x2, x3 ... .) certain movable time of being engaged in for certain individuality.
S
i=(S1, S2, S3 ... .) be certain individual all time of social activities.
More than describe data digging system 2 of the present invention in detail.
According to same inventive concept, the present invention also provides a kind of data digging method, as shown in Figure 4, it comprises, at first at step S401, gather user's position data, such as the position data of the mobile terminal device 101 that the gathers described user position data as described user.Then, at step S402, based on geography information, such as the geography information of pre-save in geography information unit 203, the position data of analysis user, to obtain the ontoanalysis information about described user.Finally, at step S403, according to the data mining demand, whole users' ontoanalysis information is carried out to data mining, to obtain target information.
Above embodiment is only in order to technical scheme of the present invention to be described, but not is limited.Therefore, in the situation that do not deviate from spirit of the present invention and essence thereof, those skilled in the art can make various changes, replacement and modification.Obviously, but within these changes, replacement and modification all should be covered by the protection domain of the claims in the present invention.
Claims (9)
1. a data digging system, is characterized in that, comprising:
Data acquisition unit (202), for gathering user's position data;
Geography information unit (203), the information that its pre-save geodata is relevant;
Data analysis unit (204), for the information based on geography information unit (203), the position data of analysis user, to obtain the ontoanalysis information about described user; And
Data mining unit (205), for according to the data mining demand, carry out data mining to whole users' ontoanalysis information, to obtain target information.
2. data digging system as claimed in claim 1, is characterized in that, also comprises public database (201), for the position data of preserving the user and the analysis result data of data analysis unit (204).
3. data digging system as claimed in claim 1, is characterized in that, described user's position data obtains by the position data of the described user's of collection mobile terminal device (101).
4. data digging system as claimed in claim 1, is characterized in that, the geography information that described geography information unit (203) adopts the gridding information storage mode to store the geographic position marker.
5. data digging system as claimed in claim 1, is characterized in that, it is for excavating urban traffic conditions or city planning.
6. a data digging method, is characterized in that, comprises step:
Gather user's position data;
Based on geography information, the position data of analysis user, to obtain the ontoanalysis information about described user; And
According to the data mining demand, whole users' ontoanalysis information is carried out to data mining, to obtain target information.
7. data digging method as claimed in claim 6, is characterized in that, gathers described user's the position data of mobile terminal device (101) as described user's position data.
8. data digging method as claimed in claim 6, is characterized in that, described geography information pre-save is in geography information unit (203).
9. data digging method as claimed in claim 6, is characterized in that, it is for excavating urban traffic conditions or city planning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013103500673A CN103440278A (en) | 2013-08-12 | 2013-08-12 | Data mining system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013103500673A CN103440278A (en) | 2013-08-12 | 2013-08-12 | Data mining system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103440278A true CN103440278A (en) | 2013-12-11 |
Family
ID=49693970
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2013103500673A Pending CN103440278A (en) | 2013-08-12 | 2013-08-12 | Data mining system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103440278A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105959476A (en) * | 2016-05-11 | 2016-09-21 | 上海电机学院 | System and method for acquiring mobile positioning behavior data |
| CN106228187A (en) * | 2016-07-21 | 2016-12-14 | 贵州力创科技发展有限公司 | Individual recognizer model based on multiple user's detail data and treatment technology |
| CN108171532A (en) * | 2017-12-11 | 2018-06-15 | 扬州大学 | A kind of user group distribution forecasting method and system |
| CN109087218A (en) * | 2018-08-10 | 2018-12-25 | 重庆市南岸区瑜目网络科技有限责任公司 | A kind of system being collected and analyze user's tourism data according to satellite positioning |
| WO2020244601A1 (en) * | 2019-06-07 | 2020-12-10 | Beijing Didi Infinity Technology And Development Co., Ltd. | Estimating passenger income level on a ridesharing platform |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070198184A1 (en) * | 2005-04-08 | 2007-08-23 | Mototaka Yoshioka | Map information updating apparatus and map information updating method |
| CN101673282A (en) * | 2009-10-15 | 2010-03-17 | 杭州北海信息系统有限公司 | Automatic positioning system method of electronic map |
| CN101692309A (en) * | 2009-09-04 | 2010-04-07 | 北京工业大学 | Traffic trip computing method based on mobile phone information |
| CN202853620U (en) * | 2012-09-29 | 2013-04-03 | 南京林业大学 | Mobile equipment-based resident trip surveying device |
| CN103150156A (en) * | 2012-12-06 | 2013-06-12 | 江苏省公用信息有限公司 | Method and system, based on geographic model and moving track, for obtaining characteristic crowd in real time |
-
2013
- 2013-08-12 CN CN2013103500673A patent/CN103440278A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070198184A1 (en) * | 2005-04-08 | 2007-08-23 | Mototaka Yoshioka | Map information updating apparatus and map information updating method |
| CN101692309A (en) * | 2009-09-04 | 2010-04-07 | 北京工业大学 | Traffic trip computing method based on mobile phone information |
| CN101673282A (en) * | 2009-10-15 | 2010-03-17 | 杭州北海信息系统有限公司 | Automatic positioning system method of electronic map |
| CN202853620U (en) * | 2012-09-29 | 2013-04-03 | 南京林业大学 | Mobile equipment-based resident trip surveying device |
| CN103150156A (en) * | 2012-12-06 | 2013-06-12 | 江苏省公用信息有限公司 | Method and system, based on geographic model and moving track, for obtaining characteristic crowd in real time |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105959476A (en) * | 2016-05-11 | 2016-09-21 | 上海电机学院 | System and method for acquiring mobile positioning behavior data |
| CN106228187A (en) * | 2016-07-21 | 2016-12-14 | 贵州力创科技发展有限公司 | Individual recognizer model based on multiple user's detail data and treatment technology |
| CN108171532A (en) * | 2017-12-11 | 2018-06-15 | 扬州大学 | A kind of user group distribution forecasting method and system |
| CN109087218A (en) * | 2018-08-10 | 2018-12-25 | 重庆市南岸区瑜目网络科技有限责任公司 | A kind of system being collected and analyze user's tourism data according to satellite positioning |
| WO2020244601A1 (en) * | 2019-06-07 | 2020-12-10 | Beijing Didi Infinity Technology And Development Co., Ltd. | Estimating passenger income level on a ridesharing platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106096631B (en) | A kind of floating population's Classification and Identification analysis method based on mobile phone big data | |
| Chen et al. | Understanding the impacts of human mobility on accessibility using massive mobile phone tracking data | |
| Chen et al. | Identifying urban spatial structure and urban vibrancy in highly dense cities using georeferenced social media data | |
| Ahas et al. | Daily rhythms of suburban commuters’ movements in the Tallinn metropolitan area: Case study with mobile positioning data | |
| CN106931974B (en) | Method for calculating personal commuting distance based on mobile terminal GPS positioning data record | |
| Toole et al. | The path most traveled: Travel demand estimation using big data resources | |
| Hong et al. | Hierarchical community detection and functional area identification with OSM roads and complex graph theory | |
| CN106462627B (en) | Analyzing semantic places and related data from multiple location data reports | |
| Yuan et al. | Correlating mobile phone usage and travel behavior–A case study of Harbin, China | |
| Ahas et al. | Mobile positioning in space–time behaviour studies: social positioning method experiments in Estonia | |
| CN107194525A (en) | A kind of down town appraisal procedure based on mobile phone signaling | |
| CN110414732A (en) | A kind of trip Future Trajectory Prediction method, apparatus, storage medium and electronic equipment | |
| Gartner et al. | Location based services and telecartography | |
| Cesario et al. | An approach for the discovery and validation of urban mobility patterns | |
| Demissie et al. | Analysis of the pattern and intensity of urban activities through aggregate cellphone usage | |
| CN103440278A (en) | Data mining system and method | |
| Šveda et al. | Daily commuting in the Bratislava metropolitan area: case study with mobile positioning data | |
| Yuan et al. | Recognition of functional areas based on call detail records and point of interest data | |
| Cao et al. | Understanding metropolitan crowd mobility via mobile cellular accessing data | |
| Alhazzani et al. | Urban Attractors: Discovering patterns in regions of attraction in cities | |
| Zhou et al. | Village-town system in suburban areas based on cellphone signaling mining and network hierarchy structure analysis | |
| Huo et al. | Exploring the influencing factors on demand-responsive customized bus ridership using multi-source data | |
| Wei et al. | Examining transportation network structures through mobile signaling data in urban China: a case study of Yixing | |
| Zhou et al. | Commuting versus consumption: The role of core city in a metropolitan area | |
| Phithakkitnukoon et al. | Temporary migration flow inference and analysis from perspective of mobile phone network data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20131211 |