CN112187961A - Real mac acquisition mechanism system and method - Google Patents
Real mac acquisition mechanism system and method Download PDFInfo
- Publication number
- CN112187961A CN112187961A CN201910605184.7A CN201910605184A CN112187961A CN 112187961 A CN112187961 A CN 112187961A CN 201910605184 A CN201910605184 A CN 201910605184A CN 112187961 A CN112187961 A CN 112187961A
- Authority
- CN
- China
- Prior art keywords
- ssid
- mac
- data
- probe
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007246 mechanism Effects 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 title claims abstract description 20
- 239000000523 sample Substances 0.000 claims abstract description 52
- 238000012545 processing Methods 0.000 claims abstract description 44
- 238000013499 data model Methods 0.000 claims abstract description 30
- 230000004044 response Effects 0.000 claims abstract description 7
- 230000008569 process Effects 0.000 claims abstract description 5
- 230000002093 peripheral effect Effects 0.000 claims description 3
- 238000004140 cleaning Methods 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 3
- 238000013480 data collection Methods 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L2101/00—Indexing scheme associated with group H04L61/00
- H04L2101/60—Types of network addresses
- H04L2101/618—Details of network addresses
- H04L2101/622—Layer-2 addresses, e.g. medium access control [MAC] addresses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Abstract
The invention provides a real mac collecting mechanism system and a method, which comprises the following steps that a proper ssid list is selected according to the geographical position of a pointer, a probe response is constructed to reply to a mobile device, when the mobile device receives the probe response, if the ssid in the probe is connected, the mobile device tries to connect, the real device mac is returned to a probe, the probe automatically creates a hidden ssid at the same time, a direct probe request returned by the mobile device comprises a previously connected ssid of a terminal, the direct probe request is externally placed to obtain the real mac, a data processing module 200 processes the probe request collected by the probe, the mac is stored in a data middleware module 500, the ssid is stored in a ssid library 300, the id is continuously updated according to the weight and is stored in the ssid library 300, the data processing module 200 processes the data, the obtained mac is sent to a data middleware module 500, and the data middleware module 500 judges the authenticity of the real mac by using a real mac data model 400, the real mac acquisition mechanism is low in identification cost and high in automation degree, and can acquire the real mac through the pseudo mac, so that interference of garbage data is reduced.
Description
Technical Field
The invention relates to the information technology industry, which is used for carrying out data cleaning after data are acquired through wireless sniffing, and reducing the influence of ios and android system pseudo mac mechanisms on the accuracy and integrity of data analysis.
Background
The mac address is a unique identity used for identifying the network equipment, equipment with a wifi function also belongs to the network equipment, and therefore the equipment also has a globally unique mac address, the existing identification method needs to rely on a plurality of preconditions, on one hand, the implementation cost is high, on the other hand, the accuracy is problematic, the existing identification method is not dynamic due to the fact that the pseudo mac mechanism is updated in a long time mode, mainly, the true and false macs are distinguished, and the true macs corresponding to the pseudo macs cannot be obtained, so that data analysis is completed better.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a real mac acquisition mechanism system and a real mac acquisition mechanism method, which can self-adjust for the update of a pseudo mac mechanism, reduce the interference of garbage data, have low identification cost and high automation degree of the identification mechanism, and can acquire the real mac through the pseudo mac.
In order to achieve the above purposes, the invention adopts the technical scheme that: a real mac collection mechanism system is provided, which comprises a data collection module 100, a data processing module 200, an ssid library 300, a real mac identification data model 400 and a data middleware module 500;
the ssid library 300 is a database containing location information, weights, ssids, and bssid wireless hotspots;
the data acquisition module 100 analyzes the mac layer and above protocols of all mobile terminal wireless data packets through the ssid library 300, classifies the data, and sends the analyzed data to the data processing module 200;
the data processing module 200 performs data processing, sends the obtained real mac to the data middleware module 500, and stores the updated ssid into the ssid library 300;
the data middleware module 500 analyzes and identifies the authenticity mac using the real mac recognition data model 400.
The data acquisition module 100, the data processing module 200 and the data middleware module 500 interact with each other in sequence to complete wifi data acquisition and preliminary processing such as mac and ssid.
The data acquisition module 100, the data processing module 200, the ssid library 300 and the real mac identification data model 400 interact in sequence, the ssid library 300 and the real mac identification data model 400 are continuously optimized according to the sniffing hit rate, the updated ssid is stored in the ssid library 300, and the real mac is stored in the real mac identification data model 400.
The data processing module 200, the ssid library 300, the real mac identification data model 400 and the data middleware module 500 interact with each other in sequence to realize the extraction of the real mac and the judgment of authenticity of the mac.
The data acquisition module is a wifi data packet acquisition protocol and is used for analyzing the mac layer and the protocol data above the mac layer by capturing all mobile terminal wireless data packets in the probe area, classifying the data and sending the analyzed data to the data processing module.
The data processing module is a real-time streaming standardized processing mode of data cleaning, data processing and data association, and is used for sending processed data to the data middleware module, realizing accumulation and storage of wireless data, converting unstructured data into structured data after cleaning and association, and facilitating development of subsequent data analysis processes based on business angles.
The ssid library is a database containing wireless hotspots such as location information, weight, ssid, bssid and the like, and is used for determining an acquisition strategy of the data acquisition module.
The real mac data model is a real mac judgment model, comprises a clustering model and a basic real mac database, and is used for carrying out clustering analysis on the acquired macs, labeling according to true macs and false macs and marking confidence coefficients.
The data middleware module is a method for storing, analyzing and processing data and is used for providing further support for business requirements.
A real mac acquisition mechanism method comprises the following steps:
s1, selecting a proper ssid list from the ssid library 300 according to the geographic position of the probe mobile device, and constructing a probe response for each ssid to reply to the mobile device;
s2, automatically creating a hidden ssid by the probe;
s3, in the data processing module 200, processing the probe request collected by the probe, storing mac in the data middleware module 500, and storing the ssid in the ssid library 300;
s4, the data middleware module 500 sniffs the probe to acquire the mobile device mac using the real mac data model 400.
The goods single-piece separating device has the advantages that the ssid library 300 is used for determining the collection strategy of the data collection module 100, the data collection module 100 is used for classifying and analyzing the data and sending the data to the data processing module 200, the data processing module 200 realizes the accumulation and storage of wireless data, the true mac identification data model 400 is used for distinguishing the authenticity mac, the data middleware module 500 temporarily stores the data, the identification mechanism of the system has high automation degree, the self-adjustment can be carried out aiming at the update of the fake mac mechanism, the ssid list is selected according to the position of the probe, each ssid is constructed into a probe response to reply the mobile equipment, a hidden ssid is automatically created at the same time, the true mac address is obtained, the updated ssid is stored in the ssid library 300, the true mac is distinguished by the true mac identification data model 400 in the data middleware module 500, the identification mechanism cost is low, true and false mac identification can be completed by bumping the library without accumulating a large number of true mac libraries, and can be completed by the true mac identification data model 400, so that not only can true and false macs be identified, but also corresponding true macs can be obtained from the obtained false macs.
Wherein step s1 includes the steps of:
s11, the mobile device comprises ios and android, and when a probe response is received, the ssid in the probe is the one the device has been connected to, and connection is attempted;
and S12, acquiring the ssid information of the peripheral space by the probe, and adding the ssid information into the ssid list.
Wherein step s2 further includes the steps of:
s21, the hidden ssid automatically created by the probe triggers the mobile equipment to send a directed probe request with the ssid;
s22, the direct probe request returned by the mobile equipment comprises the last ssid list connected by the terminal, the probe constructs an ssid according to the ssid list, the real mac is obtained by external playing, the external playing determines a carousel rule through the ssid weight, and the carousel rule is played.
Wherein step s3 further includes the steps of:
s31, the data processing module determines the weight of the ssid at a certain physical position according to the position of the ssid, the success rate of putting the ssid signal to hit the real mac, and the passing weight of the ssid in other areas, updates the weight to the ssid library 300, and stores the ssid into the ssid library, wherein the higher the weight is, the higher the duration and frequency of putting the ssid by the probe are.
Wherein s4 further comprises the steps of:
s41, the collected mac comprises a real mac and a suspected fake mac, authenticity of the mac is judged according to the occurrence frequency and frequency of the mac, the real mac is stored in the real mac identification data model 400, the collected mac comprises the real mac and the suspected fake mac, the real mac and the suspected fake mac are matched according to a real mac library, the matched mac is marked as the real mac, other macs are marked as the suspected macs, the suspected macs are processed, and macs which cannot be matched with equipment manufacturers are removed according to matching of a mac oui library. Then processing the remaining suspected false mac, judging the authenticity of the mac according to the occurrence frequency and frequency of the mac, and storing the true mac into the real mac identification data model 400;
and S42, regularly performing authenticity operation on the collected pseudo mac at the same time, and avoiding omission.
Drawings
Fig. 1 is a flowchart of the process of the present embodiment.
Detailed Description
For a more clear understanding of the technical features, objects and effects of the present invention, embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
Fig. 1 is a flowchart of steps of a real mac collecting mechanism system and method according to the present invention, which includes a data collecting module 100, a data processing module 200, an ssid library 300, a real mac identification data model 400, and a data middleware module 500;
the ssid library 300 is a database containing location information, weights, ssids, and bssid wireless hotspots;
the data acquisition module 100 analyzes the mac layer and above protocols for all mobile terminal wireless data packets through the ssid library 300, classifies the data, and sends the analyzed data to the data processing module 200;
the data processing module 200 performs data processing, sends the obtained real mac to the data middleware module 500, and stores the updated ssid into the ssid library 300;
the data middleware module 500 analyzes and identifies the authenticity mac using the real mac recognition data model 400.
The data acquisition module 100, the data processing module 200 and the data middleware module 500 interact in sequence to complete the acquisition and preliminary processing of wifi data such as mac and ssid.
The data acquisition module 100, the data processing module 200, the ssid library 300 and the real mac identification data model 400 interact in sequence, the ssid library 300 and the real mac identification data model 400 are continuously optimized according to the sniff hit rate, the updated ssid is stored in the ssid library 300, and the real mac is stored in the real mac identification data model 400.
The data processing module 200, the ssid library 300, the real mac identification data model 400 and the data middleware module 500 interact in sequence to realize the extraction of the real mac and the judgment of authenticity of the mac.
A real mac acquisition mechanism method comprises the following steps:
and S1, selecting a proper ssid list from the ssid library according to the geographic position of the probe, and constructing a probe request for each ssid to reply to the mobile equipment.
S2, the probe automatically creates a hidden ssid at the same time,
s3, in the data processing module, processing the probe request collected by the probe, storing mac in the data middleware module, and storing the ssid in an ssid library;
and S4, the data middleware module utilizes the real mac data model to sniff the probe and collect the mobile equipment mac.
Wherein step s1 further includes the steps of:
s11, the mobile equipment comprises ios and android, and when a probe request is received, the ssid in the probe is the one the equipment has been connected and tries to connect;
s12, the probe simultaneously needs to collect the ssid information of the peripheral space and adds the ssid information into an ssid list;
wherein step s2 further includes the steps of:
s21, automatically creating a hidden ssid by the probe, and triggering the mobile equipment to send a directed probe request with the ssid;
and the directed probe request returned by the S22 contains the ssid list which is connected with the terminal, and the probe constructs an ssid according to the ssid list, and the real mac is obtained by external amplification.
Wherein step s3 further includes the steps of:
s31, the data processing module (200) determines the weight of the ssid at a certain physical position according to the position of the ssid, the success rate of putting out the ssid signal to hit the real mac, and the past weight of the ssid in other areas, updates the weight to the ssid library (300), and stores the ssid in the ssid library 300.
Wherein step s4 further includes the steps of:
s41, judging the authenticity of the mac according to the occurrence frequency and frequency of the mac, storing the true mac into a true mac identification data model 400, wherein the acquired mac comprises the true mac and a suspected pseudo mac, firstly matching the true mac with the suspected pseudo mac according to a true mac library, marking the matched mac as the true mac and marking other macs as the suspected macs, then processing the suspected macs, eliminating macs which cannot be matched with equipment manufacturers according to the matching of a mac oui library, continuously processing the remaining suspected pseudo macs, and judging the authenticity of the mac according to the occurrence frequency and frequency of the mac;
and S42, regularly performing authenticity operation on the collected pseudo mac at the same time, and avoiding omission.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (9)
1. A real mac collection mechanism system is characterized in that: the system comprises a data acquisition module (100), a data processing module (200), an ssid library (300), a real mac identification data model (400) and a data middleware module (500);
the ssid library (300) is a database containing location information, weight, ssid, bssid wireless hotspot;
the data acquisition module (100) analyzes the mac layer and the protocols of all the mobile terminal wireless data packets through the ssid library (300), classifies the data and sends the analyzed data to the data processing module (200);
the data processing module (200) performs data processing, sends the obtained real mac to the data middleware module (500), and stores the updated ssid into an ssid library (300);
the data middleware module (500) analyzes and identifies the authenticity mac by using the real mac recognition data model (400).
2. The actual mac acquisition mechanism system as claimed in claim 1, wherein: the data acquisition module (100), the data processing module (200) and the data middleware module (500) are sequentially interacted to acquire and preliminarily process wifi data such as mac and ssid.
3. The actual mac acquisition mechanism system as claimed in claim 1, wherein: the data acquisition module (100), the data processing module (200), the ssid library (300) and the real mac identification data model (400) are sequentially interacted, the ssid library (300) and the real mac identification data model (400) are continuously optimized according to the sniffing hit rate, the updated ssid is stored in the ssid library (300), and the real mac is stored in the real mac identification data model (400).
4. The actual mac acquisition mechanism system as claimed in claim 1, wherein: the data processing module (200), the ssid library (300), the real mac recognition data model (400) and the data middleware module (500) interact in sequence to extract the real mac and judge the authenticity of the mac.
5. A real mac acquisition mechanism system and method are characterized in that: the real mac acquisition mechanism system according to any one of the preceding claims 1 to 4, comprising the following steps:
s1, selecting a proper ssid list from the ssid library (300) according to the geographic position of the probe mobile equipment, and constructing a probe response for each ssid to reply to the mobile equipment;
s2, automatically creating a hidden ssid by the probe;
s3, in the data processing module (200), processing the probe request collected by the probe, storing the mac in the data middleware module (500), and storing the ssid in the ssid library (300);
s4, the data middleware module (500) sniffs the probe to acquire the mobile device mac using the real mac data model (400).
6. The system and method for real mac collection according to claim 5, wherein step s1 further includes the following steps:
s11, the mobile device comprises ios and android, and when a probe response is received, the ssid in the probe is the one the device has been connected to, and connection is attempted;
and S12, acquiring the ssid information of the peripheral space by the probe, and adding the ssid information into the ssid list.
7. The method according to claim 5, wherein the step S2 further comprises the following steps:
s21, the hidden ssid automatically created by the probe triggers the mobile equipment to send a directed probe request with the ssid;
s22, the direct probe request returned by the mobile equipment comprises the last ssid list connected by the terminal, the probe constructs an ssid according to the ssid list, the real mac is obtained through outward playing, the outward playing determines the carousel rule through the weight of the ssid, and the carousel rule is played.
8. The method according to claim 5, wherein the step S3 further comprises the following steps:
s31, the data processing module (200) determines the weight of the ssid at a certain physical position according to the position of the ssid, the success rate of putting out the ssid signal to hit the real mac, and the past weight of the ssid in other areas, updates the weight to the ssid library (300), and stores the ssid in the ssid library (300).
9. The method according to claim 5, wherein the step S4 further comprises the following steps:
s41, the collected mac comprises a real mac and a suspected fake mac, the authenticity of the mac is judged according to the occurrence frequency and frequency of the mac, and the real mac is stored in a real mac identification data model (400);
and S42, periodically carrying out authenticity operation on the collected pseudo mac.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910605184.7A CN112187961A (en) | 2019-07-05 | 2019-07-05 | Real mac acquisition mechanism system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910605184.7A CN112187961A (en) | 2019-07-05 | 2019-07-05 | Real mac acquisition mechanism system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112187961A true CN112187961A (en) | 2021-01-05 |
Family
ID=73914657
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910605184.7A Pending CN112187961A (en) | 2019-07-05 | 2019-07-05 | Real mac acquisition mechanism system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112187961A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102438238A (en) * | 2011-12-28 | 2012-05-02 | 武汉虹旭信息技术有限责任公司 | Method for detecting illegal AP (Assembly Program) under centralized WLAN (Wireless Local Area Network) environment |
| CN104219670A (en) * | 2014-09-03 | 2014-12-17 | 珠海市君天电子科技有限公司 | Method and system for identifying false wifi (wireless fidelity), client side and server side |
| CN106211210A (en) * | 2016-07-21 | 2016-12-07 | 深圳奇迹智慧网络有限公司 | A kind of mobile terminal MAC data acquisition method |
| CN107094293A (en) * | 2017-06-27 | 2017-08-25 | 南京赢纳信息科技有限公司 | A kind of device and method for obtaining WiFi terminal real MAC address |
| CN107623754A (en) * | 2017-09-28 | 2018-01-23 | 武汉虹旭信息技术有限责任公司 | WiFi acquisition systems and its method based on true and false MAC identifications |
| CN108966163A (en) * | 2018-06-22 | 2018-12-07 | 上海意视信息科技有限公司 | A kind of WIFI probe device, data collection system and method |
| CN109219050A (en) * | 2018-11-27 | 2019-01-15 | 武汉虹旭信息技术有限责任公司 | A kind of WIFI acquisition system and its method based on virtual AP |
-
2019
- 2019-07-05 CN CN201910605184.7A patent/CN112187961A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102438238A (en) * | 2011-12-28 | 2012-05-02 | 武汉虹旭信息技术有限责任公司 | Method for detecting illegal AP (Assembly Program) under centralized WLAN (Wireless Local Area Network) environment |
| CN104219670A (en) * | 2014-09-03 | 2014-12-17 | 珠海市君天电子科技有限公司 | Method and system for identifying false wifi (wireless fidelity), client side and server side |
| CN106211210A (en) * | 2016-07-21 | 2016-12-07 | 深圳奇迹智慧网络有限公司 | A kind of mobile terminal MAC data acquisition method |
| CN107094293A (en) * | 2017-06-27 | 2017-08-25 | 南京赢纳信息科技有限公司 | A kind of device and method for obtaining WiFi terminal real MAC address |
| CN107623754A (en) * | 2017-09-28 | 2018-01-23 | 武汉虹旭信息技术有限责任公司 | WiFi acquisition systems and its method based on true and false MAC identifications |
| CN108966163A (en) * | 2018-06-22 | 2018-12-07 | 上海意视信息科技有限公司 | A kind of WIFI probe device, data collection system and method |
| CN109219050A (en) * | 2018-11-27 | 2019-01-15 | 武汉虹旭信息技术有限责任公司 | A kind of WIFI acquisition system and its method based on virtual AP |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107623754B (en) | WiFi acquisition system and method based on authenticity MAC identification | |
| CN102497667B (en) | Method for positioning WiFi (wireless fidelity) application focused areas, signal acquisition terminal and system | |
| CN112040432A (en) | Method and device for identifying mobile terminal user type | |
| CN109246685B (en) | Passenger flow volume statistical method, device and storage medium based on MAC address | |
| US10984040B2 (en) | Collection and provision method, device, system and server for vehicle image data | |
| CN110493363B (en) | System and method for distinguishing random MAC address of smart phone | |
| CN104270275A (en) | Auxiliary analysis method for causes of exceptions, server and intelligent equipment | |
| US20170180940A1 (en) | System and method for geography-based correlation of cellular and wlan identifiers | |
| CN109451486B (en) | WiFi acquisition system based on detection request frame and WiFi terminal detection method | |
| CN112235722B (en) | Information output method and device, computer equipment and readable storage medium | |
| CN101964813A (en) | Method and system for detecting terminal information in GPRS network | |
| US11553450B2 (en) | Methods and apparatus for locating mobile devices using wireless signals in mixed mode | |
| CN110868436B (en) | Internet of things data acquisition method and device | |
| WO2022121972A1 (en) | Information output method and apparatus, computer device, and readable storage medium | |
| CN107071708A (en) | Intelligent mobile terminal passive wireless signal acquisition and localization method | |
| CN109842851B (en) | Storage article real-time positioning method and storage article positioning system | |
| CN112187961A (en) | Real mac acquisition mechanism system and method | |
| CN113691483B (en) | Method, device and equipment for detecting abnormal user equipment and storage medium | |
| CN103546900B (en) | Electronic characteristic acquisition method based on authentication | |
| CN105959911A (en) | Method and device for identifying user | |
| CN109151827A (en) | WiFi Location fraud detection method and device based on radio-frequency fingerprint | |
| KR20170098997A (en) | Mac address information collection apparatus and method | |
| CN111212376B (en) | Method, apparatus, device and medium for correlating real-time location and voice quality results | |
| WO2020114137A1 (en) | Message processing method, device and system | |
| KR102521985B1 (en) | OUI automatic collecting computer device and method of mobile device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210105 |