CN111552727A - Big data investigation application system - Google Patents
Big data investigation application system Download PDFInfo
- Publication number
- CN111552727A CN111552727A CN202010336477.2A CN202010336477A CN111552727A CN 111552727 A CN111552727 A CN 111552727A CN 202010336477 A CN202010336477 A CN 202010336477A CN 111552727 A CN111552727 A CN 111552727A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- resources
- information
- public security
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
Abstract
The invention discloses a big data investigation application system, which comprises data resources, network intercommunication, a data warehouse, an ESB (enterprise service bus) and a front-end application five-layer framework and a client, wherein the data resources comprise the resource integration of a government affair network, the Internet and a public security network, and by utilizing the public police PKI (public key infrastructure) inquiry authority, an inquiry task is established under a DotnetSpider framework to obtain the inquiry result of each platform of the public security intranet; the network intercommunication is realized, a data synchronization program and a timing task based on JAVA of Sql Job are established, and social data and internet data of local hospitals and government affair service platforms are synchronized to a public security intranet database through data boundary equipment; the invention realizes the organic integration and application of three major types of data of public security network resources, social resources and internet resources, breaks through the problem of data barrier of the public security network, and deeply excavates and integrates various public security network data resources; a social data resource library is established, and various social data resources and data accumulated by daily work of policemen are effectively integrated.
Description
Technical Field
The invention relates to the technical field of network science, in particular to a big data investigation application system.
Background
At present, public security institutions at all levels are implementing public security big data strategy and strongly promote data police construction, but some universal bottleneck problems still exist in the aspect of big data resource service investigation actual combat, firstly, public security intranet system platforms are numerous, nearly 200 public security departments, provincial halls, city bureaus, county bureaus and system platforms of four layers are needed, the inquiry needs to be logged in one by one, items are easily missed, and each system platform inquires out data information and needs to be manually compared, researched and judged, and the intellectualization is low. Secondly, a large amount of social data and daily accumulated working data of policemen do not have a unified data repository and analysis application platform, and are in an isolated deep sleep state, so that the application efficiency is low. And thirdly, the data of the internal network and the external network are separated, the data of the internet and the data of the public security network are split, and the scientific and efficient integrated application resultant force is not formed.
Disclosure of Invention
The present invention is directed to a big data investigation application system to solve the above problems in the background art.
In order to achieve the purpose, the invention provides the following technical scheme: the big data investigation application system comprises data resources, network intercommunication, a data warehouse, an ESB (enterprise service bus), a front-end application five-layer architecture and a client;
the data resources comprise the resource integration of a government affair network, the Internet and a public security network, and a query task is established under a Dotnet Spider framework by utilizing the public police PKI query authority to obtain a query result of each platform of the public security intranet;
the network intercommunication is realized, a data synchronization program and a timing task based on JAVA of Sql Job are established, and social data and internet data of local departments such as hospitals, government affair service platforms and the like are synchronized to a public security intranet database through data boundary equipment;
the data warehouse comprehensively utilizes social data, internet data and intranet data to carry out cleaning, conversion, standardization, labeling and quality control, and establishes the data warehouse with organic, integral and panoramic information after arrangement;
the ESB, platform uses MuleSB to support integration of existing systems such as JMS, WebServices, JDBC, HTTP and other technologies;
the front-end application develops an intelligent search module, a data uploading module, a same-element analysis module, a collaboration alliance module, a background management module, a hyperlink module and a resource integration and community discussion module on the basis of an interface provided by Mule;
the client comprises a real-name registration and login module.
Preferably, the ETL is a data extraction, cleaning, conversion and loading process, a source-open ETL tool button is adopted to extract required data from a data source, and the data is cleaned and loaded into a data warehouse according to a predefined data warehouse model.
Preferably, data tagging, data management requirements and data application requirements are integrated, a tag system of a multi-coordinate system space is established to describe data, tags between each coordinate system do not have necessary relation, the relation between the tags is weak coupling and relatively independent, a tag "value" corresponding to the data, namely, a tag item of the data is obtained through a spatial position of the data, and the current coordinate system for dividing the tag space has: the data classification method comprises four dimensions of data inherent attributes, data classification attributes, data application attributes and data control attributes.
Preferably, the intelligent search module comprises personnel checking, personnel archives and export printing, wherein any one record in the results is queried secondarily or clicked according to the results, and information printing is carried out according to a fixed PDF format through export printing.
Preferably, the staff records include basic information of staff (household registration, biological features, historical photo information, etc.), relationship information (the same household, neighbors, etc.), track information (lodging, internet bar, airplane, train, etc.), internet information (cell phone number, WeChat, QQ, microblog, etc.), social information (hospital, transportation, bank, etc.).
Preferably, the data uploading module comprises social information and link uploading, the social information supports data files in various formats such as word, txt, CSV, excel and the like, the system automatically matches file formats, after uploading, the system automatically identifies the data formats to preview, clicks to confirm, uploads to a back-end database, describes and uploads a system needing hyperlink, and clicks an icon in the hyperlink module to directly enter the system.
Preferably, the same-element analysis module aims at mining and analyzing the internal association relationship of the same element, performs association query and retrieval based on the grasped identity information resources such as the identity card number, the mobile phone number, the license plate number and the like of the suspect, provides a batch comparison function, can perform collision comparison according to the relationship information of the same family, the same live, the same row, the same passenger and the like in the database, and realizes the analysis of the association relationship between two or more numbers.
Preferably, the hyperlink logs in other platforms in a hyperlink mode, and logs in other platforms of the intranet in a single point mode, so that a user can conveniently log in one platform and can link to other platforms for use.
Preferably, the collaboration alliance module issues task collaborations among multiple police, multiple departments and multiple regions, establishes collaboration alliances, and achieves internal circulation of information clues, and the community discussion module provides a community forum function and achieves internal technical communication and opinions and suggestions on a platform.
Preferably, the background management module realizes account management, log management and authority management.
Compared with the prior art, the invention has the beneficial effects that: the system platform adopts an open-source C # acquisition SaaS platform, utilizes a Fiddler tool to realize multi-platform and multi-thread one-stop acquisition of query results of various data sources of each system in a public security intranet, automatically cleans the query results in a CDC mode through an ETL tool button, and stores high-quality data into a MongoDB database, so that the platform realizes organic integration and application of three major types of data including public security network resources, social resources and internet resources, breaks the problem of data barrier of the public security network, and deeply excavates and integrates various public security network data resources; the platform has the advantages that the social data resource library is established, various social data resources and daily work accumulated data of policemen are effectively integrated, the importing and uploading efficiency is improved, the data efficiency is exerted, internet data are communicated to enter a public security network channel, the internet big data acquisition and storage center is established, the internet information resources are obtained in real time through data boundaries, the mining application level of the internet data resources is improved, the platform practically enhances the research and judgment analysis capability of case handling policemen, the research and judgment analysis efficiency of the case handling policemen is effectively enhanced, and the capability and level of the public security bureau big data service for detecting actual combat are comprehensively improved.
Drawings
FIG. 1 is an overall system block diagram of the present invention;
FIG. 2 is a block diagram of a front-end application architecture module of the present invention;
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1 to 2, the present invention provides a technical solution: the big data investigation application system comprises data resources, network intercommunication, a data warehouse, an ESB (enterprise service bus), a front-end application five-layer architecture and a client; the data resources comprise the resource integration of a government affair network, the Internet and a public security network, and a query task is established under a Dotnet Spider framework by utilizing the public police PKI query authority to obtain a query result of each platform of the public security intranet; the network intercommunication is realized, a data synchronization program and a timing task based on JAVA of Sql Job are established, and social data and internet data of local departments such as hospitals, government affair service platforms and the like are synchronized to a public security intranet database through data boundary equipment; the data warehouse comprehensively utilizes social data, internet data and intranet data to carry out cleaning, conversion, standardization, labeling and quality control, the data warehouse is established by organic, integral and panoramic information after the cleaning, and the data cleaning ETL is a data extraction, cleaning, conversion and loading process; data tagging, data management requirements and data application requirements are integrated, a tag system of a multi-coordinate system space is established to describe data, tags between each coordinate system are not necessarily connected, the relationship between the tags is weak coupling and relatively independent, a tag 'value' corresponding to the data is obtained through the spatial position of the data, namely, a tag item of the data, and the current coordinate system for dividing the tag space has the following steps: the method comprises the following steps of (1) data inherent attributes, data classification attributes, data application attributes and data control attributes; the ESB, platform uses MuleSB to support integration of existing systems such as JMS, WebServices, JDBC, HTTP and other technologies; the front-end application develops an intelligent search module, a data uploading module, a same-element analysis module, a collaboration alliance module, a background management module, a hyperlink module, a resource integration and community discussion module on the basis of an interface provided by Mule, the intelligent search module comprises personnel checking, personnel files and export printing, wherein according to results, any one record in secondary query or click results is recorded into the personnel files, and information printing is carried out according to a fixed PDF format through export printing, the personnel files comprise basic data (household roll, biological characteristics, historical photo information and the like), relationship information (same household, neighbors and the like), track information (accommodation, internet bar, airplane, train and the like), internet information (mobile phone number, WeChat, QQ, microblog and the like), social information (hospital, traffic, bank and the like), the data uploading module comprises social information and link uploading, the social information supports data files in various formats such as word, txt, CSV, excel and the like, the system automatically matches file formats, after uploading, the system automatically identifies the data formats to preview, clicks and confirms, uploads to a back-end database, describes and uploads a system needing hyperlink, clicks an icon in a hyperlink module to directly enter the system, the same element analysis module takes the mining and analysis of the internal association relationship of the same element as a target, performs association query and retrieval based on the grasped identity information resources such as identity numbers, mobile phone numbers, license plate numbers and the like of suspects, provides a batch comparison function, can perform collision comparison according to the relationship information such as the same family, the same residence, the same row, the same multiplication and the like in the database, realizes the analysis of the association relationship between two or more numbers, the hyperlink logs on other platforms in the form of hyperlink, and logs on other platforms in a single point mode in the other platforms of an intranet, the user can conveniently log in one platform to be linked to other platforms for use, the cooperative alliance module issues task cooperation among multiple police, multiple departments and multiple regions, establishes a cooperative alliance and realizes internal circulation of information clues, and the community discussion module provides a community forum function and realizes internal technical communication and opinions and suggestions on the platform;
the client comprises a real-name registration and login module, provides double verification of a PKI certificate and an account secret, acquires a PKI inquiry authority and logs in a system.
The platform mainly aims at a big data investigation application system, the system platform adopts an open-source C # acquisition SaaS platform, a Fiddler tool is utilized to realize multi-platform and multi-thread one-stop acquisition of query results of various data sources of each system in a public security intranet, an ETL tool button is utilized to automatically clean the query results in a CDC mode, and high-quality data is stored in a MongoDB database; the platform has the advantages that the social data resource library is established, various social data resources and daily work accumulated data of policemen are effectively integrated, the importing and uploading efficiency is improved, the data efficiency is exerted, internet data are communicated to enter a public security network channel, the internet big data acquisition and storage center is established, the internet information resources are obtained in real time through data boundaries, the mining application level of the internet data resources is improved, the platform practically enhances the research and judgment analysis capability of case handling policemen, the research and judgment analysis efficiency of the case handling policemen is effectively enhanced, and the capability and level of the public security bureau big data service for detecting actual combat are comprehensively improved.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. Big data investigation application system, its characterized in that: the system comprises data resources, network intercommunication, a data warehouse, an ESB (enterprise service bus), a front-end application five-layer architecture and a client;
the data resources comprise the resource integration of a government affair network, the Internet and a public security network, and a query task is established under a Dotnet Spider framework by utilizing the public police PKI query authority to obtain a query result of each platform of the public security intranet;
the network intercommunication is realized, a data synchronization program and a timing task based on JAVA of Sql Job are established, and social data and internet data of local departments such as hospitals, government affair service platforms and the like are synchronized to a public security intranet database through data boundary equipment;
the data warehouse comprehensively utilizes social data, internet data and intranet data to carry out cleaning, conversion, standardization, labeling and quality control, and establishes the data warehouse with organic, integral and panoramic information after arrangement;
the ESB, platform uses MuleSB to support integration of existing systems such as JMS, WebServices, JDBC, HTTP and other technologies;
the front-end application develops an intelligent search module, a data uploading module, a same-element analysis module, a collaboration alliance module, a background management module, a hyperlink module and a resource integration and community discussion module on the basis of an interface provided by Mule;
the client comprises a real-name registration and login module.
2. The big data scout application system according to claim 1, wherein: the ETL is a process of data extraction, cleaning, conversion and loading, and the required data is extracted from a data source by adopting an open-source ETL tool button, and is loaded into a data warehouse according to a predefined data warehouse model after data cleaning.
3. The big data scout application system according to claim 1, wherein: data tagging, data management requirements and data application requirements are integrated, a tag system of a multi-coordinate system space is established to describe data, tags between each coordinate system are not necessarily connected, the relationship between the tags is weak coupling and relatively independent, a tag 'value' corresponding to the data is obtained through the spatial position of the data, namely, a tag item of the data, and the current coordinate system for dividing the tag space has the following steps: the data classification method comprises four dimensions of data inherent attributes, data classification attributes, data application attributes and data control attributes.
4. The big data scout application system according to claim 1, wherein: the intelligent search module comprises personnel checking, personnel archives and export printing, wherein any one record in the results is inquired secondarily or clicked according to the results, enters the personnel archives, and information printing is carried out according to a fixed PDF format through export printing.
5. The big data spy application system of claim 4, wherein: the personnel file comprises basic data (household register, biological characteristics, historical photo information and the like) of personnel, relationship information (the same household, neighbors and the like), track information (accommodation, internet bar, airplane, train and the like), internet information (mobile phone number, WeChat, QQ, microblog and the like), and social information (hospital, transportation, bank and the like).
6. The big data scout application system according to claim 1, wherein: the data uploading module comprises social information and link uploading, the social information supports data files in various formats such as word, txt, CSV, excel and the like, the system automatically matches file formats, after uploading, the system automatically identifies the data formats to preview, clicks to confirm, uploads to a back-end database, describes and uploads a system needing hyperlink, and clicks an icon to directly enter the system in the hyperlink module.
7. The big data scout application system according to claim 1, wherein: the same-element analysis module aims at mining and analyzing the internal association relationship of the same element, performs association query and retrieval based on the grasped identity information resources such as identity numbers, mobile phone numbers, license plate numbers and the like of suspects, provides a batch comparison function, can perform collision comparison according to relationship information such as the same family, the same live, the same row, the same riding and the like in a database, and realizes the analysis of the association relationship between two or more numbers.
8. The big data scout application system according to claim 1, wherein: the hyperlink logs in other platforms in a hyperlink mode, single sign-on is carried out on other platforms of the intranet, and a user can conveniently log in one platform and then can link to other platforms for use.
9. The big data scout application system according to claim 1, wherein: the community discussion module provides a community forum function and realizes internal technical exchange and opinions and suggestions on the platform.
10. The big data scout application system according to claim 1, wherein: the background management module realizes account management, log management and authority management.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010336477.2A CN111552727A (en) | 2020-04-26 | 2020-04-26 | Big data investigation application system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010336477.2A CN111552727A (en) | 2020-04-26 | 2020-04-26 | Big data investigation application system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111552727A true CN111552727A (en) | 2020-08-18 |
Family
ID=72005745
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010336477.2A Pending CN111552727A (en) | 2020-04-26 | 2020-04-26 | Big data investigation application system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111552727A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105554070A (en) * | 2015-12-09 | 2016-05-04 | 北京中科云集科技有限公司 | Method based on police affair big data center service construction |
| CN105574657A (en) * | 2015-12-11 | 2016-05-11 | 山东新网格信息技术有限公司 | Intelligent police actual combat comprehensive application platform |
| CN108334635A (en) * | 2018-02-27 | 2018-07-27 | 浪潮软件集团有限公司 | Content customizable smart city comprehensive presentation system |
| CN108446395A (en) * | 2018-03-26 | 2018-08-24 | 北京神州泰岳软件股份有限公司 | A kind of police service information processing method and system based on big data |
| CN109522342A (en) * | 2018-11-30 | 2019-03-26 | 北京百度网讯科技有限公司 | Police affairs management method, device, equipment and storage medium |
-
2020
- 2020-04-26 CN CN202010336477.2A patent/CN111552727A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105554070A (en) * | 2015-12-09 | 2016-05-04 | 北京中科云集科技有限公司 | Method based on police affair big data center service construction |
| CN105574657A (en) * | 2015-12-11 | 2016-05-11 | 山东新网格信息技术有限公司 | Intelligent police actual combat comprehensive application platform |
| CN108334635A (en) * | 2018-02-27 | 2018-07-27 | 浪潮软件集团有限公司 | Content customizable smart city comprehensive presentation system |
| CN108446395A (en) * | 2018-03-26 | 2018-08-24 | 北京神州泰岳软件股份有限公司 | A kind of police service information processing method and system based on big data |
| CN109522342A (en) * | 2018-11-30 | 2019-03-26 | 北京百度网讯科技有限公司 | Police affairs management method, device, equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 冯 前: "警务信息综合应用平台实施及适应性改造", pages 13 - 15 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105138661B (en) | A kind of network security daily record k-means cluster analysis systems and method based on Hadoop | |
| Cooper et al. | Challenges for database management in the internet of things | |
| CN110990664B (en) | Big data operation management system | |
| CN105554070A (en) | Method based on police affair big data center service construction | |
| CN101743542A (en) | Collecting and presenting temporal-based action information | |
| CN109189782A (en) | A kind of indexing means in block chain commodity transaction inquiry | |
| CN105556552A (en) | Fraud detection and analysis | |
| CN103995807B (en) | Magnanimity data query and the method for after-treatment under a kind of framework based on Web | |
| CN104933070A (en) | Catalog management system used for government affairs information platform | |
| CN111427968A (en) | Key person holographic archive construction method and device based on knowledge graph | |
| US20100174386A1 (en) | Method and apparatus for routing data in an automatic identification system | |
| CN102508833A (en) | Data monitoring device and data monitoring method | |
| US20190050435A1 (en) | Object data association index system and methods for the construction and applications thereof | |
| US20160092452A1 (en) | Large-scale processing and querying for real-time surveillance | |
| CN104680265A (en) | Public employment service management system and equipment | |
| CN105117824A (en) | Cloud service system used for project management | |
| CN106528794A (en) | Electronic document filing method based on archive management system | |
| CN114238388A (en) | Heterogeneous data collection and retrieval system based on multiple protocols | |
| CN102523309A (en) | Method and equipment for responding request of client side and acquiring and returning real-time data | |
| CN103345383A (en) | Method and device for comparing multithreading data | |
| CN111552727A (en) | Big data investigation application system | |
| Basaille et al. | Towards a twitter observatory: A multi-paradigm framework for collecting, storing and analysing tweets | |
| Obalı et al. | A real time data warehouse approach for data processing | |
| CN116821106A (en) | Enterprise data digital management system and method | |
| CN111861830B (en) | Information cloud platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |