CN109344190A - A kind of police service data processing method and device - Google Patents
A kind of police service data processing method and device Download PDFInfo
- Publication number
- CN109344190A CN109344190A CN201811149077.XA CN201811149077A CN109344190A CN 109344190 A CN109344190 A CN 109344190A CN 201811149077 A CN201811149077 A CN 201811149077A CN 109344190 A CN109344190 A CN 109344190A
- Authority
- CN
- China
- Prior art keywords
- data
- library
- police service
- service data
- police
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the present application provides a kind of police service data processing method and device, firstly, by the police service tidal data recovering of separate sources into acquisition library, realize data collect and rough sort；Then, the data item that user pays close attention to is extracted from the police service data that acquisition library is issued, library is extracted in deposit；Then, data cleansing is carried out to the police service data for extracting library publication, by the police service data deposit cleaning library after cleaning, realizes purification and the disaggregated classification of data；Then, data analysis is carried out to the police service data of cleaning library publication, and analysis result data is deposited into fusion library；Finally, exporting analysis result data from fusion library according to the business demand of police system.Method provided by the embodiments of the present application as a result, can be according to collecting police service data, and extracts the police service data of user's concern according to the demand of user and successively classified and analyzed, thus it is versatile, it can satisfy police to the different demands of police service data.
Description
Technical field
This application involves technical field of data processing more particularly to a kind of police service data processing method and devices.
Background technique
With the arrival of big data era, the data volume generated daily in society is more and more huger, these data are related to respectively
A industry, such as: population information data, houseclearing data, marriage data etc..These data due to being related to different industries,
And enormous amount, therefore it is difficult manually means from the number relationship hidden between mining data in these data.
The prior art attempt by some technological means from the data of magnanimity between mining data hide data relationship,
But the prior art is difficult to carry out data relationship complete by excavating specified data relationship generally only for some specific requirements
It excavates to face, bad adaptability.And in some usage scenarios, the demand of user changes frequent occurrence, and those skilled in the art are not
It obtains not as the change of user demand redesigns new technical solution to meet the needs of new, to reduce data mining
Efficiency.For example, in police work, the demand of community people's police, household register people's police, policeman in charge of the case to police service data is different
, such as: community people's police wish to obtain the data such as Community Population information, occupancy information, houseclearing, and household register people's police then wish
Obtain mobile personnel's information, the data such as personnel's family relationship, therefore in order to meet police to the different demands of police service data,
The excavation scheme of police service data will design extremely complex, both reduce the treatment effeciency of police service data, and also be unfavorable for data
It is shared.
Therefore, how a kind of data processing scheme with versatility is provided, to adapt to wider data processing needs,
Referred to as those skilled in the art's technical problem urgently to be resolved.
Summary of the invention
The embodiment of the present application provides a kind of police service data processing method and device, to solve at data in the prior art
Reason method poor universality, the problem of being unable to satisfy different demands of the police to police service data.
In a first aspect, according to an embodiment of the present application, providing a kind of police service data processing method, comprising:
By the police service tidal data recovering of separate sources into acquisition library, and it is distributed to and extracts library；
The data item that user pays close attention to is extracted from the police service data that acquisition library is issued, library is extracted in deposit, and is distributed to cleaning
Library；
Data cleansing is carried out to the police service data for extracting library publication, by the police service data deposit cleaning library after cleaning, concurrently
Cloth gives fusion library；
Data analysis is carried out to the police service data of cleaning library publication, and analysis result data is deposited into fusion library；
According to the business demand of police system, analysis result data is exported from fusion library.
Second aspect provides a kind of police service data processing equipment according to an embodiment of the present application, comprising:
Acquisition module, for by the police service tidal data recovering of separate sources to acquisition library in, and be distributed to extract library；
Library is extracted in abstraction module, the data item paid close attention to for extracting user from the police service data that acquisition library is issued, deposit,
And it is distributed to cleaning library；
Cleaning module deposits the police service data after cleaning for carrying out data cleansing to the police service data for extracting library publication
Enter and clean library, and is distributed to fusion library；
Analysis module for carrying out data analysis to the police service data of cleaning library publication, and analysis result data is stored in
To fusion library；
Export module exports analysis result data from fusion library for the business demand according to police system.
From the above technical scheme, the embodiment of the present application provides a kind of police service data processing method and device, firstly,
By the police service tidal data recovering of separate sources to acquisition library in, and be distributed to extract library, realize data collect and rough sort；So
Afterwards, the data item that user pays close attention to is extracted from the police service data that acquisition library is issued, library is extracted in deposit, and is distributed to cleaning library, real
The extraction of current family focused data；Then, data cleansing is carried out to the police service data for extracting library publication, by the police service number after cleaning
Library is cleaned according to deposit, and is distributed to fusion library, realizes purification and the disaggregated classification of data；Then, to the police service number of cleaning library publication
It is deposited into fusion library according to progress data analysis, and by analysis result data, the data of each classification are analyzed in realization；Most
Afterwards, according to the business demand of police system, analysis result data is exported from fusion library, is realized the analysis knot of the application arrived
Fruit data application is into the business of police system.Method provided by the embodiments of the present application as a result, can be according to collecting police service number
According to, and extract the police service data of user's concern according to the demand of user and classify and analyze, thus versatile, Neng Gouman
Different demands of the sufficient police to police service data.
Detailed description of the invention
In order to illustrate more clearly of the technical solution of the application, letter will be made to attached drawing needed in the embodiment below
Singly introduce, it should be apparent that, for those of ordinary skills, without any creative labor,
It is also possible to obtain other drawings based on these drawings.
Fig. 1 is a kind of flow chart of police service data processing method provided by the embodiments of the present application；
Fig. 2 is a kind of database schema figure of police service data processing method provided by the embodiments of the present application；
Fig. 3 is a kind of flow chart of police service data processing method steps S110 provided by the embodiments of the present application；
Fig. 4 is a kind of flow chart of police service data processing method steps S120 provided by the embodiments of the present application；
Fig. 5 is a kind of flow chart of police service data processing method steps S140 provided by the embodiments of the present application；
Fig. 6 is a kind of flow chart of police service data processing method steps S150 provided by the embodiments of the present application；
Fig. 7 is a kind of flow chart of the first time data reparation of police service data processing method provided by the embodiments of the present application；
Fig. 8 is a kind of flow chart of second of data reparation of police service data processing method provided by the embodiments of the present application；
Fig. 9 is a kind of schematic diagram of police service data processing equipment provided by the embodiments of the present application.
Specific embodiment
In order to make those skilled in the art better understand the technical solutions in the application, below in conjunction with the application reality
The attached drawing in example is applied, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described implementation
Example is merely a part but not all of the embodiments of the present application.Based on the embodiment in the application, this field is common
The application protection all should belong in technical staff's every other embodiment obtained without making creative work
Range.
With the arrival of big data era, the technical staff of various industries is attempting to improve industry using big data technology
Interior production efficiency creates higher social value.For example, day accumulates the moon to police in the daily work in police work
A large amount of police service data are had accumulated tiredly, such as: population information data, houseclearing data, marriage data, mobile personnel's letter
Breath, personnel's family relationship data etc..From a wealth of sources, the relative distribution of these data, may but contain deeper data
Relationship often play a role in the tasks such as demographic census, analysis case in police, and with the work of police
Make task difference, the demand to police service data is also different.
In order to meet police to the different demands of police service data, this application provides a kind of police service data processing methods
And device.
Here is the present processes embodiment.This method can be applied to server, PC (PC), plate electricity
In the plurality of devices such as brain, mobile phone, virtual reality device and intelligent wearable device.
Fig. 1 is a kind of flow chart of police service data processing method provided by the embodiments of the present application.
Fig. 2 is a kind of database schema figure of police service data processing method provided by the embodiments of the present application.
As depicted in figs. 1 and 2, method includes the following steps:
Step S110 by the police service tidal data recovering of separate sources into acquisition library, and is distributed to and extracts library.
Wherein, police service data may include: population information, occupancy information, houseclearing etc..Further, population information
Population essential information (for example, name, age, identification card number of the permanent personnel in community or mobile personnel), people can be specifically included
Member's social relationships information (such as: work address, work unit, the social place often entered and left, the personnel to contact in the recent period letter
Breath), personnel's family relationship information (such as kinsfolk's information), personnel's contact information (such as: telephone number, social matchmaker
The letter such as body account), etc.；Occupancy information can specifically include the home address information registered in household register system, practical family
Certificate address information, once certificate address information etc.；Houseclearing can specifically include the self-owned house information of personnel, believe with other people shared houses
Breath, information of renting a house certainly, information of renting a house jointly with other people, lease room house-owner information etc..
In addition, police service data can have multiple sources.For example, obtaining some local population from public security household register system
Information；From personnel's social relationships information and the house letter obtained in the statistical data of grass-roots police affairs department or community management department
Breath；The personnel's social relationships information obtained from the statistical data of labour department；From the system of real name authentication information of social media
Personnel's contact information of acquisition；The personnel's information of vehicles etc. obtained from vehicle management department.
Further, police service data can also by police using police service data acquisition equipment it is daily it is on duty,
It is obtained in visiting, check and investigating.Police service information collecting device can be hand-held mobile collection equipment, such as: mobile phone, plate
Computer, scanner, laptop, law-enforcing recorder, camera, video camera, sound pick-up outfit etc. can also be fixed acquisition
Equipment, such as: have any one or more information input function such as keyboard input, mouse input, video input and voice input
The computer equipment etc. of energy.
Further, police service data can be a variety of data modes such as structural data and unstructured data.Specifically may be used
To be the database file (such as: the database file etc. of CSV, XLS format) of structured storage data；It can also be non-structural
The database file (such as database file of TXT, DB format etc.) of the storing data of change；Can also be media file (such as
Image file, voice document etc.), when police service data are media files, automatic speech recognition technology can be used
(automatic speech recognition, ASR), optical character recognition technology (optical character
Recognition, OCR) the police service data for including in media file conversion is convenient for the data file comprising character of processing.
In a kind of selectable embodiment, explanation can be further described in conjunction with Fig. 3 in step S110.
Step S120 extracts the data item that user pays close attention to from the police service data that acquisition library is issued, and library is extracted in deposit, concurrently
Cloth gives cleaning library.
Police service data in acquisition library are collected from multiple data sources, it is understood that there may be a variety of data formats, and may be used also
There can be the data that some users are not concerned with, be not suitable for directly progress data analysis and therefore in the step s 120, need basis
The business demand of user extracts the data item that user pays close attention to, and the data item that will be drawn into from the police service data that acquisition library is issued
Data value be stored in a unified format extract library corresponding data item in.Wherein, user can be subscribed to by addition, setting is matched
Set the data item that its concern is arranged in the modes such as file.
In a kind of selectable embodiment, explanation can be further described in conjunction with Fig. 4 in step S120.
Step S130 carries out data cleansing to the police service data for extracting library publication, the police service data after cleaning is stored in clear
Library is washed, and is distributed to fusion library.
Since the acquisition of police service data is from each department of police system, but be mostly by police in daily work
Accumulation obtains, therefore, can inevitably be generated in the typing of police service data accumulated over a long period, acquisition and keeping some redundancies,
Mistake and incomplete problem data, these problems data can be pooled in acquisition library, and further be drawn into extraction
In library.These problems data can interfere subsequent data analysis process, influence the accuracy of data analysis result, therefore, it is necessary to
It is removed by way of data cleansing.
As some selectable embodiments, the embodiment of the present application carries out data cleansing to the police service data extracted in library
It can specifically include:
1, by the association between police service data, completion is carried out to the data item of missing.
For example, extracting in " Community Police Affairs _ population information _ population information extracts data " in library comprising such as next data value
The data value of the data of missing, " Contact " data item in the data is sky:
| Name | Age | ID card | Contact |
| Open XX | 45 | XXXXXXXXXXXXXXXXXX |
For the field contents in completion " Contact ", it is possible, firstly, to find phase from other tables of data for extracting library
Associated data.For example, finding from " Community Police Affairs _ occupancy information _ occupancy information extracts data " such as next data value
For the associated data of " XX ":
| Name | Residential quarter | Number | Type of house | Contact method |
| Open XX | Harmonious home | 16-1-22-1 | It is own | 18XXXXXXXXX |
Then, data item " correspondent party corresponding with data item " contact " is found from this houseclearing data
Formula ", and the data value " 18XXXXXXXXX " of " contact method " is used to carry out completion to data item " contact ", such as:
| Name | Age | ID card | Contact |
| Open XX | 45 | XXXXXXXXXXXXXXXXXX | 18XXXXXXXXX |
In another example extracting in " Community Police Affairs _ population information _ population information extracts data " in library comprising in such as next
Hold the data of missing, the data value of the data item " Age " in the data is sky:
Since the data item " ID card " of the data has recorded the ID card No. of " horse XX ", it can be according to body
Date of birth information in part card number calculates age of " horse XX ", and completion is into " Age " data item, such as:
| Name | Age | ID card | Contact |
| Horse XX | 28 | XXXXXX19900501XXXX | 17XXXXXXXXX |
2, duplicate removal processing is carried out to duplicate police service data.
For example, obtaining the confidence level of each data value respectively, and only protect when a data item has multiple data values
The highest data value of indwelling reliability.Wherein, the confidence level of data value can be determined according to the source of data value, for example, to coming from
Higher confidence level is arranged in data value in public security household register system database, to the data value setting from community management department compared with
Low confidence level.Illustratively, the age of " the king XX " that acquires from Community Police Affairs platform is 45 years old, is acquired from happy administration commission of community
" king XX " age be 44 years old, then remove data value " 44 ", retain data value " 45 ".
3, according to preset judgment rule, legitimacy analysis is carried out to police service data, only retains and meets the judgment rule
Valid data value.
Illustratively, for data item such as " Age " " ID card " " Contact ", the judgment rule that can be set is to analyze it
Whether data value is pure digi-tal character string, and if it is pure digi-tal character string, then data value is legal, if data value includes nonnumeric
Character, then data value is illegal.
Illustratively, for data item " ID card ", can also make a concrete analysis of whether its data value is for the word of 18 bit digitals
Symbol string, if it is, data value is legal, if it is not, then data value is illegal.Further, number can also be made a concrete analysis of
Numerical value it is whether legal, for example, data value does not conform to if the first bit digital of the data value of data item " ID card " is 0
Method.
In addition, the embodiment of the present application to police service data carry out data cleansing after, can to the police service data after cleaning into
The classification of one step, and it is deposited into cleaning library, believe for example, police service data are divided into population essential information, houseclearing, personnel house
The classifications such as breath, personnel's certificate address information, personnel's contact method, personnel's family relationship, personnel's social relationships and personnel's unit information.So
Afterwards, data-interface is established to the police service data of each classification, gives police service data publication to fusion library by data-interface.
As a result, by carrying out data cleansing to the police service data extracted in library, problem data is washed away, and by the police after cleaning
In data of being engaged in deposit cleaning library, to keep the police service data cleaned in library accurately legal, it can be directly used for data and analyze and protect
Demonstrate,prove precision of analysis.
Step S140 carries out data analysis according to police service data of the subscription demand of user to the publication of cleaning library, and will divide
Analysis result data is deposited into fusion library.
Illustratively, merge the analysis result data that saves in library may include: population essential information _ floating population's information,
Houseclearing _ per capita living space information, personnel's houseclearing _ with other people shared house property informations, personnel's certificate address information _ are now lived
Location, personnel's contact method _ with phone, personnel's family relationship _ doubtful sham divorce, etc..
In a kind of selectable embodiment, explanation can be further described in conjunction with Fig. 5 in step S140.
Step S150 exports analysis result data from fusion library according to the business demand of police system.
For example, all analysis result datas are exported in full-text search library, freely retrieved for polices at different levels；It will stream
Dynamic demographic data, which exports, gives floating population's service management platform；Or it exports in other external systems.
In a kind of selectable embodiment, explanation can be further described in conjunction with Fig. 6 in step S150.
From the above technical scheme, the embodiment of the present application provides a kind of police service data processing method, firstly, by different
The police service tidal data recovering in source to acquisition library in, and be distributed to extract library, realize data collect and rough sort；Then, from adopting
The data item of user's concern is extracted in the police service data of Ji Ku publication, library is extracted in deposit, and is distributed to cleaning library, realizes that user is closed
Infuse the extraction of data；Then, data cleansing is carried out to the police service data for extracting library publication, the police service data after cleaning is stored in clear
Library is washed, and is distributed to fusion library, realizes purification and the disaggregated classification of data；Then, the police service data of cleaning library publication are counted
It is deposited into fusion library according to analysis, and by analysis result data, the data of each classification are analyzed in realization；Finally, according to police
The business demand of business system exports analysis result data from fusion library, realize by the application to analysis result data answer
It uses in the business of police system.Method provided by the embodiments of the present application as a result, can according to collecting police service data, and according to
The demand of user extracts the police service data of user's concern and classifies and analyze, thus it is versatile, it can satisfy police service people
Different demands of the member to police service data.
Fig. 3 is a kind of flow chart of police service data processing method steps S110 provided by the embodiments of the present application.Due to police service
Department may be very huge in the police service quantity of accumulation, in order to improve police service tidal data recovering to the speed and effect in acquisition library
Rate, step S110 may comprise steps of:
Step S111, to the police service data creation data queue of separate sources.
Specifically, police service data to be collected can come from multiple data sources, have plenty of with the data comprising a plurality of data
The form in library exists, and has plenty of and exists in the form of scrappy single data, and the application is ranked up these police service data, raw
At the data queue of police service data, consequently facilitating these police service data are distributed to multiple collecting threads, and pass through multiple acquisitions
Thread parallel is pooled in acquisition library.
Illustratively, data queue is as shown in table 1:
| Queue number | Data name | Data source | Data volume | Size of data |
| 1 | Population information | Community Police Affairs platform | 10000 | 200MB |
| 2 | Occupancy information | Community Police Affairs platform | 5000 | 150MB |
| 3 | Houseclearing | Housing management department | 3000 | 100MB |
| 4 | Occupancy information | Police's acquisition | 10 | 300KB |
| 5 | Population information | Police's acquisition | 2 | 40KB |
| …… | …… | …… | …… | …… |
The data queue of 1 police service data of table
Police service data in data queue are divided into multiple data sectionals according to preset chopping rule by step S112.
Specifically, police service data in data queue are divided into multiple data sectionals comprising preset M data.Example
Such as, the population information of sequence number 1 is divided into 10 segmentations, each segmentation includes 1000 datas.Such as sequence is compiled
For population information of numbers 4 number of data lower than 1000, the police service data of multiple sequence numbers can be gathered together enough into 1000 numbers
It is used as 1 data sectional according to rear, or is merged into other adjacent data sectionals.
Furthermore it is also possible to preset the size of data S of each segmentation, (that is: each segmentation is shared in physical storage medium
The byte number of amount of physical memory, for example, setting S=50MB), and the police service data in data queue are divided into multiple data
Size is the data sectional of S.For example, as S=50MB, the population information of sequence number 1 can be divided into 4 points in total
Section；It, can be by the police service number of multiple sequence numbers for the size of data of such as sequence number 4 is lower than 50MB population information
According to gathering together enough as 1 data sectional after 50MB data, or it is merged into other adjacent data sectionals.
The data sectional is distributed to multiple preset collecting threads by step S113, passes through the data sectional more
A collecting thread is pooled to parallel in the acquisition library.
Specifically, it is determined as the number of the data sectional of each collecting thread distribution according to the processing capacity of each collecting thread
Amount.For example, data sectional can be averagely allocated to each gathering line when the processing capacity of all collecting threads is all identical
Journey；When the processing capacity difference of each collecting thread, more data can be distributed for the higher collecting thread of processing capacity
Segmentation is that the lower collecting thread of processing capacity distributes less data segmentation, makes all collecting thread load balancing.
It illustratively, can be by the 1st to the when the quantity of collecting thread is 10, and the sum of data sectional is 1000
100 data segment assignments give the 1st thread, the 101st to the 200th data segment assignments are given to the 2nd thread, with such
It pushes away.To which each collecting thread is responsible for acquiring 100 data sectionals, when this 10 collecting thread parallel acquisitions, with single thread
Acquisition is compared, and acquisition speed can be improved 10 times, and when number of threads is more, acquisition speed, which can also access, further to be mentioned
It rises.
In the embodiment of the present application, to the police service data creation data queue of separate sources, then, by the police in data queue
Business data are divided into multiple data sectionals, finally, data sectional is distributed to multiple preset collecting threads, make the data point
Section is pooled in the acquisition library parallel by multiple collecting threads, to greatly improve the acquisition speed of police service data.Work as police
When data of being engaged in increase or update, due to acquiring the promotion of speed, the renewal speed for acquiring library can significantly be accelerated, and guarantee acquisition library
In police service data timeliness, reduce the quantity of stale data.
The police service data classification acquired in library is distributed to and extracts library by step S114.
Specifically, can be classified according to the source and content of police service data to police service data, such as: it will acquire from society
The population information of area's police service platform is named as " Community Police Affairs _ population information " as a classification；It will acquire from Community Police Affairs
The occupancy information of platform is named as " Community Police Affairs _ occupancy information " as a classification；Self-housing administrative department will be acquired
Houseclearing is named as " housing management _ houseclearing " as a classification.Then, the police service data of each classification are built
Vertical data-interface, and police service data are issued by data-interface.
Fig. 4 is a kind of flow chart of police service data processing method steps S120 provided by the embodiments of the present application.
As shown in figure 4, step S120 may comprise steps of in a kind of selectable embodiment:
Step S121, the data item of configuration user concern in extracting library.
Specifically, each user can be according to the classification of the police service data of acquisition library publication and the data need of combination itself
The data item of configuration oneself concern, and corresponding addition configuration file are asked, makes to extract library according to the configuration file of user, creation includes
The tables of data of user's focused data item.For example, when user needs to obtain the population information of " happy community " library can be being extracted
The population information tables of data of the happy community of middle creation one, and corresponding data item is configured in tables of data, such as shown in table 2:
| Name | Age | ID card | Contact |
Table 2 extracts the tables of data (example) in library
Wherein, name (name), the age (age), ID card No. of the Community Population of user's concern are contained in table 2
The data item such as (ID card) and contact method (contact).
Step S122 establishes from the data item for acquiring library and extracts the mapping between the data item in library.
Due to including the police service data of separate sources in acquisition library, data format also can be varied, such as have
It is structural data, has plenty of unstructured data；Also, can exist expressing the data item of same category information different
Name or expression-form, for example, some is named as phone number, and some is named as phone for this data item of contact method
Number, some are named as contact method.Therefore, in order to accurately extract the data item that user pays close attention to from acquisition library,
Need to establish the mapping between the data item for acquiring library and the data item in extraction library.
Illustratively, it acquires in library and has issued following data (data source 1):
The street XX population information
" name: XX, the age: 45 years old, affiliated community: happy community, ID card No.: XXXXXXXXXXXXXXXXXX,
Contact method: (sky) "
" name: Zhao X, the age: 34 years old, affiliated community: peaceful community, ID card No.: XXXXXXXXXXXXXXXXXX,
Contact method: 159XXXXXXXX "
" name: horse XX, the age: 28 years old, affiliated community: happy community, ID card No.: XXXXXXXXXXXXXXXXXX,
Contact method: 171XXXXXXXX "
" name: Liu XX, the age: 26 years old, affiliated community: happy community, ID card No.: XXXXXXXXXXXXXXXXXX,
Contact method: 137XXXXXXXX "
……
And also issue following data (data source 2):
| Name | Age | Gender | Building generic term for a building, e. g. Apartment, store, a movie theater, etc. | Marital status | Telephone number |
| Open XX | 45 | Male | 16 | It is married | 182XXXXXXXX |
| Pacify X | 26 | Female | 8 | It is unmarried | 136XXXXXXXX |
| Marquis XX | 35 | Male | 5 | It is married | 151XXXXXXXX |
The happy community's permanent population register information of table 3
So, establish from the data item of data source 1 and data source 2 to the mapping of the data item of table 2, so that it may include: by
" name " data item of data source 1 and " name " data item for extracting library (table 2), which are established, to be mapped, by " age " number of data source 1
Mapping is established according to item and " age " data item for extracting library (table 2), by " ID card No. " data item of data source 1 and extracts library
" ID card " data item of (table 2) establishes mapping, by " contact method " data item of data source 1 and extracts library (table 2)
" contact " data item establishes mapping, and " telephone number " data item of data source 2 and " contact " that extracts library (table 2) are counted
Mapping, etc. is established according to item.
Step S123 extracts the data item that user pays close attention to according to the mapping from the police service data that acquisition library is issued
Data value.
Illustratively, according to mapping relations, the people for living in " happy community " is extracted from the population information that acquisition library is issued
Name, age, ID card No. and the contact information of member.
For example, being extracted from data source 1:
Such as: it is extracted from data source 2:
The data value being drawn into is stored in the corresponding data item for extracting library by step S124.
Illustratively, it will be saved in and extracted in library from the data value that data source 1 and data source 2 extract, such as following table (table 4) institute
Show:
| Name | Age | ID card | Contact |
| Open XX | 45 | XXXXXXXXXXXXXXXXXX | 18XXXXXXXXX |
| Horse XX | 28 | XXXXXXXXXXXXXXXXXX | 17XXXXXXXXX |
| Liu XX | 26 | XXXXXXXXXXXXXXXXXX | 13XXXXXXXXX |
4 population information data of table
The police service data classification extracted in library is distributed to fusion library by step S125.
Specifically, can be classified using identical mode classification to police service data with acquisition library, such as: it will be from acquisition
The population information that library is extracted is named as " Community Police Affairs _ population information _ population information extracts data " as a classification；From
The occupancy information of library extraction is acquired as a classification, and is named as " Community Police Affairs _ occupancy information _ occupancy information extraction number
According to "；Using the houseclearing extracted from acquisition library as a classification, and it is named as " housing management _ houseclearing _ houseclearing
Extract data ".Then, data-interface is established to the police service data of each classification, and police service data is issued by data-interface.
As a result, in the embodiment of the present application, each user can extract oneself concern with the data requirements of itself from acquisition library
Data item, and adjust when the data requirements of itself changes the data item of oneself concern, thus, without for different
Data requirements designs different data pick-up schemes, thus, the versatility of the data processing scheme greatlyd improve.
Fig. 5 is a kind of flow chart of police service data processing method steps S140 provided by the embodiments of the present application.
As shown in figure 5, step S140 may comprise steps of in a kind of selectable embodiment:
Step S141, from police service data needed for the police service data pick-up data analysis that cleaning library is issued.
Specifically, police can subscribe to different analysis demands according to their business needs, then according to analysis need
It asks and extracts police service data from cleaning library.For example, the personnel address in cleaning library can be extracted in order to analyze the address of personnel
Information；In order to analyze the marital status of personnel, the family relationship information of personnel etc. in cleaning library can be extracted.
Step S142 carries out data analysis using police service data of the preset analyzer to extraction, generates analysis number of results
According to, and analysis result data is deposited into fusion library.
Wherein, analyzer is the software module that can complete data analysis task, for executing the data in data analysis
Collision, serial data simultaneously wait operation, and generate analysis result.Each user can configure multiple analyzers, and each analysis device is used for
Complete the data analysis task under a kind of business demand.
Fig. 6 is a kind of flow chart of police service data processing method steps S150 provided by the embodiments of the present application.
As shown in fig. 6, step S150 may comprise steps of in a kind of selectable embodiment:
Step S151, creates the data export interface from fusion library to each police system, and the data export interface is used
In the analysis result data is converted into specified data format according to business demand.
Step S152 exports the analysis result data by the data export interface.
In addition, in a kind of selectable embodiment, a kind of step of data processing method provided by the embodiments of the present application
Rapid S130 further includes the process that police service data are carried out with the reparation of first time data.
Fig. 7 is a kind of flow chart of the first time data reparation of police service data processing method provided by the embodiments of the present application.
As shown in fig. 7, first time data reparation may comprise steps of in a kind of selectable embodiment:
Police service data underproof after cleaning are stored in and clean abnormal library by step S201.
Specifically, after cleaning underproof police service data may include: can not during data cleansing completion missing
Data item, with multiple data values and can not duplicate removal data item (such as: having the confidence level of multiple data values identical), exist
The data item of illegal data value；Etc..
It is cleaned after abnormal library in addition, underproof police service data are stored in by the application, it can also be to the pipe of police service data
Reason personnel distribute work order, remind administrative staff to confirm data exception situation and repair abnormal data.
Step S202 carries out data reparation to the police service data cleaned in abnormal library.
Specifically, data reparation can be carried out by data correlation means and preset cleaning rule, alternatively, to management
After personnel distribute work order, data reparation is carried out by administrative staff.
Police service data after reparation are newly stored into the extraction library by step S203.
In addition, in a kind of selectable embodiment, a kind of step of data processing method provided by the embodiments of the present application
Rapid S140 further includes the process that police service data are carried out with second of data reparation.
Fig. 8 is a kind of flow chart of second of data reparation of police service data processing method provided by the embodiments of the present application.
As shown in figure 8, second of data reparation may comprise steps of in a kind of selectable embodiment:
Abnormal library is analyzed in the police service data deposit of step S301, the abnormal results data that analysis is generated.
Specifically, abnormal results data may include: it is obvious with it is general recognize be not inconsistent data (such as: per-capita housing face
Product is 500 square metres, address be there is no address or non-residential address), there are the data of multiple mutually exclusive outcomes (A and B
The shared house property of certain is F house property, and the shared house property of C and D are also F house property)；Etc..
In addition, abnormal results data are stored in after analyzing abnormal library by the application, it can also be to the administrator of police service data
Member distributes work order, reminds administrative staff to confirm data exception situation and repairs abnormal data.
Step S302 carries out data reparation to the police service data analyzed in abnormal library.
Specifically, data reparation can be carried out by preset reparation rule, alternatively, distributing work order to administrative staff
Afterwards, data reparation is carried out by administrative staff.
Police service data after reparation are newly stored into the cleaning library by step S303.
The application has carried out data reparation twice to police service data in data cleansing and data analysis process as a result, from
And the data volume of abnormal data is reduced, improve precision of analysis.
In addition, the application generates daily record data in real time in data processing, and will in a kind of selectable mode
Daily record data is stored in log library, thus, user can understand the process of data processing by the daily record data in log library, and
It is noted abnormalities in time reason when data processing is abnormal by daily record data.
Here is the Installation practice of the application, can be used for executing the present processes embodiment.This method can be applied
In the plurality of devices such as server, PC (PC), tablet computer, mobile phone, virtual reality device and intelligent wearable device.It is right
The undisclosed details in the application Installation practice, please refers to the application embodiment of the method.
Fig. 9 is a kind of schematic diagram of police service data processing equipment provided by the embodiments of the present application.
It is shown in Figure 9, the apparatus may include:
Acquisition module 410, for by the police service tidal data recovering of separate sources to acquisition library in, and be distributed to extract library；
Abstraction module 420, the data item paid close attention to for extracting user from the police service data that acquisition library is issued, deposit are extracted
Library, and it is distributed to cleaning library；
Cleaning module 430, for carrying out data cleansing to the police service data for extracting library publication, by the police service data after cleaning
Deposit cleaning library, and it is distributed to fusion library；
Analysis module 440 for carrying out data analysis to the police service data of cleaning library publication, and analysis result data is deposited
Enter to fusion library；
Export module 450 exports analysis result data from fusion library for the business demand according to police system.
From the above technical scheme, the embodiment of the present application provides a kind of police service data processing equipment, and being used for will be different
The police service tidal data recovering in source to acquisition library in, and be distributed to extract library, realize data collect and rough sort；Then, from adopting
The data item of user's concern is extracted in the police service data of Ji Ku publication, library is extracted in deposit, and is distributed to cleaning library, realizes that user is closed
Infuse the extraction of data；Then, data cleansing is carried out to the police service data for extracting library publication, the police service data after cleaning is stored in clear
Library is washed, and is distributed to fusion library, realizes purification and the disaggregated classification of data；Then, the police service data of cleaning library publication are counted
It is deposited into fusion library according to analysis, and by analysis result data, the data of each classification are analyzed in realization；Finally, according to police
The business demand of business system exports analysis result data from fusion library, realize by the application to analysis result data answer
It uses in the business of police system.Method provided by the embodiments of the present application as a result, can according to collecting police service data, and according to
The demand of user extracts the police service data of user's concern and classifies and analyze, thus it is versatile, it can satisfy police service people
Different demands of the member to police service data.
The application can be used in numerous general or special purpose computing system environments or configuration.Such as: personal computer, service
Device computer, handheld device or portable device, laptop device, multicomputer system, microprocessor-based system, top set
Box, programmable consumer-elcetronics devices, network PC, minicomputer, mainframe computer, including any of the above system or equipment
Distributed computing environment etc..
The application can describe in the general context of computer-executable instructions executed by a computer, such as program
Module.Generally, program module includes routines performing specific tasks or implementing specific abstract data types, programs, objects, group
Part, data structure etc..The application can also be practiced in a distributed computing environment, in these distributed computing environments, by
Task is executed by the connected remote processing devices of communication network.In a distributed computing environment, program module can be with
In the local and remote computer storage media including storage equipment.
It should be noted that, in this document, the relational terms of such as " first " and " second " or the like are used merely to one
A entity or operation with another entity or operate distinguish, without necessarily requiring or implying these entities or operation it
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant are intended to
Cover non-exclusive inclusion, so that the process, method, article or equipment for including a series of elements not only includes those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or setting
Standby intrinsic element.
Those skilled in the art will readily occur to its of the application after considering specification and practicing application disclosed herein
Its embodiment.This application is intended to cover any variations, uses, or adaptations of the application, these modifications, purposes or
Person's adaptive change follows the general principle of the application and including the undocumented common knowledge in the art of the application
Or conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the application are by following
Claim is pointed out.
It should be understood that the application is not limited to the precise structure that has been described above and shown in the drawings, and
And various modifications and changes may be made without departing from the scope thereof.Scope of the present application is only limited by the accompanying claims.
Claims (10)
1. a kind of police service data processing method, which is characterized in that the described method includes:
By the police service tidal data recovering of separate sources into acquisition library, and it is distributed to and extracts library；
The data item that user pays close attention to is extracted from the police service data that acquisition library is issued, library is extracted in deposit, and is distributed to cleaning library；
Data cleansing is carried out to the police service data for extracting library publication, by the police service data deposit cleaning library after cleaning, and is distributed to
Merge library；
Data analysis is carried out according to police service data of the subscription demand of user to the publication of cleaning library, and analysis result data is stored in
To fusion library；
According to the business demand of police system, analysis result data is exported from fusion library.
2. the method according to claim 1, wherein according to the subscription demand of user to the police service of cleaning library publication
Data carry out data analysis, and analysis result data is deposited into fusion library, comprising:
According to the subscription demand of user, from police service data needed for the police service data pick-up data analysis that cleaning library is issued；
Data analysis is carried out using police service data of the presupposition analysis device to extraction, generates analysis result data, and result will be analyzed
Data are deposited into fusion library.
3. the method according to claim 1, wherein the police service tidal data recovering by separate sources to acquisition library
In, and be distributed to and extract library, comprising:
To the police service data creation data queue of separate sources；
According to preset chopping rule, the police service data in the data queue are divided into multiple data sectionals；
The data sectional is distributed into multiple preset collecting threads, the data sectional is made to pass through multiple collecting threads
It is pooled in the acquisition library parallel；
The police service data classification acquired in library is distributed to and extracts library.
4. the method according to claim 1, wherein described extract user from the police service data that acquisition library is issued
Library is extracted in the data item of concern, deposit, and is distributed to cleaning library, comprising:
The data item of configuration user concern in extracting library；
Establish from the data item for acquiring library and extract the mapping between the data item in library；
According to the mapping, the data value of user's focused data item is extracted from the police service data that acquisition library is issued；
The data value being drawn into is stored in the corresponding data item for extracting library；
The police service data classification extracted in library is distributed to fusion library.
5. the method according to claim 1, wherein the data cleansing, comprising:
By the association between police service data, completion is carried out to the data item of missing；And
Duplicate removal processing is carried out to duplicate police service data；And
According to preset judgment rule, legitimacy analysis is carried out to police service data, only retains and meets the legal of the judgment rule
Data value.
6. the method according to claim 1, wherein the business demand according to police system, from fusion library
Middle export analysis result data, comprising:
The data export interface from fusion library to each police system is created, the data export interface is used for according to business demand
The analysis result data is converted into specified data format；
The analysis result data is exported by the data export interface.
7. method according to claim 1 or 5, which is characterized in that the described pair of police service data for extracting library publication count
According to cleaning, further includes:
Abnormal library is cleaned in underproof police service data deposit after cleaning；
Data reparation is carried out to the police service data cleaned in abnormal library；
Police service data after reparation are newly stored into the extraction library.
8. method according to claim 1 or 2, which is characterized in that the police service data of described pair of cleaning library publication count
According to analysis, further includes:
Abnormal library is analyzed in the police service data deposit for the abnormal results data that analysis is generated；
Data reparation is carried out to the police service data analyzed in abnormal library；
Police service data after reparation are newly stored into the cleaning library.
9. the method according to claim 1, wherein further include:
The daily record data generated in record data handling procedure, the daily record data is stored in log library.
10. a kind of police service data processing equipment, which is characterized in that described device includes:
Acquisition module, for by the police service tidal data recovering of separate sources to acquisition library in, and be distributed to extract library；
Library is extracted in abstraction module, the data item paid close attention to for extracting user from the police service data that acquisition library is issued, deposit, concurrently
Cloth gives cleaning library；
Police service data after cleaning are stored in clear by cleaning module for carrying out data cleansing to the police service data for extracting library publication
Library is washed, and is distributed to fusion library；
Analysis module for carrying out data analysis to the police service data of cleaning library publication, and analysis result data is deposited into and is melted
Close library；
Export module exports analysis result data from fusion library for the business demand according to police system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811149077.XA CN109344190A (en) | 2018-09-29 | 2018-09-29 | A kind of police service data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811149077.XA CN109344190A (en) | 2018-09-29 | 2018-09-29 | A kind of police service data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109344190A true CN109344190A (en) | 2019-02-15 |
Family
ID=65307969
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811149077.XA Pending CN109344190A (en) | 2018-09-29 | 2018-09-29 | A kind of police service data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109344190A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111212051A (en) * | 2019-12-27 | 2020-05-29 | 柏科数据技术（深圳）股份有限公司 | Narrow-band transmission system and application method thereof |
| CN111582693A (en) * | 2020-04-28 | 2020-08-25 | 重庆中科云从科技有限公司 | Population management method, system, machine readable medium and device |
| CN112000710A (en) * | 2020-07-20 | 2020-11-27 | 华动泰越科技有限责任公司 | Social security control system and method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020107871A1 (en) * | 2001-02-05 | 2002-08-08 | Knowledge Computing Corporation | Method and system for database migration and association |
| CN102495885A (en) * | 2011-12-08 | 2012-06-13 | 中国信息安全测评中心 | Method for integrating information safety data based on base-networking engine |
| CN105554070A (en) * | 2015-12-09 | 2016-05-04 | 北京中科云集科技有限公司 | Method based on police affair big data center service construction |
| CN105574657A (en) * | 2015-12-11 | 2016-05-11 | 山东新网格信息技术有限公司 | Intelligent police actual combat comprehensive application platform |
| CN107506428A (en) * | 2017-08-19 | 2017-12-22 | 合肥智贤智能化科技有限公司 | A kind of police service data handling utility system |
-
2018
- 2018-09-29 CN CN201811149077.XA patent/CN109344190A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020107871A1 (en) * | 2001-02-05 | 2002-08-08 | Knowledge Computing Corporation | Method and system for database migration and association |
| CN102495885A (en) * | 2011-12-08 | 2012-06-13 | 中国信息安全测评中心 | Method for integrating information safety data based on base-networking engine |
| CN105554070A (en) * | 2015-12-09 | 2016-05-04 | 北京中科云集科技有限公司 | Method based on police affair big data center service construction |
| CN105574657A (en) * | 2015-12-11 | 2016-05-11 | 山东新网格信息技术有限公司 | Intelligent police actual combat comprehensive application platform |
| CN107506428A (en) * | 2017-08-19 | 2017-12-22 | 合肥智贤智能化科技有限公司 | A kind of police service data handling utility system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111212051A (en) * | 2019-12-27 | 2020-05-29 | 柏科数据技术（深圳）股份有限公司 | Narrow-band transmission system and application method thereof |
| CN111582693A (en) * | 2020-04-28 | 2020-08-25 | 重庆中科云从科技有限公司 | Population management method, system, machine readable medium and device |
| CN112000710A (en) * | 2020-07-20 | 2020-11-27 | 华动泰越科技有限责任公司 | Social security control system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2022204452B2 (en) | Verification of electronic identity components | |
| CN103294592A (en) | Leveraging user-to-tool interactions to automatically analyze defects in it services delivery | |
| US8255392B2 (en) | Real time data collection system and method | |
| CN111612038B (en) | Abnormal user detection method and device, storage medium and electronic equipment | |
| CN109344190A (en) | A kind of police service data processing method and device | |
| WO2021159834A1 (en) | Abnormal information processing node analysis method and apparatus, medium and electronic device | |
| CN112311612B (en) | Information construction method and device and storage medium | |
| KR101982756B1 (en) | System and Method for processing complex stream data using distributed in-memory | |
| CN108776857A (en) | NPS short messages method of investigation and study, system, computer equipment and storage medium | |
| CN106022640B (en) | Electric quantity index checking system and method | |
| CN112925899B (en) | Ordering model establishment method, case clue recommendation method, device and medium | |
| Li et al. | Vandalism detection in OpenStreetMap via user embeddings | |
| CN116028467A (en) | Intelligent service big data modeling method, system, storage medium and computer equipment | |
| CN113538011B (en) | Method for associating non-booked contact information with booked user in electric power system | |
| CN115423361A (en) | Data processing method and device for risk view, storage medium and equipment | |
| CN115907649A (en) | Gridding government affair data management system | |
| CN115062725A (en) | Hotel income abnormity analysis method and system | |
| CN115829073A (en) | Object recognition method, device, equipment, readable storage medium and program product | |
| CN1588405A (en) | Data processing device and method for risk control system | |
| CN115147117A (en) | Method, device and equipment for identifying account group with abnormal resource use | |
| CN111858598A (en) | Mass data comprehensive management system and method | |
| Koteshwariah et al. | A fuzzy version of generalized DBSCAN clustering algorithm | |
| CN104331481A (en) | Method for acquiring business model data and physical model data relation based on large-scale data collision | |
| CN111368550A (en) | Public opinion information management system | |
| KR102470731B1 (en) | Method and apparatus for providing dataset using ontology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190905 Address after: Room 630, 6th floor, Block A, Wanliu Xingui Building, 28 Wanquanzhuang Road, Haidian District, Beijing Applicant after: DINFO (BEIJING) SCIENCE DEVELOPMENT Co.,Ltd. Address before: Room 601, Block A, Wanliu Xingui Building, 28 Wanquanzhuang Road, Haidian District, Beijing Applicant before: ULTRAPOWER SOFTWARE Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 230000 zone B, 19th floor, building A1, 3333 Xiyou Road, hi tech Zone, Hefei City, Anhui Province Applicant after: Dingfu Intelligent Technology Co.,Ltd. Address before: Room 630, 6th floor, Block A, Wanliu Xingui Building, 28 Wanquanzhuang Road, Haidian District, Beijing Applicant before: DINFO (BEIJING) SCIENCE DEVELOPMENT Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190215 |