CN111324646A - Policy information importing and analyzing method based on dynamic mapping - Google Patents
Policy information importing and analyzing method based on dynamic mapping Download PDFInfo
- Publication number
- CN111324646A CN111324646A CN202010000226.7A CN202010000226A CN111324646A CN 111324646 A CN111324646 A CN 111324646A CN 202010000226 A CN202010000226 A CN 202010000226A CN 111324646 A CN111324646 A CN 111324646A
- Authority
- CN
- China
- Prior art keywords
- data
- file
- fields
- uploading
- template
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013507 mapping Methods 0.000 title claims abstract description 18
- 238000000034 method Methods 0.000 title claims description 15
- 238000004458 analytical method Methods 0.000 claims abstract description 12
- 238000005516 engineering process Methods 0.000 claims abstract description 7
- 238000003672 processing method Methods 0.000 claims abstract description 6
- 230000000007 visual effect Effects 0.000 claims description 7
- 238000007405 data analysis Methods 0.000 claims description 3
- 238000007418 data mining Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 abstract description 2
- 238000012795 verification Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 3
- 238000005065 mining Methods 0.000 description 2
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a policy information data processing method, which comprises the following steps: creating a dynamic mapping template; step two: uploading data by using a data uploading module; step three: the data classification module is used for classifying the data, the data classification module can classify and arrange available data according to requirements, the technology can set different dynamic mapping templates according to policy data from different sources, the templates can be modified to meet the change of the requirements, the effectiveness verification and analysis are carried out on the input data, and the incorrect policy data are arranged to generate an error data file to be fed back to a user.
Description
Technical Field
The invention relates to the field of computer data processing, in particular to a method for importing and analyzing policy information based on dynamic mapping.
Background
With the rapid advance of modern economy, people have more and more contact to insurance, and the demand is more and more urgent. For the insurance brokerage company, different insurance company products are acted, even between different organizations and groups of the company, insurance data in various formats are gathered together, and the insurance data have respective description names and need a large amount of manual combination, arrangement and analysis. A complete solution for importing and parsing policy information with different names becomes more urgent. Excel can make a report, use a formula and a function to perform complex operation, and use a powerful function of visually and clearly representing data by an icon, so that repeated actions can be solved, the efficiency is improved, the time is saved, and more people can trust. With the advent of the big data era, more and more insurance agency companies pay attention to the importance of policy data, and potential customers and values of mining hiding, and then the policy data needs to be put into a container in batches for sorting and analyzing. However, the sources of the policy data are inconsistent, the description names are inconsistent, and the Excel tool used at ordinary times cannot flexibly identify different characters to form the same data field, and still needs a large amount of manual identification. The time efficiency and the service quality are the targets of pursuit and survival of modern enterprises, and the aim of reasonably utilizing an Excel tool to improve the working efficiency is deeply pursued.
Disclosure of Invention
In order to solve the problems in the prior art and achieve the above object, the present invention provides a method for importing and analyzing policy information based on dynamic mapping.
The technical scheme adopted by the invention is as follows: a policy information data processing method, the method comprising:
the method comprises the following steps: creating a dynamic mapping template; step two: uploading data by using a data uploading module; step three: the data classification module is used for classifying the data, and the data which can be utilized needs to be classified and sorted out according to requirements.
The method comprises the steps of establishing a dynamic mapping template, uploading an excel template file, extracting fields in the template by a system, displaying the fields on a page, acquiring all the fields in the system by the page, freely selecting the system fields and the template fields by a user to establish a corresponding relation, and if the required system fields do not exist, selectively adding the system fields and setting field attributes.
The preferred technical scheme comprises the following steps: the data uploading module is used for uploading data, the service data file is uploaded, the system can check whether the file is uploaded according to md5, interference of repeated data is avoided, the uploading is successful, the source file is stored in the cloud server, the database stores the current file batch information, then the system uses the thread pool technology to asynchronously analyze the file, and visual file analysis progress display is provided.
In the third step of the preferred technical scheme, the system provides a function of classification according to field rules. The user selects the classified fields, the classification rules are set, the system can extract data according to the classification of different dimensions, and reports and charts of various dimensions are generated for deep data analysis and mining. The optimized technical scheme corresponds to the characteristics of field configuration, checks the validity of data line by line, stores complete data into a database, arranges and generates error files according to policy records which do not meet conditions, returns the error files to a user, distributes different threads according to the number of records in average for analysis, considers that all threads of a system can be consumed when the records of the files are overlarge, does not distribute more threads after the number of records exceeds a set limit, and sequentially executes the remaining data by the original distributed threads, thus the invention has the advantages that: the technology can set different dynamic mapping templates according to policy data from different sources, can modify the templates to meet the change of requirements, carry out validity verification and analysis on the input data, and generate an error data file for incorrect policy data sorting to feed back to a user, thereby meeting the requirements of the existing insurance agency company on policy data batch sorting and deep mining analysis. The technology reasonably utilizes the characteristics of Excel, can provide various variable format dynamic mapping templates for different insurance data sources, efficiently solves the data management problem of each insurance agent company, reduces the manual maintenance cost and simultaneously improves the company service quality.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein: fig. 1 is a flowchart of a method for importing and parsing policy information based on dynamic mapping, fig. 2 is a dynamic mapping template interface diagram, fig. 3 is a visual file parsing progress presentation diagram interface, fig. 4 is a visual file parsing progress presentation diagram interface, fig. 5 is a thread parsing program language structure diagram, and fig. 6 is a service distribution interface diagram.
Detailed Description
Example embodiments will now be described more fully with reference to fig. 1, which is a flow diagram of a method for importing and parsing policy information based on dynamic mapping. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations or operations have not been shown or described in detail to avoid obscuring aspects of the invention. The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means. A method for importing and analyzing policy information based on dynamic mapping comprises the first step of creating a dynamic mapping template, and if a user creates the template before, the user can skip the template and enter an uploading template file module. The contents of this module are as follows: and uploading a template file (excel), and extracting fields in the template by the system and displaying the fields on a page. And simultaneously, the page acquires all the existing fields in the system. And the user freely selects the system field and the template field to establish the corresponding relation. If the required system field does not exist, the system field can be added optionally, and the field attribute (whether the field data can be modified, whether the field data is displayed or not, whether the field data needs to be filled, the data format and the data repetition check rule are set). The field attribute in the database table holds all the special attributes of the field. The dynamic mapping template interface diagram is shown in fig. 2.
And step two, a data uploading module selects an organization and a department, the template created in the step one is used for uploading the service data file, and the system can check whether the file is uploaded according to md5 so as to avoid the interference of repeated data. Uploading is successful, the source file is saved in the cloud server, the database saves the current file (batch) information, then the system utilizes the thread pool technology to asynchronously analyze the file, and a visual file analysis progress display is provided, for example, as shown in fig. 3, a visual file analysis progress display chart interface is provided.
And (3) checking the validity of the data line by line according to the characteristics of the corresponding field configuration in the step one, storing the complete data in a database, sorting the policy records which do not meet the conditions to generate an error file, and returning the error file to the user, such as a visual file analysis progress display chart interface shown in fig. 4.
In order to analyze files more quickly, each file is averagely allocated to different threads for analysis according to the number of records, all threads of a system are probably consumed when the file records are overlarge, the threads are not allocated any more after the number of records exceeds a set limit, and the original allocated threads sequentially execute the rest data. FIG. 5 is a diagram of a thread parser language architecture. And if the uploading fails, returning an error prompt to the user. And step three, a data classification module, namely, the policy data input in the step two are relatively chaotic, and the usable data need to be classified and sorted out according to requirements. The present system provides functionality for classification according to field rules. The user selects the classified fields, the classification rules are set, the system can extract data according to the classification of different dimensions, and reports and charts of various dimensions are generated for deep data analysis and mining. Fig. 6 is a service distribution interface diagram showing a service distribution diagram of the insurance agency nationwide and a service distribution situation of a certain province: the technology is based on a micro service architecture (springboot/closed), is issued by jenkins, is deployed in a docker container, and is easy to develop and maintain in the later period. The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation. With respect to the problems of the prior art, it is to be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (5)
1. A policy information data processing method, the method comprising: the method comprises the following steps: creating a dynamic mapping template; step two: uploading data by using a data uploading module; step three: the data classification module is used for classifying the data, and the data which can be utilized needs to be classified and sorted out according to requirements.
2. The policy information data processing method according to claim 1, wherein: creating a dynamic mapping template, uploading an excel template file, extracting fields in the template by a system, displaying the fields on a page, simultaneously acquiring all the existing fields in the system by the page, freely selecting the system fields and the template fields by a user to establish a corresponding relation, and if the required system fields do not exist, selectively adding the system fields and setting field attributes.
3. The policy information data processing method according to claim 1, wherein: the second step is as follows: the data uploading module is used for uploading data, the service data file is uploaded, the system can check whether the file is uploaded according to md5, interference of repeated data is avoided, the uploading is successful, the source file is stored in the cloud server, the database stores the current file batch information, then the system uses the thread pool technology to asynchronously analyze the file, and visual file analysis progress display is provided.
4. The policy information data processing method according to claim 1, wherein: and step three, the user selects classified fields, sets classification rules, and the system can extract data according to the classification of different dimensions and generate reports and charts of various dimensions for deep data analysis and mining.
5. The method according to claim 2, wherein the validity of the data is checked row by row corresponding to the field attribute, the complete data is stored in the database, the policy records that do not meet the condition are sorted to generate error files, the error files are returned to the user, and each file is averagely distributed to different threads for analysis according to the number of records.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010000226.7A CN111324646A (en) | 2020-01-02 | 2020-01-02 | Policy information importing and analyzing method based on dynamic mapping |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010000226.7A CN111324646A (en) | 2020-01-02 | 2020-01-02 | Policy information importing and analyzing method based on dynamic mapping |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111324646A true CN111324646A (en) | 2020-06-23 |
Family
ID=71170907
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010000226.7A Pending CN111324646A (en) | 2020-01-02 | 2020-01-02 | Policy information importing and analyzing method based on dynamic mapping |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111324646A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115496156A (en) * | 2022-09-29 | 2022-12-20 | 成都安世亚太科技有限公司 | Extensible test data batch analysis and structured management method based on template |
-
2020
- 2020-01-02 CN CN202010000226.7A patent/CN111324646A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115496156A (en) * | 2022-09-29 | 2022-12-20 | 成都安世亚太科技有限公司 | Extensible test data batch analysis and structured management method based on template |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109359028B (en) | Code quality monitoring method, device, computer equipment and storage medium | |
| CN110990053A (en) | Method for creating and using machine learning scheme template and device | |
| CN111126019B (en) | Report generation method and device based on mode customization and electronic equipment | |
| CN110543303A (en) | Visual business platform | |
| CN108733635B (en) | Text information display method and device | |
| CN108121579B (en) | Method and device for modifying rule file | |
| CN116226159A (en) | Metadata blood-edge relationship analysis method, system, equipment and storage medium | |
| CN111090976A (en) | Method and device for generating inspection document based on dynamic data | |
| CN111324646A (en) | Policy information importing and analyzing method based on dynamic mapping | |
| CN110287459B (en) | Form batch processing method, system, device and storage medium | |
| CN114722789B (en) | Data report integrating method, device, electronic equipment and storage medium | |
| CN115510361A (en) | Data chart page generation method based on visual configuration and related equipment | |
| CN112700093B (en) | Project progress Gantt chart construction method, terminal device and storage medium | |
| CN114238724A (en) | Data visualization method and device | |
| CN113705184A (en) | Method and device for generating custom report, storage medium and electronic equipment | |
| US20170060647A1 (en) | Event-based data management method and device | |
| CN114596044A (en) | Tool and method for project process approval | |
| JP6496078B2 (en) | Analysis support device, analysis support method, and analysis support program | |
| CN116931889B (en) | Software modeling method and system based on object tree | |
| CN116501318B (en) | Form creation method, form creation device, electronic equipment and computer readable storage medium | |
| Palo | Requirements Traceability | |
| CN110019109B (en) | Method and apparatus for processing data warehouse data | |
| Swidan et al. | Semi-automatic extraction of cross-table data from a set of spreadsheets | |
| CN117874379A (en) | Multi-report data verification method and device based on SpreadJS technology | |
| CN117371880A (en) | Service data analysis method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200623 |