CN116401323A - Index data processing method and device and computer equipment - Google Patents
Index data processing method and device and computer equipment Download PDFInfo
- Publication number
- CN116401323A CN116401323A CN202310339638.7A CN202310339638A CN116401323A CN 116401323 A CN116401323 A CN 116401323A CN 202310339638 A CN202310339638 A CN 202310339638A CN 116401323 A CN116401323 A CN 116401323A
- Authority
- CN
- China
- Prior art keywords
- task
- index
- data
- index data
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 90
- 238000000034 method Methods 0.000 claims abstract description 55
- 238000013506 data mapping Methods 0.000 claims abstract description 35
- 238000013515 script Methods 0.000 claims abstract description 27
- 238000004364 calculation method Methods 0.000 claims description 51
- 238000013507 mapping Methods 0.000 claims description 40
- 238000004590 computer program Methods 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 12
- 238000013500 data storage Methods 0.000 claims description 7
- 230000000694 effects Effects 0.000 claims description 4
- 230000000875 corresponding effect Effects 0.000 description 27
- 238000010586 diagram Methods 0.000 description 11
- 238000007726 management method Methods 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 7
- 238000011161 development Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000008676 import Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000012550 audit Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
- G06F40/18—Editing, e.g. inserting or deleting of tables; using ruled lines of spreadsheets
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/186—Templates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Strategic Management (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Software Systems (AREA)
- Game Theory and Decision Science (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure relates to the technical field of data processing, and particularly discloses an index data processing method, an index data processing device and computer equipment, wherein the method comprises the following steps: acquiring original index data, and processing the original index data by executing a pre-configured database script to obtain processed index data; writing the processed index data into an index database; responding to an index task processing instruction, and acquiring a task electronic table template corresponding to a task object associated with the index task; according to the task object and the data mapping relation, reading index data corresponding to the index task from the index database, writing the read index data into the task electronic table template, and generating a task electronic table instance; and executing task execution logic pre-configured in the task electronic table instance to acquire a task execution result. The method and the device improve the index data reading and writing efficiency and meet the processing requirements of personalized index tasks.
Description
Technical Field
The disclosure relates to the technical field of data processing, and in particular relates to an index data processing method, an index data processing device and computer equipment.
Background
With the rapid development of global economy and science technology, the development of information technology and internet technology is rapid, and the development of information technology and internet technology promotes the data to the era of big data. Wherein, data processing is throughout various fields of social production and social life. The development of data processing technology and the breadth and depth of application thereof greatly influence the progress of human society development, and particularly the application of the data processing technology in a multi-dimensional processing scene for data according to different data characteristics (such as indexes).
In the related art, when large-scale data is processed according to data indexes and the index data is further processed, various indexes are often subjected to different statistical modes; in addition, after the index data is obtained, the index data processing modes are different due to different sources and characteristics of the index data, so that the processing efficiency is low, and the method is difficult to adapt to large-scale and multi-characteristic index data processing scenes.
Disclosure of Invention
In view of the foregoing, it is desirable to provide an index data processing method, apparatus, computer device, storage medium, and computer program product.
In a first aspect, the present disclosure provides an index data processing method. The method comprises the following steps:
Acquiring original index data, and processing the original index data by executing a pre-configured database script to obtain processed index data;
writing the processed index data into an index database;
responding to an index task processing instruction, and acquiring a task electronic table template corresponding to a task object associated with the index task;
according to the task object and the data mapping relation, reading index data corresponding to the index task from the index database, writing the read index data into the task electronic table template, and generating a task electronic table instance; the data mapping relation comprises a data field and a data storage position relation between the pre-configured task electronic table template and the index database;
and executing task execution logic pre-configured in the task electronic table instance to acquire a task execution result.
In one embodiment, the acquiring the original index data performs processing on the original index data by executing a pre-configured database script to obtain processed index data:
subscribing original index data associated with an index task, and storing the subscribed original index data into a processing database; the processing database is provided with the database script;
Traversing and executing the database script to generate index data; the processed index data comprises index data records taking task object numbers and task time as keywords;
the index data record is inserted into the index database.
In one embodiment, the method further comprises:
acquiring association configuration information of the task electronic table template and the task object, and establishing a corresponding relation between the task electronic table template and the task object according to the association configuration information;
and acquiring mapping configuration information of the task electronic table template and the index database, and establishing a data mapping relation of the task electronic table template and the index database according to the mapping configuration information.
In one embodiment, the mapping configuration information includes a plurality of mapping relationship records, and the mapping relationship records at least include page names and field names of the index data in the task electronic table template, and data table names and field names of the index data in the index database.
In one embodiment, the method further comprises:
receiving a task electronic form template or an electronic form update file of a task electronic form instance;
Under the condition that the electronic form update file meets the authority rule, the electronic form update file takes effect and generates a flow record; the flow record includes a copy file of the spreadsheet update file.
In one embodiment, the executing the task execution logic configured in advance in the task electronic table instance, and the obtaining the task execution result includes:
extracting the first line data of the task electronic form instance, and analyzing and obtaining a calculation rule template according to the first line data;
according to the calculation rule template, a calculation formula is adaptively written in the task electronic form example according to a line number;
calculating the index data in the task electronic form example according to the calculation formula to obtain the task execution result;
and filling the task execution result into the task electronic table instance to obtain a task result file.
In one embodiment, the method further comprises:
and reading a task execution result in the task result file, and storing the task execution result into the index database according to the data mapping relation.
In a second aspect, the present disclosure further provides an index data processing apparatus. The device comprises:
The data processing module is used for acquiring original index data, and processing the original index data by executing a pre-configured database script to obtain processed index data;
the index database module is used for writing the processed index data into an index database;
the task instruction module is used for responding to the index task processing instruction and acquiring a task electronic table template corresponding to a task object associated with the index task;
the task electronic form module is used for reading the index data corresponding to the index task from the index database according to the task object and the data mapping relation, writing the read index data into the task electronic form template and generating a task electronic form instance; the data mapping relation comprises a data field and a data storage position relation between the pre-configured task electronic table template and the index database;
and the task result module is used for executing the task execution logic pre-configured in the task electronic table instance and obtaining a task execution result.
In one embodiment, the data processing module includes:
The subscription unit is used for subscribing the original index data associated with the index task and storing the subscribed original index data into the processing database; the processing database is provided with the database script;
the script execution unit is used for traversing and executing the database script to generate index data; the processed index data comprises index data records taking task object numbers and task time as keywords;
and the record inserting unit is used for inserting the index data record into the index database.
In one embodiment, the apparatus further comprises:
the association configuration module is used for acquiring association configuration information of the task electronic form template and the task object and establishing a corresponding relation between the task electronic form template and the task object according to the association configuration information;
and the mapping configuration module is used for acquiring the mapping configuration information of the task electronic table template and the index database, and establishing a data mapping relation between the task electronic table template and the index database according to the mapping configuration information.
In one embodiment, the mapping configuration information includes a plurality of mapping relationship records, and the mapping relationship records at least include page names and field names of the index data in the task electronic table template, and data table names and field names of the index data in the index database.
In one embodiment, the apparatus further comprises:
the update file module is used for receiving a task electronic form template or an electronic form update file of a task electronic form instance;
the update validation module is used for validating the electronic form update file and generating a flow record under the condition that the electronic form update file meets the permission rule; the flow record includes a copy file of the spreadsheet update file.
In one embodiment, the task result module includes:
the computing and extracting unit is used for extracting the first line data of the task electronic form instance and analyzing and obtaining a computing rule template according to the first line data;
the formula writing unit is used for adaptively writing a calculation formula in the task electronic table example according to the calculation rule template and the row number;
the calculation execution unit is used for carrying out calculation processing on the index data in the task electronic table instance according to the calculation formula to obtain the task execution result;
and the result writing unit is used for filling the task execution result into the task electronic table instance to obtain a task result file.
In one embodiment, the index database module is further configured to read a task execution result in the task result file, and store the task execution result to the index database according to the data mapping relationship.
In a third aspect, the present disclosure also provides a computer device. The computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the steps of the index data processing method when executing the computer program.
In a fourth aspect, the present disclosure also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the index data processing method described above.
In a fifth aspect, the present disclosure also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of the index data processing method described above.
The index data processing method, the index data processing device, the computer equipment, the storage medium and the computer program product at least comprise the following beneficial effects:
according to the method, on one hand, the obtained original index data are processed to obtain the index data, the index data are stored in the index database, on the other hand, the task electronic table template corresponding to the task object is determined according to the task object of the index task by responding to the index task processing instruction, and further, the index data required by the index task can be accurately and rapidly written into the task electronic table template from the index database according to the data mapping relation, the task electronic table template executes task execution logic aiming at the task object, and finally, the task execution result is obtained, so that the reading and writing efficiency of the index data is greatly improved, the processing process of the index task is standardized by executing the index task more intuitively and transversely through the task electronic table template, and the processing efficiency is improved; meanwhile, the task spreadsheet template corresponds to the task object, so that the processing requirement of the personalized index task is met, and the processing accuracy and flexibility are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments or the conventional techniques of the present disclosure, the drawings required for the descriptions of the embodiments or the conventional techniques will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present disclosure, and other drawings may be obtained according to the drawings without inventive effort to those of ordinary skill in the art.
FIG. 1 is a diagram of an application environment for a method of processing index data in one embodiment;
FIG. 2 is a diagram of an application environment for the step of obtaining index data in one embodiment;
FIG. 3 is an application environment diagram showing the steps of obtaining a task execution result in one embodiment;
FIG. 4 is a flow chart of a method of processing index data in one embodiment;
FIG. 5 is a flowchart illustrating a step of acquiring index data according to one embodiment;
FIG. 6 is a flowchart illustrating steps for obtaining configuration information in one embodiment;
FIG. 7 is a flow chart of an update file configuration step in one embodiment;
FIG. 8 is a flowchart illustrating steps for obtaining a task execution result in one embodiment;
FIG. 9 is a block diagram of an index data processing device in one embodiment;
FIG. 10 is a block diagram of the data processing module in one embodiment;
FIG. 11 is a block diagram of a pointer data processing apparatus in another embodiment;
FIG. 12 is a block diagram of a finger data processing device according to yet another embodiment;
FIG. 13 is a block diagram of the structure of a task result module in one embodiment;
FIG. 14 is a block diagram of the internal architecture of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in the description of the disclosure herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure.
The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, it is not excluded that additional identical or equivalent elements may be present in a process, method, article, or apparatus that comprises a described element. For example, if first, second, etc. words are used to indicate a name, but not any particular order.
As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," and/or the like, specify the presence of stated features, integers, steps, operations, elements, components, or groups thereof, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof. Also, in this specification, the term "and/or" includes any and all combinations of the associated listed items.
As described in the background, in a large-scale and multi-feature index data processing scene, there is often a limitation that the processing efficiency is low and it is difficult to flexibly configure according to the scene requirement. Taking a performance assessment scene of human resources as an example, the index data processing method of the disclosure is explained.
The index data processing method provided by the embodiment of the disclosure can be applied to an application environment as shown in fig. 1. The index task system 100 communicates with the external service system 102 and the client 103104 through a network, and the index task system 100 may receive raw index data associated with performance assessment (index task) generated by the service system. The index task system 100 includes an external data subscription module 104, an index processing and warehousing module 106, a file management module 108, a mapping configuration module 110, a mapping read-write module 112, a performance evaluation planning module 114, an operation auditing module 116, a comprehensive query module 118, a performance template management module 120, an operation pipelining module 122, a template formula calculation module 124, a performance import and export module 126, and a performance analysis module 128.
Specifically, as shown in fig. 2, the external data subscription module 104 is configured to interact with the service system 102, and obtain, by means of subscription, raw index data associated with performance assessment. The external data subscription module 104 may import the subscribed raw index data into a process database with database scripts deployed.
The index processing and warehousing module 106 is configured to execute a pre-configured database script, read the original index data in the processing database 130 to perform processing, and calculate to obtain the index data. The index data of this embodiment may be used to characterize performance data of the user. The index processing and warehousing module 106 may store the obtained index data in an index database.
The file management module 108 is used to provide file storage management capabilities for the index task system 100. The file management module 108 may implement its functions through the operation object store. Object storage is commonly used for object-based storage and is a generic term used to describe methods of resolving and processing discrete units, which are referred to as objects. Objects will no longer have a hierarchy in a hierarchy. Each object is in the same level of a flat address space called a memory pool, and one object does not belong to the next level of another object. The storage object may store a task spreadsheet template file.
The mapping configuration module 110 is configured to receive a configuration instruction of the client 103, and configure a data mapping relationship between the task spreadsheet template and the index database. Each mapping relation record at least comprises four key information: (1) Page names in the task electronic form template where the index data are located; (2) The field names in the task electronic form template where the index data are located; (3) A data table name of index data in the index database; field names of the index data in the index database.
The mapping read-write module 112 is configured to implement writing of index data from the index database into the task spreadsheet template and writing of index data or task execution result data in the task spreadsheet instance into the index database according to the data mapping relationship between the task spreadsheet template and the index database.
As shown in fig. 3, the performance evaluation plan module 114 is configured to initiate a performance assessment task (index task) according to a configuration instruction of the client 103 at a preset period timing according to the configuration instruction.
The operation auditing module 116 is configured to implement modification of the task electronic form template and the associated configuration information of the task object, the mapping configuration information of the index database and the task electronic form template, or the task electronic form template by the client 103, and the operation auditing module 116 may audit the modification authority and take effect on the modification instruction of the client 103 having the administrator authority.
The comprehensive query module 118 is configured to provide the client 103 with query services such as an original index data query, an index data query, and a task execution result data query.
The performance template management module 120 is configured to provide a management function of the task electronic form template for the client 103, and may perform corresponding actions according to a series of instructions of uploading, modifying, deleting, etc. the task electronic form template and associated configuration information of the task electronic form template by the client 103. The performance template management module 120 can also manage task cycle information for the task spreadsheet template. The performance template management module 120 can upload the determined task spreadsheet template file to the file management module 108, which can store the table template file in the object store by the file management module 108.
The operation flow module 122 is used for recording operation flow of the electronic form update file uploaded by the client 103. The client 103 may upload a spreadsheet update file, wherein the spreadsheet update file may be a spreadsheet update file for a task spreadsheet template or a task spreadsheet instance that includes index data. In the case that the spreadsheet update file is for a task spreadsheet template, the spreadsheet update file is used as a new task spreadsheet template to alter the calculation formula of the task spreadsheet template. In the case where the spreadsheet update file is for a task spreadsheet instance, the spreadsheet update file is used to modify the index data in the task spreadsheet instance. In the event that the operation flow module 122 receives the spreadsheet update file, the modification authority may be audited by the operation audit module 116 for the client 103 account that uploaded the spreadsheet update file. And under the condition of meeting the authority requirement, the electronic form update file is enabled to be effective, and meanwhile, a copy file of the electronic form update file is established, and the operation serial number can be used as the name of the copy file.
The template formula calculation module 124 is configured to extract a first row of each page in the task electronic form instance, take the first row as a column of a formula to extract features and generate a calculation formula template, and fill the calculation formula into an effective row in the task electronic form template, where the calculation formula can be adaptively filled according to a row number of the task electronic form template, so as to implement a formula drop-down function. The template formula calculation module 124 triggers calculation of all calculation formulas in the task electronic form template after filling, and assessment result data is obtained.
The performance import and export module 126 is configured to implement import and export of the task execution result, so that the client 103 can query the task execution result according to keywords such as task time and task objects.
The performance analysis module 128 is configured to analyze the task execution result, and may display the task execution result, task ranking, history comparison information, etc. of the task object through imaging, tabulating, etc.
The client 103 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, etc. The index task system 100 may be implemented by a stand-alone server or a server cluster composed of a plurality of servers.
In some embodiments of the present disclosure, as shown in fig. 4, an index data processing method is provided, and the method is applied to the index task system in fig. 1 for illustration, and includes the following steps:
Where an index generally refers to the concept of an overall composite quantitative feature. For example, in a performance assessment scenario of human resources, the total number of business people, sales of business people, performance, etc. may be used as indicators, all reflecting the overall quantitative characteristics from various aspects.
For example, the index task system may periodically acquire the original index data from the service system according to a preset period. The original index data may refer to data related to an index in data daily generated by a service system, for example, in a performance assessment scenario of human resources, and the original index data may include sales list records of service personnel. After the original index data is obtained, the original index data can be processed by executing a database script, the original index data is summarized and calculated, for example, sales sum, performance and the like of business personnel are obtained according to sales list record calculation of the business personnel, and the processed index data is obtained.
And step 420, writing the processed index data into an index database.
For example, the index task system may store the resulting index data into an index database.
For example, the index task system may respond to the index task processing instructions after receiving the processing instructions of the index task. The index task processing instruction can be automatically initiated by the periodic timing of the index task system, or can receive the indication of the client side so as to initiate the index task processing instruction. The index task processing instruction may include index task information, for example, the index task information may include a task object, a task time, and the like. The task object associated with the index task can be determined through the index task processing instruction, and a task electronic table template corresponding to the task object is further obtained according to the task object of the index task. The task spreadsheet template may correspond to a task object; or may correspond to a particular attribute of the task object, such as a level of the task object. The task electronic form template may be a preset form template file (for example, an excel file) used for filling the index data and adapting to the index data. The index task may refer to a task that processes index data according to a pre-configured task execution logic, such as a performance assessment task.
Illustratively, the data mapping relationship may characterize a data mapping relationship between the task spreadsheet template and the index database, including mapping information of the data fields and mapping information of the data storage locations. According to the task object and data mapping relation related to the index task, index data (such as index data related to the task object) corresponding to the index task can be read from an index database and written into a corresponding position of the task electronic table template. The task spreadsheet template, after writing the index data, may generate a task spreadsheet instance for the index task.
Illustratively, after the task electronic table instance is generated, the index data in the task electronic table instance is processed according to the pre-configured task execution logic, for example, the index data can be calculated through a calculation formula, so as to obtain a task execution result.
According to the index data processing method, the index data are obtained through processing the obtained original index data, the index data are stored in the index database, and the task electronic table template corresponding to the task object is determined according to the task object of the index task by responding to the index task processing instruction, so that the index data required by the index task can be accurately and rapidly written into the task electronic table template from the index database according to the data mapping relation, the task electronic table template executes task execution logic aiming at the task object, and finally, the task execution result is obtained, so that the reading and writing efficiency of the index data is greatly improved, the task processing process of the index task is standardized through executing the index task more intuitively and transversely through the task electronic table template, and the processing efficiency is improved; meanwhile, the task spreadsheet template corresponds to the task object, so that the processing requirement of the personalized index task is met, and the processing accuracy and flexibility are improved.
In some embodiments of the present disclosure, as shown in fig. 5, step 410 includes:
For example, the original index data associated with the index task may be acquired by a subscription manner, and the original index data associated with the index task may be source data of the index data involved in the execution process of the index task. The subscription mode can adopt a periodic timing subscription mode or a mode of triggering subscription by data updating of a service system. After subscribing to the original index data, the original index data may be stored in a process database, which is pre-deployed with a database script.
By way of example, by performing calculation processing on the original index data by traversing the database script of the execution processing database, a plurality of index data records with the task object number and the task time as keywords can be generated. And inserting the generated index data record into an index database, and storing the index data.
In the embodiment, the processing database is used as a buffer, and the original index data is processed through the database script to obtain the index data, so that the processing process of the index data is standardized, and the index data can be inquired and positioned through the keywords.
In some embodiments of the present disclosure, as shown in fig. 6, the method further comprises:
Illustratively, the index task system may receive, by a client, associated configuration information for a task spreadsheet template and a task object. The association configuration information at least includes correspondence information between the task electronic table templates and task objects, for example, the task electronic table templates may be in one-to-one correspondence with task objects or one-to-one correspondence with task objects of a certain class. The index task system can configure the corresponding relation between the task electronic table template and the task object according to the association configuration information.
Illustratively, the index task system may receive mapping configuration information for the task spreadsheet template and the index database through the client. The mapping configuration information at least comprises data mapping information of the task electronic table template and the index database, such as data field mapping relation information and storage position mapping relation information of index data in the task electronic table template and in the index data. The index task system can configure the data mapping relation between the task electronic table template and the index database according to the mapping configuration information.
According to the method and the device, the association configuration information and the mapping configuration information sent by the client are obtained, so that the corresponding relation between the task electronic form template and the task object is configured according to the association configuration information, and the data mapping relation between the task electronic form template and the index database is configured according to the mapping configuration information, so that the task electronic form template can be highly suitable for the task object and the index database, and adaptability and flexibility are greatly improved.
In some embodiments of the present disclosure, the mapping configuration information includes a number of mapping relationship records including at least page names and field names of the index data in the task spreadsheet template, and data table names and field names of the index data in the index database. That is, each mapping record includes at least four key information: (1) Page names in the task electronic form template where the index data are located; (2) The field names in the task electronic form template where the index data are located; (3) A data table name of index data in the index database; field names of the index data in the index database. The method is beneficial to accurately and quickly positioning the index data in the task electronic table template and the index database.
In some embodiments of the present disclosure, as shown in fig. 7, the method further comprises:
Illustratively, the index task system may receive a spreadsheet update file for a task spreadsheet template or task spreadsheet instance through a client. In the case that the spreadsheet update file is for a task spreadsheet template, the spreadsheet update file is used as a new task spreadsheet template to alter the calculation formula of the task spreadsheet template. In the case where the spreadsheet update file is for a task spreadsheet instance, the spreadsheet update file is used to modify the index data in the task spreadsheet instance.
Illustratively, the index task system may perform authority verification on the client account that sends the electronic form update file after receiving the electronic form update file uploaded by the client. And judging whether the client account meets the set permission rule. Under the condition that the electronic form update file meets the authority rule, the electronic form update file is enabled to be effective and used as a formal task electronic form template or a task electronic form instance, and a corresponding flow record is generated. The flow record comprises a copy file of the electronic form update file, and the file name of the copy file can adopt a flow number.
The embodiment supports the update of the task electronic form template and the task electronic form instance by the client, updates the electronic form update file in a mode of uploading the electronic form update file, and verifies the authority of the electronic form update file, so that the electronic form update file is enabled to be more safely effective, and meanwhile, a running record is generated for the effective electronic form update file, so that backtracking of a data copy is facilitated, and data stability is ensured.
In some embodiments of the present disclosure, as shown in fig. 8, step 450 includes:
For example, after generating the task spreadsheet instance, the index task system may execute pre-configured task execution logic on the task spreadsheet instance. The target row of the task spreadsheet instance may be extracted, and the target row of behavior data header row may be used to characterize the characteristics or attributes of the index data of the corresponding column. The target row is typically the first row of the task spreadsheet template. After the first line data of the task electronic form instance is extracted, the first line data is analyzed, and the columns are used as calculation factors to obtain a calculation rule template corresponding to the task electronic form instance. The calculation rule template may comprise a calculation formula. Optionally, in some embodiments, the first column data of the task electronic table instance may be extracted, and the row is used as a calculation factor to obtain a calculation rule template corresponding to the task electronic table instance.
Illustratively, the calculation formula is written in the active row in the task spreadsheet instance according to the determined calculation rule template. The calculation formula can be filled in a self-adaptive mode according to the line number of the task electronic form template, and a formula pull-down function is achieved. The valid row may refer to a row in the task electronic table instance where no blank or abnormal row data exists.
And 456, calculating the index data in the task electronic table example according to the calculation formula to obtain the task execution result.
The method includes the steps that after the calculation formula is filled, index data in a task electronic table instance are triggered to be calculated according to the calculation formula, and a task execution result is obtained.
And step 458, filling the task execution result into the task electronic table instance to obtain a task result file.
For example, the task execution result obtained by calculation may be filled in a corresponding position in the task electronic form instance, so as to obtain a task result file.
According to the embodiment, the first line data in the task electronic table instance is extracted and analyzed, so that the calculation rule template corresponding to the task electronic table instance is determined, a calculation formula suitable for the task electronic table instance is obtained, the task electronic table instance can be rapidly calculated, and a task execution result is obtained.
In some embodiments of the present disclosure, the method further comprises:
and reading a task execution result in the task result file, and storing the task execution result into the index database according to the data mapping relation.
Illustratively, the metrics database is used not only to store metrics data, but also to store task execution results. The data mapping relation also comprises a data mapping relation between the task execution result and the index database. After the task spreadsheet instance generates the task result file, the index task system may read the task execution result from the task result file and store it to the index database.
According to the embodiment, the task execution result is read from the task result file and stored in the index database, so that the task execution result can be quickly stored, and the follow-up positioning inquiry is facilitated.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the disclosure also provides an index data processing device for implementing the index data processing method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation of one or more embodiments of the index data processing device provided below may refer to the limitation of the index data processing method hereinabove, and will not be repeated herein.
In some embodiments of the present disclosure, as shown in fig. 9, an index data processing device is provided. The apparatus 900 includes:
the data processing module 910 is configured to obtain original index data, and process the original index data by executing a pre-configured database script to obtain processed index data;
an index database module 920, configured to write the processed index data into an index database;
a task instruction module 930, configured to respond to an index task processing instruction, and obtain a task electronic table template corresponding to a task object associated with the index task;
the task electronic table module 940 is configured to read, from the index database, index data corresponding to the index task according to the task object and the data mapping relationship, and write the read index data into the task electronic table template to generate a task electronic table instance; the data mapping relation comprises a data field and a data storage position relation between the pre-configured task electronic table template and the index database;
And a task result module 950, configured to execute the task execution logic configured in advance in the task electronic table instance, and obtain a task execution result.
In some embodiments of the present disclosure, as shown in fig. 10, the data processing module 910 includes:
a subscription unit 912, configured to subscribe to original index data associated with an index task, and store the subscribed original index data to a processing database; the processing database is provided with the database script;
a script execution unit 914, configured to perform traversal of the database script, and generate index data; the processed index data comprises index data records taking task object numbers and task time as keywords;
a record insertion unit 916 for inserting the index data record into the index database.
In some embodiments of the present disclosure, as shown in fig. 11, the apparatus 900 further includes:
the association configuration module 960 is configured to obtain association configuration information of the task electronic form template and the task object, and establish a corresponding relationship between the task electronic form template and the task object according to the association configuration information;
the mapping configuration module 970 is configured to obtain mapping configuration information of the task electronic table template and the index database, and establish a data mapping relationship between the task electronic table template and the index database according to the mapping configuration information.
In some embodiments of the present disclosure, the mapping configuration information includes a number of mapping relationship records including at least page names and field names of the index data in the task spreadsheet template, and data table names and field names of the index data in the index database.
In some embodiments of the present disclosure, as shown in fig. 12, the apparatus 900 further includes:
an update file module 980 for receiving a spreadsheet update file for a task spreadsheet template or task spreadsheet instance;
an update validation module 990, configured to validate the electronic form update file and generate a flow record if the electronic form update file meets the permission rule; the flow record includes a copy file of the spreadsheet update file.
In some embodiments of the present disclosure, as shown in fig. 13, the task result module 950 includes:
the calculation extraction unit 952 is configured to extract first line data of the task electronic table instance, and obtain a calculation rule template according to the first line data;
a formula writing unit 954, configured to adaptively write a calculation formula according to a row number in the task electronic table instance according to the calculation rule template;
A calculation execution unit 956, configured to perform calculation processing on the index data in the task electronic table instance according to the calculation formula to obtain the task execution result;
and a result writing unit 958, configured to fill the task execution result into the task electronic table instance to obtain a task result file.
In some embodiments of the present disclosure, the index database module is further configured to read a task execution result in the task result file, and store the task execution result to the index database according to the data mapping relationship.
The respective modules in the index data processing device described above may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules. It should be noted that, in the embodiment of the present disclosure, the division of the modules is merely a logic function division, and other division manners may be implemented in actual practice.
Based on the foregoing description of the embodiments of the index data processing method, in another embodiment provided by the present disclosure, a computer device is provided, which may be a server, and an internal structure diagram thereof may be as shown in fig. 14. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing data. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of index data processing.
It will be appreciated by those skilled in the art that the structure shown in fig. 14 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the computer device to which the present application applies, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
Based on the foregoing description of the embodiments of the index data processing method, in another embodiment provided in the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor, implements the steps of the method embodiments described above.
Based on the foregoing description of the embodiments of the index data processing method, in another embodiment provided by the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method embodiments described above.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
In the description of the present specification, reference to the terms "some embodiments," "other embodiments," "desired embodiments," and the like, means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic descriptions of the above terms do not necessarily refer to the same embodiment or example.
It should be understood that, in the present specification, each embodiment of the method is described in a progressive manner, and the same/similar parts of each embodiment are referred to each other, where each embodiment focuses on a difference from other embodiments. For relevance, reference should be made to the description of other method embodiments.
The technical features of the above embodiments may be arbitrarily combined, and for brevity, all of the possible combinations of the technical features of the above embodiments are not described, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present disclosure, which are described in more detail and are not to be construed as limiting the scope of the claims. It should be noted that variations and modifications can be made by those skilled in the art without departing from the spirit of the disclosure, which are within the scope of the disclosure. Accordingly, the scope of protection of the present disclosure should be determined by the following claims.
Claims (10)
1. A method of processing index data, the method comprising:
acquiring original index data, and processing the original index data by executing a pre-configured database script to obtain processed index data;
writing the processed index data into an index database;
responding to an index task processing instruction, and acquiring a task electronic table template corresponding to a task object associated with the index task;
according to the task object and the data mapping relation, reading index data corresponding to the index task from the index database, writing the read index data into the task electronic table template, and generating a task electronic table instance; the data mapping relation comprises a data field and a data storage position relation between the pre-configured task electronic table template and the index database;
and executing task execution logic pre-configured in the task electronic table instance to acquire a task execution result.
2. The method of claim 1, wherein the obtaining the raw index data, performing a processing process on the raw index data by executing a pre-configured database script, and obtaining the processed index data comprises:
Subscribing original index data associated with an index task, and storing the subscribed original index data into a processing database; the processing database is provided with the database script;
traversing and executing the database script to generate index data; the processed index data comprises index data records taking task object numbers and task time as keywords;
the index data record is inserted into the index database.
3. The method according to claim 1, wherein the method further comprises:
acquiring association configuration information of the task electronic table template and the task object, and establishing a corresponding relation between the task electronic table template and the task object according to the association configuration information;
and acquiring mapping configuration information of the task electronic table template and the index database, and establishing a data mapping relation of the task electronic table template and the index database according to the mapping configuration information.
4. A method according to claim 3, wherein the mapping configuration information comprises a number of mapping records including at least page names and field names of the index data in the task spreadsheet template, and data table names and field names of the index data in the index database.
5. A method according to claim 3, characterized in that the method further comprises:
receiving a task electronic form template or an electronic form update file of a task electronic form instance;
under the condition that the electronic form update file meets the authority rule, the electronic form update file takes effect and generates a flow record; the flow record includes a copy file of the spreadsheet update file.
6. The method of claim 1, wherein the executing the task execution logic pre-configured in the task spreadsheet instance, the obtaining the task execution result comprises:
extracting the first line data of the task electronic form instance, and analyzing and obtaining a calculation rule template according to the first line data;
according to the calculation rule template, a calculation formula is adaptively written in the task electronic form example according to a line number;
calculating the index data in the task electronic form example according to the calculation formula to obtain the task execution result;
and filling the task execution result into the task electronic table instance to obtain a task result file.
7. The method of claim 6, wherein the method further comprises:
And reading a task execution result in the task result file, and storing the task execution result into the index database according to the data mapping relation.
8. An index data processing device, characterized in that the device comprises:
the data processing module is used for acquiring original index data, and processing the original index data by executing a pre-configured database script to obtain processed index data;
the index database module is used for writing the processed index data into an index database;
the task instruction module is used for responding to the index task processing instruction and acquiring a task electronic table template corresponding to a task object associated with the index task;
the task electronic form module is used for reading the index data corresponding to the index task from the index database according to the task object and the data mapping relation, writing the read index data into the task electronic form template and generating a task electronic form instance; the data mapping relation comprises a data field and a data storage position relation between the pre-configured task electronic table template and the index database;
And the task result module is used for executing the task execution logic pre-configured in the task electronic table instance and obtaining a task execution result.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310339638.7A CN116401323A (en) | 2023-03-31 | 2023-03-31 | Index data processing method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310339638.7A CN116401323A (en) | 2023-03-31 | 2023-03-31 | Index data processing method and device and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116401323A true CN116401323A (en) | 2023-07-07 |
Family
ID=87009769
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310339638.7A Pending CN116401323A (en) | 2023-03-31 | 2023-03-31 | Index data processing method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116401323A (en) |
-
2023
- 2023-03-31 CN CN202310339638.7A patent/CN116401323A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10747762B2 (en) | Automatic generation of sub-queries | |
| US8370303B1 (en) | Generating snapshots of data tables | |
| CN110472068A (en) | Big data processing method, equipment and medium based on heterogeneous distributed knowledge mapping | |
| CN115544183A (en) | Data visualization method and device, computer equipment and storage medium | |
| CN114238085A (en) | Interface testing method and device, computer equipment and storage medium | |
| CN116401238A (en) | Deviation monitoring method, apparatus, device, storage medium and program product | |
| CN115858471A (en) | Service data change recording method, device, computer equipment and medium | |
| US9092472B1 (en) | Data merge based on logical segregation | |
| CN115495452A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN116401323A (en) | Index data processing method and device and computer equipment | |
| CN112835886A (en) | Data table field adding method and device | |
| CN112402955B (en) | Game log recording method and system | |
| CN116880927A (en) | Rule management method, device, computer equipment and storage medium | |
| CN117370160A (en) | Code auditing method, apparatus, computer device, storage medium and program product | |
| CN117785674A (en) | Interface use case generation method, device, computer equipment and storage medium | |
| CN116091209A (en) | Credit service processing method, apparatus, computer device and storage medium | |
| CN117093759A (en) | Data processing method, device, computer equipment and storage medium | |
| CN115629958A (en) | Universal field level automatic checking method and device for different service interfaces | |
| CN117077635A (en) | Report file generation method, report file generation device, computer equipment and storage medium | |
| CN118152504A (en) | Unstructured data indexing method, device, apparatus, medium and program product | |
| CN117875284A (en) | Form processing method, form processing device, computer equipment and storage medium | |
| CN116880852A (en) | Code data checking method and device based on machine learning and computer equipment | |
| CN117555487A (en) | Data splitting method, device, computer equipment and storage medium | |
| CN117236298A (en) | Report data generation method, report data generation device, computer equipment, report data generation medium and program product | |
| CN117312306A (en) | Financial business data sheet conversion method, apparatus, device, medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |