CN117709909A - Business data processing method based on large language model - Google Patents

Abstract

The invention discloses a business data processing method based on a large language model, which comprises the following steps: s1, modularizing a subsystem according to department work items; s2, making a file subset classification table; the invention relates to the technical field of electric digital data processing. According to the business data processing method based on the large language model, various business data files of different departments and different personnel in an enterprise are integrated uniformly, uniform rules are formulated, a subset classification table is established to carry out ordered classification, identification classification and marking can be carried out when data are uploaded, and further when a main system processes the data, the direction to be processed can be quickly found out, such as classified storage record or sending to other subsystems, the difficulty of staff operation is saved, the error rate is reduced, a log table is simultaneously identified and processed and generated, a reminding mechanism is matched, the repeated operation problem is effectively avoided, the working efficiency is improved, meanwhile, the tracing is convenient, and the business data management is convenient.

Description

A business data processing method based on large language model

Technical field

The invention relates to the technical field of electronic digital data processing, and is specifically a business data processing method based on a large language model.

Background technique

In the course of work, enterprises usually process a large number of complex data files, and there are many types of existing data. For different departments of the entire enterprise, the types of data files that need to be processed are even more complex.

CN102368311B discloses a business data processing device, which can flexibly process its own data according to user-defined business rules for application businesses with different specific business rules in the same link, support subsequent processes, and take into account resource sharing, without repeated development, and Solved the problem of repeated user operations caused by the introduction of multiple business rules.

The above technology extracts unified rules for the same type of documents with different rules, and then manages all documents through unified rules. However, for some enterprises, the types of documents may be more diverse, such as drawing documents, financial statements, It is impossible to find unified rules to define and manage all types of documents such as personnel information, customer information, etc. Therefore, it is difficult to effectively handle different documents during the work process;

Even some files exist in different departments at the same time, and even the same file will have subtle differences. For example, the same file in different departments has deviations in the folder name, and the terminal devices of different departments and different employees of the same department usually have resources. It is also not shared. If it needs to be uploaded to the system, the system will only identify whether the outermost folder name is consistent. Even if only one word is added, it will be judged to be inconsistent. That is, the problem of file duplication will occur, thus resulting in the duplication of a lot of work and repeated operations. It may lead to incorrect data covering up correct data, which may cause a series of subsequent troubles.

Contents of the invention

In view of the shortcomings of the existing technology, the present invention provides a business data processing method based on a large language model, which solves the problem of multiple accounts in an enterprise processing a large amount of data, which is prone to repeated operations, and may even lead to data errors in the end. Issues with follow-up work.

In order to achieve the above objectives, the present invention is implemented through the following technical solutions: a business data processing method based on a large language model, which specifically includes the following steps:

S1. Modularly develop subsystems based on department work projects;

S2. Develop a file subset classification table: first create a subset of file data, and its formula is expressed as:

S={x|x∈A,P(x)};

Among them, S represents the created subset, A represents the original data set, and P(x) represents the condition or rule used for classification, which means that S is a subset composed of elements x in A that satisfy the condition P(x);

Then establish a file subset classification table based on the subset, and process and classify the new file names. Let the first-level classification be A, including two categories, large and small, respectively A1 and A2; the second-level classification is B, including both large and small. The class categories are B1 and B2 respectively, then the formula of the classification table can be expressed as:

A={A1, A2}, B={B1, B2};

Among them, A1 and A2 are the sub-category numbers of A, B1 and B2 are the sub-category numbers of B, and n numbers are further divided under the categories A1, A2, B1 and B2 to refer to specific keywords;

S3. Before uploading enterprise data to the main system, the subsystem performs keyword tagging;

S4. The data of the main system is identified, classified and processed through the data management unit. At the same time, for the files uploaded for the first time, all subsystems are traversed to find similar files, and reminders are issued to subsystems with similar files to eliminate duplication of work;

S5. Data identification combines data marking and coverage strategies to identify and process data, and at the same time cooperates with user interaction to deeply participate in the processing process;

S6. If the main database file is not accessed after the set time is reached, it means that the project is over, all associated subsystem database data will be automatically deleted, and the data corresponding to the main database will be transferred to the completion library;

S7. When searching for data, ask through the large language interaction module. The large language interaction module will feedback the data download items and attach the data history log.

Preferably, the S2 specifically includes the following steps:

(1) File name division: Divide the file name into numbers, letters and nouns according to spaces and symbols to obtain a word list; perform part-of-speech tagging for each word in the word list;

(2) Historical identification records: Based on historical file classification records and specific file data information, file name sentences are extracted and invalid features are eliminated based on artificially set rules, while valid features are retained to assist in the subsequent automatic classification process;

(3) Set rule classification tags: Manually formulate a file subset classification table based on the department and file type to which the business data file belongs, and manually set rules to edit the extracted features, edit the subset codes to replace the features, and add codes to the file names. Category tags associate file names with corresponding categories;

(4) Algorithm integration: Integrate effective features, historical identification records and formulated file subset classification tables into an algorithm model, and label and classify new file names.

Preferably, the S4 specifically includes:

(1) Word segmentation processing and keyword extraction:

1. Word segmentation processing: replace non-alphanumeric characters in the file name with spaces, and use regular expressions to split the file name according to words;

2. Extract keywords: For each segmented word, compare it with the keywords in the file subset classification table, eliminate redundant words, and use the processed words as keywords in the file name;

(2) Use the classification algorithm model to classify the extracted keywords against the subset classification table;

(3) Data storage, specifically including:

1. Store in the main system database;

2. Send to other subsystem databases;

3. Generate log table.

Preferably, the S5 specifically includes: reminding the data for which the mark has not been recognized, extracting the log table to determine whether it is the first upload, and prompting the mark option based on the direction of the subsystem and the number of uploads, and the uploader chooses to add the mark. , the data not uploaded for the first time will overwrite the original data after distribution, and the original data will be transferred to the temporary storage station for temporary storage.

Preferably, the S3 subsystem needs to perform a series of data processing and classification before uploading data to the main system, which specifically includes the following steps:

(1) Divide the data packets uploaded by the subsystem. If the data packet class contains different types of data, classify the different types of data into different subsets;

(2) According to the rules in the subset classification table, add coding to each file to mark it. During the marking process, the subset files are superimposed based on the identified characteristics and the order of classification marks in the file subset classification table. Classification tag;

(3) Upload the marked and coded data to the main system.

Preferably, the subsystem is divided into several department subsystems according to enterprise departments, and each employee account in the department is divided into the same department subsystem.

Preferably, the main system includes a data management unit, a main system database, a large language interaction module and a subsystem authority management module. The data management unit and the large language interaction module upload, download and view data by accessing the main system database, so The subsystem permission management module described above is used to manage the account permissions of department subsystems.

Preferably, the subsystem is connected to the main system through an enterprise intranet, and when the subsystem is located on the external network, it enters the intranet through VPN to access the main system.

Preferably, the department subsystem includes a subsystem database, a system framework, a communication mechanism, an expansion module and a modular functional unit. The modular functional unit also includes several functional modules and an interface definition module that defines several functional module interfaces.

Preferably, the communication mechanism is used to connect the functional modules integrated by the modular functional units to the system framework to form a complete system, and the expansion module is used to load new functional data and then expand into new functional modules, or when the original functional module fails. The functional module data is loaded for temporary use, and the subsystem database is used to store local data and modular data of functional data.

beneficial effects

The present invention provides a business data processing method based on a large language model. Compared with existing technology, it has the following beneficial effects:

1. This business data processing method based on the large language model uniformly integrates a variety of business data files from different departments and different personnel within the enterprise, formulates unified rules, and establishes a subset classification table for orderly classification. When uploading data, it can be identified, classified and marked. Then when the main system processes the data, it can quickly find the direction that should be processed, such as classifying and storing records or sending them to other subsystems, which saves the difficulty of employee operations and reduces errors. efficiency, identify, process and generate log tables at the same time, and cooperate with the reminder mechanism and subsystem automatic search function to prompt subsystems with common data, effectively avoiding repeated operation problems, improving work efficiency, and also facilitating traceability and management. Business data, while formulating time rules, regularly cleaning stored files, and cleaning subsystems to avoid redundant subsystem data, thus reducing the workload of employees in processing data and effectively avoiding repeated operations.

2. This business data processing method based on the large language model can build a floating window in the corner of the operation interface by setting up the large language interaction module. As the operation interface is pulled, it will always remain at a fixed position on the display interface, making it convenient for the operator to use. The large language interaction module can be used for inquiries through dialogue, and can perform system-recorded operations such as searching data, searching history, and operating rules. Enterprises in need can also be equipped with artificial intelligence plug-ins such as chatgpt to assist in office work, which effectively improves the efficiency of enterprise data processing. efficiency.

3. This business data processing method based on the large language model divides the large system into several subsystems according to the nature of the department, and manages the accounts of employees within the department respectively. The content of each department's subsystem is different, which facilitates targeted management and Business data processing uses both internal enterprise intranet access and external VPN access to the intranet, further ensuring the security of enterprise business data.

4. This business data processing method based on the large language model establishes department subsystems as functional modules, makes each function independent, and uses the communication mechanism to cooperate with the interface definition module to be mounted on the system framework, so that it can be conveniently based on different functions. Depending on the nature of the department's business, targeted function combinations have been carried out, eliminating many unnecessary functions. The department subsystem interface settings have been optimized to make it simple and easy to operate, making it easier for newcomers to get started, and also reducing the error rate of business data operations.

Description of the drawings

Figure 1 is a general system block diagram of the present invention;

Figure 2 is a system block diagram of the department subsystem of the present invention;

Figure 3 is a schematic flow diagram of the present invention;

Figure 4 is a logical flow chart of long-term file processing in the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

Figure 1-4 The present invention provides three technical solutions:

The first implementation mode: a business data processing method based on a large language model, specifically including the following steps:

S1. Modularly develop subsystems based on department work projects;

S2. Specifically includes the following steps:

(1) File name division: Divide the file name into numbers, letters and nouns according to spaces and symbols to obtain a word list; perform part-of-speech tagging for each word in the word list;

(2) Historical identification records: Based on historical file classification records and specific file data information, file name sentences are extracted and invalid features are eliminated based on artificially set rules, while valid features are retained to assist in the subsequent automatic classification process;

(3) Set rule classification tags: Manually formulate a file subset classification table based on the department and file type to which the business data file belongs, and manually set rules to edit the extracted features, edit the subset codes to replace the features, and add codes to the file names. Category tags associate file names with corresponding categories;

Developing a file subset classification table includes: first creating a subset of file data, the formula is expressed as:

S={x|x∈A,P(x)};

Among them, S represents the created subset, A represents the original data set, and P(x) represents the condition or rule used for classification, which means that S is a subset composed of elements x in A that satisfy the condition P(x);

Then establish a file subset classification table based on the subset, and process and classify the new file names. Let the first-level classification be A, including two categories, large and small (classifications can be added according to the development of the enterprise), which are A1 and A2 respectively; The secondary classification is B, which includes two categories, large and small, B1 and B2 respectively. The formula of the classification table can be expressed as:

A={A1, A2}, B={B1, B2};

Among them, A1 and A2 are the sub-category numbers of A, B1 and B2 are the sub-category numbers of B, and the categories A1, A2, B1 and B2 are further divided into n numbers to refer to specific keywords, such as A1 and the following include {A1 ⁰¹ , A1 ⁰² ,..., A1 ⁿ }, A1 represents financial engineering materials, A1 ⁰¹ represents engineering investment estimation materials, A1 ⁰² represents engineering design budget materials;

(4) Algorithm integration: Integrate effective features, historical identification records and formulated file subset classification tables into algorithm models, and use machine learning, deep learning or other related technologies to mark, process and classify new file names;

Before S3 and the subsystem upload enterprise data to the main system, a series of data processing and classification are required, including the following steps:

(1) Divide the data packets uploaded by the subsystem. If the data packet class contains different types of data, classify the different types of data into different subsets;

(2) According to the rules in the subset classification table, add coding to each file to mark. During the marking process, the subset files are superimposed based on the identified characteristics and the order of classification marks in the file subset classification table. Classification tag;

(3) Upload the marked and coded data to the main system;

S4. The data of the main system is identified, classified and processed through the data management unit, including:

1. Word segmentation processing: Replace non-alphanumeric characters in the file name with spaces, and use regular expressions to split the file names according to words; regular expressions are a method used to describe string patterns, and they can be used to match , search and replace strings in text.

2. Extract keywords: For each segmented word, compare it with the keywords in the file subset classification table, eliminate redundant words, and use the processed words as keywords in the file name;

For example: 2023.11.29 Zhang San Finance Department of an enterprise 202345286 - Project accounting report - phase control information copy. The extracted keywords include: 2023.11.29 (date), Zhang San (name of person handling the matter), Finance Department (department), A certain enterprise (company name), 202345286 (code), engineering accounting report (file category), phase control information, copy, and spaces and symbols. Among them, Zhang San, Finance Department, a certain enterprise, and engineering accounting report can be matched to the file subset classification features in the table, the remaining keywords are invalid features and will be deleted.

(2) Use the classification algorithm model to classify the extracted keywords against the subset classification table. For example, if the identified keywords belong to A1 ⁰² under the A1 category and B2 ⁰¹ under the B2 category, mark A1 ⁰² B2 ⁰¹ before the file name; Common classification algorithm models include: logistic regression, decision tree random forest, naive Bayes, support vector machine, K nearest neighbor algorithm, and neural network, all of which are existing mature classification algorithm models.

(3) Data storage, specifically including:

1. Store in the main system database;

2. Send to other subsystem databases;

3. Generate log table.

At the same time, for files uploaded for the first time, all subsystems are traversed to find similar files, and reminders are issued to subsystems where similar files exist to eliminate duplication of work;

S5. Data identification combines data marking and coverage strategies to identify and process data, and at the same time cooperates with user interaction to deeply participate in the processing process, including: reminding data that has not been marked, and extracting the log table to determine whether it is the first upload, and based on the child The system prompts the tag option according to the direction and the number of uploads. The uploader chooses to add tags. The data that is not uploaded for the first time will overwrite the original data after distribution, and the original data will be transferred to the temporary storage station for temporary storage (the temporary storage station and the previous storage space can Is it a cloud platform or a company’s internal large-capacity storage device);

S6. If the main database file is not accessed after reaching the set time, it means that the project is over, all associated subsystem database data will be automatically deleted, and the data corresponding to the main database will be transferred to the completion library (the completion library is preferably a cloud platform. More and longer data can be stored for easy traceability). The steps are:

(1) First set the threshold time, such as 30 days;

(2) Traverse all files in the file system;

(3) For each file, check its last access time. You can use the API provided by the operating system to obtain the last access time of the file;

(4) If the file was last accessed for more than 30 days, mark it as "needs deletion";

(5) After completing the traversal, traverse all the files that need to be deleted and delete them.

As shown in Figure 4, assume that the last access date is marked as T _f , the current date is T _d , and the unvisited time is T _w, , then: T _w, = T _d - T _f , if T _w, 30, transfer the file in the main system and delete the corresponding file in the subsystem, otherwise keep the file.

S7. When searching for data, ask through the large language interaction module. The large language interaction module will feedback the data download items and attach the data history log.

By integrating various business data files from different departments and personnel within the enterprise, and formulating unified rules, establishing a subset classification table for orderly classification, identification, classification and labeling can be performed when uploading data, and then in When the main system processes data, it can quickly find the direction that should be processed, such as classifying and storing records or sending them to other subsystems, which saves the difficulty of employee operations and reduces the error rate. It also identifies and processes and generates log tables, and cooperates with the reminder mechanism. And the automatic search function of subsystems prompts subsystems with common data, which effectively avoids repeated operations, improves work efficiency, and avoids incorrect data overwriting correct data. It also facilitates traceability, management of business data, and formulation of Time rules regularly clean up stored files and subsystems to avoid redundant subsystem data, thereby reducing the workload of employees in processing data and effectively avoiding repeated operations.

Making the large language interactive module icon always float in the corner of the interface can be achieved using CSS and HTML:

First, create a div element containing the icon in your HTML file, for example:

<div class="floating-icon">

<img src="icon.png" alt="Floating Icon">

</div>

Then, style the div element in the CSS file so that it always floats in the corner of the interface, for example:

.floating-icon {

position: fixed;

bottom: 20px; /* distance from the bottom */

right: 20px; /* distance from the right */

}

Through the above method, you can make the icon always float in the corner of the interface, no matter how the user scrolls the page or resizes the window.

By setting up a large language interaction module, a floating window can be built in a corner of the operation interface. As the operation interface is pulled, it will always remain at a fixed position on the display interface, making it convenient for the operator to use. The large language interaction module can be asked through dialogue, and can be The system has recorded operations such as data search, search history, and operation rules. Enterprises in need can also use artificial intelligence plug-ins such as chatgpt to assist in office work, which effectively improves the efficiency of enterprise data processing.

The main difference between the second implementation mode and the first implementation mode is that the subsystem is divided into several department subsystems according to enterprise departments, and each employee account in the department is divided into the same department subsystem.

The main system includes a data management unit, a main system database, a large language interaction module and a subsystem authority management module. The data management unit and the large language interaction module upload, download and view data by accessing the main system database. The subsystem The permission management module is used to manage the account permissions of department subsystems.

The subsystem is connected to the main system through the enterprise intranet. When the subsystem is located on the external network, it enters the intranet through VPN to access the main system.

By dividing the large system into several subsystems according to the nature of the department, and managing the accounts of employees within the department respectively, the content of each department's subsystem is different, which facilitates targeted management and business data processing, while using the enterprise's internal intranet access and external VPN access to the intranet further ensures the security of corporate business data.

The main difference between the third implementation mode and the second implementation mode is that the department subsystem includes a subsystem database, a system framework, a communication mechanism, an expansion module and a modular functional unit, and the modular functional unit also includes several Function modules and interface definition modules that define several functional module interfaces;

The communication mechanism is used to connect the functional modules integrated by the modular functional units to the system framework to form a complete system. The expansion module is used to load new functional data and then expand it into a new functional module, or load the function when the original functional module fails. Module data is temporarily used, and the subsystem database is used to store local data and modular data of functional data.

By establishing the department subsystem as a functional modular method, each function is independent, and the communication mechanism is used in conjunction with the interface definition module to be mounted on the system framework, which can facilitate targeted function combinations based on the different business nature of different departments. It eliminates many unnecessary functions and optimizes the department subsystem interface settings to make it simple and easy to operate, making it easier for newcomers to get started and reducing the error rate of business data operations.

At the same time, contents not described in detail in this specification belong to the prior art known to those skilled in the art.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are mutually exclusive. any such actual relationship or sequence exists between them. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment.

Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art will understand that various changes, modifications, and substitutions can be made to these embodiments without departing from the principles and spirit of the invention. and modifications, the scope of the invention is defined by the appended claims and their equivalents.

Claims (10)

1. A business data processing method based on a large language model, which is characterized by: specifically including the following steps: S1. Modularly develop subsystems based on department work projects; S2. Develop a file subset classification table: first create a subset of file data, and its formula is expressed as: S={x|x∈A,P(x)}; Among them, S represents the created subset, A represents the original data set, and P(x) represents the conditions or rules used for classification. The conditions or rules for classification are set by human participation, which means that S is formed from A that satisfies the condition P( A subset composed of elements x of x); Then establish a file subset classification table based on the subset, and process and classify the new file names. Let the first-level classification be A, including two categories, large and small, respectively A1 and A2; the second-level classification is B, including both large and small. The class categories are B1 and B2 respectively, then the formula of the classification table can be expressed as: A={A1, A2}, B={B1, B2}; Among them, A1 and A2 are the sub-category numbers of A, B1 and B2 are the sub-category numbers of B, and n numbers are further divided under the categories A1, A2, B1 and B2 to refer to specific keywords; S3. Before uploading enterprise data to the main system, the subsystem performs keyword tagging; S4. The data of the main system is identified, classified and processed through the data management unit. At the same time, for the files uploaded for the first time, all subsystems are traversed to find similar files, and reminders are issued to subsystems with similar files to eliminate duplication of work; S5. Data identification combines data marking and coverage strategies to identify and process data, and at the same time cooperates with user interaction to deeply participate in the processing process; S6. If the main database file is not accessed after the set time is reached, it means that the project is over, all associated subsystem database data will be automatically deleted, and the data corresponding to the main database will be transferred to the completion library; S7. Use CSS and HTML to load the large language interactive module into the system operation interface in a floating manner. When searching for data, ask through the large language interactive module, and the large language interactive module will feed back the corresponding data. 2. A business data processing method based on a large language model according to claim 1, characterized in that: the S2 specifically includes the following steps: (1) File name division: Divide the file name into numbers, letters and nouns according to spaces and symbols to obtain a word list; perform part-of-speech tagging for each word in the word list; (2) Historical identification records: Based on historical file classification records and specific file data information, file name sentences are extracted and invalid features are eliminated based on artificially set rules, while valid features are retained to assist in the subsequent automatic classification process; (3) Set rule classification tags: Manually formulate a file subset classification table based on the department and file type to which the business data file belongs, and manually set rules to edit the extracted features, edit the subset codes to replace the features, and add codes to the file names. Category tags associate file names with corresponding categories; (4) Algorithm integration: Integrate effective features, historical identification records and formulated file subset classification tables into an algorithm model, and label and classify new file names. 3. A business data processing method based on a large language model according to claim 1, characterized in that: the S4 specifically includes: (1) Word segmentation processing and keyword extraction: 1. Word segmentation processing: replace non-alphanumeric characters in the file name with spaces, and use regular expressions to split the file name according to words; 2. Extract keywords: For each segmented word, compare it with the keywords in the file subset classification table, eliminate redundant words, and use the processed words as keywords in the file name; (2) Use the classification algorithm model to classify the extracted keywords against the subset classification table; (3) Data storage, specifically including: 1. Store in the main system database; 2. Send to other subsystem databases; 3. Generate log table. 4. A business data processing method based on a large language model according to claim 1, characterized in that: the S5 specifically includes: reminding the data that has not been marked, and extracting the log table to determine whether it is the first upload. , and prompts the marking option based on the subsystem's direction and the number of uploads. The uploader chooses to add a mark. The non-first-time uploaded data will overwrite the original data after distribution, and the original data will be transferred to the temporary storage station for temporary storage. 5. A business data processing method based on a large language model according to claim 1, characterized in that: the S3 subsystem needs to perform a series of data processing and classification before uploading data to the main system, specifically including the following: steps: (1) Divide the data packets uploaded by the subsystem. If the data packet class contains different types of data, classify the different types of data into different subsets; (2) According to the rules in the subset classification table, add coding to each file to mark. During the marking process, the subset files are superimposed based on the identified characteristics and the order of classification marks in the file subset classification table. Classification tag; (3) Upload the marked and coded data to the main system. 6. A business data processing method based on a large language model according to claim 1, characterized in that: the subsystem is divided into several department subsystems according to enterprise departments, and each employee account in the department is divided into the same department subsystem. within the system. 7. A business data processing method based on a large language model according to claim 6, characterized in that: the main system includes a data management unit, a main system database, a large language interaction module and a subsystem authority management module. The data management unit and the large language interaction module upload, download and view data by accessing the main system database. The subsystem authority management module is used to manage the account authority of the department subsystem. 8. A business data processing method based on a large language model according to claim 6, characterized in that: the subsystem is connected to the main system through the enterprise intranet, and when the subsystem is located on the external network, it enters the internal network through VPN. Network access to the main system. 9. A business data processing method based on a large language model according to claim 6, characterized in that: the department subsystem includes a subsystem database, a system framework, a communication mechanism, an expansion module and a modular functional unit, so The modular functional unit also includes several functional modules and an interface definition module that defines several functional module interfaces. 10. A business data processing method based on a large language model according to claim 9, characterized in that: the communication mechanism is used to connect functional modules integrated with modular functional units to the system framework to form a complete system. The expansion module is used to load new function data and then expand it into a new function module, or to load the function module data for temporary use when the original function module fails. The subsystem database is used to store local data and modular data of the function data.