CN117648998A - Large language model federal pre-training method based on trusted execution environment - Google Patents

Abstract

The invention discloses a large language model federated pre-training method based on a trusted execution environment, which includes the following steps: Step 1: Create a large language model joint pre-training task, determine joint modeling participants, prepare data, and create Computing and storage network resources for joint modeling; Step 2: Perform joint pre-training of large language models; Step 3: Optimize the large language model obtained by joint pre-training. Based on the actual scenario of multi-party federated modeling applied to large language model pre-training, RDMA and CXL technologies are fully utilized to build a cross-domain trusted execution environment cluster in a distributed environment, and divide the memory into shared areas and private areas. The area forms a large memory so that it can accommodate large language models and their training data as well as intermediate training results, overcoming the communication bottlenecks of trusted modeling and insufficient resource utilization at the scale of large models and big data.

Description

A federated pre-training method for large language models based on a trusted execution environment

Technical field

The invention belongs to the technical field of large language model federated pre-training, and specifically relates to a large language model federated pre-training method based on a trusted execution environment.

Background technique

In the context of today's rapid development of artificial intelligence, natural language processing, as an important direction of artificial intelligence, has shown a wide range of application prospects in various fields, such as machine translation, sentiment analysis, intelligent dialogue, text generation and other application prospects are becoming more and more widespread, and Continue to empower and promote the development of medical, financial, government and other industries.

Pre-training large models is a key technology for natural language processing. By pre-training on large-scale data, the model can learn rich language knowledge and patterns, providing strong support for various downstream tasks. However, this technology also faces a series of serious challenges, which not only involve technical aspects, but also relate to issues in multiple key areas such as data privacy, computing efficiency, and security.

First of all, the data required for pre-training of large language models is huge, covering multiple data sources and data owners. These data may contain sensitive information, such as personal privacy and business secrets, so how to conduct large models in the context of multi-party cooperation? Training, ensuring data privacy has become a crucial issue to avoid data leakage and misuse.

Secondly, the pre-training process of large language models requires huge storage space, such as the storage of data sets for training, the storage of large language models, and the storage of intermediate training results, which places high demands on computing and storage resources. When resources are limited, It may lead to a decrease in training speed or even an inability to meet training needs. At the same time, GPU acceleration plays an important role in the training of NLP large language models. However, the credibility of the GPU is challenged and may introduce security risks. How to ensure the credibility and security of the GPU needs to be solved. In addition, the pre-training process of large language models takes a long time, and node failures and abnormal interruptions may cause the training process to be interrupted. In addition, all data is stored in memory, and the risk of intermediate data loss is significant.

The large language model is based on Transformer. The idea of pre-training is that the model parameters are no longer randomly initialized, but are pre-trained through tasks to obtain a set of model parameters, and then use this set of parameters to initialize the model before training. The large language model pre-training process mainly uses the above words to predict the next word. It is an unsupervised pre-training, including AR models (that is, models that learn from left to right), in-context learning technology, etc. With the emergence of ChatGPT, after completing the pre-training of the large language model, supervised learning, reward models and reinforcement learning will be used for further fine-tuning.

In this case, how to make full use of cutting-edge technologies such as Trusted Execution Environment (TEE), Compute Express Link (CXL) technology, and Remote Direct Memory Access (RDMA), combined with key management, encrypted transmission, and data isolation, to target In large language model pre-training scenarios, multi-party joint and efficient modeling under the premise of compliance has become an urgent problem that needs to be solved.

Contents of the invention

In order to overcome the shortcomings of the above-mentioned existing technologies, the purpose of the present invention is to provide a large language model federated pre-training method based on a trusted execution environment, so as to be applied to the actual scenario of large language model pre-training multi-party federated modeling, and make full use of RDMA and CXL technology build a cross-domain Trusted Execution Environment (TEE) cluster in a distributed environment, and divide the memory into shared areas and private areas. By forming the shared area into a large memory, it can accommodate large language models and their training. data and intermediate training results to overcome the communication bottlenecks and resource underutilization of trusted modeling at the scale of large models and big data.

In order to achieve the above objects, the technical solution adopted by the present invention is:

A federated pre-training method for large language models based on a trusted execution environment, including the following steps:

Step 1. Create a large language model joint pre-training task, determine joint modeling participants, prepare data, and create computing and storage network resources for joint modeling;

Step 2. Conduct joint pre-training of large language models;

Step 3. Optimize the large language model obtained by joint pre-training.

The step 1 specifically includes the following steps:

Step 101. Confirm the pre-training tasks: Define the specific tasks of large language model pre-training, including the initial parameter configuration of the large language model and training data requirements;

Step 102. Determine the joint modeling participants: Determine the joint modeling participants participating in the joint pre-training of the large language model, including the owners and data providers of the large language model;

Step 103. Build a cross-domain TEE cluster: In a distributed environment, build a cross-domain Trusted Execution Environment (TEE) cluster. Each joint modeling participant starts the TEE management node and plans shared areas, private areas, and GPU resources in the TEE. , and load the access permission information into the FPGA to control access to RDMA and CXL;

Step 104. Set the global clock: Introduce the global clock as a unified time scale and distribute N random number seeds to all joint modeling participants to ensure time consistency and data security;

Step 105. Encryption and MAC authentication: The initiator of the large language model federated pre-training task selects a random seed to obtain the global clock generation key, encrypts the initial model, generates a MAC message authentication code, and sends the encrypted initial model and related identifiers Information is placed into the shared memory area to prepare for model distribution;

The shared memory area is divided into a private area and a shared public area. The private area stores the private data of the participants participating in the federated modeling joint modeling, which is sensitive data. These data do not leave the domain; the shared public area is used for Efficient and fast sharing of non-sensitive data, metadata in the modeling process, global model parameters, and public data sets;

Step 106. Training data loading and processing: Each of the joint modeling participants loads the data into the private area and shared area of the TEE according to the data sensitivity, tokenizes the data to form a vector representation, obtains the global clock, and selects Local random seeds encrypt data in the shared area to enhance data confidentiality;

Step 107. Data aggregation and obfuscation: The task initiator aggregates the data in the TEE shared area, generates a key based on the random seed number identified by the metadata and the global clock, and uses the key to decrypt the data, and then performs Data obfuscation blurs the source of the data. By obtaining the global clock and selecting local random seeds, the data in the shared area is re-encrypted to generate a public data set and store it in the shared memory area;

The step 107 specifically includes:

Data aggregation: The task initiator obtains data from the TEE shared area of each joint modeling participant (these data are preprocessed, tokenized, encrypted vector representations), and the task initiator uses the global clock and random metadata identification Several sub-serial numbers generate keys, and using the generated keys, the task initiator decrypts the data obtained from each TEE. This step restores the original, encrypted data to its original state;

Data obfuscation: On the basis of decryption, the task initiator obfuscates the data (the purpose of obfuscation is to obscure the source of the data and increase the privacy of the data). The obfuscation process can use different technologies, and it can be obfuscated through sampling. , you can also use the deduplication method to mix them all together, you can add some noise, perturb the data, or use other techniques to introduce randomness.

Re-encryption: The obfuscated data is re-encrypted, and a new key is generated again using the global clock and the local random seed of the task initiator. The encryption ensures the security of the obfuscated data during the storage process, while still being able to be used in the next step. are decrypted and used for further training of the model.

A public dataset is generated: the processed, obfuscated, and re-encrypted data are combined into a common dataset. (A public data set is a collection that contains information from different joint modeling participants, but the specific source of individual data has become unclear due to obfuscation. The public data set is stored in a shared memory area, So that other joint modeling participants can access and use it.)

The goal of this step is to ensure that in federated pre-training, the data can be used for collaborative modeling without exposing individual privacy. The process of encryption and obfuscation provides additional security during the transmission and storage of data, and by aggregating this data, the model can obtain more comprehensive information from many aspects.

Step 108. Pulling public data sets: Each of the joint modeling participants pulls the data in the shared area into their respective local environments through the RDMA protocol to prepare for subsequent training;

Step 109. GPU initialization: Each of the joint modeling participants initializes the GPU to make it a trusted computing resource for accelerating the training process of the large language model.

The step 2 specifically includes the following steps:

Step 201. Model initialization and memory transfer: Each of the joint modeling participants passes the initial model through the FPGA controller and uses the RDMA protocol to directly copy the memory of the model to the memory of the respective TEE cluster to ensure that the model is loaded into a trusted environment. middle;

Step 202. Data selection and preparation: The joint modeling participants randomly select a part of the data from the public data and use the user's private data as the training data for this round of training;

The public data refers to user non-sensitive data or corresponding data desensitization, or data that has been published from the outside. These data are sent by each user to the central node for data obfuscation, and then formed a basic public data set; mainly In order to accelerate the convergence speed of federated learning (essentially solving the problem of NonIID, the distribution of the public data set is the same as the final joint modeling target-oriented data distribution).

Step 203. Accelerated calculation and data transmission: The FPGA controller locks the GPU for exclusive use, routes part of the program and training data that need to be accelerated through the FPGA, and performs a decryption operation. The data is transmitted to the GPU using CXL technology so that High-speed calculation and processing;

Step 204. Distributed parallel training: In the local TEE cluster, multiple sets of data are used for distributed parallel training. The joint modeling participants use different strategies to process the data in the public area and the private area, calculate the gradient, and perform the training in the private area. Gradient aggregation is completed in the TEE cluster, and then the global clock and the selected random seed are used to generate a key for encryption, the result is stored in the public shared memory, and all joint modeling participants are notified;

The shared memory is an area that forms a large memory for each node to directly read and write the memory to improve processing speed;

Step 205. Clear sensitive memory: After this round of calculation is completed, use the FPGA controller to clear the sensitive memory on the GPU chip to ensure data security;

Step 206. Global aggregation and model update: The aggregation node is responsible for aggregating the gradient data in the shared memory and then updating the model parameters;

The gradient data is used by the joint modeling participants of federated modeling to use local data sets to perform local forward calculations, then calculate the gradients of parameters according to the set loss function, and then upload these gradients to the aggregation node. You can refer to the FedAvg algorithm. The central aggregation node performs a weighted average of the gradient parameter values from each node and updates the global model. This is a common processing method for federated learning algorithms.

Step 207. Model distribution: The aggregation node puts the updated model into the public area, encrypts it using a random number seed and a global clock, and prepares it for model distribution for subsequent training;

Step 208. Download and train the updated model: Each of the joint modeling participants downloads the new model from the aggregation node, and the aggregation node updates the global model and sends it directly to the joint modeling participants, who then pull it. The method adopted is used to accelerate execution and be used for a new round of model training until the model reaches a convergence state;

Step 209. Encrypted storage of intermediate results: The task is triggered regularly, and the aggregation node will continuously iterate for training, and the data of the model and training status of the intermediate results of continuously aggregating parameters will generate three copies, and use random keys for encrypted storage; then , using Shamir's Secret Sharing Algorithm to distribute random key fragments to each joint modeling participant for future key recovery;

The Shamir secret sharing algorithm is a method that divides secret information into multiple parts, each part is distributed to joint modeling participants, and the original secret can only be restored when a certain threshold is reached. This algorithm was proposed by Adi Shamir in 1979 and is an application of threshold cryptography.

The following is the basic principle of Shamir's secret sharing algorithm:

Secret division: Suppose there is a secret S, and the algorithm divides the secret into N parts, of which at least K parts are needed to restore the original secret;

Generating polynomial: starting from a high-degree polynomial, the constant term of the polynomial is the secret S, and the coefficients of the polynomial are randomly selected from a finite field (such as integer modulus);

Compute shards: By choosing different X values on the polynomial, calculate the coordinates of multiple points, each coordinate corresponding to a secret part;

Distribution part: Distribute the coordinates of these points as secret parts to different joint modeling participants. Each joint modeling participant only knows the coordinate values they hold;

Restore the secret: At least K different parts are needed, and the original polynomial is restored through these parts using interpolation method to obtain the original secret.

Shamir's method provides good security and resilience, and the secret can only be restored when the threshold K is reached. This method has wide applications in key management, data storage, and distributed system security.

Step 210. Abnormal situation processing: When an abnormal situation occurs, the joint modeling participants jointly recover the secret fragments they own for key recovery and perform decryption operations to recover the intermediate results and ensure the continuity and accuracy of training. reliability.

Abnormal situations refer to abnormal situations such as network failures and hardware failures. The aggregation node as a central node has the risk of single point failure. Since the training of large language models often requires a long time and computing resources, aggregation nodes fail or a large number of participants participate in the construction. If the module node goes offline, a plan needs to be handled.

Described step 3 specifically includes the following steps;

Step 301: After completing the training of the large language model joint pre-training task, deploy it to the actual application environment for use;

Step 302: Collect feedback data, continue to increase training data, and continuously improve model accuracy and training efficiency.

Beneficial effects of the present invention:

This invention is intended to be applied to the actual scenario of large language model pre-training multi-party federated modeling, making full use of RDMA and CXL technology to build a cross-domain trusted execution environment (TEE) cluster in a distributed environment, and dividing shared areas and memory areas. The private area, by forming a shared area into a large memory, can accommodate large language models and their training data as well as intermediate training results, overcoming the bottleneck of trusted modeling communication and insufficient resource utilization at the scale of large models and big data. This method can ensure effective utilization and isolation of resources, reduce memory latency, increase pre-training speed, and enhance data confidentiality, integrity, and security.

The present invention adopts the remote direct memory access (RDMA) protocol to realize high-speed communication between different TEEs, and introduces CXL technology to realize high-speed interconnection between GPU and TEE, reduce memory delay, and improve pre-training speed.

In order to ensure the security of the GPU, the present invention adds a TPM chip to the FPGA to form a trusted controller, and dynamically distributes permission policies according to the training tasks. By verifying the integrity and correctness of the GPU driver, it ensures that the GPU starts up The process is in a safe state and the exclusivity of the GPU execution process is maintained to prevent malicious attacks and data tampering. At the same time, after the GPU completes the task, it is routed to the TEE environment through the FPGA and the sensitive data in the GPU on-chip memory is cleared to solve the problem of video memory Residual issues.

This method introduces a global clock, a multi-party key agreement mechanism and random number seed distribution to perform fragmentation and obfuscation of shared memory data, and combines encryption and MAC authentication to enhance data confidentiality, integrity and security. In the large model pre-training stage, public data and private data are selected for federated modeling, and a hierarchical aggregation strategy is adopted, and finally gradient aggregation is performed in the public area, thus reducing the risk of gradient leakage.

The present invention adopts multi-party secret sharing and multi-copy encrypted storage and persistence methods to ensure the security and recoverability of intermediate data during model training and reduce the risk of data loss.

In summary, the present invention can effectively deal with many challenges in large language model pre-training, provides a credible and efficient solution for practical application scenarios of large model multi-party federated modeling, and has high practical value and wide application. prospect.

Description of drawings

Figure 1 is a schematic diagram of the node composition of the large model federated modeling of the present invention.

Figure 2 is a schematic connection diagram of the memory sharing protocol of the present invention.

Figure 3 is a schematic diagram of the node function composition of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings.

As shown in Figure 1, according to the actual needs of large language model pre-training multi-party federated modeling tasks, a cross-domain Trusted Execution Environment (TEE) cluster is constructed in a distributed environment, and the memory is divided into shared areas and private areas. The shared area is formed into a large memory to load large models and training data for pre-training, thus overcoming the problem of trusted modeling resource limitations in the scale of large models and big data.

Set the global clock to achieve high-speed memory access through RDMA and CXL protocols, make full use of GPU and TEE environment resources, and effectively reduce performance bottlenecks caused by memory delays. Using FPGA controller and security module, it integrates multiple mechanisms such as multi-party key negotiation, random number seed distribution, data fragmentation obfuscation, encryption and MAC authentication to ensure the credibility of the GPU, as well as the privacy and security during the distributed training process. safety. In addition, a hierarchical aggregation strategy is adopted to reduce the risk of gradient leakage. Checkpoints are set up and multi-copy encrypted persistent storage is used for intermediate results to improve training reliability.

Among them, the main function of the large language model is to process, understand and generate natural language, obtain rich language knowledge and patterns in the pre-training stage, and provide strong support for various downstream NLP tasks; the joint modeling participants are those who participate in the model Organizations or institutions that jointly train, contribute training data, model parameters and computing resources, and collaboratively train large language models to improve pre-training performance;

The TEE environment provides an isolated and secure computing environment. Each TEE cluster node has a private memory area and a shared memory area. The TEE has functions such as remote authentication, authority control, security module, and secure interconnection to ensure the security of the computing environment;

The memory division means that the TEE cluster nodes divide the memory into private areas and shared areas to ensure the isolation and confidentiality of data between different nodes;

The remote authentication function ensures that TEE nodes can perform remote authentication, verify the identities of other nodes, and ensure that only legal nodes can access their resources; the security module is a security-related function built into the TEE environment, including encryption and decryption, MAC operations, secret The sharing function is used for secure transmission and processing of data, as well as privacy protection.

As shown in Figure 2: Secure interconnection realizes secure high-speed communication between TEE cluster nodes and between TEE and GPU through FPGA controller, supports encryption and authentication, and ensures the security of high-speed data transmission and authority authentication.

The FPGA controller implements GPU trustworthiness and secure communication and authority authentication between the TEE and the outside. By verifying the integrity and correctness of the GPU driver, it ensures that the GPU is in a safe state during the startup process and maintains the integrity of the GPU during execution. Exclusivity to prevent malicious attacks and data tampering. At the same time, after the GPU completes the task, it is routed to the TEE environment through the FPGA, and sensitive data in the GPU on-chip memory is cleared to solve the problem of residual video memory, which mainly includes TPM chip management, RDMA memory sharing, CXL secure communication and other functions;

The TPM chip management is used by the TPM chip in the FPGA controller to manage the relevant keys and certificates of trusted computing to ensure the security and credibility of the TEE cluster;

The RDMA memory sharing is through the RDMA protocol and the FPGA controller realizes high-speed memory sharing between TEE clusters to accelerate data transmission and communication;

The CXL secure communication is the introduction of CXL technology into the FPGA controller to achieve high-speed and secure interconnection between GPU and TEE, improve pre-training speed and reduce memory latency.

The GPU is responsible for accelerating training of large language models to improve training efficiency and speed, and is controlled and verified by the FPGA controller to achieve GPU credibility.

The TEE cluster is composed of multiple trusted execution environments (TEEs). The TEE cluster belongs to different joint modeling participants participating in model joint training. The federated modeling joint modeling participants design the interconnection of TEE nodes. Provide remote authentication and permission control functions; the TEE cluster remote authentication ensures that only certified nodes can participate in the pre-training process, and designates some nodes to interact with other external joint modeling participants TEE cluster environment; the TEE cluster permissions The main control function is that the TEE cluster can dynamically allocate permissions based on different training tasks and joint modeling participants to ensure that each node can only access the data and resources it has permissions for.

As shown in Figure 3: the large language model joint pre-training task runs in the TEE cluster involving multiple parties, including computing nodes and aggregation/management nodes;

The computing nodes are used to perform model training and model training acceleration tasks, and perform training in a trusted execution environment to ensure the security and credibility of model training; the computing nodes are participating nodes and are responsible for local model training calculations.

The aggregation/management node is responsible for the management and scheduling of distributed training of large language models as well as gradient aggregation and model distribution, including model loading, data loading, data obfuscation, gradient aggregation, parameter update, model distribution, key management and model persistence;

The model loading is an operation of loading a large language model from the storage medium to the TEE cluster;

The data loading is to load the data required for large language model pre-training into the designated memory of the TEE cluster according to privacy security; the data obfuscation is responsible for performing a joint obfuscation operation on the data stored in the shared area to Eliminate the referentiality and identification of data and use a unified key for encryption;

The gradient aggregation function is responsible for aggregating the gradients calculated by the distributed training TEE nodes to update the model parameters;

The parameter update function is to use the gradient obtained by aggregation to update the model parameters for use in the next round of training;

The model distribution is to distribute the updated model parameters to each TEE training node using the RDMA protocol;

The key management function is to introduce a global clock into the system as a time reference input, combine it with a random number seed to generate a shared symmetric key, and manage it;

The model persistence is to store intermediate results such as model parameters in multiple copies encrypted and persistently, so that the training progress can be restored and the risk of data loss can be restored when an abnormality occurs during the training process. The global clock serves as the clock synchronization mechanism of the distributed system to ensure the time consistency of each node and is used for the area identification of the memory as part of the key.

The method process provided by the present invention will be described in detail below with reference to specific examples.

Step 1. Preparing the large language model joint pre-training modeling environment:

Preparing the large language model joint pre-training modeling environment includes the following steps:

Step 101. Confirm the pre-training tasks: Define the specific tasks for pre-training the large language model, including the initial parameter configuration of the model and training data requirements.

Step 102. Determine the joint modeling participants: Determine the joint modeling participants participating in the joint pre-training of the large language model, including model owners and data providers.

Step 103. Build a cross-domain TEE cluster: In a distributed environment, build a cross-domain Trusted Execution Environment (TEE) cluster. Each joint modeling participant starts the TEE management node and plans shared areas, private areas, and GPU resources in the TEE. , and load the access permission information into the FPGA to control access to RDMA and CXL.

Step 104. Set the global clock: Introduce the global clock as a unified time scale and distribute N random number seeds to all joint modeling participants to ensure time consistency and data security.

Step 105. Encryption and MAC authentication: The initiator of the large language model federated pre-training task selects a random seed to obtain the global clock generation key, encrypts the initial model, generates a MAC message authentication code, and combines the encrypted initial model and Relevant identification information is placed into the shared memory area to prepare for model distribution.

Step 106. Training data loading and processing: Each joint modeling participant loads the data into the private area and shared area of the TEE according to the data sensitivity, tokenizes the data to form a vector representation, obtains the global clock, and selects local random Seeds encrypt data in shared areas to enhance data confidentiality.

Step 107. Data aggregation and obfuscation: The initiator of the large language model federated pre-training task aggregates the data in the TEE shared area, generates a key based on the random seed number identified by the metadata and the global clock, and uses the key to convert the data Decrypt, then perform data obfuscation to blur the source of the data. By obtaining the global clock, select local random seeds to re-encrypt the data in the shared area, generate a public data set, and store it in the shared memory area.

Step 108. Public data set pulling: Each joint modeling participant pulls the data in the shared area into their respective local environments through the RDMA protocol to prepare for subsequent training.

Step 109. GPU initialization: Each joint modeling participant initializes the GPU to make it a trusted computing resource to accelerate the model training process.

The present invention provides a large language model federated pre-training method based on a trusted execution environment, which is used for joint pre-training modeling of large language models, including:

Step 2. Joint pre-training modeling of large language models:

Joint pre-training modeling of large language models includes the following steps:

Step 201. Model initialization and memory transfer: Each joint modeling participant passes the initial model through the FPGA controller and uses the RDMA protocol to directly copy the memory of the model to the memory of the respective TEE cluster to ensure that the initial model is loaded into a trusted environment. .

Step 202. Data selection and preparation: The joint modeling participants randomly select a part of the data from the public data and use the user's private data as the training data for this round of training.

Step 203. Accelerated calculation and data transmission: The FPGA controller locks the GPU for exclusive use, routes part of the program and training data that need to be accelerated through the FPGA, and performs decryption operations. The data is transmitted to the GPU using CXL technology for high-speed processing. Calculation and processing.

Step 204. Distributed parallel training: In the local TEE cluster, multiple sets of data are used for distributed parallel training. The joint modeling participants use different strategies to process the data in the public area and the private area, calculate the gradient, and perform the training in the private area. Gradient aggregation is completed in the TEE cluster, and then the key is generated through the global clock and the selected random seed for encryption, the result is stored in the public shared memory, and all joint modeling participants are notified.

Step 205. Clear sensitive memory: After this round of calculation is completed, use the FPGA controller to clear the sensitive memory on the GPU chip to ensure data security.

Step 206. Global aggregation and model update: The aggregation node is responsible for aggregating the gradient data in the shared memory and then updating the model parameters.

Step 207. Model distribution: The aggregation node puts the updated model into the public area, encrypts it using a random number seed and a global clock, and prepares it for model distribution for subsequent training.

Step 208. Download and train the updated model: Each of the joint modeling participants downloads the new model and uses it for a new round of model training until the model reaches a convergence state.

Step 209. Encrypted storage of intermediate results: The task is triggered regularly. The aggregation node generates three copies of the model and training status data of the intermediate results, and uses random keys for encrypted storage. Then, the Shamir secret sharing algorithm is used to store the random keys. Key shards are distributed to each joint modeling participant for future key recovery.

Step 210. Abnormal situation handling: When an abnormal situation occurs, the parties jointly restore the secret fragments they own and perform decryption operations to restore the intermediate results and ensure the continuity and reliability of training.

Step 3. Optimization of joint pre-training large language model:

The optimization of jointly pretrained large language models includes the following steps:

Step 301: After completing the training of the large language model joint pre-training task, deploy it to the actual application environment for use;

Step 302: Collect feedback data, continue to increase training data, and continuously improve model accuracy and training efficiency.

The above-described embodiment is only one of the specific implementation modes of the present invention, and ordinary changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims (6)

1. A large language model federated pre-training method based on a trusted execution environment, which is characterized by including the following steps: Step 1. Create a large language model joint pre-training task, determine joint modeling participants, prepare data, and create computing and storage network resources for joint modeling; Step 2. Conduct joint pre-training of large language models; Step 3. Optimize the large language model obtained by joint pre-training. 2. A large language model federated pre-training method based on a trusted execution environment according to claim 1, characterized in that said step 1 specifically includes the following steps: Step 101. Confirm the pre-training tasks: Define the specific tasks of large language model pre-training, including the initial parameter configuration of the large language model and training data requirements; Step 102. Determine the joint modeling participants: Determine the joint modeling participants participating in the joint pre-training of the large language model, including the owners and data providers of the large language model; Step 103. Build a cross-domain TEE cluster: In a distributed environment, build a cross-domain Trusted Execution Environment (TEE) cluster. Each joint modeling participant starts the TEE management node and plans shared areas, private areas, and GPU resources in the TEE. , and load the access permission information into the FPGA to control access to RDMA and CXL; Step 104. Set the global clock: Introduce the global clock as a unified time scale and distribute N random number seeds to all joint modeling participants to ensure time consistency and data security; Step 105. Encryption and MAC authentication: The initiator of the large language model federated pre-training task selects a random seed to obtain the global clock generation key, encrypts the initial model, generates a MAC message authentication code, and sends the encrypted initial model and related identifiers Information is placed into the shared memory area to prepare for model distribution; The shared memory area is divided into a private area and a shared public area. The private area stores private data of participants participating in federated modeling and joint modeling, which is sensitive data; the shared public area is used for non-sensitive data and modeling processes. Efficient and fast sharing of metadata, global model parameters, and public data sets; Step 106. Training data loading and processing: Each of the joint modeling participants loads the data into the private area and shared area of the TEE according to the data sensitivity, tokenizes the data to form a vector representation, obtains the global clock, and selects Local random seeds encrypt data in the shared area; Step 107. Data aggregation and obfuscation: The task initiator aggregates the data in the TEE shared area, generates a key based on the random seed number identified by the metadata and the global clock, and uses the key to decrypt the data, and then performs data obfuscation. , blur the source of the data, obtain the global clock, select local random seeds to re-encrypt the data in the shared area, generate a public data set, and store it in the shared memory area; Step 108. Pulling public data sets: Each of the joint modeling participants pulls the data in the shared area into their respective local environments through the RDMA protocol to prepare for subsequent training; Step 109. GPU initialization: Each of the joint modeling participants initializes the GPU to make it a trusted computing resource for accelerating the training process of the model. 3. A large language model federated pre-training method based on a trusted execution environment according to claim 2, characterized in that the step 107 specifically includes; Data aggregation: The task initiator obtains data from the TEE shared area of each joint modeling participant. The task initiator uses the global clock and the random seed number identified by the metadata to generate a key. Using the generated key, the task initiator pairs the data from The data obtained by each TEE is decrypted and the original, encrypted data is restored to its original state; Data obfuscation: Based on decryption, the task initiator obfuscates the data; Re-encryption: The obfuscated data is re-encrypted, and a new key is generated again using the global clock and the local random seed of the task initiator; A public dataset is generated: the processed, obfuscated, and re-encrypted data are combined into a common dataset. 4. A large language model federated pre-training method based on a trusted execution environment according to claim 2, characterized in that step 2 specifically includes the following steps: Step 201. Model initialization and memory transfer: Each of the joint modeling participants passes the initial model through the FPGA controller and uses the RDMA protocol to directly copy the memory of the initial model to the memory of their respective TEE clusters to ensure that the initial model is loaded into a trusted environment. middle; Step 202. Data selection and preparation: The joint modeling participants randomly select a part of the data from the public data and use the user's private data as the training data for this round of training; The public data refers to user non-sensitive data or corresponding data desensitization, or data that has been published from the outside. These data are sent by each user to the central node for data obfuscation, and then form a basic public data set; Step 203. Accelerated calculation and data transmission: The FPGA controller locks the GPU for exclusive use, routes part of the program and training data that need to be accelerated through the FPGA, and performs a decryption operation. The data is transmitted to the GPU using CXL technology for processing. calculation and processing; Step 204. Distributed parallel training: In the local TEE cluster, multiple sets of data are used for distributed parallel training. The joint modeling participants use different strategies to process the data in the public area and the private area, calculate the gradient, and perform the training in the private area. Gradient aggregation is completed in the TEE cluster, and then the key is generated through the global clock and the selected random seed for encryption, the results are stored in the shared memory in the public area, and all joint modeling participants are notified; The shared memory is an area that forms a large memory for each node to directly read and write the memory to improve processing speed; Step 205. Clear sensitive memory: After this round of calculation is completed, use the FPGA controller to clear the sensitive memory on the GPU chip to ensure data security; Step 206. Global aggregation and model update: The aggregation node is responsible for aggregating the gradient data in the shared memory and then updating the model parameters; The gradient data is used by the joint modeling participants of federated modeling to perform local forward calculations using local data sets, and then calculates the gradients of parameters according to the set loss function, and then uploads these gradients to the aggregation node; Step 207. Model distribution: The aggregation node puts the updated model into the public area, encrypts it using random number seeds and global clocks, and prepares it for model distribution for subsequent training; Step 208. Download and train the updated model: Each of the joint modeling participants downloads the new model from the aggregation node, and the aggregation node updates the global model and sends it directly to the joint modeling participants, who then pull it. The method adopted is used to accelerate execution and be used for a new round of model training until the model reaches a convergence state; Step 209. Encrypted storage of intermediate results: The task is triggered regularly, and the aggregation node will continue to iteratively train, and the model of the intermediate results of the parameters and the data of the training situation will be continuously aggregated to generate three copies, and a random key will be used for encrypted storage; then, use Shamir secret sharing algorithm distributes random key fragments to each joint modeling participant for future key recovery; Step 210. Abnormal situation handling: When an abnormal situation occurs, each of the joint modeling participants jointly restores the secret fragments they own and performs decryption operations to restore the intermediate results and ensure the continuity and continuity of training. reliability. 5. A large language model federated pre-training method based on a trusted execution environment according to claim 4, characterized in that the FPGA controller implements GPU credibility and secure communication and authority authentication between the TEE and the outside, FPGA controller includes TPM chip management, RDMA memory sharing, and CXL secure communication; The TPM chip management is used by the TPM chip in the FPGA controller to manage the relevant keys and certificates of trusted computing to ensure the security and credibility of the TEE cluster; The RDMA memory sharing is through the RDMA protocol and the FPGA controller realizes high-speed memory sharing between TEE clusters to accelerate data transmission and communication; The CXL secure communication realizes high-speed and secure interconnection between GPU and TEE, improves pre-training speed and reduces memory latency; The GPU is responsible for accelerating training of large language models to improve training efficiency and speed, and is controlled and verified by the FPGA controller to achieve GPU credibility. 6. A large language model federated pre-training method based on a trusted execution environment according to claim 4, characterized in that said step 3 specifically includes the following steps; Step 301: After completing the training of the large language model joint pre-training task, deploy it to the actual application environment for use; Step 302: Collect feedback data, continue to increase training data, and continuously improve model accuracy and training efficiency.