JP2024040614A - Technology to reduce switching time of learning models and networks in deep learning - Google Patents

Abstract

[Problem] To provide a method for reducing the service time for users while suppressing the increase in the number of computers by switching between a trained model and an inference network that match desired characteristics without delay on a single computer. [Solution] The method involves loading multiple pairs of inference networks and trained data that differ for each algorithm into the CPU/GPU memory of a single computer, switching between the inference set work and the trained data, and using a single computing unit. Make inferences. [Selection diagram] Figure 2

Description

The present invention is a software technology for switching between a trained model and an inference network without loss of time in deep learning (hereinafter abbreviated as DL).

When inferring images, sounds, and sentences using DL, images, sounds, and sentences unique to the model are inferred from a trained model based on the characteristics of the model. If the training data differs, the characteristics of the trained model will also change. To obtain the characteristics necessary for inference, select a trained model with the desired characteristics and process it on a computer with an inference network that can process it. When changing a trained model or inference network to change the desired characteristics, either reload the trained model or inference network to the computer or prepare multiple computers with a single trained model and inference network. Must. In the former case, switching time occurs each time the desired characteristics differ, and in the latter case, it is necessary to increase the number of computers as the number of trained models increases.

In this invention, we have developed a method that reduces service time to users while suppressing the increase in the number of computers by switching between trained models and inference networks that match the desired characteristics without delay on a single computer.

The size of trained models varies greatly depending on the size of training data and algorithm. Additionally, the inference network is larger in size, resulting in large delays when switching. Figure 1 shows the processing flow of the conventional method. In the reloading method, switching can be easily achieved by reloading the trained model or inference network to the computer, but there is a delay. As a means of solving the problem of reducing delays, if you switch computers using the conventional technique of preparing a computer for each trained model or inference network, you can switch computers without reloading and without delay. A computer is required for each model and inference network, which increases computer resources. In this invention, we focus on the rapid improvement in computing power of computers and the large expansion of computer resources per computer, and we load the necessary trained models and inference networks into the memory resources of expanded computers in advance and classify them. We devised a method to switch between trained models and inference networks without reloading them by specifying them from the processing software. Figure 2

By using this invention, delays in switching trained models and switching inference networks to obtain inference data with required characteristics can be reduced, and multiple This makes it possible to switch between trained models and inference networks without delay, greatly contributing to improved operability, reduced user waiting time when providing services, and reduced processing time in industrial computer systems that use deep learning models. It became so.

FIG. 2 is an explanatory diagram of a conventional method using a trained model and an inference network. FIG. 2 is an explanatory diagram of a method of using a trained model and an inference network improved by the present invention. (Example 1) FIG. 2 is an explanatory diagram of a usage method for switching only the trained model in the trained model and inference network improved by the present invention. (Example 2)

The invention has a program code as a means for reading a trained model and a means for classifying it, and requires computer resources capable of reading all necessary trained models, that is, main memory of a CPU or GPU memory, a trained model, and an inference network. Consists of software (selector) for switching.

FIG. 2 (Embodiment 1) is a specific data processing flow for utilizing the present invention. A trained model and its inference network form a pair, and inference processing is executed when both exist on a computer. Furthermore, the inference network is different for each learning algorithm, and a trained model exists for each inference network. In the present invention, a single computer has large computer resources, a trained model and an inference network are read in advance as a single combination, and the calculation unit uses a selector to combine the trained model and inference network within the computer. By using them properly, you can process multiple inference networks and trained models.

FIG. 3 (Embodiment 2) An infinite number of learned models based on a specific algorithm can be generated for each learning data. On the other hand, since there is only one type of inference network for each algorithm, a single inference network and multiple trained data are read and classified, and the calculation unit uses a selector to switch between trained models and perform inference. It is possible to perform inference by switching between trained models without using multiple computers. In this embodiment, there is no waste in the operation of the arithmetic unit in the computer, and no switching delay occurs.

Conventionally, a relatively large system was required to handle multiple inference networks and multiple trained models, but according to the present invention, a large number of complex trained models can be handled with a small computer system. , the applications of deep learning will expand. For example, in facial recognition and fingerprint recognition processing systems, it is possible to simultaneously perform face and fingerprint authentication without adding additional computers.

1 Method by reloading 2 Computer switching 3 Trained model 4 Latent network 5 Selector 6 CPU/GPU
7 Main memory 8 GPU memory

Claims (2)

For each pair of inference network and trained data that differs for each algorithm, multiple pairs are loaded into the CPU/GPU memory of a single computer, and inference is performed with a single computing unit by switching between the inference set work and the trained data. Loading a single inference network and multiple trained data into the CPU/GPU memory of a single computer, switching the trained data and performing inference with a single computing unit.

Images