CN116881717A - Data training method, device and medium - Google Patents
Data training method, device and medium Download PDFInfo
- Publication number
- CN116881717A CN116881717A CN202310867942.9A CN202310867942A CN116881717A CN 116881717 A CN116881717 A CN 116881717A CN 202310867942 A CN202310867942 A CN 202310867942A CN 116881717 A CN116881717 A CN 116881717A
- Authority
- CN
- China
- Prior art keywords
- training
- data
- file
- identifier
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 383
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000012986 modification Methods 0.000 claims abstract description 42
- 230000004048 modification Effects 0.000 claims abstract description 42
- 238000004590 computer program Methods 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 10
- 230000003631 expected effect Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 abstract description 10
- 239000002699 waste material Substances 0.000 abstract description 9
- 238000007792 addition Methods 0.000 description 7
- 238000013473 artificial intelligence Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the technical field of data processing, and discloses a data training method, a device and a medium, wherein training data is firstly obtained, training tasks are run according to the training data, and training results are obtained; when the training result is detected to meet the preset condition, acquiring a modification file; the modification file includes a parameter file and a data file. Compared with the prior art, when parameter adjustment or data addition is carried out, the training model needs to carry out data training again, and time and resources are wasted. By adopting the technical scheme, when the training model is adjusted, the modified file is uploaded to the corresponding directory position according to the type of adjustment, and the initial position of training data through the modified file is confirmed according to the input identifier so as to perform data training. In the technical scheme, the training of the data is continued from the position corresponding to the identifier, so that the time and resource waste caused by the training from the beginning are avoided.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data training method, apparatus, and medium.
Background
The artificial intelligence platform is a service platform of a machine learning integrated development environment for reasoning and training, and is used for providing product design, optimization and training for other artificial intelligence products. A good artificial intelligence product requires an excellent algorithmic support. The advent of an excellent algorithm often requires extensive computation and is perfected by numerous parameter corrections and data additions.
In the prior art, in order to quickly acquire the accuracy of the training model in the initial stage of training the algorithm model, smaller data are often used, and the parameters of the model are modified or the training data size is increased according to the training result each time. Along with the stability of the training model, the whole training period is long, and when parameter adjustment or data increase is carried out in the later period, the trained training data needs to be retrained, so that more time and resources are consumed.
It can be seen how to avoid repeated training of historical training data, and avoiding time and resource waste is a problem to be solved by those skilled in the art.
Disclosure of Invention
The invention aims to provide a data training method, a data training device and a medium, which are used for avoiding repeated training of historical training data and avoiding time and resource waste.
In order to solve the technical problems, the present invention provides a data training method, including:
acquiring training data;
running a training task according to the training data and acquiring a training result;
when the training result is detected to meet the preset condition, acquiring a modification file; the modification file comprises a parameter file and a data file;
uploading the modified file to a corresponding directory position according to the type of the modified file;
and confirming the starting position of training data through the modification file according to the input identifier so as to perform data training.
In one aspect, the modification file is a parameter file;
correspondingly, the training result meeting the preset condition comprises:
the training result does not conform to the expected output;
correspondingly, the uploading the modified file to the corresponding directory location according to the type of the modified file includes:
and uploading the parameter file to a catalog position mounted after the training task is operated.
On the other hand, the identifier is: marking a training result, and outputting a identifier used by a data position which accords with an expected effect;
further, the input identifier is: among the respective identifiers, the one closest to the current time is the one.
On the other hand, the modification file is a data file;
correspondingly, the training result meeting the preset condition comprises:
the training result of the training data accords with the expected output;
correspondingly, the uploading the modified file to the corresponding directory location according to the type of the modified file includes:
and uploading the data file to the same directory location as the training data.
On the other hand, the identifier is: an end identifier of the training data;
further, the method further comprises the following steps: the start position of the data file is marked with the end identifier.
In another aspect, there is coincident data between the training data and the data in the data file.
In another aspect, the method further comprises:
acquiring a task running state instruction;
if the task running state instruction is an ending instruction, stopping training after the training task is completed;
and if the task running state instruction is a continuing instruction, continuing training of other training tasks after the training task is completed.
In order to solve the technical problem, the present invention further provides a data training device, including:
the acquisition module is used for acquiring training data;
the operation module is used for operating the training task according to the training data and obtaining a training result;
the detection module is used for acquiring a modification file when the training result is detected to meet the preset condition; the modification file comprises a parameter file and a data file;
the uploading module is used for uploading the modified file to a corresponding directory position according to the type of the modified file;
and the confirming module is used for confirming the starting position of the training data through the modification file according to the input identifier so as to carry out data training.
On the other hand, the detection module is used for acquiring a parameter file when the training result does not accord with the expected output;
correspondingly, the uploading module is used for uploading the parameter file to the catalog position mounted after the training task is operated.
On the other hand, the identifier in the confirmation module is: marking a training result, and outputting a identifier used by a data position which accords with an expected effect;
further, the input identifier is: among the respective identifiers, the one closest to the current time is the one.
On the other hand, the detection module is used for acquiring a data file when the training result of the training data accords with the expected output;
correspondingly, the uploading module is used for uploading the data file to the same directory position as the training data.
On the other hand, the identifier in the confirmation module is: an end identifier of the training data;
further, the method further comprises the following steps: and the marking module is used for marking the starting position of the data file by using the ending identifier.
On the other hand, the data in the data file acquired in the detection module and the training data have coincident data.
In another aspect, the method further comprises:
the processing module is used for acquiring a task running state instruction; if the task running state instruction is an ending instruction, stopping training after the training task is completed; and if the task running state instruction is a continuing instruction, continuing training of other training tasks after the training task is completed.
In order to solve the technical problem, the invention also provides a data training device, which comprises a memory for storing a computer program;
and a processor for implementing the steps of the data training method as described above when executing the computer program.
To solve the above technical problem, the present invention further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the data training method as described above.
According to the data training method provided by the invention, training data is firstly obtained, training tasks are run according to the training data, and training results are obtained; when the training result is detected to meet the preset condition, acquiring a modification file; the modification file comprises a parameter file and a data file; uploading the modified file to a corresponding directory position according to the type of the modified file; based on the inputted identifier, the start position of the training data by modifying the file is confirmed to perform data training. Compared with the prior art, when parameter adjustment or data addition is carried out, the training model needs to carry out data training again, and time and resources are wasted. By adopting the technical scheme, when the training model is adjusted, the modified file is uploaded to the corresponding directory position according to the type of adjustment, and the initial position of training data through the modified file is confirmed according to the input identifier so as to perform data training. In the technical scheme, the training of the data is continued from the position corresponding to the identifier, so that the time and resource waste caused by the training from the beginning are avoided.
In addition, the data training device and the medium provided by the invention correspond to the data training method, and have the same effects.
Drawings
For a clearer description of embodiments of the present invention, the drawings that are required to be used in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.
FIG. 1 is a flowchart of a data training method according to an embodiment of the present invention;
FIG. 2 is a flowchart of an application of parameter file adjustment according to an embodiment of the present invention;
FIG. 3 is a flowchart of an application of data file adjustment according to an embodiment of the present invention;
FIG. 4 is a block diagram of a data training device according to an embodiment of the present invention;
fig. 5 is a block diagram of another data training apparatus according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without making any inventive effort are within the scope of the present invention.
The core of the invention is to provide a data training method, a device and a medium, which are used for avoiding repeated training of historical training data and avoiding time and resource waste.
In order to better understand the aspects of the present invention, the present invention will be described in further detail with reference to the accompanying drawings and detailed description.
Firstly, it should be noted that the data training method provided by the invention is mainly applied to an artificial intelligent platform, the establishment of an algorithm model is realized through the artificial intelligent platform, and a more accurate and convenient data base is provided for the algorithm model through the data training method provided by the invention. The execution main body of the data training method provided by the invention can be a data training device, which can be a processor and the like, and a technician can realize data training through man-machine interaction with the data training device.
Fig. 1 is a flowchart of a data training method according to an embodiment of the present invention, as shown in fig. 1, where the method includes:
s10: acquiring training data;
s11: running a training task according to the training data and obtaining a training result;
s12: when the training result is detected to meet the preset condition, acquiring a modification file; the modification file comprises a parameter file and a data file;
s13: uploading the modified file to a corresponding directory position according to the type of the modified file;
s14: based on the inputted identifier, the start position of the training data by modifying the file is confirmed to perform data training.
In a specific implementation, when training one training data, in order to more quickly confirm the accuracy of the training model, a small part of data can be trained first, and then the data size is increased when the training result accords with the expected effect. Therefore, specifically, in this embodiment, after the training data is obtained, the method further includes: dividing training data into a plurality of parts, and further sequentially submitting running training tasks; when the training result of a certain training task does not accord with the expected output, the parameter information is timely adjusted, and the accuracy of the algorithm model is ensured. When running training tasks, the training tasks need to be submitted in turn according to the sequence of training data so as to ensure the consistency and accuracy of the training model.
After the training tasks are run, the log file records training results of each training task, and can record the success or failure of training of the training tasks or record names or marks of the training tasks which succeed in training. The judgment of success or failure of the data training can be based on whether the output result of the training meets the expected output.
The embodiment provides a method for logging training results, which marks the positions of data training through case_id, and can confirm the positions of data training through reading the case_id in the running of training tasks.
When the training result is detected to meet the preset condition, acquiring a modification file; the modification file includes a parameter file and a data file. The parameter file is mainly a parameter in a training model and is used for guaranteeing the accuracy of an algorithm. The data file is mainly newly added training data and is used for expanding the data volume of training and improving the accuracy of an algorithm. The detection of whether the training result meets the preset condition can be that a technician inputs a related instruction or that a certain condition is detected by the data training device. Specifically, the obtained modification file is mainly aimed at parameter adjustment and training data quantity adjustment, so that the training result preset condition is also aimed at the condition that parameters need to be adjusted or data quantity needs to be increased. For example, the training results deviate, parameters need to be adjusted at this time, or training data training is completed, and new data needs to be added for training at this time.
It will be appreciated that for different situations, different modified files need to be obtained, and their storage locations are different according to the types of the modified files, so that the modified files need to be uploaded to corresponding directory locations. Such as the location of the parameter deposit and the location of the data deposit.
After uploading the modified file to the corresponding position, the technician can start the training task from the position corresponding to the identifier after inputting the identifier, and continue data training through the modified file. The identifier here may be a symbol for marking the training position of the data in the above description, or may be another symbol (for example, a symbol for marking the end or start of the data). Through the identifier, technicians and the data training device can confirm the starting position of the training task.
According to the data training method provided by the embodiment of the invention, training data is firstly obtained, training tasks are run according to the training data, and training results are obtained; when the training result is detected to meet the preset condition, acquiring a modification file; the modification file comprises a parameter file and a data file; uploading the modified file to a corresponding directory position according to the type of the modified file; based on the inputted identifier, the start position of the training data by modifying the file is confirmed to perform data training. Compared with the prior art, when parameter adjustment or data addition is carried out, the training model needs to carry out data training again, and time and resources are wasted. By adopting the technical scheme, when the training model is adjusted, the modified file is uploaded to the corresponding directory position according to the type of adjustment, and the initial position of training data through the modified file is confirmed according to the input identifier so as to perform data training. In the technical scheme, the training of the data is continued from the position corresponding to the identifier, so that the time and resource waste caused by the training from the beginning are avoided.
Based on the above embodiments, this embodiment provides a specific model adjustment method. In this embodiment, the modification file is a parameter file;
correspondingly, the training result meeting the preset condition comprises:
the training result does not conform to the expected output;
correspondingly, uploading the modified file to the corresponding directory location according to the type of the modified file includes:
and uploading the parameter file to a catalog position mounted after the training task is operated.
It can be appreciated that in the present invention, since the technician can acquire the training result in real time through the log, when the training result is found to be inconsistent with the expected output, the parameter adjustment is required. Thus, it is necessary to obtain a parameter file for modifying parameters in the current model by training results.
Fig. 2 is an application flowchart of parameter file adjustment according to an embodiment of the present invention, where, as shown in fig. 2, the application flowchart includes:
s20: submitting a training task;
s21: whether to run a training task; if yes, go to step S220, if no, go to step S221;
s220: outputting an operation result; s221: waiting for operation;
s23: whether to update the parameters; if yes, go to step S24; if not, the process proceeds to step S25.
S24: uploading the parameter file to the corresponding directory position, and inputting the identifier;
s25: the task runs.
It will be appreciated that in a specific implementation, after the training data is obtained and the training task is submitted, the technician may select whether to start training or whether to run the training task, which may be actively controlled by the technician, for example, by inputting a related instruction, or clicking a corresponding button to indicate that the training task is run, or starting to run the training task when other judgment conditions are met (for example, reaching a set time), or else waiting. After the training task is run, a running result can be obtained, and a technician can confirm whether to update parameters according to the running result, and can specifically confirm whether the training result accords with expected output. When the parameters need to be updated, a technician uploads the parameter file to the corresponding directory position, inputs the identifier, clicks the task to run, and the training model can continue to train data from the position corresponding to the identifier.
The directory location refers to a directory location where the parameter file is uploaded to be mounted after the training task is run. The identifiers are: marking a training result, and outputting a identifier used by a data position which accords with an expected effect; further, the input identifier is: among the respective identifiers, the one closest to the current time is the one.
It can be understood from the description of the above embodiment that, in this embodiment, after the parameter is changed, the data training is continued from the position corresponding to the input identifier, while in this embodiment, the position of the identifier is marked by the position where the training result accords with the expected output, after each output of a training result that accords with the expected output, one identifier is added to the position, after the parameter is updated, the user can input the latest identifier, and the training task can start training from the position, thereby implementing retraining on the data that fails to be trained, and avoiding repeated training on the data that succeeds in training. Of course, in other embodiments, training may also be started from the first few identifiers at the current time, as desired by the technician.
Based on the above embodiments, this embodiment provides another specific method for model adjustment. In this embodiment, the modification file is a data file;
correspondingly, the training result meeting the preset condition comprises:
the training result of the training data accords with the expected output;
correspondingly, uploading the modified file to the corresponding directory location according to the type of the modified file includes:
the data file is uploaded to the same directory location as the training data.
When data addition is performed, the corresponding identifiers are: an end identifier of the training data;
further, the method further comprises the following steps: the start position of the data file is marked with an end identifier.
The above embodiment provides a data training method after parameter adjustment, and the present embodiment provides a training method after adding data. It will be appreciated from the description of the above embodiments that, in order to more quickly confirm the training model, the training data may be divided into multiple pieces and multiple training tasks for training. Thus, in this embodiment, after one training task is trained, other training tasks need to be trained. Or after the whole training data is trained, new data are acquired for training. At this time, the acquired modification file is a data file. The premise of being able to continue data training is that the training of the historical training data is in accordance with the expected output, so in this embodiment, the data file is acquired to continue training after the training result of the training data is in accordance with the expected output. The uploading position of the data file is the same directory position as the training data so as to realize that the training of the data in the data file is continued after the training of the training data is completed.
FIG. 3 is a flowchart of an application of data file adjustment according to an embodiment of the present invention, similar to FIG. 2, including:
s30: submitting a training task;
s31: whether to run a training task; if yes, go to step S320, if no, go to step S321;
s320: outputting an operation result; s321: waiting for operation;
s33: whether to update the parameters; if yes, go to step S34; if not, the process proceeds to step S35.
S34: uploading a data file to a corresponding directory position, and inputting a identifier, a name of training data and a name of the data file;
s35: the task runs.
In this embodiment, the application flowchart of the data file adjustment is different from the application flowchart of the parameter file adjustment in that when the parameter adjustment is performed, training is continued for the training data from the position of the identifier, and when the data adjustment is performed, after the newly added data is acquired, the name of the training data needs to be confirmed, the name of the data file is checked, and training is started through the identifier, where the identifier is the coincidence of the training data and the data file, and when the identifier of the training data is detected, the training is performed by switching to the position corresponding to the identifier of the data file. The identifier here may still be a location where the training is successful, but in a specific implementation, there is overlapping data between the training data and the data in the data file in order to ensure the accuracy of the training result. Thus, the identifier may be an end identifier of the training data, which may mark a position before the last data. For example, the training data is audio data, and the length is 10 seconds, and the position of the identifier mark is 9 seconds. The 9-10 second part is the data overlapped with the data file.
On the basis of the above embodiment, in this embodiment, further includes:
acquiring a task running state instruction;
if the task running state instruction is an ending instruction, stopping training after the training task is completed;
if the task running state instruction is a continuing instruction, continuing training of other training tasks after the training task is completed.
In this embodiment, before the training task is run, the task may be selected to be run repeatedly after the running is completed or the task may be ended. The judgment can be specifically performed according to the task running state instruction, and when the task running state instruction is a continuing instruction, the input identifier can be selected to run from a designated position or from the beginning. And stopping training after the training task is completed when the task running state instruction is an ending instruction, and releasing resources.
To sum up, in order to facilitate understanding, the present embodiment provides a specific data training scenario, where in the present embodiment, data to be trained is audio or video and may be divided into several segments. Firstly, submitting tasks and training audio data, and firstly, training by using audio with the duration of 1 minute. The task running state instruction selects a continue instruction.
And after the task is operated, according to a training result, when the deviation of the audio identification is found, finding the case_id which accords with the expected latest according to the case_id printed by the log, adjusting training parameters, and uploading the adjusted file to a specified directory (generally, putting the file under the mounted directory after the task is operated).
Selecting a task operation field, adjusting parameters, inputting a case_id, clicking to determine, and waiting for a task to run from the case_id.
When the audio training result of 1 minute duration meets the expectations, the subsequent audio is required to continue training, and the subsequent audio is uploaded to a specified catalog (generally placed under the same catalog as the audio data of 1 minute duration).
And clicking update data in a task operation field, inputting data_tag, old training data name and new training data name which are automatically generated from the acquired system, and clicking and determining.
And if task training is to be completed, selecting to change the task running ending state from a continuing instruction to an ending instruction.
And according to the output result of the task log, when the training result does not accord with the expected output, repeating the steps.
In the above embodiments, the data training method is described in detail, and the invention further provides a corresponding embodiment of the data training device. It should be noted that the present invention describes an embodiment of the device portion from two angles, one based on the angle of the functional module and the other based on the angle of the hardware.
Fig. 4 is a block diagram of a data training device according to an embodiment of the present invention, as shown in fig. 4, where the device includes:
an acquisition module 10 for acquiring training data;
the operation module 11 is used for operating the training task according to the training data and acquiring a training result;
the detection module 12 is used for acquiring a modification file when the training result is detected to meet the preset condition; the modification file comprises a parameter file and a data file;
an uploading module 13, configured to upload the modified file to a corresponding directory location according to the type of the modified file;
a confirmation module 14, configured to confirm the start position of the training data by modifying the file according to the inputted identifier for training the data.
In other embodiments, the detection module is configured to obtain a parameter file when the training result does not conform to an expected output;
correspondingly, the uploading module is used for uploading the parameter file to the catalog position mounted after the training task is operated.
In other embodiments, the identifier in the validation module is: marking a training result, and outputting a identifier used by a data position which accords with an expected effect;
in other embodiments, the input identifiers are: among the respective identifiers, the one closest to the current time is the one.
In other embodiments, the detection module is configured to obtain a data file when a training result of the training data accords with an expected output;
correspondingly, the uploading module is used for uploading the data file to the same directory position as the training data.
In other embodiments, the identifier in the validation module is: an end identifier of the training data;
further, the method further comprises the following steps: and the marking module is used for marking the starting position of the data file by using the ending identifier.
In other embodiments, the data in the data file acquired in the detection module and the training data have coincident data.
In other embodiments, further comprising:
the processing module is used for acquiring a task running state instruction; if the task running state instruction is an ending instruction, stopping training after the training task is completed; and if the task running state instruction is a continuous instruction, continuing training of other training tasks after the training task is completed.
Since the embodiments of the apparatus portion and the embodiments of the method portion correspond to each other, the embodiments of the apparatus portion are referred to the description of the embodiments of the method portion, and are not repeated herein.
The data training device provided by the embodiment of the invention firstly acquires training data, runs training tasks according to the training data and acquires training results; when the training result is detected to meet the preset condition, acquiring a modification file; the modification file comprises a parameter file and a data file; uploading the modified file to a corresponding directory position according to the type of the modified file; based on the inputted identifier, the start position of the training data by modifying the file is confirmed to perform data training. Compared with the prior art, when parameter adjustment or data addition is carried out, the training model needs to carry out data training again, and time and resources are wasted. By adopting the technical scheme, when the training model is adjusted, the modified file is uploaded to the corresponding directory position according to the type of adjustment, and the initial position of training data through the modified file is confirmed according to the input identifier so as to perform data training. In the technical scheme, the training of the data is continued from the position corresponding to the identifier, so that the time and resource waste caused by the training from the beginning are avoided.
Fig. 5 is a block diagram of another data training apparatus according to an embodiment of the present invention, as shown in fig. 5, where the apparatus includes: a memory 20 for storing a computer program;
the processor 21 is configured to implement the steps of the data training method according to the above embodiment when executing the computer program.
The data training device provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like.
Processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in hardware in at least one of a digital signal processor (Digital Signal Processor, DSP), a Field programmable gate array (Field-Programmable Gate Array, FPGA), a programmable logic array (Programmable Logic Array, PLA). The processor 21 may also comprise a main processor, which is a processor for processing data in an awake state, also called central processor (Central Processing Unit, CPU), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with an image processor (Graphics Processing Unit, GPU) for taking care of rendering and rendering of the content that the display screen is required to display. In some embodiments, the processor 21 may also include an artificial intelligence (Artificial Intelligence, AI) processor for processing computing operations related to machine learning.
Memory 20 may include one or more computer-readable storage media, which may be non-transitory. Memory 20 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used for storing a computer program 201, which, when loaded and executed by the processor 21, is capable of implementing the relevant steps of the data training method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may further include an operating system 202, data 203, and the like, where the storage manner may be transient storage or permanent storage. The operating system 202 may include Windows, unix, linux, among others. The data 203 may include, but is not limited to, an identifier or the like.
In some embodiments, the data training device may further include a display 22, an input/output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.
Those skilled in the art will appreciate that the configuration shown in fig. 5 is not limiting of the data training apparatus and may include more or fewer components than shown.
The data training device provided by the embodiment of the invention comprises a memory and a processor, wherein the processor can realize the following method when executing a program stored in the memory: acquiring training data; running a training task according to the training data and obtaining a training result; when the training result is detected to meet the preset condition, acquiring a modification file; the modification file comprises a parameter file and a data file; uploading the modified file to a corresponding directory position according to the type of the modified file; based on the inputted identifier, the start position of the training data by modifying the file is confirmed to perform data training.
The data training device provided by the embodiment of the invention firstly acquires training data, runs training tasks according to the training data and acquires training results; when the training result is detected to meet the preset condition, acquiring a modification file; the modification file comprises a parameter file and a data file; uploading the modified file to a corresponding directory position according to the type of the modified file; based on the inputted identifier, the start position of the training data by modifying the file is confirmed to perform data training. Compared with the prior art, when parameter adjustment or data addition is carried out, the training model needs to carry out data training again, and time and resources are wasted. By adopting the technical scheme, when the training model is adjusted, the modified file is uploaded to the corresponding directory position according to the type of adjustment, and the initial position of training data through the modified file is confirmed according to the input identifier so as to perform data training. In the technical scheme, the training of the data is continued from the position corresponding to the identifier, so that the time and resource waste caused by the training from the beginning are avoided.
Finally, the invention also provides a corresponding embodiment of the computer readable storage medium. The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps as described in the method embodiments above.
It will be appreciated that the methods of the above embodiments, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored on a computer readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium for performing all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The computer readable storage medium provided by the embodiment of the invention firstly acquires training data, runs training tasks according to the training data and acquires training results; when the training result is detected to meet the preset condition, acquiring a modification file; the modification file comprises a parameter file and a data file; uploading the modified file to a corresponding directory position according to the type of the modified file; based on the inputted identifier, the start position of the training data by modifying the file is confirmed to perform data training. Compared with the prior art, when parameter adjustment or data addition is carried out, the training model needs to carry out data training again, and time and resources are wasted. By adopting the technical scheme, when the training model is adjusted, the modified file is uploaded to the corresponding directory position according to the type of adjustment, and the initial position of training data through the modified file is confirmed according to the input identifier so as to perform data training. In the technical scheme, the training of the data is continued from the position corresponding to the identifier, so that the time and resource waste caused by the training from the beginning are avoided.
The data training method, the data training device and the data training medium provided by the invention are described in detail. In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the invention can be made without departing from the principles of the invention and these modifications and adaptations are intended to be within the scope of the invention as defined in the following claims.
It should also be noted that in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A method of data training, comprising:
acquiring training data;
running a training task according to the training data and acquiring a training result;
when the training result is detected to meet the preset condition, acquiring a modification file; the modification file comprises a parameter file and a data file;
uploading the modified file to a corresponding directory position according to the type of the modified file;
and confirming the starting position of training data through the modification file according to the input identifier so as to perform data training.
2. The data training method of claim 1, wherein the modification file is a parameter file;
correspondingly, the training result meeting the preset condition comprises:
the training result does not conform to the expected output;
correspondingly, the uploading the modified file to the corresponding directory location according to the type of the modified file includes:
and uploading the parameter file to a catalog position mounted after the training task is operated.
3. The data training method of claim 2, wherein the identifier is: marking a training result, and outputting a identifier used by a data position which accords with an expected effect;
further, the input identifier is: among the respective identifiers, the one closest to the current time is the one.
4. The data training method of claim 1, wherein the modification file is a data file;
correspondingly, the training result meeting the preset condition comprises:
the training result of the training data accords with the expected output;
correspondingly, the uploading the modified file to the corresponding directory location according to the type of the modified file includes:
and uploading the data file to the same directory location as the training data.
5. The data training method of claim 4, wherein the identifier is: an end identifier of the training data;
further, the method further comprises the following steps: the start position of the data file is marked with the end identifier.
6. The data training method of claim 5, wherein there is coincident data between the training data and the data in the data file.
7. The data training method of claim 1, further comprising:
acquiring a task running state instruction;
if the task running state instruction is an ending instruction, stopping training after the training task is completed;
and if the task running state instruction is a continuing instruction, continuing training of other training tasks after the training task is completed.
8. A data training device, comprising:
the acquisition module is used for acquiring training data;
the operation module is used for operating the training task according to the training data and obtaining a training result;
the detection module is used for acquiring a modification file when the training result is detected to meet the preset condition; the modification file comprises a parameter file and a data file;
the uploading module is used for uploading the modified file to a corresponding directory position according to the type of the modified file;
and the confirming module is used for confirming the starting position of the training data through the modification file according to the input identifier so as to carry out data training.
9. A data training device comprising a memory for storing a computer program;
processor for implementing the steps of the data training method according to any of claims 1 to 7 when executing said computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the data training method according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310867942.9A CN116881717A (en) | 2023-07-14 | 2023-07-14 | Data training method, device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310867942.9A CN116881717A (en) | 2023-07-14 | 2023-07-14 | Data training method, device and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116881717A true CN116881717A (en) | 2023-10-13 |
Family
ID=88258226
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310867942.9A Pending CN116881717A (en) | 2023-07-14 | 2023-07-14 | Data training method, device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116881717A (en) |
-
2023
- 2023-07-14 CN CN202310867942.9A patent/CN116881717A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10642468B2 (en) | Accessibility lock and accessibility pause | |
| CN108656558A (en) | Apply to the speech control system and sound control method of 3D printer | |
| CN111569427B (en) | Resource processing method and device, storage medium and electronic device | |
| CN109656595A (en) | The method, apparatus and system that client-side program updates | |
| CN116028377A (en) | Automatic test method, device and medium based on jenkins | |
| CN109274709B (en) | Method and system for synchronizing programming works on education operation system | |
| CN106445479B (en) | Information-pushing method and device | |
| CN112269588B (en) | Algorithm upgrading method, device, terminal and computer readable storage medium | |
| CN111401566B (en) | Machine learning training method and system | |
| CN107885781A (en) | A kind of method for edition management and system | |
| CN116881717A (en) | Data training method, device and medium | |
| CN117311784A (en) | AI algorithm container visual manufacturing method, system, medium and equipment | |
| CN110784353A (en) | Network element equipment configuration data migration method and device | |
| CN113741931B (en) | Software upgrading method and device, electronic equipment and readable storage medium | |
| CN115145634A (en) | System management software self-adaption method, device and medium | |
| CN112668293B (en) | Data report processing method | |
| CN111399866A (en) | VNF (virtual network function) packet updating method and device | |
| CN112084753B (en) | Method and system for assisting in editing document | |
| CN111399871B (en) | System updating method, device, equipment and medium of HBA card | |
| JP2004139559A (en) | Device for providing knowledge information | |
| CN114022105A (en) | Data processing method and device, electronic equipment and readable storage medium | |
| JP2017220142A (en) | Programming device | |
| CN112130840A (en) | Intelligent programming implementation method and device for obtaining type of operating equipment | |
| CN104780148A (en) | Server, terminal, system and method for document online operation | |
| CN114860611B (en) | BIOS upgrading test method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |