CN114861937A - Data identification and training method - Google Patents
Data identification and training method Download PDFInfo
- Publication number
- CN114861937A CN114861937A CN202210505216.8A CN202210505216A CN114861937A CN 114861937 A CN114861937 A CN 114861937A CN 202210505216 A CN202210505216 A CN 202210505216A CN 114861937 A CN114861937 A CN 114861937A
- Authority
- CN
- China
- Prior art keywords
- training
- data
- recognition
- identification
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 153
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000011161 development Methods 0.000 claims abstract description 21
- 230000008569 process Effects 0.000 claims abstract description 18
- 230000000694 effects Effects 0.000 claims abstract description 9
- 238000001514 detection method Methods 0.000 claims abstract description 6
- 238000012795 verification Methods 0.000 claims description 22
- 101100153581 Bacillus anthracis topX gene Proteins 0.000 claims description 6
- 101150041570 TOP1 gene Proteins 0.000 claims description 6
- 230000007246 mechanism Effects 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000012790 confirmation Methods 0.000 claims description 2
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000018109 developmental process Effects 0.000 description 16
- 238000002372 labelling Methods 0.000 description 13
- 230000007547 defect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000011218 segmentation Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000009825 accumulation Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 206010021703 Indifference Diseases 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000010992 reflux Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a data identification and training method, which comprises the following steps: step 1, carrying out authentication check on data of the identification request, and judging whether the service exists or not through a service identifier identified by the request after the authentication check is successful; step 2: obtaining the service through the identification mark to form result feedback; and step 3: generating a data set from the data by identifying the historical records, wherein the data set provides a data basis for model training; and 4, step 4: starting a training data set, uploading a training result model after training is finished, and releasing occupied computing resources; and 5: training process data for real-time viewing of training effects; step 6: and the successfully trained model is issued to the cloud platform, and the system automatically schedules the model to a corresponding server node according to the resources required by identification for online preview detection of the training effect. The invention has the beneficial effects that: the dependency of the algorithm on the sample is reduced, the AI development efficiency is greatly improved through cloud training and publishing, and the AI development difficulty is reduced.
Description
Technical Field
The invention relates to the field of AI artificial intelligence, in particular to a data identification and training method.
Background
In the field of AI artificial intelligence, the problem that early defect identification and inspection of the segmentation industry does not have training data and the identification service is available and the AI research and development deployment uses a high gate bar is also ensured. In a specific subdivision industry, training data cannot be acquired in the early stage, and defect detection cannot be provided; the defect sample data is less, the model training result is not ideal, and reliable identification service cannot be provided; at present, the training and the publishing of the AI algorithm can be operated by very specialized personnel, and the development and the use of the doorsill are very high.
The current solutions are: the whole process of generating corresponding models by algorithm development training and issuing the models to a server to provide services is as follows: sufficient training data are collected firstly, then the data are labeled, then programming training is carried out by professional algorithm post personnel, finally a training model is obtained, the model is verified, and the model meeting the requirements is packed into a mirror image by developers and is issued to the service.
The defects of the above process are that: without enough data before the model is trained, it is impossible to train and provide recognition services. The manual identification can collect the marking training data while providing identification service, and can respond to the requirements of users more quickly.
Generally, when the algorithm provides identification service, identified data are not collected, and the data cannot be effectively accumulated.
Disclosure of Invention
In order to overcome the defects in the technology, the scheme of the invention is as follows: the early recognition service performs manual recognition in the absence of data for model training, realizes the indifference between the manual recognition and the machine recognition, labels the data by the manual recognition and returns a recognition result, firstly provides available services, and simultaneously recognizes the labeled image as a data base for AI online training in the future. The method is realized through the following technical scheme.
A method of data recognition and training comprising the steps of:
step 2: acquiring whether the service is manually identified or automatically identified through the identification mark to form result feedback;
and step 3: recording identification data in the whole process from the beginning of the identification request to the end of the identification, storing annotation information and images in the identification data, and generating a data set from the data through identification history records, wherein the data set provides a data basis for model training;
and 4, step 4: starting a training data set, uploading a training result model after training is finished, and releasing occupied computing resources;
and 5: the training process data is displayed in the training details in a chart form for checking the training effect in real time;
step 6: and the successfully trained model is issued to the cloud platform, and the system automatically schedules the model to a corresponding server node according to the resources required by identification for online preview detection of the training effect.
Further, the manual identification is to send the data to be identified to the identification personnel in a task form for secondary personnel confirmation, the identification personnel label the image on line, the system converts the label information into an identification result, and the identification result is asynchronously fed back to the calling party in a callback form.
Further, the algorithm identification is carried out, an algorithm identification interface is called, and identification result feedback is obtained.
Furthermore, the generated labeling information is identified by the data set, and after the labeling information is confirmed and modified by training personnel, the labeled data set is imported into the system, so that development and training can be performed.
Further, before development training, firstly, a training type is determined, whether training conditions are met or not is determined through a resource management and control mechanism, development training meeting the conditions is performed, and the pod is dispatched to the node with the GPU display card through the k8s cluster.
Furthermore, the successfully trained model can be released on line, and relevant information of the model is filled in through a newly added recognition service function and then uploaded to a cloud platform.
Further, the data training method comprises the following steps:
downloading a data set to a local training container according to a training request, converting the data set into a voc format, and dividing the data set into a training set, a verification set and a test set;
checking the training parameters provided by the initiated training request, and if the checking is passed, continuing training;
loading a training set and a verification set in a voc format, and carrying out batch loading according to the training parameters;
defining a network model structure to be trained, and selecting whether to load an open-source pre-training model parameter according to the training parameter;
carrying out forward propagation on the loaded training set through a network model structure, and solving the loss value loss, top1 probability and accuracy index of the forward propagation of the training set;
the updating optimizer is used for substituting the loss value loss into the network model structure to carry out back propagation updating training parameters;
transmitting the loaded verification set to an updated network model structure, calculating the indexes of top1 probability and accuracy of the verification set, comparing the indexes of different types of models, and storing the network model structure with the optimal index;
and sending the indexes of the training set and the indexes of the verification set to an AI service in a certain data format in an http request mode so that the AI stores the training data and makes a training process data table.
The invention has the beneficial effects that:
and storing the data and the labeling information of manual identification and AI identification to generate a data set, providing a data base for model training, and along with the more use times, the more the accumulated and precipitated data is, and the higher the identification accuracy of the model training is. Through online training and online publishing, the model obtained through training is very conveniently published to the cloud platform, and available recognition service is quickly formed.
When the sample is less, the combination mode of pattern recognition plus advanced learning is adopted, so that the dependency of the algorithm on the sample is reduced, and thus, training can be performed through a small amount of samples, an initial algorithm with ideal recognition accuracy is obtained, early application is issued, along with the accumulation of the use time and the number, the recognition data is converted into a data set, one-key online training and issuing are performed, and the recognition accuracy is continuously improved.
The recognition data and the offline uploaded data are managed in a data set mode in a unified mode, data backflow is formed, and the AI model training precision is improved.
The recognition capability is provided under the scene without a relevant recognition algorithm, the model is trained again in a data set form to improve the recognition accuracy while the number of the accumulated data is accumulated, the AI development efficiency is greatly improved through the training and the publishing of the cloud, the AI development difficulty is reduced, and the AI development time and the cost are saved.
Drawings
Fig. 1 is a diagram of a data recognition flow structure according to an embodiment of the present invention.
Fig. 2 is a diagram of a data training flow structure according to an embodiment of the present invention.
FIG. 3 is a flow chart of selecting manual recognition and algorithmic recognition in an embodiment of the present invention.
FIG. 4 is a block diagram of training types in an embodiment of the invention.
FIG. 5 is a schematic diagram of a training process data coordinate system according to an embodiment of the present invention.
Detailed Description
The present invention is further illustrated by the following examples, which are only a part of the examples of the present invention, and these examples are only for explaining the present invention and do not limit the scope of the present invention.
The flow diagrams depicted in the figures are merely illustrative and do not necessarily include all of the elements and operations/steps, nor do they necessarily have to be performed in the order depicted. For example, some operations/steps may be decomposed, combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
Fig. 1 is a flow chart of data identification according to an embodiment of the present invention.
Step 1: the data of the identification request is stored firstly, so that subsequent log examination is facilitated, and then authentication verification is performed, so that invalid requests and attacks are prevented.
And after the authentication is successful, judging whether the service exists or not through the service identifier identified by the request, and ending the process without direct feedback.
Step 2: whether the service is identified manually or automatically can be obtained through the identification mark, and structural feedback is formed.
If the data to be recognized is manually recognized, the data to be recognized is distributed to a recognizer in a task form and confirmed by secondary personnel, the recognizer performs online marking on the image, and the system converts marking information into a recognition result and asynchronously feeds back the recognition result to a calling party in a callback form. If the algorithm identification is carried out, the algorithm identification interface is directly called to obtain the identification result feedback.
In an embodiment of the invention:
as shown in fig. 3, it is first determined whether the conditions for algorithm identification are met, that is, whether a large amount of basic data exists, if the basic data is small, manual identification is performed, and if the basic data is large, algorithm identification is performed.
Manual identification: and drawing points, lines, frames, polygons and the like on the picture to be identified, and labeling the marked and drawn data.
Automatic algorithm identification: the recognition result includes coordinate data of points, lines, frames and polygons, and is converted into labeling data through format mapping, and the recognition result is used as a label.
The training of the algorithm model needs a large amount of basic data, and under the condition that the basic data and the trained model do not exist, the system does not have the recognition capability, and is firstly completed by manual recognition, and the data are accumulated.
The online identification is an online labeling process, a page labeling tool is used for drawing a dot line frame and the like on a picture to be identified (image classification is directly labeled without labeling, target detection is that rectangular labeling is used and labeled, semantic segmentation and example segmentation are used and polygonal labeling and labeled), the image after label identification is stored in a system, a part of label and picture data stored in the system is selected to be made into a subset, the subset is a data set, the data set is used as a data base, model training is started, a trained model is obtained after training is completed, the trained model and an algorithm can be issued to a cloud platform to start, and in the time, due to the fact that the data quantity is small, the identification precision is low, but as the use times are increased, the accumulated basic data are more, training is carried out again, and the precision is higher.
In an embodiment of the invention: a certain requirement has no algorithm recognition capability on a platform, the service is still provided firstly, a background manually recognizes pictures, the pictures are labeled on line, a recognition result is fed back to a calling party firstly, then a data set A is generated from the pictures, an algorithm model is trained on the basis of the data set A, the platform is published after the training is finished, the model algorithm can recognize the type requirement at the moment, the accuracy is low, the data accumulation is more and more along with the increase of the usage amount, a data set B (the number: B > A) is generated, the model trained on the basis of the data set B is more accurate at the moment, the model is published again to the platform to replace the original recognition algorithm, and the virtuous circle is achieved sequentially.
And step 3: the identification data is recorded in the whole process from the beginning of the identification request to the end of the identification, the labeling information and the image in the identification data are stored in both manual and algorithm identification, the data can be generated into a data set through the identification history, and the data set provides a data basis for model training. The labeling information generated by the automatic identification part in the data set can be confirmed and modified by a training person. The data set supports online import, offline data are imported into the system, and development training can be carried out after manual marking is completed.
And storing the data and the labeling information of manual identification and AI identification to generate a data set, providing a data base for model training, and along with the more use times, the more the accumulated and precipitated data is, and the higher the identification accuracy of the model training is. Through online training and online publishing, the model obtained through training is very conveniently published to the cloud platform, and available recognition service is quickly formed.
And 4, step 4: on-line development training first determines the training type, as shown in fig. 4, which is currently divided into training types of image classification, target detection, semantic segmentation, and instance segmentation, where different types correspond to different models and labeled data, and the format adopts the form of resnet + number for differential classification. After newly added development training, the system determines whether the training conditions are met through a resource management and control mechanism, the qualified training scheduling schedules pod to the node with the GPU video card through a k8s cluster, the system starts to train a data set, a training result model is uploaded after training is finished, and occupied computing resources are released.
And 5: and the training process data is displayed in the training details in a chart form, so that the user can check the training effect in real time. As shown in fig. 5, which is a graph of loss of the training set, average accuracy of the validation set, and average cross-over ratio of the validation set, the data is displayed in the graph by the embodiment, and the data in the training process can be very intuitively understood.
Step 6: the successfully trained model supports online release, information such as description, example pictures and resource use related to the model is filled through the newly added identification service function, the information is released to the cloud platform in a one-key mode, the system is automatically dispatched to a corresponding server node according to the resources required by identification, and the training effect can be detected online through the preview function of the identification service.
The effect of the steps is as follows: when the samples are few, a combined mode of pattern recognition plus advanced learning is adopted, so that the dependence of the algorithm on the samples is reduced, and therefore the algorithm can be trained through a small number of samples, an initial version algorithm with ideal recognition accuracy is obtained, early application is published, recognition data are converted into a data set along with the accumulation of the using time and the number, one-key online training and publishing are carried out, and the recognition accuracy is continuously improved.
And (4) uniformly managing the identification data and the offline uploaded data in a data set form to form data reflux, and improving the AI model training precision by grouping again.
The recognition capability is provided under the scene without a relevant recognition algorithm, the model is trained again in a data set form to improve the recognition accuracy while the number of the accumulated data is accumulated, the AI development efficiency is greatly improved through the training and the publishing of the cloud, the AI development difficulty is reduced, and the AI development time and the cost are saved.
Under the condition that the service does not do enough training data and algorithm support temporarily, the manual identification can meet the urgent identification requirement, the service which is not different from AI identification is provided, and the data is labeled during the manual identification. Compared with the traditional AI development training, the development and deployment difficulty of the AI recognition function is greatly reduced through the standard flow and release of the on-line training.
As shown in fig. 2, for the identified data, a training process is also involved.
The first step is that according to the data set information provided by the training request, the data set is downloaded to the designated miniIO address to be local to the training container, the data set is converted into a voc format, and the data set is divided into a training set, a verification set and a test set according to requirements so as to be loaded in the following process.
And step two, the training parameters provided by the training request are verified, if the verification is passed, the training is continued, otherwise, the training is stopped, and a parameter verification failure message is fed back.
And thirdly, loading the training set and the verification set after the format conversion in the first step, and carrying out batch loading according to the training parameters in the second step.
And step four, defining a network model structure to be trained, and selecting whether to load the pre-training model parameters of the open source or not according to the training parameters in the step two.
And fifthly, transmitting the training set loaded in the third step to the network model in the fourth step for forward propagation, and solving related indexes such as loss value loss, top1 probability, accuracy and the like of the forward propagation of the training set.
Sixthly, updating the optimizer, carrying the loss calculated in the fifth step into the network in 4 for carrying out back propagation and updating the training parameters,
and seventhly, transmitting the verification set loaded in the third step to the updated network model in the fifth step, calculating related indexes such as top1 probability, accuracy and the like of the verification set, and storing the trained network model according to specific indexes of different models if the indexes become better.
And eighthly, sending the training set indexes in the fifth step and the verification set indexes in the seventh step to an AI service in a certain data format in an http request mode so that the AI stores the training data and makes a training process data table.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (7)
1. A method of data recognition and training, comprising the steps of:
step 1, storing data of an identification request, then carrying out authentication verification, and judging whether the service exists or not through a service identifier of the identification request after the authentication verification is successful;
step 2: acquiring whether the service is manually identified or automatically identified through the identification mark to form result feedback;
and step 3: recording identification data in the whole process from the beginning of the identification request to the end of the identification, storing annotation information and images in the identification data, and generating a data set from the data through identification history records, wherein the data set provides a data basis for model training;
and 4, step 4: starting a training data set, uploading a training result model after training is finished, and releasing occupied computing resources;
and 5: the training process data is displayed in the training details in a chart form for checking the training effect in real time;
step 6: and the successfully trained model is issued to the cloud platform, and the system automatically schedules the model to a corresponding server node according to the resources required by identification for online preview detection of the training effect.
2. The data recognition and training method as claimed in claim 1, wherein in the manual recognition, the data to be recognized is distributed to the recognition personnel in a task form for secondary personnel confirmation, the recognition personnel perform online annotation on the image, and the system converts annotation information into a recognition result and asynchronously feeds back the recognition result to the calling party in a callback form.
3. The data recognition and training method of claim 1, wherein the algorithm recognition calls an algorithm recognition interface to obtain recognition result feedback.
4. The data recognition and training method of claim 1, wherein the labeled information generated by the data set recognition is confirmed and modified by a trainer, and the labeled data set is imported into a system for development and training.
5. The data recognition and training method of claim 4, wherein before development training, a training type is determined, whether a training condition is met is determined through a resource management and control mechanism, and the qualified development training is used for dispatching the pod to the node with the GPU graphics card through the k8s cluster.
6. The data recognition and training method as claimed in claim 1, wherein the successfully trained model can be released on line, and through adding a recognition service function, relevant information of the model is filled in and then uploaded to the cloud platform.
7. The data recognition and training method of claim 1, wherein in the step 4, the data training method comprises the following steps:
downloading a data set to a local training container according to a training request, converting the data set into a voc format, and dividing the data set into a training set, a verification set and a test set;
checking the training parameters provided by the initiated training request, and if the checking is passed, continuing training;
loading a training set and a verification set in a voc format, and carrying out batch loading according to the training parameters;
defining a network model structure to be trained, and selecting whether to load an open-source pre-training model parameter according to the training parameter;
carrying out forward propagation on the loaded training set through a network model structure, and solving the loss value loss, top1 probability and accuracy index of the forward propagation of the training set;
the updating optimizer is used for substituting the loss value loss into the network model structure to carry out back propagation updating training parameters;
transmitting the loaded verification set to an updated network model structure, calculating the indexes of top1 probability and accuracy of the verification set, comparing the indexes of different types of models, and storing the network model structure with the optimal index;
and sending the indexes of the training set and the indexes of the verification set to an AI service in a certain data format in an http request mode so that the AI stores the training data and makes a training process data table.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210505216.8A CN114861937A (en) | 2022-05-10 | 2022-05-10 | Data identification and training method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210505216.8A CN114861937A (en) | 2022-05-10 | 2022-05-10 | Data identification and training method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114861937A true CN114861937A (en) | 2022-08-05 |
Family
ID=82638283
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210505216.8A Pending CN114861937A (en) | 2022-05-10 | 2022-05-10 | Data identification and training method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114861937A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115525556A (en) * | 2022-09-29 | 2022-12-27 | 成都万创科技股份有限公司 | AI decision system testing method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10296848B1 (en) * | 2018-03-05 | 2019-05-21 | Clinc, Inc. | Systems and method for automatically configuring machine learning models |
| US20190318208A1 (en) * | 2016-12-27 | 2019-10-17 | Cloudminds (Shenzhen) Robotics Systems Co., Ltd | Image identification system and image identification method |
| CN112906902A (en) * | 2020-12-22 | 2021-06-04 | 上海有个机器人有限公司 | Robot data collection iterative training method and system based on active learning technology and storage medium |
| CN112966772A (en) * | 2021-03-23 | 2021-06-15 | 之江实验室 | Multi-person online image semi-automatic labeling method and system |
| CN113505805A (en) * | 2021-05-25 | 2021-10-15 | 平安银行股份有限公司 | Sample data closed loop generation method, device, equipment and storage medium |
| CN113515632A (en) * | 2021-06-30 | 2021-10-19 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Text classification method based on graph path knowledge extraction |
-
2022
- 2022-05-10 CN CN202210505216.8A patent/CN114861937A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190318208A1 (en) * | 2016-12-27 | 2019-10-17 | Cloudminds (Shenzhen) Robotics Systems Co., Ltd | Image identification system and image identification method |
| US10296848B1 (en) * | 2018-03-05 | 2019-05-21 | Clinc, Inc. | Systems and method for automatically configuring machine learning models |
| CN112906902A (en) * | 2020-12-22 | 2021-06-04 | 上海有个机器人有限公司 | Robot data collection iterative training method and system based on active learning technology and storage medium |
| CN112966772A (en) * | 2021-03-23 | 2021-06-15 | 之江实验室 | Multi-person online image semi-automatic labeling method and system |
| CN113505805A (en) * | 2021-05-25 | 2021-10-15 | 平安银行股份有限公司 | Sample data closed loop generation method, device, equipment and storage medium |
| CN113515632A (en) * | 2021-06-30 | 2021-10-19 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Text classification method based on graph path knowledge extraction |
Non-Patent Citations (1)
| Title |
|---|
| 李英英;纪昌杰;: "基于信息熵加权去噪的半监督SVM分类器", 电脑知识与技术, no. 25, 5 September 2013 (2013-09-05), pages 145 - 147 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115525556A (en) * | 2022-09-29 | 2022-12-27 | 成都万创科技股份有限公司 | AI decision system testing method and system |
| CN115525556B (en) * | 2022-09-29 | 2023-06-20 | 成都万创科技股份有限公司 | Test method and system of AI decision system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111917878B (en) | Message processing method, device, equipment and storage medium | |
| CN112507973B (en) | Text and picture recognition system based on OCR technology | |
| CN114003690A (en) | Information labeling method, model training method, electronic device and storage medium | |
| CN111949795A (en) | Work order automatic classification method and device | |
| CN111835926A (en) | Intelligent voice outbound method, device, equipment and medium based on voice interaction | |
| WO2024032781A1 (en) | Algorithm testing method and apparatus, and storage medium | |
| CN113449698A (en) | Automatic paper document input method, system, device and storage medium | |
| CN110874405A (en) | Service quality inspection method, device, equipment and computer readable storage medium | |
| CN114861937A (en) | Data identification and training method | |
| CN110991279B (en) | Document Image Analysis and Recognition Method and System | |
| CN112749649A (en) | Method and system for intelligently identifying and generating electronic contract | |
| CN110929032B (en) | User demand processing system and method for software system | |
| CN116409575A (en) | Warehouse-in and warehouse-out management and control system of intelligent storage shelf | |
| US20210201014A1 (en) | Extracting values from images of documents | |
| CN109800776A (en) | Material mask method, device, terminal and computer readable storage medium | |
| CN117911039A (en) | Control method, equipment and storage medium for after-sales service system | |
| CN112541150A (en) | Job submitting method, job processing method and electronic equipment | |
| CN112613367A (en) | Bill information text box acquisition method, system, equipment and storage medium | |
| CN111047731A (en) | AR technology-based telecommunication room inspection method and system | |
| CN110659348A (en) | Group enterprise universe risk fusion analysis method and system based on knowledge reasoning | |
| US11816167B1 (en) | Knowledge base platform | |
| CN116341657A (en) | Knowledge extraction platform construction method based on active learning | |
| CN112182413B (en) | Intelligent recommendation method and server based on big teaching data | |
| CN115330200A (en) | Apparatus, system and method for operating artificial intelligence model | |
| CN111881106B (en) | Data labeling and processing method based on AI (advanced technology attachment) test |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |