CN112085078A - Image classification model generation system, method and device and computer equipment - Google Patents
Image classification model generation system, method and device and computer equipment Download PDFInfo
- Publication number
- CN112085078A CN112085078A CN202010893131.2A CN202010893131A CN112085078A CN 112085078 A CN112085078 A CN 112085078A CN 202010893131 A CN202010893131 A CN 202010893131A CN 112085078 A CN112085078 A CN 112085078A
- Authority
- CN
- China
- Prior art keywords
- image
- training
- classification model
- image classification
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/40—Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Abstract
The application relates to an image classification model generation system, method, device, computer equipment and storage medium, comprising: the image training data storage module is used for verifying an image training data uploading request sent by the terminal and acquiring first image training data and a category label; obtaining second image training data carrying class labels according to the first image training data and the corresponding class labels; the image classification model training module is used for analyzing an image classification model training request sent by the terminal to obtain parameter configuration information; extracting a corresponding algorithm mirror image from an algorithm mirror database according to the parameter configuration information, and determining a candidate training server cluster; running an algorithm mirror image to generate a corresponding image classification model; and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the second image training data until the image classification model is trained completely. The system improves the efficiency of image classification model generation.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a system, a method, an apparatus, a computer device, and a storage medium for generating an image classification model.
Background
In recent years, 3C products emerge endlessly, and the products need to be supported by a great variety of parts; the parts have the characteristics of small volume and irregular appearance, and the parts need to be detected and classified by manual visual inspection.
With the popularization of artificial intelligence technology, deep learning technology is also introduced into the 3C industry; through the image classification model, the work of classifying 3C products can be replaced by manpower. However, a lot of time cost is consumed for manually training and debugging the model, the type of the model which can be trained is single, the overall timeliness is poor, and the diversified requirements of customers cannot be met in time. Therefore, the existing image classification model is low in generation efficiency.
Disclosure of Invention
In view of the above, it is necessary to provide an image classification model generation system, method, apparatus, computer device, and storage medium capable of improving the image classification model generation efficiency in view of the above technical problems.
An image classification model generation system, the system comprising: the image classification model training module is used for training the image classification model; the image training data storage module is in communication connection with the image classification model training module;
the image training data storage module is used for verifying an image training data uploading request sent by a terminal, and acquiring first image training data uploaded by the terminal and a class label corresponding to the first image training data if the verification is passed; obtaining second image training data carrying the class label according to the first image training data and the corresponding class label; sending the second image training data to the image classification model training module;
the image classification model training module is used for analyzing an image classification model training request sent by a terminal to obtain parameter configuration information; extracting a corresponding algorithm mirror image from an algorithm mirror database according to the parameter configuration information, and determining a candidate training server cluster; running the algorithm mirror image to generate a corresponding image classification model; and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the second image training data until the training of the image classification model is completed.
In one embodiment, the image classification model generation system further comprises an account alteration module; the account changing module comprises an account creating module and an authority adjusting module;
the account creating module is used for responding to an account creating request sent by the terminal, generating corresponding account information and distributing the account information to a corresponding account group;
and the permission adjusting module is used for adjusting permission parameters corresponding to the account group.
In one embodiment, the image training data storage module further comprises a request verification module;
the request checking module is used for analyzing the image training data uploading request to obtain the account information; determining an account group corresponding to the account information; if the authority parameter of the account group is smaller than a preset authority threshold value, refusing to execute the image training data uploading request; and if the authority parameter of the account group is greater than or equal to the preset authority threshold, acquiring the first image training data and the corresponding class label according to the image training data uploading request.
In one embodiment, the image classification model training module further comprises a cluster scheduling module;
the cluster scheduling module is used for acquiring the running states of a plurality of training server clusters; determining a load state parameter of the training server cluster according to the running state of the training server cluster; and taking the training server cluster with the load state parameter smaller than a preset load threshold value as the candidate training server cluster.
In one embodiment, the image classification model training module is further configured to perform an image classification test on the image classification model after a single training to obtain an image classification accuracy; and when the image classification accuracy reaches a preset accuracy threshold, determining that the training of the image classification model is finished.
In one embodiment, the image classification model training module is further configured to send the trained image classification model to a terminal corresponding to the image classification model training request; the terminal is used for calling the trained image classification model, classifying the images to be processed and obtaining image classification results.
A method of generating an image classification model, the method comprising:
acquiring parameter configuration information carried in an image classification model training request sent by a terminal, and determining an algorithm mirror image and a candidate training server cluster according to the parameter configuration information;
running the algorithm mirror image to generate a corresponding image classification model;
and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the image training data with the class labels until the training of the image classification model is completed.
An image classification model generation apparatus, the apparatus comprising:
the parameter acquisition module is used for acquiring parameter configuration information carried in an image classification model training request sent by a terminal and determining an algorithm mirror image and a candidate training server cluster according to the parameter configuration information;
the model generation module is used for operating the algorithm mirror image to generate a corresponding image classification model;
and the model training module is used for scheduling the candidate training server cluster and repeatedly training the image classification model based on the image training data carrying the class label until the training of the image classification model is finished.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring parameter configuration information carried in an image classification model training request sent by a terminal, and determining an algorithm mirror image and a candidate training server cluster according to the parameter configuration information;
running the algorithm mirror image to generate a corresponding image classification model;
and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the image training data with the class labels until the training of the image classification model is completed.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring parameter configuration information carried in an image classification model training request sent by a terminal, and determining an algorithm mirror image and a candidate training server cluster according to the parameter configuration information;
running the algorithm mirror image to generate a corresponding image classification model;
and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the image training data with the class labels until the training of the image classification model is completed.
The image classification model generation system, method, device, computer equipment and storage medium comprise: the image classification model training module is used for training the image classification model; the image training data storage module is in communication connection with the image classification model training module; the image training data storage module is used for verifying an image training data uploading request sent by the terminal, and acquiring first image training data uploaded by the terminal and a class label corresponding to the first image training data if the verification is passed; obtaining second image training data carrying class labels according to the first image training data and the corresponding class labels; sending the second image training data to an image classification model training module; the image classification model training module is used for analyzing an image classification model training request sent by the terminal to obtain parameter configuration information; extracting a corresponding algorithm mirror image from an algorithm mirror database according to the parameter configuration information, and determining a candidate training server cluster; running an algorithm mirror image to generate a corresponding image classification model; and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the second image training data until the image classification model is trained completely. The system determines a proper algorithm mirror image according to parameter configuration information carried in an image classification model training request, and establishes a corresponding model according to the algorithm mirror image, so that the model requirement for distinguishing different image categories can be met; by selecting the candidate training server cluster, the training of a plurality of image classification models can be realized without manual participation, and the efficiency of generating the image classification models is improved.
Drawings
FIG. 1 is a diagram of an environment in which an image classification model generation system is applied in one embodiment;
FIG. 2 is a schematic diagram of an interface of an image classification model generation system in one embodiment;
FIG. 3 is a schematic diagram of a page of an item detail page in one embodiment;
FIG. 4 is a schematic diagram of a page for account creation in one embodiment;
FIG. 5 is a system diagram of an image classification model generation system according to an embodiment;
FIG. 6 is a flowchart illustrating a method for generating an image classification model according to an embodiment;
FIG. 7 is a block diagram showing the configuration of an image classification model generation apparatus according to an embodiment;
FIG. 8 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The image classification model generation system provided by the application can be applied to the application environment as shown in fig. 1. The terminal 11 communicates with the image classification model generation system 12 via a network. The image classification model generation system 12 includes: an image training data storage module 121 and an image classification model training module 122; the image training data storage module 121 is in communication connection with the image classification training module 122; the image training data storage module 121 is configured to verify an image training data uploading request sent by the terminal 11, and if the verification is passed, obtain first image training data uploaded by the terminal 11 and a category label corresponding to the first image training data; the image training data storage module 121 obtains second image training data carrying class labels according to the first image training data and the corresponding class labels; the image training data storage module 121 sends the second image training data to the image classification model training module 122; the image classification model training module 122 is configured to analyze an image classification model training request sent by the terminal 11 to obtain parameter configuration information; the image classification model training module 122 extracts a corresponding algorithm image from the algorithm image library 1221 according to the parameter configuration information, and determines a candidate training server cluster 1222; the image classification model training module 122 runs an algorithm mirror to generate a corresponding image classification model; the image classification model training module 122 schedules the candidate training server cluster 1222 to iteratively train the image classification model based on the second image training data until the image classification model training is complete.
The terminal 11 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the image classification model generation system 12 and each internal module may be implemented by an independent server or a server cluster composed of a plurality of servers. The image classification model generation system 12 may support a plurality of deployment modes, such as standalone deployment, cluster deployment, online deployment, and the like, to maximize computational efficiency.
The image classification model generation system can provide a web-like management and control interface shown in an interface schematic diagram of fig. 2 for a user, wherein in a web page of the type, a middle area is a preview area, details of a single picture or multiple pictures can be displayed, data annotation and annotation result viewing are supported, the pictures can be controlled by a mouse to be rolled, amplified or reduced, and the annotation processing by the user is facilitated. The left side is an operation area which can be used for uploading image data; the upper part of the left side is respectively provided with a training button, a verifying button and a testing button, and the buttons are respectively connected with corresponding modules and can provide operation inlets related to model generation; the right part is a display area which mainly displays the condition of adding image data labels, namely labeling information, and a training report and a test report which are derived after model training to a user.
A user can log in the image classification model generation system through a terminal to perform corresponding operation, and each module in the image classification model generation system can provide corresponding data support according to the operation of the user, so that the training and generation processes of the image classification model are realized.
In one embodiment, the image training data storage module is used for verifying an image training data uploading request sent by a terminal, and acquiring first image training data uploaded by the terminal and a class label corresponding to the first image training data if the verification is passed; obtaining second image training data carrying class labels according to the first image training data and the corresponding class labels; and sending the second image training data to an image classification model training module.
The type of the image training data can be in common formats such as JPEG, TIFF, PNG and the like, and the terminal can upload the image training data in a compressed packet form. The class label is an annotation of a class to which the target object belongs in the first image training data. The second image training data is image training data carrying class labels formed by combining the first image training data and the corresponding class labels.
Specifically, a user enters an image classification model generation system through terminal connection, and sends an image training data uploading request to the image classification model generation system, wherein the image training data uploading request carries account information corresponding to the user; the image training data storage module can determine a corresponding account group through the account information, judge whether the authority parameters of the account group reach a threshold value capable of uploading image training data, and if so, judge that the authentication passes, and acquire the image training data stored in the terminal.
When the image is uploaded or after the image is uploaded, the image training data storage module can add annotation information to the uploaded image data through the image data storage module according to the user; the image training data storage module takes the images uploaded by the user through the terminal as first image training data, the images can be sequentially arranged and displayed in a list form, and the user can sequentially add corresponding category labels to the upper image training data on the system interface according to the list sequence to finish labeling. The image training data storage module combines the first image training data with the class labels to obtain second image training data with the class labels, and the second image training data are used as training data of the image classification model and sent to the image classification model training module.
According to the image classification model generation method and device, the second image training data carrying the class labels are obtained by obtaining the first image training data and the corresponding labeling information, data accumulation required by image classification model generation is completed, model requirements for distinguishing different types of images can be met, and the image classification model generation efficiency is improved.
In one embodiment, the image classification model training module is used for analyzing an image classification model training request sent by a terminal to obtain parameter configuration information; extracting a corresponding algorithm mirror image from an algorithm mirror database according to the parameter configuration information, and determining a candidate training server cluster; running an algorithm mirror image to generate a corresponding image classification model; and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the second image training data until the image classification model is trained completely.
The image classification model training module is an execution module for specifically training and generating the image classification model. The parameter configuration information is specific setting of an image classification model training task and can comprise categories such as data enhancement configuration and training parameter configuration; the data enhancement configuration comprises a plurality of options for preprocessing the training data, such as image blurring, image cutting, image noise processing, image sharpening, image color adjustment, image angle adjustment and the like, so that the images can be uniformly optimized, and the identifiability of the image data is improved; the model size, learning times, hardware configuration, algorithm and the like can be selected in the training parameter configuration. A large number of algorithm image files are stored in the algorithm image library, wherein the algorithm image files comprise industry customized algorithms, general algorithms and the like, and can be pulled according to needs, and corresponding models are generated through the algorithm image files.
Specifically, the image classification model training module acquires an image classification model training request sent by a terminal, acquires parameter configuration information from the image classification model training request, and determines a suitable algorithm mirror image and a candidate training server cluster from an algorithm mirror database according to the parameter configuration information. And then establishing a corresponding image classification model according to the algorithm mirror image, and scheduling the candidate training server cluster to perform a training process on the image classification model based on the second image data until the training of the image classification model is completed.
In the embodiment, a proper algorithm mirror image is determined through parameter configuration information carried in the image classification model training request, and a corresponding model is established according to the algorithm mirror image, so that the model requirements for classifying different types of images can be met; by selecting the candidate training server cluster, the training of a plurality of image classification models can be realized without manual participation, and the efficiency of generating the image classification models is improved.
The image classification model generation system includes: the image classification model training module is used for training the image classification model; the image training data storage module is in communication connection with the image classification model training module; the image training data storage module is used for verifying an image training data uploading request sent by the terminal, and acquiring first image training data uploaded by the terminal and a class label corresponding to the first image training data if the verification is passed; obtaining second image training data carrying class labels according to the first image training data and the corresponding class labels; sending the second image training data to an image classification model training module; the image classification model training module is used for analyzing an image classification model training request sent by the terminal to obtain parameter configuration information; extracting a corresponding algorithm mirror image from an algorithm mirror database according to the parameter configuration information, and determining a candidate training server cluster; running an algorithm mirror image to generate a corresponding image classification model; and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the second image training data until the image classification model is trained completely. The system determines a proper algorithm mirror image according to parameter configuration information carried in an image classification model training request, and establishes a corresponding model according to the algorithm mirror image, so that the model requirement for distinguishing different image categories can be met; by selecting the candidate training server cluster, the training of a plurality of image classification models can be realized without manual participation, and the efficiency of generating the image classification models is improved.
In one embodiment, the image training data storage module comprises one or more of an image acquisition module, an image annotation module, a visualization module and a project module; the modules are mutually matched to realize the steps or the method executed by the image training data storage module.
The image acquisition module can acquire the image sent by the terminal according to an image training data uploading request sent by the terminal, wherein the image training data uploading request comprises information such as an image data list, image data size, image type, whether to label and the like; the image acquisition module establishes communication connection with the terminal after verifying an image training data uploading request sent by the terminal, and acquires an image corresponding to the image training data uploading request; the image training data uploading request can also directly carry a corresponding image, and the corresponding image can be directly obtained from the image training data uploading request as the first image training data after the verification is passed.
The image labeling module can enable the terminal to label the first image training data, sequentially display each picture to a terminal user in a preset interface of the image classification model generation system, the user respectively checks each picture, and corresponding category labels are added to each first image training data through the terminal. The image labeling module can also call a pre-trained general image classification model to identify part of target objects in the first image training data and correspondingly add the class labels, and the user modifies and confirms the class labels to finish the addition of the class labels to the first image training data. The image labeling module can also identify a target object to be identified in the image, highlight the edge of the target object, generate an input frame in the area near the target object, and enable a user to directly input category information in the input frame as a category label.
The image labeling module can also detect the class labels and the image training data, can reduce the image when the image to be labeled is too large, and can enlarge the image when the image is too small; on the contrary, if the added category label is too small, the category label can be amplified, and if the added category label is too large, the category label can be reduced; the size decision can be derived from the ratio between the class label and the image data or the target object area. The shape of the category label is not limited and can be rectangular, linear, and irregular images.
The project module is used for viewing and managing the whole process of image model generation, and the functions of the project module comprise project creation, project modification, project deletion and the like. The new project requires a user to log in a image classification model generation system through a terminal to enter parameters of the new project so as to preset basic attributes of the project, the parameters include but are not limited to project name, project operation authority, task type and other information, a project module can generate a corresponding project according to the parameters of the new project, allocate an identifier (project ID) to the project, and display the project in a project detail page provided by the image classification model generation system, as shown in a project detail page schematic diagram of fig. 3, the display content includes information such as the project ID, the project name, a project creator (i.e. creating account information corresponding to the project), project state, image classification model number, task type and the like. Subsequent steps of image training data acquisition, image classification model training and the like can be carried out by taking the item as a unit.
The visualization module can generate a visualized chart according to the type, the size, the labeling condition and the like of the stored image training data. For example, according to the newly added amount of the second image training data, a line graph reflecting the amount change trend in a certain time range is generated, or a pie graph is generated according to the category label of the image. Furthermore, the data visualization module can analyze and summarize the image training data to form a visualization report of the image training data.
In one embodiment, the image classification model training module comprises one or more of a resource monitoring module, a cluster scheduling module, an algorithm mirroring module, a training module and a data management module.
The resource monitoring module is used for acquiring resource utilization states, cluster states and the like of the training servers and the training server clusters, generating corresponding running states and mastering the health degree of the training servers and the training server clusters. And the cluster scheduling module is used for calling the training server cluster according to the running state of the training server cluster and executing the training task of the corresponding image classification model. The algorithm mirror module can manage and control the version of the algorithm mirror stored in the algorithm mirror library, and comprises the steps of importing, exporting, updating iteration, deleting, modifying and the like of the algorithm mirror; the algorithm mirror image module extracts an algorithm mirror image according to the parameter configuration information, and an image classification model to be trained is obtained after operation; the algorithm mirror comprises a general algorithm and a plurality of industry customized algorithms, and different selections can be carried out according to specific requirements. And the training module trains the image classification model according to the second image training data until the image classification model reaches the training standard. The resource monitoring module can acquire progress information of image classification model training and display the progress information as a project running state on a project detail page in forms of progress bars and the like, so that a user can know the progress of the image classification model training conveniently.
In one embodiment, the image classification model generation system further comprises an account alteration module; the account changing module comprises an account creating module and a permission adjusting module; the account creating module is used for responding to an account creating request sent by the terminal, generating corresponding account information and distributing the account information to a corresponding account group; the authority adjusting module is used for adjusting the authority parameters corresponding to the account group.
The account changing module is a module for configuring an account for a system user. The account creating module can create an account for a new user and also can delete and modify account information; the classification and the hierarchical management of the user accounts can be realized in an account group mode, and the batch management of the accounts can be realized by adjusting the authority corresponding to the account group through the authority parameter.
Specifically, the account creation module verifies an account creation request sent by the terminal, and generates corresponding account information according to the account creation request after the verification is passed; as shown in the account creation diagram of fig. 4, the account information may include a user name, an account group, a password, a permission, a creation time, and the like; meanwhile, an identifier can be allocated to the account information to serve as an account identifier; the authority adjusting module is used for adjusting the authority parameters corresponding to the account group, and the adjusting object of the authority parameters can also be the personal authority parameters corresponding to the account information. Different privilege parameters correspond to different modules that can operate.
In the embodiment, the management of multiple accounts is realized through the account creating module and the authority adjusting module, the efficiency of the user for operating the image classification model generating system is improved, and the safety and the stability of the image classification model generating system are also improved by monitoring the authority of the user.
In one embodiment, the image training data storage module further comprises a request checking module; the request checking module is used for analyzing the image training data uploading request to obtain account information; determining an account group corresponding to the account information; if the authority parameter of the account group is smaller than the preset authority threshold value, refusing to execute the image training data uploading request; and if the authority parameter of the account group is greater than or equal to the preset authority threshold, acquiring the first image training data and the corresponding class label according to the image training data uploading request.
Specifically, a plurality of modules in the image classification model generation system can limit the modules which can be operated by a user through a preset authority threshold value, so that the transmission safety of data of each module is maintained. Taking an image training data acquisition module as an example, when an image training data uploading request sent by a terminal is received, account information carried in the request is acquired, and authority parameters of a corresponding account group are determined; for example, the preset authority threshold of the image data acquisition module is 5, the authority parameter of the account group corresponding to the account information is 0, the authority parameter is smaller than the preset authority threshold, and the account group in which the account information is located does not have the authority for uploading the image training data, so that the image training data uploading request sent by the terminal can be rejected. And if the authority parameter of the account group corresponding to the account information of the other terminal is 10 and the authority parameter is larger than the preset authority threshold, the account information has the qualification of uploading the image training data.
According to the embodiment, the permission parameters of the account group corresponding to the account information and the preset permission threshold value limit the modules which can be operated by different account information, and the safety of the image classification model generation system is improved. The image training data acquisition module also avoids random uploading of image training data by setting a preset authority threshold value, and improves the usability of the whole data of the system.
In one embodiment, the image classification model training module further comprises a cluster scheduling module; the cluster scheduling module is used for acquiring the running states of a plurality of training server clusters; determining a load state parameter of the training server cluster according to the running state of the training server cluster; and taking the training server cluster with the load state parameter smaller than the preset load threshold value as a candidate training server cluster.
Specifically, the candidate training server cluster is better in overall operation performance due to the fact that the load state parameter is smaller than the preset load threshold, and is preferentially selected during training. The cluster scheduling module realizes load balance among a plurality of training server clusters by determining the load state parameters of the training server clusters, and improves the overall operation efficiency of the image classification model generation system.
In one embodiment, the image classification model training module is further configured to perform an image classification test on the image classification model after a single training to obtain an image classification accuracy; and when the image classification accuracy reaches a preset accuracy threshold, determining that the training of the image classification model is finished.
Specifically, the image classification model training module can determine the progress of image classification model training according to the obtained image classification accuracy by performing image classification test on the trained image classification model; and when the image classification accuracy rate does not reach the preset recognition rate threshold value, continuing training, and if the image classification accuracy rate does not reach the preset recognition rate threshold value after image classification detection is carried out for multiple times and for a long time, packaging information such as parameters generated in the training process, generating early warning information, and sending the early warning information to a person corresponding to the terminal for reason investigation. When the image classification accuracy reaches the preset recognition threshold after the image classification test is carried out for multiple times, the training of the image classification model can be considered to be completed.
In the embodiment, the image classification test is carried out on the trained image classification model, the progress of the training of the image classification model is mastered through the image classification accuracy, and the accuracy of the trained image classification model is ensured.
In one embodiment, the image classification model training module is further configured to send the trained image classification model to a terminal corresponding to the image classification model training request; and the terminal is used for calling the trained image classification model and classifying the images to be processed to obtain an image classification result.
Specifically, after the training of the image classification model is completed, the image classification model training module can send the trained image classification model to the corresponding terminal. And the terminal can finish the image classification work through the trained image classification model. The image classification model training request can contain information of a demand side of the image classification model, and the image classification model training module can determine a target sending terminal of the image classification model according to the information of the demand side. Or the image classification model training module stores the trained image classification model to the image classification model generating system, and the demand terminal can log in the image classification model generating system to obtain the image classification model. According to the embodiment, the efficiency of acquiring the image classification model by the terminal is improved by sending the trained image classification model to the corresponding terminal.
In an embodiment, in order to more clearly illustrate the technical solution provided by the embodiment of the present application, an architecture of the system will be described below with reference to fig. 5, which includes the following specific contents:
the image classification model generation system can be divided into a foreground part and a background part, and the foreground part and the background part can be communicated through http/tcp; the foreground part comprises an image training data storage module, wherein the image training data storage module comprises an image acquisition module, an image labeling module and a data visualization module, and a user can upload image data required by training of an image classification model through the image acquisition module and add corresponding class labels through the image labeling module; and then, a visualization report containing a chart can be derived by utilizing the data visualization module, and the image data and the labeling condition are determined. The background part mainly comprises an account changing module and an image classification model training module. The account management module comprises an account creating module and a permission adjusting module. The system can modify the account information of the user logging in the system through the account changing module, such as account creation, account deletion, account group distribution and the like, and set the authority parameters of the account group through the authority adjusting module so as to adjust the operable module range of the user and ensure the overall safety of the system.
The image classification model training module comprises a resource monitoring module, a training module, an algorithm mirror module, a data management module and a cluster scheduling module. The system determines the running state of each training server cluster through a resource monitoring module to obtain a cluster state and sends the cluster state to a training module, the training module determines a training server cluster to be called according to the cluster state, and the cluster scheduling module carries out scheduling operation; after a training server cluster capable of performing model training is scheduled, an image classification model to be trained is established according to corresponding mirror images and pulled from an algorithm mirror image module, and then image data of a foreground part is acquired from a data management module for training to obtain the trained image classification model.
In one embodiment, as shown in fig. 6, an image classification model generation method is provided, which is described by taking the method as an example applied to the image classification model generation system 12 in fig. 1, and includes the following steps:
and step 63, scheduling the candidate training server cluster, and repeatedly training the image classification model based on the image training data with the class labels until the training of the image classification model is completed.
In the image classification model generation method, the server determines a corresponding algorithm mirror image and a candidate training server cluster according to parameter configuration information carried in an image classification model training request; and running an algorithm mirror image to generate an image classification model to be trained, acquiring image training data carrying annotation information, and training the image classification model on the candidate training server cluster to obtain the trained image classification model. The method can quickly carry out the generation process of various different image classification models; the scheduling candidate training server cluster can realize the training of a plurality of image classification models with different purposes without manual participation, and the generation efficiency of the image classification models is improved.
It should be understood that, although the steps in the flowchart of fig. 6 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 6 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
In one embodiment, as shown in fig. 7, there is provided an image classification model generation apparatus including: a parameter acquisition module 71, a model generation module 72 and a model training module 73, wherein:
the parameter obtaining module 71 is configured to obtain parameter configuration information carried in an image classification model training request sent by a terminal, and determine an algorithm mirror image and a candidate training server cluster according to the parameter configuration information;
a model generation module 72, configured to run an algorithm mirror image to generate a corresponding image classification model;
and the model training module 73 is used for scheduling the candidate training server cluster and repeatedly training the image classification model based on the image training data carrying the class label until the training of the image classification model is completed.
For specific limitations of the image classification model generation apparatus, reference may be made to the above limitations of the image classification model generation method, which are not described herein again. The modules in the image classification model generation device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 8. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing image classification model generation data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an image classification model generation method.
Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring parameter configuration information carried in an image classification model training request sent by a terminal, and determining an algorithm mirror image and a candidate training server cluster according to the parameter configuration information;
running an algorithm mirror image to generate a corresponding image classification model;
and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the image training data carrying the class labels until the training of the image classification model is completed.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring parameter configuration information carried in an image classification model training request sent by a terminal, and determining an algorithm mirror image and a candidate training server cluster according to the parameter configuration information;
running an algorithm mirror image to generate a corresponding image classification model;
and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the image training data carrying the class labels until the training of the image classification model is completed.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. An image classification model generation system, characterized in that the system comprises: the image classification model training module is used for training the image classification model; the image training data storage module is in communication connection with the image classification model training module;
the image training data storage module is used for verifying an image training data uploading request sent by a terminal, and acquiring first image training data uploaded by the terminal and a class label corresponding to the first image training data if the verification is passed; obtaining second image training data carrying the class label according to the first image training data and the corresponding class label; sending the second image training data to the image classification model training module;
the image classification model training module is used for analyzing an image classification model training request sent by a terminal to obtain parameter configuration information; extracting a corresponding algorithm mirror image from an algorithm mirror database according to the parameter configuration information, and determining a candidate training server cluster; running the algorithm mirror image to generate a corresponding image classification model; and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the second image training data until the training of the image classification model is completed.
2. The system of claim 1, wherein the image classification model generation system further comprises an account alteration module; the account changing module comprises an account creating module and an authority adjusting module;
the account creating module is used for responding to an account creating request sent by the terminal, generating corresponding account information and distributing the account information to a corresponding account group;
and the permission adjusting module is used for adjusting permission parameters corresponding to the account group.
3. The system of claim 2, wherein the image training data storage module further comprises a request verification module;
the request checking module is used for analyzing the image training data uploading request to obtain the account information; determining an account group corresponding to the account information; if the authority parameter of the account group is smaller than a preset authority threshold value, refusing to execute the image training data uploading request; and if the authority parameter of the account group is greater than or equal to the preset authority threshold, acquiring the first image training data and the corresponding class label according to the image training data uploading request.
4. The system of claim 1, wherein the image classification model training module further comprises a cluster scheduling module;
the cluster scheduling module is used for acquiring the running states of a plurality of training server clusters; determining a load state parameter of the training server cluster according to the running state of the training server cluster; and taking the training server cluster with the load state parameter smaller than a preset load threshold value as the candidate training server cluster.
5. The system of claim 1, wherein the image classification model training module is further configured to perform an image classification test on the image classification model after a single training to obtain an image classification accuracy; and when the image classification accuracy reaches a preset accuracy threshold, determining that the training of the image classification model is finished.
6. The system according to any one of claims 1 to 5, wherein the image classification model training module is further configured to send the trained image classification model to a terminal corresponding to the image classification model training request; the terminal is used for calling the trained image classification model, classifying the images to be processed and obtaining image classification results.
7. A method for generating an image classification model, the method comprising:
acquiring parameter configuration information carried in an image classification model training request sent by a terminal, and determining an algorithm mirror image and a candidate training server cluster according to the parameter configuration information;
running the algorithm mirror image to generate a corresponding image classification model;
and scheduling the candidate training server cluster, and repeatedly training the image classification model based on the image training data with the class labels until the training of the image classification model is completed.
8. An apparatus for generating an image classification model, the apparatus comprising:
the parameter acquisition module is used for acquiring parameter configuration information carried in an image classification model training request sent by a terminal and determining an algorithm mirror image and a candidate training server cluster according to the parameter configuration information;
the model generation module is used for operating the algorithm mirror image to generate a corresponding image classification model;
and the model training module is used for scheduling the candidate training server cluster and repeatedly training the image classification model based on the image training data carrying the class label until the training of the image classification model is finished.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of claim 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method as claimed in claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010893131.2A CN112085078A (en) | 2020-08-31 | 2020-08-31 | Image classification model generation system, method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010893131.2A CN112085078A (en) | 2020-08-31 | 2020-08-31 | Image classification model generation system, method and device and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112085078A true CN112085078A (en) | 2020-12-15 |
Family
ID=73728909
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010893131.2A Pending CN112085078A (en) | 2020-08-31 | 2020-08-31 | Image classification model generation system, method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112085078A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906554A (en) * | 2021-02-08 | 2021-06-04 | 智慧眼科技股份有限公司 | Model training optimization method and device based on visual image and related equipment |
| CN113094171A (en) * | 2021-03-31 | 2021-07-09 | 北京达佳互联信息技术有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113128601A (en) * | 2021-04-22 | 2021-07-16 | 北京百度网讯科技有限公司 | Training method of classification model and method for classifying images |
| CN116152525A (en) * | 2023-04-20 | 2023-05-23 | 北京大数据先进技术研究院 | Image classification model selection system based on remote procedure call and meta learning |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108881446A (en) * | 2018-06-22 | 2018-11-23 | 深源恒际科技有限公司 | A kind of artificial intelligence plateform system based on deep learning |
| CN110213345A (en) * | 2019-05-14 | 2019-09-06 | 深圳极视角科技有限公司 | A kind of AI algorithm management system |
| CN110554047A (en) * | 2019-09-06 | 2019-12-10 | 腾讯科技（深圳）有限公司 | method, device, system and equipment for processing product defect detection data |
| CN111190718A (en) * | 2020-01-07 | 2020-05-22 | 第四范式（北京）技术有限公司 | Method, device and system for realizing task scheduling |
| CN111242317A (en) * | 2020-01-09 | 2020-06-05 | 深圳供电局有限公司 | Method and device for managing application, computer equipment and storage medium |
-
2020
- 2020-08-31 CN CN202010893131.2A patent/CN112085078A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108881446A (en) * | 2018-06-22 | 2018-11-23 | 深源恒际科技有限公司 | A kind of artificial intelligence plateform system based on deep learning |
| CN110213345A (en) * | 2019-05-14 | 2019-09-06 | 深圳极视角科技有限公司 | A kind of AI algorithm management system |
| CN110554047A (en) * | 2019-09-06 | 2019-12-10 | 腾讯科技（深圳）有限公司 | method, device, system and equipment for processing product defect detection data |
| CN111190718A (en) * | 2020-01-07 | 2020-05-22 | 第四范式（北京）技术有限公司 | Method, device and system for realizing task scheduling |
| CN111242317A (en) * | 2020-01-09 | 2020-06-05 | 深圳供电局有限公司 | Method and device for managing application, computer equipment and storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906554A (en) * | 2021-02-08 | 2021-06-04 | 智慧眼科技股份有限公司 | Model training optimization method and device based on visual image and related equipment |
| CN112906554B (en) * | 2021-02-08 | 2022-12-23 | 智慧眼科技股份有限公司 | Model training optimization method and device based on visual image and related equipment |
| CN113094171A (en) * | 2021-03-31 | 2021-07-09 | 北京达佳互联信息技术有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113128601A (en) * | 2021-04-22 | 2021-07-16 | 北京百度网讯科技有限公司 | Training method of classification model and method for classifying images |
| CN113128601B (en) * | 2021-04-22 | 2022-04-29 | 北京百度网讯科技有限公司 | Training method of classification model and method for classifying images |
| CN116152525A (en) * | 2023-04-20 | 2023-05-23 | 北京大数据先进技术研究院 | Image classification model selection system based on remote procedure call and meta learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112085078A (en) | Image classification model generation system, method and device and computer equipment | |
| CN107766940B (en) | Method and apparatus for generating a model | |
| CN105283849B (en) | For the Parallel Tracking of performance and details | |
| CN105283852B (en) | A kind of method and system of fuzzy tracking data | |
| CN112884092B (en) | AI model generation method, electronic device, and storage medium | |
| CN112102263A (en) | Defect detection model generation system, method and device and computer equipment | |
| CN111835582B (en) | Configuration method and device of Internet of things inspection equipment and computer equipment | |
| WO2019100577A1 (en) | Automated test management method and apparatus, terminal device, and storage medium | |
| CN110990053A (en) | Method for creating and using machine learning scheme template and device | |
| US11720825B2 (en) | Framework for multi-tenant data science experiments at-scale | |
| CN111401722B (en) | Intelligent decision method and intelligent decision system | |
| CN112085019A (en) | Character recognition model generation system, method and device and computer equipment | |
| CN110704095A (en) | Workflow-based configuration processing method and device, computer equipment and storage medium | |
| CN114049185A (en) | Cost accounting method and device, computer equipment and storage medium | |
| US11941519B2 (en) | Machine learning training platform | |
| CN111459796A (en) | Automatic testing method and device, computer equipment and storage medium | |
| CN112085027A (en) | Image segmentation model generation system, method and device and computer equipment | |
| US9508006B2 (en) | System and method for identifying trees | |
| CN115270923A (en) | Scene-based visual intelligent decision method and system | |
| Zhang et al. | On the cost of interactions in interactive visual machine learning | |
| CN113192171A (en) | Three-dimensional effect graph efficient rendering method and system based on cloud rendering | |
| CN113821296B (en) | Visual interface generation method, electronic equipment and storage medium | |
| CN117495191A (en) | Quality of service evaluation method, device, computer equipment and storage medium | |
| US11830298B2 (en) | Service data transfer system and method for automotive diagnostics and service | |
| US20230410476A1 (en) | Automated image analysis using artificial intelligence techniques |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201215 |
|
| RJ01 | Rejection of invention patent application after publication |