CN111209931A - Data processing method, platform, terminal device and storage medium - Google Patents
Data processing method, platform, terminal device and storage medium Download PDFInfo
- Publication number
- CN111209931A CN111209931A CN201911338949.1A CN201911338949A CN111209931A CN 111209931 A CN111209931 A CN 111209931A CN 201911338949 A CN201911338949 A CN 201911338949A CN 111209931 A CN111209931 A CN 111209931A
- Authority
- CN
- China
- Prior art keywords
- data
- processed
- model
- information
- algorithm model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Abstract
The embodiment of the application is suitable for the field of artificial intelligence and discloses a data processing method, a platform, terminal equipment and a computer readable storage medium. The method comprises the following steps: acquiring marked data to be processed and algorithm model information; constructing a model training task according to the algorithm model information and the data to be processed; training a pre-constructed artificial intelligence algorithm model according to the model training task to obtain a trained artificial intelligence algorithm model; acquiring real-time data and model ID information; and inputting the real-time data into the trained artificial intelligence algorithm model corresponding to the model ID information to obtain an output result of the artificial intelligence algorithm model. The embodiment of the application improves the intelligent degree of data processing.
Description
Technical Field
The present application belongs to the field of artificial intelligence, and in particular, relates to a data processing method, a data processing platform, a terminal device, and a computer-readable storage medium.
Background
With the continuous development of artificial intelligence technology, the application of artificial intelligence is also more and more extensive.
At present, the application of artificial intelligence generally needs to be carried out based on an artificial intelligence algorithm model, and the artificial intelligence algorithm model needs to be trained firstly and then can be used for processing data. However, in the prior art, manual algorithm model training generally requires marking data manually and constructing a model training task manually. In addition, the trained algorithm model cannot be used to perform corresponding processing on the real-time data automatically.
That is to say, in the prior art, the process of data processing based on the artificial intelligence algorithm model needs human intervention, and the intelligence degree of data processing is low.
Disclosure of Invention
The embodiment of the application provides a data processing method, a data processing platform, terminal equipment and a computer readable storage medium, and aims to solve the problem that the existing artificial intelligence algorithm model-based data processing process needs artificial interference, so that the intelligence degree of data processing is low.
In a first aspect, an embodiment of the present application provides a data processing method, including:
acquiring marked data to be processed and algorithm model information;
constructing a model training task according to the algorithm model information and the data to be processed;
training a pre-constructed artificial intelligence algorithm model according to the model training task to obtain a trained artificial intelligence algorithm model;
acquiring real-time data and model ID information;
and inputting the real-time data into the trained artificial intelligence algorithm model corresponding to the model ID information to obtain an output result of the artificial intelligence algorithm model.
According to the method and the device, the model training task is automatically constructed through the algorithm model information and the data to be processed, the model is trained based on the model training task to obtain the trained algorithm model, the real-time data and the model ID information are automatically obtained, the algorithm model corresponding to the model ID information is used for processing the data, human participation is little or not in the data processing process, and the intelligent degree of data processing is improved.
In a possible implementation manner of the first aspect, before acquiring the data to be processed and the algorithm model information after the marking, the method further includes:
marking the data to be processed to obtain the marked data to be processed.
In a possible implementation manner of the first aspect, marking data to be processed to obtain the marked data to be processed includes:
acquiring scene description information and data index information of the data to be processed;
and labeling the data to be processed according to the scene description information and the data index information to obtain the labeled data to be processed.
In a possible implementation manner of the first aspect, marking data to be processed to obtain the marked data to be processed includes:
obtaining marking rules of the data to be processed;
and labeling the data to be processed according to the marking rule to obtain the marked data to be processed.
In a possible implementation manner of the first aspect, before marking data to be processed and obtaining the marked data to be processed, the method further includes:
and acquiring the data to be processed uploaded by the user.
In a second aspect, an embodiment of the present application provides a data processing platform, including:
the algorithm model training module is used for acquiring marked data to be processed and algorithm model information; constructing a model training task according to the algorithm model information and the data to be processed; training a pre-constructed artificial intelligence algorithm model according to the model training task to obtain a trained artificial intelligence algorithm model;
the model prediction service module is in communication connection with the algorithm model training module and is used for acquiring real-time data and model ID information; and inputting the real-time data into the trained artificial intelligence algorithm model corresponding to the model ID information to obtain an output result of the artificial intelligence algorithm model.
In a possible implementation manner of the second aspect, the system further includes a labeling module, configured to label data to be processed, and obtain the labeled data to be processed.
In a possible implementation manner of the second aspect, the labeling module is specifically configured to:
acquiring scene description information and data index information of the data to be processed;
labeling the data to be processed according to the scene description information and the data index information to obtain the labeled data to be processed;
alternatively, the first and second electrodes may be,
obtaining marking rules of the data to be processed;
and labeling the data to be processed according to the marking rule to obtain the marked data to be processed.
In a third aspect, an embodiment of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor, when executing the computer program, implements the method according to any one of the above first aspects.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the method according to any one of the above first aspects.
In a fifth aspect, embodiments of the present application provide a computer program product, which, when run on a terminal device, causes the terminal device to perform the method of any one of the first aspect.
It is understood that the beneficial effects of the second aspect to the fifth aspect can be referred to the related description of the first aspect, and are not described herein again.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a schematic diagram of a data processing platform according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a data relationship provided in an embodiment of the present application;
FIG. 3 is a diagram illustrating a data structure provided by an embodiment of the present application;
fig. 4 is a schematic block flow chart of a data processing method according to an embodiment of the present application;
fig. 5 is another schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 6 is a block diagram schematically illustrating a marking management apparatus according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
The data processing scheme provided by the embodiment of the application can be carried out based on a data processing platform, and the data processing platform can be an artificial intelligence algorithm platform integrating functions of data management, label management, data marking, algorithm model training, model prediction service and the like.
The data management means that the data processing platform can manage data by adopting a unified standard, an automatic interface is arranged to realize automatic storage of the data, and an uploading interface is arranged to realize file uploading of an operator.
The label management means that the data processing platform can create labels, manage the labels and the like according to scenes, data indexes, classification rules and the like.
The data marking means that the data processing platform can inquire data according to the associated scene, the associated index, the classification name and the time and mark a corresponding label on specific data.
The algorithm model training means that the data processing platform can select a stored artificial intelligence algorithm model, call corresponding model parameters, automatically train the model according to a model training task, and store the trained model.
The model prediction service means that a user can input real-time data into a trained model to obtain a corresponding prediction result.
To better describe the data processing platform provided in the embodiment of the present application, the following description will be made with reference to a schematic diagram of the data processing platform shown in fig. 1.
As shown in FIG. 1, the data processing platform may include marking and model generation functionality, as well as model prediction functionality. In the marking and model generating functions, if the problem is a binary problem, scene description and data types are provided for original data through scenes and indexes, and then the original data are marked based on the scene description, the data types and the like to obtain marked original data; if the problem is the multi-classification problem, marking rules are established according to the classification rules, and marking is carried out on the original data through the marking rules to obtain marked original data.
And selecting a corresponding artificial intelligence algorithm model according to the requirement, giving out the hyper-parameters of the algorithm model, and constructing a model training task by the data processing platform according to the hyper-parameters and the model information. And automatically executing the model training task to train the corresponding artificial intelligence algorithm model to obtain the trained artificial intelligence algorithm model, and storing the trained model.
The user can upload real-time data and the ID of the artificial intelligence algorithm model required to be selected through the service and the application provided by the data processing platform, the data processing platform calls the corresponding artificial intelligence algorithm model according to the model ID, the real-time data is input into the model, the result output by the model is obtained, and the model output result is returned to the user.
To better describe the data processing platform, the following description will be made in conjunction with a data relationship diagram shown in fig. 2 and a data structure diagram shown in fig. 3.
As shown in fig. 2, the scene provides a scene description for the data index, the data index provides an index description for the raw data, and provides an index description and sampling features that can be used for the marking rule. The classification provides a classification basis for marking rules, and the marking rules provide a basis for marking rules for raw data.
The scene, data index and raw data in fig. 2 have the same meaning as those in fig. 1.
In the scenario of fig. 2, the index set data describes the scenario of data, and the basic functions of the scenario include: adding scenes, inquiring scenes, modifying into using scenes and deleting unused scenes. In the data index of fig. 2, the index set describes one or more indexes in the scene, and also specifies a silent type, describing the sampling characteristics that the data should have. The index set is the basis for the raw data, the template and the marking. The basic functions of the data index include: newly adding indexes, inquiring indexes, modifying unused indexes and deleting unused indexes.
The classification in fig. 2 may define the classification in the marking rule. In the marking rule, the marking rule is divided into two-classification and multi-classification problems, and the basic functions of the marking rule comprise creating marking standards or creating classifications, inquiring the marking standards or classifications, modifying unused marking standards or classifying and deleting the unused marking standards or classifications.
In the raw data in fig. 2, data uploaded by a user when the raw data is uploaded is defined, and after the user uploads the data, the data processing platform marks according to rules, and the data processing platform includes basic functions of importing the data, querying the raw data, marking and deleting unused data.
Based on fig. 1 and fig. 2, fig. 3 shows a data structure of the data processing platform, and shows the content included in each type of data. For example, the original data includes an original data ID, an index ID, data content, a tag on which the data is stamped, status information, and the like, the scene includes a scene ID, a scene name, creation time, a scene description, status information, and the like, and the data index includes an index ID, an index name, a scene ID, a data type, a time point collection number, description information, status information, and the like. Other data structures include content as shown in fig. 3. While MOS in fig. 3 refers to MOS system, i.e. the data structure of the data processing platform is the data structure under MOS system.
As can be seen from fig. 1, fig. 2, and fig. 3, the data processing platform (or the artificial intelligence algorithm platform) provided in the embodiment of the present application integrates multiple functions, and can automatically mark raw data, automatically construct a model training task, and automatically train an artificial intelligence algorithm model according to the model training task. In addition, the data processing platform also provides a model prediction service for the user, the user only needs to upload real-time data and specify a model to be called, the data processing platform can automatically input the real-time data into the corresponding artificial intelligence algorithm model for prediction or classification, and the prediction or classification result is returned to the user. The intelligent degree of the data processing process can be improved through the artificial intelligence algorithm platform.
The technical solutions provided in the embodiments of the present application will be described below with specific embodiments.
Referring to fig. 4, a schematic flow chart of a data processing method according to an embodiment of the present application is shown, where the method includes the following steps:
and S401, acquiring marked data to be processed and algorithm model information.
It should be noted that the to-be-processed data may be data uploaded by a user, and specifically may be raw data as in fig. 1, fig. 2, and fig. 3. The above algorithm model information includes, but is not limited to, a model ID and a hyper-parameter of the model, and the model is a pre-constructed artificial intelligence algorithm model, and the type of the model may be arbitrary and is not limited herein. The algorithm model information can be specified by a user or automatically set by the data processing platform.
And S402, constructing a model training task according to the algorithm model information and the data to be processed.
Note that the above-described model training task has the same meaning as the model training task in fig. 3. Referring to fig. 3, the model training task may include, but is not limited to, a task ID, a task name, an algorithm ID, a marking rule ID, a number of samples, a number of classified samples, a start time, an end time, a model hyper-parameter, an algorithm model, a model index, a data index, description information, and the like. The algorithm model can be referred to as mos _ algorithm and AlgorithmParamPO in fig. 3, which is not described herein.
And S403, training the pre-constructed artificial intelligence algorithm model according to the model training task to obtain the trained artificial intelligence algorithm model.
Referring to fig. 2, after the model training task is constructed, the model training task is automatically executed, the artificial intelligence algorithm model is trained by using the marked original data, the trained artificial intelligence algorithm model is obtained, and the trained artificial intelligence algorithm model is stored.
And S404, acquiring real-time data and model ID information.
Referring to fig. 2, the data processing platform provides model prediction services and applications, a user can upload real-time data through the services and the applications, the real-time data is data to be processed, and the data processing platform inputs the real-time data into the artificial intelligence algorithm model corresponding to the model ID information to obtain a prediction result or a classification result output by the artificial intelligence algorithm model. The data processing platform then returns the prediction results or classification results to the user.
For example, the real-time data uploaded by the user is charging data corresponding to a charging order, the charging data is charging data of the electric vehicle recorded by the charging pile, and the model to be called is a charging curve analysis model. After the data processing platform acquires the charging data, the charging data is input into a charging curve analysis model, the type of a charging curve corresponding to the charging data is determined, and then the charging behavior of a user is identified according to the type of the charging curve (for example, a private patch board, a power adapter is wrapped, and the like)
And S405, inputting the real-time data into the trained artificial intelligence algorithm model corresponding to the model ID information to obtain an output result of the artificial intelligence algorithm model.
According to the method and the device, the model training task is automatically constructed through the algorithm model information and the data to be processed, the model is trained based on the model training task to obtain the trained algorithm model, the real-time data and the model ID information are automatically obtained, the algorithm model corresponding to the model ID information is used for processing the data, human participation is little or not in the data processing process, and the intelligent degree of data processing is improved.
Referring to another flow diagram schematic of a data processing method shown in fig. 5, the data processing method may include the steps of:
and step S501, acquiring to-be-processed data uploaded by a user.
It should be noted that the to-be-processed data has the same meaning as the raw data in fig. 1, fig. 2, and fig. 3, and is data uploaded by a user through a data interface. The type of the data to be processed may be any type, for example, the data to be processed is charging data corresponding to a charging order, the charging data is data recorded by a charging pile in the charging process of the electric vehicle, and the charging data may include charging voltage data, charging current data, charging power data, and the like.
And S502, marking the data to be processed to obtain marked data to be processed.
In some embodiments, for the binary problem, scene description information and data index information of the data to be processed may be specifically obtained; and labeling the data to be processed according to the scene description information and the data index information to obtain the marked data to be processed. The process can refer to the process of marking the raw data based on the scene and the index shown in fig. 2.
In other embodiments, if the problem is a multi-classification problem, marking rules of the data to be processed can be obtained; and according to the marking rule, labeling the data to be processed to obtain the marked data to be processed. The process may participate in the marking process of fig. 2 illustrating a multi-classification problem.
And S503, acquiring marked data to be processed and algorithm model information.
And S504, constructing a model training task according to the algorithm model information and the data to be processed.
And S505, training the pre-constructed artificial intelligence algorithm model according to the model training task to obtain the trained artificial intelligence algorithm model.
And S506, acquiring real-time data and model ID information.
And S507, inputting the real-time data into the trained artificial intelligence algorithm model corresponding to the model ID information to obtain an output result of the artificial intelligence algorithm model.
And step S508, returning the output result to the user.
It should be noted that, the same portions in the embodiment of the present application and the embodiment corresponding to fig. 4 can be referred to the corresponding contents above, and are not described again here.
The data processing method provided by the embodiment of the application is based on the data processing platform, and the data processing platform integrates the functions of data management, label management, data marking, model training, model prediction and the like, so that the intelligent degree of the data processing method is improved.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Fig. 6 shows a schematic block diagram of the marking management apparatus provided in the embodiment of the present application, and for convenience of description, only the parts related to the embodiment of the present application are shown.
Referring to fig. 6, the apparatus includes:
the algorithm model training module 61 is used for acquiring marked data to be processed and algorithm model information; constructing a model training task according to the algorithm model information and the data to be processed; training a pre-constructed artificial intelligence algorithm model according to the model training task to obtain a trained artificial intelligence algorithm model;
a model prediction service module 62 in communication with the algorithm model training module for obtaining real-time data and model ID information; and inputting the real-time data into the trained artificial intelligence algorithm model corresponding to the model ID information to obtain an output result of the artificial intelligence algorithm model.
In a possible implementation manner, the system further includes a labeling module 63, configured to label the data to be processed, and obtain the marked data to be processed.
In one possible implementation, the labeling module is specifically configured to:
acquiring scene description information and data index information of data to be processed;
according to the scene description information and the data index information, labeling the data to be processed to obtain the labeled data to be processed;
alternatively, the first and second electrodes may be,
obtaining marking rules of data to be processed;
and according to the marking rule, labeling the data to be processed to obtain the marked data to be processed.
In a possible implementation manner, the system further includes a data management module 64, configured to obtain the to-be-processed data uploaded by the user. A label management module 65 is also included for creating labels and managing labels.
The marking management device has the function of realizing the marking management method, the function can be realized by hardware, or can be realized by executing corresponding software by hardware, the hardware or the software comprises one or more modules corresponding to the function, and the modules can be software and/or hardware.
It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/modules, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and reference may be made to the part of the embodiment of the method specifically, and details are not described here.
Fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in fig. 7, the terminal device 7 of this embodiment includes: at least one processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, the processor 70 implementing the steps in any of the various serial number generation method embodiments described above when executing the computer program 72.
The terminal device 7 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor 70, a memory 71. Those skilled in the art will appreciate that fig. 7 is only an example of the terminal device 7, and does not constitute a limitation to the terminal device 7, and may include more or less components than those shown, or combine some components, or different components, for example, and may further include input/output devices, network access devices, and the like.
The Processor 70 may be a Central Processing Unit (CPU), and the Processor 70 may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 71 may in some embodiments be an internal storage unit of the terminal device 7, such as a hard disk or a memory of the terminal device 7. In other embodiments, the memory 71 may also be an external storage device of the terminal device 7, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the terminal device 7. The memory 71 is used for storing an operating system, an application program, a BootLoader (BootLoader), data, and other programs, such as program codes of the computer program. The memory 71 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the above-mentioned method embodiments.
The embodiments of the present application provide a computer program product, which when running on a terminal device, enables the terminal device to implement the steps in the above method embodiments when executed.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing apparatus/terminal apparatus, a recording medium, computer Memory, Read-Only Memory (ROM), random-access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A data processing method, comprising:
acquiring marked data to be processed and algorithm model information;
constructing a model training task according to the algorithm model information and the data to be processed;
training a pre-constructed artificial intelligence algorithm model according to the model training task to obtain a trained artificial intelligence algorithm model;
acquiring real-time data and model ID information;
and inputting the real-time data into the trained artificial intelligence algorithm model corresponding to the model ID information to obtain an output result of the artificial intelligence algorithm model.
2. The method of claim 1, prior to obtaining post-marking data to be processed and algorithm model information, further comprising:
marking the data to be processed to obtain the marked data to be processed.
3. The method of claim 2, wherein marking the data to be processed to obtain the marked data to be processed comprises:
acquiring scene description information and data index information of the data to be processed;
and labeling the data to be processed according to the scene description information and the data index information to obtain the labeled data to be processed.
4. The method of claim 2, wherein marking the data to be processed to obtain the marked data to be processed comprises:
obtaining marking rules of the data to be processed;
and labeling the data to be processed according to the marking rule to obtain the marked data to be processed.
5. The method of claim 2, wherein prior to marking the data to be processed to obtain the marked data to be processed, further comprising:
and acquiring the data to be processed uploaded by the user.
6. A data processing platform, comprising:
the algorithm model training module is used for acquiring marked data to be processed and algorithm model information; constructing a model training task according to the algorithm model information and the data to be processed; training a pre-constructed artificial intelligence algorithm model according to the model training task to obtain a trained artificial intelligence algorithm model;
the model prediction service module is in communication connection with the algorithm model training module and is used for acquiring real-time data and model ID information; and inputting the real-time data into the trained artificial intelligence algorithm model corresponding to the model ID information to obtain an output result of the artificial intelligence algorithm model.
7. The platform of claim 6, further comprising a labeling module for labeling the data to be processed to obtain the labeled data to be processed.
8. The platform of claim 7, wherein the tagging module is specifically configured to:
acquiring scene description information and data index information of the data to be processed;
labeling the data to be processed according to the scene description information and the data index information to obtain the labeled data to be processed;
alternatively, the first and second electrodes may be,
obtaining marking rules of the data to be processed;
and labeling the data to be processed according to the marking rule to obtain the marked data to be processed.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911338949.1A CN111209931A (en) | 2019-12-23 | 2019-12-23 | Data processing method, platform, terminal device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911338949.1A CN111209931A (en) | 2019-12-23 | 2019-12-23 | Data processing method, platform, terminal device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111209931A true CN111209931A (en) | 2020-05-29 |
Family
ID=70785476
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911338949.1A Pending CN111209931A (en) | 2019-12-23 | 2019-12-23 | Data processing method, platform, terminal device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111209931A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111797144A (en) * | 2020-07-09 | 2020-10-20 | 海南科技职业大学 | Data processing method and system based on artificial intelligence |
| CN112149838A (en) * | 2020-09-03 | 2020-12-29 | 第四范式（北京）技术有限公司 | Method, device, electronic equipment and storage medium for realizing automatic model building |
| CN112580706A (en) * | 2020-12-11 | 2021-03-30 | 北京地平线机器人技术研发有限公司 | Training data processing method and device applied to data management platform and electronic equipment |
| CN113255079A (en) * | 2021-06-01 | 2021-08-13 | 焦作大学 | Artificial intelligence-based package design method and device |
| CN113435597A (en) * | 2021-07-07 | 2021-09-24 | 杭州海康威视数字技术股份有限公司 | Model generation and scheduling method, system and device |
| CN113435126A (en) * | 2021-07-07 | 2021-09-24 | 魏天骐 | Knowledge sharing processing method, intelligent robot equipment, knowledge sharing system and task learning system |
| CN113537493A (en) * | 2021-07-23 | 2021-10-22 | 深圳宏芯宇电子股份有限公司 | Artificial intelligence model training method and device, remote platform and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108875963A (en) * | 2018-06-28 | 2018-11-23 | 北京字节跳动网络技术有限公司 | Optimization method, device, terminal device and the storage medium of machine learning model |
| CN108881446A (en) * | 2018-06-22 | 2018-11-23 | 深源恒际科技有限公司 | A kind of artificial intelligence plateform system based on deep learning |
| CN109074502A (en) * | 2018-07-26 | 2018-12-21 | 深圳前海达闼云端智能科技有限公司 | Method, apparatus, storage medium and the robot of training artificial intelligence model |
| CN109165249A (en) * | 2018-08-07 | 2019-01-08 | 阿里巴巴集团控股有限公司 | Data processing model construction method, device, server and user terminal |
| CN109934955A (en) * | 2019-02-28 | 2019-06-25 | 深圳智链物联科技有限公司 | Charge mode recognition methods, device, terminal device and storage medium |
| CN110033312A (en) * | 2019-03-13 | 2019-07-19 | 平安城市建设科技(深圳)有限公司 | Generation method, device, equipment and the storage medium of room rate prediction model |
-
2019
- 2019-12-23 CN CN201911338949.1A patent/CN111209931A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108881446A (en) * | 2018-06-22 | 2018-11-23 | 深源恒际科技有限公司 | A kind of artificial intelligence plateform system based on deep learning |
| CN108875963A (en) * | 2018-06-28 | 2018-11-23 | 北京字节跳动网络技术有限公司 | Optimization method, device, terminal device and the storage medium of machine learning model |
| CN109074502A (en) * | 2018-07-26 | 2018-12-21 | 深圳前海达闼云端智能科技有限公司 | Method, apparatus, storage medium and the robot of training artificial intelligence model |
| CN109165249A (en) * | 2018-08-07 | 2019-01-08 | 阿里巴巴集团控股有限公司 | Data processing model construction method, device, server and user terminal |
| CN109934955A (en) * | 2019-02-28 | 2019-06-25 | 深圳智链物联科技有限公司 | Charge mode recognition methods, device, terminal device and storage medium |
| CN110033312A (en) * | 2019-03-13 | 2019-07-19 | 平安城市建设科技(深圳)有限公司 | Generation method, device, equipment and the storage medium of room rate prediction model |
Non-Patent Citations (1)
| Title |
|---|
| 聂为之: "多模态媒体数据分析关键技术研究" * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111797144A (en) * | 2020-07-09 | 2020-10-20 | 海南科技职业大学 | Data processing method and system based on artificial intelligence |
| CN112149838A (en) * | 2020-09-03 | 2020-12-29 | 第四范式（北京）技术有限公司 | Method, device, electronic equipment and storage medium for realizing automatic model building |
| WO2022048648A1 (en) * | 2020-09-03 | 2022-03-10 | 第四范式（北京）技术有限公司 | Method and apparatus for achieving automatic model construction, electronic device, and storage medium |
| CN112580706A (en) * | 2020-12-11 | 2021-03-30 | 北京地平线机器人技术研发有限公司 | Training data processing method and device applied to data management platform and electronic equipment |
| CN113255079A (en) * | 2021-06-01 | 2021-08-13 | 焦作大学 | Artificial intelligence-based package design method and device |
| CN113435597A (en) * | 2021-07-07 | 2021-09-24 | 杭州海康威视数字技术股份有限公司 | Model generation and scheduling method, system and device |
| CN113435126A (en) * | 2021-07-07 | 2021-09-24 | 魏天骐 | Knowledge sharing processing method, intelligent robot equipment, knowledge sharing system and task learning system |
| CN113435126B (en) * | 2021-07-07 | 2024-02-02 | 魏天骐 | Knowledge sharing processing method, intelligent robot device, knowledge sharing system and task learning system |
| CN113537493A (en) * | 2021-07-23 | 2021-10-22 | 深圳宏芯宇电子股份有限公司 | Artificial intelligence model training method and device, remote platform and readable storage medium |
| CN113537493B (en) * | 2021-07-23 | 2023-12-08 | 深圳宏芯宇电子股份有限公司 | Artificial intelligence model training method, device, remote platform and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111209931A (en) | Data processing method, platform, terminal device and storage medium | |
| CN108495192A (en) | Page editing processing method and processing device | |
| CN110247811A (en) | A kind of alarm method and relevant apparatus of internet of things equipment | |
| CN108171189A (en) | A kind of method for video coding, video coding apparatus and electronic equipment | |
| CN109478311A (en) | A kind of image-recognizing method and terminal | |
| CN115549313B (en) | Power consumption monitoring method and system based on artificial intelligence | |
| CN110472109B (en) | Dynamic data quality analysis method and platform system | |
| CN107798082B (en) | File label processing method and device | |
| CN109284367A (en) | Method and apparatus for handling text | |
| CN115509765A (en) | Super-fusion cloud computing method and system, computer equipment and storage medium | |
| CN112307318A (en) | Content publishing method, system and device | |
| CN108763260A (en) | A kind of examination question searching method, system and terminal device | |
| CN107506407B (en) | File classification and calling method and device | |
| CN111522854B (en) | Data labeling method and device, storage medium and computer equipment | |
| CN110876072A (en) | Batch registered user identification method, storage medium, electronic device and system | |
| CN111046632A (en) | Data extraction and conversion method, system, storage medium and electronic equipment | |
| CN116993978A (en) | Small sample segmentation method, system, readable storage medium and computer device | |
| CN110222286A (en) | Information acquisition method, device, terminal and computer readable storage medium | |
| CN116382916A (en) | Resource scheduling method and system for cloud computer computing power cluster and electronic equipment | |
| CN114661764A (en) | Method and device for automatically generating cable production information and storage medium | |
| CN111538914B (en) | Address information processing method and device | |
| CN113434577A (en) | Service data processing method, storage medium and equipment | |
| CN112580505A (en) | Method and device for identifying opening and closing states of network points, electronic equipment and storage medium | |
| CN115114467A (en) | Training method and device of picture neural network model | |
| US20240135679A1 (en) | Method for classifying images and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |