CN112989039A - Method, system and storage medium for implementing small sample scene artificial intelligence - Google Patents
Method, system and storage medium for implementing small sample scene artificial intelligence Download PDFInfo
- Publication number
- CN112989039A CN112989039A CN202110178161.XA CN202110178161A CN112989039A CN 112989039 A CN112989039 A CN 112989039A CN 202110178161 A CN202110178161 A CN 202110178161A CN 112989039 A CN112989039 A CN 112989039A
- Authority
- CN
- China
- Prior art keywords
- data
- training
- model
- artificial intelligence
- application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
Abstract
The invention discloses a method, a system and a storage medium for realizing small sample scene artificial intelligence, wherein the method comprises the following steps: acquiring an application request instruction, and extracting semantic features according to the application request instruction; determining a training model and training data according to the semantic features and a preset semantic classification system; and training and testing the training model according to the training data to obtain an application model and application data. According to the embodiment of the invention, firstly, a semantic classification system of a model and data is established according to semantic features, after an application request instruction is obtained, a training model and training data are determined according to the semantic features of the application request instruction, and the training model is trained and tested according to the training data, so that the application model and the application data are obtained, namely, the embodiment of the invention realizes a small sample-oriented data request according to the semantic features of the application request instruction and a preset semantic classification system. The embodiment of the invention can be widely applied to the technical field of artificial intelligence.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a method, a system and a storage medium for realizing small sample scene-oriented artificial intelligence.
Background
The core problem of the artificial intelligence technology is to solve the cognition and application of various data. Under the condition of explosion of mass information of the Internet of things, how to deal with the cognition of the mass information and how to apply an artificial intelligence technology based on the cognition leads the application of the artificial intelligence in the field of the Internet of things to be more difficult than the application of the traditional artificial intelligence.
Need adopt a large amount of label data to carry out iterative training to the model in traditional artificial intelligence's the application, in the artificial intelligence application of actual thing networking, compare with traditional internet artificial intelligence technical application and have two problems: 1. the application scene faced by the field of the Internet of things is complex, and if a large number of edge side models, terminal side models and the like exist, model selection is difficult; 2. training data of the model is not easy to obtain, if various application scenes belong to different companies or units, data sharing is difficult to achieve, the fragmentation state of the scene AI of the Internet of things causes insufficient training data of the model, in addition, the marked training data are less, and the training data and the marked training data cause the problem of small samples.
Disclosure of Invention
In view of this, an object of the embodiments of the present invention is to provide a method, a system, and a storage medium for implementing small sample-oriented scene-oriented artificial intelligence, where the method is capable of determining a training model and training data according to an application request and obtaining application data when the method is oriented to a small sample.
In a first aspect, an embodiment of the present invention provides a method for implementing small sample-oriented scene-oriented artificial intelligence, including the following steps:
acquiring an application request instruction, and extracting semantic features according to the application request instruction;
determining a training model and training data according to the semantic features and a preset semantic classification system;
and training and testing the training model according to the training data to obtain an application model and application data.
Optionally, the preset semantic classification system is determined by:
and constructing an information label system, a model scene label system and a semantic expression matrix of the information to model scene support degree label system.
Optionally, determining the preset semantic classification system further includes:
and constructing scene element semantics or data set semantics of the scene artificial intelligence.
Optionally, the training model is determined by:
determining the support degree of each known model according to the semantic features and the time elements;
and determining the known model with the support degree exceeding a preset value as the training model.
Optionally, the training data is determined by:
acquiring data of the training model;
and determining training data according to the data of the training model and the data of the application request instruction.
Optionally, the determining training data according to the data of the training model and the data of the application request instruction includes:
performing error calculation on the data of the training model and the data of the application request instruction;
determining first training data according to the error;
and determining training data according to the first training and the data of the application request instruction.
Optionally, the training model or the data of the training model is updated according to the recording time and the calling frequency.
In a second aspect, an embodiment of the present invention provides a system for implementing small sample scene-oriented artificial intelligence, including:
the acquisition module is used for acquiring an application request instruction and extracting semantic features according to the application request instruction;
the determining module is used for determining a training model and training data according to the semantic features and a preset semantic classification system;
and the application module is used for training and testing the training model according to the training data to obtain an application model and application data.
In a third aspect, an embodiment of the present invention provides a system for implementing small sample scene-oriented artificial intelligence, including:
at least one processor;
at least one memory for storing at least one program;
when the at least one program is executed by the at least one processor, the at least one processor is enabled to implement the method for implementing the small sample scenario-oriented artificial intelligence.
In a fourth aspect, an embodiment of the present invention provides a storage medium, in which a processor-executable program is stored, where the processor-executable program is used to execute the implementation method of the small-sample-oriented scenarized artificial intelligence described above when executed by a processor.
The implementation of the embodiment of the invention has the following beneficial effects: according to the embodiment of the invention, firstly, a semantic classification system of a model and data is established according to semantic features, after an application request instruction is obtained, a training model and training data are determined according to the semantic features of the application request instruction, and the training model is trained and tested according to the training data, so that the application model and the application data are obtained, namely, the embodiment of the invention realizes a small sample-oriented data request according to the semantic features of the application request instruction and a preset semantic classification system.
Drawings
Fig. 1 is a schematic flowchart illustrating steps of a method for implementing small-sample scene-oriented artificial intelligence provided in an embodiment of the present invention;
fig. 2 is a schematic flowchart illustrating steps of another implementation method for small sample scene-oriented artificial intelligence according to an embodiment of the present invention;
fig. 3 is a block diagram of a small sample scene-oriented artificial intelligence implementation system according to an embodiment of the present invention;
fig. 4 is a block diagram of another implementation system for small sample scene-oriented artificial intelligence according to an embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and the specific embodiments. The step numbers in the following embodiments are provided only for convenience of illustration, the order between the steps is not limited at all, and the execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
As shown in fig. 1, an embodiment of the present invention provides a method for implementing small sample-oriented scenized artificial intelligence, which includes the following steps.
S100, acquiring an application request instruction, and extracting semantic features according to the application request instruction;
s200, determining a training model and training data according to the semantic features and a preset semantic classification system;
s300, training and testing the training model according to the training data to obtain an application model and application data.
Specifically, the preset semantic classification system is established according to a known model and data before acquiring the application request instruction, and is continuously maintained and updated according to an example in the actual application. Firstly, establishing a scene artificial intelligence semantic expression method and a corresponding identification system, namely quickly obtaining a similar scene of new artificial intelligence application based on semantic matching; then, inputting the acquired semantic features of the application request instruction into a preset semantic classification system to determine a training model and training data; and finally, training and testing the training model according to the training data to obtain an application model and application data.
Optionally, determining the preset semantic classification system further includes:
and S001, constructing scene element semantics of scene artificial intelligence.
Specifically, semantic elements of the scene include an application scene, an application field, an application device type, an application data type, characteristics of the application data type, processing capability of the application device, and the like; the application equipment type comprises a data acquisition equipment type, an equipment type derived by a model and the like; the application data types comprise images, videos, voices, characters and the like; establishing a finer corresponding expression by each scenarized AI application according to the characteristics of the application data type, such as the characteristics of a real scene scenarized AI data set; the processing capability of the application device comprises a processing chip, an operating system for running the model, computing resources and the like.
Optionally, determining the preset semantic classification system further includes:
and S002, constructing the data set semantics of the scene artificial intelligence.
Specifically, methods of computing semantic correlations of data sets include, but are not limited to: the scene-based artificial intelligence applies actual data corresponding to a front-end scene, such as data types, formats and coding rules, such as images, videos, audios, texts and scene characteristics; the front-end scenario corresponds to a terminal type, such as a feature of the device, a hardware feature of the device, and an operating system feature.
Optionally, the preset semantic classification system is determined by:
s003, constructing an information label system, a model scene label system and a semantic expression matrix of the information to model scene support degree label system.
It should be noted that the information label system is divided into label systems of various information such as images, videos, voices, audios, texts and the like; the model scene label system comprises label systems such as scene expression suitable for the model, a suitable operating system, equipment conditions suitable for running and the like; the information-to-model scene support degree label system covers various information resource sets and model scene support degree label systems.
It should be noted that each type of information resource set is a generalized concept, and specifically, there may be several subsets in a large set, and there may be finer subsets in the subsets; each subset has the data quality of the subset and semantic expression and description of the adaptive scene; each subset also includes training support conditions of a specific model of the subset adapted scene, such as how the model trained by using the subset behaves on the test set of the scene, and how the model trained by using the subset behaves in practical applications. In the actual construction and use of the semantic classification system, a semantic expression matrix is constructed by all semantic expressions to form a semantic index module, and all functions of the semantic index module are protected.
Specifically, the semantic expression matrix comprises a model name, a model evaluation index, model precision requirements, data set characteristics for generating a model, a method for deducing and processing the data set characteristics and calculating similar data based on small sample data; setting a scene artificial intelligence data semantic support degree calculation method, iterating to obtain a proper training and testing data set, and the like.
Optionally, the training model is determined by:
s201, determining the support degree of each known model according to the semantic features and the time elements;
s202, determining the known model with the support degree exceeding a preset value as the training model.
It should be noted that, in practical use, the time of data generation in each data set is recorded, and the service quality invoked by the front end of the information set training model is recorded. The quality of service called by the front end mainly considers the time element, the accessed information and the result evaluation of the time marking factor. The time elements comprise the time of information generation, the frequency of information updating, the time and the frequency of information used by a user for searching, the relation between the time and a query keyword thereof and the like. Time factors are considered in the information correlation calculation model, and the time elements are stored as a label, so that more accurate information service can be provided. Query goals such as working hours and off-hours may differ using a search engine.
Specifically, receiving an artificial intelligence application sent by a user, analyzing an artificial intelligence application request, and extracting key elements corresponding to user requirements, such as application scene characteristics, data characteristics, model operating environment characteristics and the like; calling a semantic scene artificial intelligence data classification system and a data item set according to the scene artificial intelligence application elements, and calculating the support degree of each field subset to the artificial intelligence elements; judging whether the support degree of each field subset to the scene artificial intelligence elements reaches a preset value, and sending the search elements to a field artificial intelligence model training engine and a data generation engine corresponding to one or more field subsets reaching the preset value; training and testing the artificial intelligence training engines in each field according to the obtained data elements, and returning artificial intelligence model training results; and returning logic according to a preset result and issuing the result of the trained artificial intelligence model.
Optionally, the training data is determined by:
s203, acquiring data of the training model;
and S204, determining training data according to the data of the training model and the data of the application request instruction.
Optionally, the determining training data according to the data of the training model and the data of the application request instruction includes:
s2041, carrying out error calculation on the data of the training model and the data of the application request instruction;
s2042, determining first training data according to the error;
s2043, determining training data according to the first training data and the data of the application request instruction.
Specifically, the first training data generation method includes two methods: the method comprises the steps that firstly, when data corresponding to a returned result during access of scene front-end equipment are not accurate, semantic distances between the data and a training model are calculated, and existing data in a database are obtained based on the semantic distances and serve as first training data; the method enlarges the search range of the training data; acquiring data which is close to the semantic distance of the data in a database, thereby generating a batch of data; this approach can accurately obtain data similar to inaccurate data.
It should be noted that the semantic distance calculation methods for different types of information are different, and the semantic distance calculation methods for images, voices, texts, and the like are all different.
Optionally, the training model or the data of the training model is updated according to the recording time and the calling frequency.
Specifically, when a semantic expression matrix is established, recording time, calling frequency and the like of a training model and a data set corresponding to the training model are recorded; updating data corresponding to the model according to a time attenuation strategy of global semantic value, and rewarding and calling the model with high frequency and the corresponding data set from the perspective of semantic support according to the frequency and the calling time according to the model selected by a certain scene and the corresponding data set; and updating the support degree of various models and related data sets on the scene and the like, thereby providing related basic semantic information for the semantic search models and data sets in the future. It should be noted that the update principle may be set as: the closer the recording time is to the calling time or the higher the calling frequency is, the higher the support degree of the model and the data set is; therefore, data with less calling frequency or long recording time is eliminated, and the accuracy of the semantic classification system is improved.
Specifically, an embodiment of the present invention provides a specific embodiment, as shown in fig. 2, extracting artificial intelligence scene semantic features according to a user request, and performing model feature discrimination and data feature discrimination according to the semantic features respectively; judging the closest model and acquiring training data of the closest model according to model features in the semantic features, such as deployment requirements, operating environment requirements, functional requirements, performance requirements, functional evaluation indexes, performance evaluation indexes, artificial intelligence targets and the like; acquiring a closest data set in the existing semantic data set according to data features in the semantic features, such as data types, actual data features, differences between the existing data set and the actual data and the like; comprehensively calculating according to the training data of the closest model and the closest data set to obtain a data set and generate a training set and a test set; and training and testing the closest model by using the generated training set and the test set to obtain a new application model and application data.
Specifically, the embodiment of the present invention provides another specific embodiment, which includes the following specific steps:
and S1, obtaining the scene artificial intelligence demand request data.
S2, constructing a scene artificial intelligence requirement semantic matching matrix, including but not limited to scene equipment, scene requirements and scene request data types, such as image comparison, image recognition, voice recognition, keyword recognition, semantic understanding requests, search requests, scene execution equipment types and processor operation requirements.
And S3, matching the semantic expression of the existing data resources by the scene semantic matching matrix.
S4, selecting the former resources as candidate sets, and calculating or obtaining the semantic center of each candidate set or obtaining the representative data in the candidate sets.
S5, calculating the semantic distance between the request data and the candidate set, and calculating the semantic distance between the request data and the data represented by the candidate set.
And S6, selecting a candidate set with small semantic distance according to the calculation result to carry out model training.
S7, testing the training model and recording the precision of the model, including but not limited to accuracy, recall, MRR (Mean reciprocal rank), and the like. Note that the evaluation index is different for different scenes or different processing information.
And S8, updating the semantic expression matrix according to the result.
The implementation of the embodiment of the invention has the following beneficial effects: according to the embodiment of the invention, firstly, a semantic classification system of a model and data is established according to semantic features, after an application request instruction is obtained, a training model and training data are determined according to the semantic features of the application request instruction, and the training model is trained and tested according to the training data, so that the application model and the application data are obtained, namely, the embodiment of the invention realizes a small sample-oriented data request according to the semantic features of the application request instruction and a preset semantic classification system.
As shown in fig. 3, an embodiment of the present invention provides a system for implementing small sample-oriented scenized artificial intelligence, including:
the acquisition module is used for acquiring an application request instruction and extracting semantic features according to the application request instruction;
the determining module is used for determining a training model and training data according to the semantic features and a preset semantic classification system;
and the application module is used for training and testing the training model according to the training data to obtain an application model and application data.
It can be seen that the contents in the foregoing method embodiments are all applicable to this system embodiment, the functions specifically implemented by this system embodiment are the same as those in the foregoing method embodiment, and the advantageous effects achieved by this system embodiment are also the same as those achieved by the foregoing method embodiment.
As shown in fig. 4, an embodiment of the present invention provides a system for implementing small sample-oriented scenized artificial intelligence, including:
at least one processor;
at least one memory for storing at least one program;
when the at least one program is executed by the at least one processor, the at least one processor is enabled to implement the method steps for implementing the small sample scenario-oriented artificial intelligence.
It can be seen that the contents in the foregoing method embodiments are all applicable to this system embodiment, the functions specifically implemented by this system embodiment are the same as those in the foregoing method embodiment, and the advantageous effects achieved by this system embodiment are also the same as those achieved by the foregoing method embodiment.
In addition, the embodiment of the application also discloses a computer program product or a computer program, and the computer program product or the computer program is stored in a computer readable storage medium. The processor of the computer device can read the computer program from the computer readable storage medium, and the processor executes the computer program, so that the computer device executes the implementation method for the small sample scenario-oriented artificial intelligence. Likewise, the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the advantageous effects achieved by the present storage medium embodiment are also the same as those achieved by the above method embodiments.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A method for realizing small sample scene-oriented artificial intelligence is characterized by comprising the following steps:
acquiring an application request instruction, and extracting semantic features according to the application request instruction;
determining a training model and training data according to the semantic features and a preset semantic classification system;
and training and testing the training model according to the training data to obtain an application model and application data.
2. The method for implementing small sample-oriented scenarized artificial intelligence according to claim 1, wherein the preset semantic classification system is determined by the following steps:
and constructing an information label system, a model scene label system and a semantic expression matrix of the information to model scene support degree label system.
3. The method for implementing artificial intelligence oriented to small sample scenario according to claim 2, wherein determining the preset semantic classification system further includes:
and constructing scene element semantics or data set semantics of the scene artificial intelligence.
4. The method for implementing artificial intelligence oriented to small sample scene in claim 1, wherein the training model is determined by the following steps:
determining the support degree of each known model according to the semantic features and the time elements;
and determining the known model with the support degree exceeding a preset value as the training model.
5. The method for implementing artificial intelligence oriented to small sample scene in claim 4, wherein the training data is determined by the following steps:
acquiring data of the training model;
and determining training data according to the data of the training model and the data of the application request instruction.
6. The method for implementing small sample scenario-oriented artificial intelligence, according to claim 5, wherein the determining training data according to the data of the training model and the data of the application request instruction includes:
performing error calculation on the data of the training model and the data of the application request instruction;
determining first training data according to the error;
and determining training data according to the first training data and the data of the application request instruction.
7. The method for implementing small-sample-oriented scenarized artificial intelligence as claimed in claim 5, wherein the training model or the data of the training model is updated according to a recording time and a calling frequency.
8. The utility model provides a realization system towards small sample scene artificial intelligence which characterized in that includes:
the acquisition module is used for acquiring an application request instruction and extracting semantic features according to the application request instruction;
the determining module is used for determining a training model and training data according to the semantic features and a preset semantic classification system;
and the application module is used for training and testing the training model according to the training data to obtain an application model and application data.
9. The utility model provides a realization system towards small sample scene artificial intelligence which characterized in that includes:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, the at least one program causes the at least one processor to implement the method for implementing small sample scenario-oriented artificial intelligence as claimed in any one of claims 1-7.
10. A storage medium having stored therein a program executable by a processor, wherein the program executable by the processor is configured to perform the method for implementing small sample scenario-oriented artificial intelligence according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110178161.XA CN112989039A (en) | 2021-02-08 | 2021-02-08 | Method, system and storage medium for implementing small sample scene artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110178161.XA CN112989039A (en) | 2021-02-08 | 2021-02-08 | Method, system and storage medium for implementing small sample scene artificial intelligence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112989039A true CN112989039A (en) | 2021-06-18 |
Family
ID=76392701
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110178161.XA Pending CN112989039A (en) | 2021-02-08 | 2021-02-08 | Method, system and storage medium for implementing small sample scene artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112989039A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106557461A (en) * | 2016-10-31 | 2017-04-05 | 百度在线网络技术（北京）有限公司 | Semantic analyzing and processing method and device based on artificial intelligence |
| US20200193206A1 (en) * | 2018-12-18 | 2020-06-18 | Slyce Acquisition Inc. | Scene and user-input context aided visual search |
| CN111613212A (en) * | 2020-05-13 | 2020-09-01 | 携程旅游信息技术（上海）有限公司 | Speech recognition method, system, electronic device and storage medium |
| CN111695670A (en) * | 2019-03-11 | 2020-09-22 | 深圳市茁壮网络股份有限公司 | Neural network model training method and device |
-
2021
- 2021-02-08 CN CN202110178161.XA patent/CN112989039A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106557461A (en) * | 2016-10-31 | 2017-04-05 | 百度在线网络技术（北京）有限公司 | Semantic analyzing and processing method and device based on artificial intelligence |
| US20200193206A1 (en) * | 2018-12-18 | 2020-06-18 | Slyce Acquisition Inc. | Scene and user-input context aided visual search |
| CN111695670A (en) * | 2019-03-11 | 2020-09-22 | 深圳市茁壮网络股份有限公司 | Neural network model training method and device |
| CN111613212A (en) * | 2020-05-13 | 2020-09-01 | 携程旅游信息技术（上海）有限公司 | Speech recognition method, system, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108804641B (en) | Text similarity calculation method, device, equipment and storage medium | |
| CN109947909B (en) | Intelligent customer service response method, equipment, storage medium and device | |
| CN110163376B (en) | Sample detection method, media object identification method, device, terminal and medium | |
| CN111382248B (en) | Question replying method and device, storage medium and terminal equipment | |
| CN110110726A (en) | The recognition methods of power equipment nameplate, device, computer equipment and storage medium | |
| CN112732883A (en) | Fuzzy matching method and device based on knowledge graph and computer equipment | |
| CN111274822A (en) | Semantic matching method, device, equipment and storage medium | |
| CN114168841A (en) | Content recommendation method and device | |
| WO2021001517A1 (en) | Question answering systems | |
| CN113657510A (en) | Method and device for determining data sample with marked value | |
| CN111143515B (en) | Text matching method and device | |
| CN112434533A (en) | Entity disambiguation method, apparatus, electronic device, and computer-readable storage medium | |
| JP7256357B2 (en) | Information processing device, control method, program | |
| CN112989039A (en) | Method, system and storage medium for implementing small sample scene artificial intelligence | |
| CN113312396B (en) | Metadata processing method and device based on big data | |
| CN114780700A (en) | Intelligent question-answering method, device, equipment and medium based on machine reading understanding | |
| CN114676237A (en) | Sentence similarity determining method and device, computer equipment and storage medium | |
| CN110309273A (en) | Answering method and device | |
| CN111666770B (en) | Semantic matching method and device | |
| CN114282513A (en) | Text semantic similarity matching method and system, intelligent terminal and storage medium | |
| CN114090869A (en) | Target object processing method and device, electronic equipment and storage medium | |
| CN113434653A (en) | Method, device and equipment for processing query statement and storage medium | |
| CN112579774A (en) | Model training method, model training device and terminal equipment | |
| CN116992111B (en) | Data processing method, device, electronic equipment and computer storage medium | |
| CN111382246A (en) | Text matching method, matching device and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |