CN117456416B - Method and system for intelligently generating material labels - Google Patents
Method and system for intelligently generating material labels Download PDFInfo
- Publication number
- CN117456416B CN117456416B CN202311454740.8A CN202311454740A CN117456416B CN 117456416 B CN117456416 B CN 117456416B CN 202311454740 A CN202311454740 A CN 202311454740A CN 117456416 B CN117456416 B CN 117456416B
- Authority
- CN
- China
- Prior art keywords
- label
- custom
- model
- material data
- universal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000000463 material Substances 0.000 title claims abstract description 144
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000012549 training Methods 0.000 claims abstract description 67
- 238000007781 pre-processing Methods 0.000 claims abstract description 31
- 238000011156 evaluation Methods 0.000 claims description 17
- 238000003058 natural language processing Methods 0.000 claims description 11
- 238000012360 testing method Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 238000013095 identification testing Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010030 laminating Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
Abstract
The embodiment of the invention discloses a method and a system for intelligently generating material labels, which are used for acquiring a certain amount of material data and preprocessing the material data to obtain preprocessed data; constructing a training set based on the preprocessed data; constructing a custom tag identification model, wherein the custom tag identification model is related to a general tag identification model based on identification task requirements; inputting the training set into a custom tag recognition model for training to obtain a trained custom tag recognition model; inputting the material data to be identified into a universal label identification model to obtain a universal label of the material data to be identified; inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified; and reasoning to obtain a final label of the material data to be identified based on the universal label and the custom label. The method for intelligently generating the material labels solves the problem that the material labels cannot be accurately generated based on label generation requirements in the prior art.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method, a system, electronic equipment and a storage medium for intelligently generating material labels.
Background
The tag is a keyword describing articles and user preference, is used for describing an objective attribute and user interests of the articles, is a bridge connecting the users and the articles, and when the users upload media materials such as pictures, videos and the like, the AI large model can automatically extract some general tags for the materials for retrieval, and the general AI large model has identification models such as styles, buildings, characters, commodities, vehicles and the like, and is trained based on general large data, so that the AI large model can identify the tags, but the identified tags are too generalized, can not accurately express user material information, is unfavorable for material management and transmission, and still needs a large amount of human intervention cost.
For a multi-tenant SaaS platform, materials of different users have unique industry attributes, the traditional mode is to customize different service interfaces for different enterprises directly at an application layer, and when the tenants grow, the customized code quantity can expand rapidly, and finally, the support and maintenance are difficult. In addition, the technology of simply selecting a certain manufacturer in the traditional mode is difficult to cover any field, an excellent scheme is difficult to output to a user, and the method is also difficult to adapt to various deployment environments.
Therefore, a method capable of intelligently generating material tags based on tag generation requirements is needed.
Disclosure of Invention
The embodiment of the invention aims to provide a method, a system, electronic equipment and a storage medium for intelligently generating material labels, which are used for solving the problem that tax returns of the month to which the same tax belongs cannot be classified based on a tax return mode and then summarized in the prior art.
In order to achieve the above object, an embodiment of the present invention provides a method for intelligently generating a material tag, where the method specifically includes:
acquiring a certain amount of material data, and preprocessing the material data to obtain preprocessed data;
constructing a training set based on the preprocessing data;
constructing a custom tag identification model, wherein the custom tag identification model is related to a general tag identification model based on identification task requirements;
inputting the training set into the custom tag recognition model for training to obtain a trained custom tag recognition model;
inputting the material data to be identified into the universal label identification model to obtain a universal label of the material data to be identified;
Inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified;
And reasoning to obtain a final label of the material data to be identified based on the universal label and the custom label.
Based on the technical scheme, the invention can also be improved as follows:
Further, the obtaining a certain amount of material data, and preprocessing the material data to obtain preprocessed data includes:
Judging whether the material data are video data or not, if so, dividing the video data into a plurality of sub-videos according to a lens;
determining at least one video frame from the sub-video as a key frame of the sub-video;
And constructing a training set based on the key frames.
Further, the building of the custom tag recognition model, wherein the custom tag recognition model is related to the general tag recognition model based on the recognition task requirement, comprises:
Classifying the universal label identification model based on an industry identifier;
determining an industry identifier and a platform identifier corresponding to the identification task;
and determining a universal label recognition model to be associated based on the industry identifier, the platform identifier and the recognition task, wherein the number of the universal label recognition models is at least one.
Further, the building of the custom tag recognition model, wherein the custom tag recognition model is related to the general tag recognition model based on the recognition task requirement, further comprises:
acquiring url calling addresses corresponding to the universal label identification models to be associated;
And associating the customized tag identification model with the to-be-associated universal tag identification model based on the url call address.
Further, the inputting the training set into the custom tag recognition model for training to obtain a trained custom tag recognition model includes:
Dividing the preprocessing data into a training set, a verification set and a test set;
Training the custom tag recognition model based on the training set;
performing performance evaluation on the trained custom tag recognition model based on the verification set to obtain a custom tag recognition model meeting performance conditions;
And evaluating the identification result of the customized label identification model meeting the performance condition based on the test set to obtain an evaluation index corresponding to the customized label identification model.
Further, the inputting the material data to be identified into the universal label identification model to obtain the universal label of the material data to be identified includes:
performing character recognition, scene recognition and style recognition on the material data to be recognized based on the universal tag recognition model, and obtaining a recognition result;
and extracting the identification result to obtain the universal label of the material data to be identified.
Further, the reasoning to obtain the final tag of the material data to be identified based on the universal tag and the custom tag includes:
Inputting the universal label and the custom label into a natural language processing model, and reasoning and matching the user own label from a user own label system database through the natural language processing model;
and obtaining a final label based on the user own label, the universal label and the custom label.
A system for intelligently generating material tags, comprising:
the preprocessing module is used for acquiring a certain amount of material data and preprocessing the material data to obtain preprocessed data;
A first construction module for constructing a training set based on the preprocessing data;
The second construction module is used for constructing a customized tag identification model, wherein the customized tag identification model is related to a general tag identification model based on identification task requirements;
the training module is used for inputting the training set into the customized label recognition model for training to obtain a trained customized label recognition model;
inputting the material data to be identified into the universal label identification model to obtain a universal label of the material data to be identified;
Inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified;
and the reasoning module is used for reasoning and obtaining the final label of the material data to be identified based on the universal label and the custom label.
An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method when the computer program is executed.
A non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method.
The embodiment of the invention has the following advantages:
According to the method for intelligently generating the material labels, a certain amount of material data are obtained, and the material data are preprocessed to obtain preprocessed data; constructing a training set based on the preprocessing data; constructing a custom tag identification model, wherein the custom tag identification model is related to a general tag identification model based on identification task requirements; inputting the training set into the custom tag recognition model for training to obtain a trained custom tag recognition model; inputting the material data to be identified into the universal label identification model to obtain a universal label of the material data to be identified; inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified; and the final label of the material data to be identified is obtained by reasoning based on the universal label and the customized label, so that the problem that the material label cannot be accurately generated based on the label generation requirement in the prior art is solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It will be apparent to those skilled in the art from this disclosure that the drawings described below are merely exemplary and that other embodiments may be derived from the drawings provided without undue effort.
The structures, proportions, sizes, etc. shown in the present specification are shown only for the purposes of illustration and description, and are not intended to limit the scope of the invention, which is defined by the claims, so that any structural modifications, changes in proportions, or adjustments of sizes, which do not affect the efficacy or the achievement of the present invention, should fall within the scope of the invention.
FIG. 1 is a flow chart of a method of intelligently generating material tags in accordance with the present invention;
FIG. 2 is a first architecture diagram of the system for intelligently generating material tags of the present invention;
fig. 3 is a schematic diagram of an entity structure of an electronic device according to the present invention.
Wherein the reference numerals are as follows:
Preprocessing module 10, first building module 20, second building module 30, training module 40, reasoning module 50, electronics 60, processor 601, memory 602, bus 603.
Detailed Description
Other advantages and advantages of the present invention will become apparent to those skilled in the art from the following detailed description, which, by way of illustration, is to be read in connection with certain specific embodiments, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Examples
Fig. 1 is a flowchart of an embodiment of a method for intelligently generating a material tag according to the present invention, as shown in fig. 1, and the method for intelligently generating a material tag according to the embodiment of the present invention includes the following steps:
s101, acquiring a certain amount of material data, and preprocessing the material data to obtain preprocessed data;
Specifically, judging whether the material data is video data or not, if so, dividing the video data into a plurality of sub-videos according to a lens;
determining at least one video frame from the sub-video as a key frame of the sub-video;
And constructing a training set based on the key frames.
S102, constructing a training set based on preprocessing data;
S103, constructing a custom tag identification model, wherein the custom tag identification model is related to a general tag identification model based on the identification task requirement;
specifically, classifying the universal label identification model based on industry identification;
determining an industry identifier and a platform identifier corresponding to the identification task;
and determining a universal label recognition model to be associated based on the industry identifier, the platform identifier and the recognition task, wherein the number of the universal label recognition models is at least one.
Acquiring url calling addresses corresponding to the universal label identification models to be associated;
and associating the custom tag identification model with the to-be-associated universal tag identification model based on the url call address.
The universal extraction module adopts a universal label recognition model, the loading core of the universal label recognition model is abstracted and designed into a universal base, and the main stream large model of each cloud platform and the open source model on the market can be integrated into the universal extraction module in theory. The label can be used for any cloud platform and large models with optimal open source communities in different fields, and the problems of pre-identification and generalization of customized identification can be solved.
The industry identification of the generic label is added, which is used for identifying the industry of the universal label identification model, such as commodities and automobiles. Further subdivision industry identification is used to distinguish specific scenarios, such as IP images, commodity categories, vehicle brands. The platform identification is used to represent different cloud platforms, for example: arian cloud, baidu cloud, hua as cloud and open source platform.
A general abstract interface is designed for an Arian cloud, a Baidu cloud, a Hua-Yuan cloud, an open source platform and the like in a tag identification system;
All platform generic tag identification models must follow the definition of the abstraction layer to achieve final access. Specifically including input, output data formats, output success and error codes, and the like.
The model layer is used for storing based on the general label identification model data structure design, the mainstream cloud platform provides an API service generally, the open source platform provides a model file, and the model file can be deployed as an API service. Thus, the model layer can unify the call address of the external service url, and the models are loaded and configured according to the realization of the abstract layer.
The generic tag identification model data, because of the generalized scenario, is allowed to be shared across enterprises, each enterprise user can create configuration entries that associate multiple large models for implementing the identification needs of different specific scenarios.
One embodiment is: in a vehicle type recognition scene, whether the picture contains a vehicle, the position of the picture where the vehicle is located and the general vehicle type of the vehicle can be recognized through a general large model in the vehicle industry, such as: suv, mpv, cars and trucks, but the generic tag identification model has no way to obtain what series, year, model, power and characteristics these vehicles are.
The series, year, model, power and characteristics of the vehicle are identified through the customized tag identification model, for example, the vehicle a can be identified through data training of the model a of the brand A of the vehicle enterprise.
S104, inputting the training set into the custom tag recognition model for training to obtain a trained custom tag recognition model.
Specifically, the preprocessing data is divided into a training set, a verification set and a test set;
Training the custom tag recognition model based on the training set;
performing performance evaluation on the trained custom tag recognition model based on the verification set to obtain a custom tag recognition model meeting performance conditions;
And evaluating the identification result of the customized label identification model meeting the performance condition based on the test set to obtain an evaluation index corresponding to the customized label identification model.
Performing performance evaluation on the trained custom tag recognition model based on the verification set to obtain a custom tag recognition model meeting performance conditions; and evaluating the identification result of the customized label identification model meeting the performance condition based on the test set to obtain an evaluation index corresponding to the customized label identification model. Performing performance evaluation on the custom tag identification model to obtain a percent score (namely, the highest score is 100 points and the lowest score is 0 points), and determining the custom tag identification model with the score larger than a set value based on the percent score, for example, the custom tag identification model with the score larger than 90 points is the custom tag identification model meeting the performance condition;
And performing evaluation index calculation on the customized tag identification model meeting the performance condition to obtain evaluation indexes of the customized tag identification model, and calculating to obtain an evaluation value corresponding to each evaluation index, wherein the evaluation value is used for representing the capability value of the customized tag identification model on the evaluation indexes.
S105, inputting the material data to be identified into a universal label identification model to obtain a universal label of the material data to be identified;
Specifically, text recognition, scene recognition and style recognition are carried out on the material data to be recognized based on the universal tag recognition model, and recognition results are obtained;
and extracting the identification result to obtain the universal label of the material data to be identified.
S106, inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified;
Specifically, the customized extraction module adopts a hybrid model, can be applied to different extraction tasks such as classification, target detection and the like, meanwhile, the customized extraction module is different from the general module in that the customized extraction module supports customized training, a user can extract data from the customized extraction module and train a dedicated model according to the user data, and the customized extraction module still provides a general base which allows model AP I services generated by training of each cloud platform and offline model loading obtained by training of an open source large model to be integrated in unlike the traditional mode. Another important feature is that when the front-end hybrid general extraction module is allowed, for example, when vehicles are identified, the positions and coordinates of the vehicles are judged based on vehicle detection, then the vehicle enterprise brand large model is based, finally the custom label identification model is combined based on a plurality of front-end large models, and an accurate brand and model label is output to users.
And the custom tag identification model data structure is designed by multiplexing a general large model data structure on a data structure and only adding an identification to distinguish custom and general.
Customizing a definition of a label identification model multiplexing general label identification model layer;
The main stream cloud platform training output of the tag identification model is also an AP I service, the open source platform provides a model file, and the model file can be deployed as an AP I service. The model layer may load and configure the models according to the implementation of the abstraction layer.
The custom tag identification model differs from the generic tag identification model in that the custom tag identification model is unique to a single enterprise user, does not support sharing, and is data isolated by enterprise identification. Multiple generic recognition tasks may be associated in a custom recognized task, here by defining a workflow. For example, before the custom identification test, the custom identification test is started, and after the custom identification test, a plurality of general tag identification models are allowed to be applied in sequence.
And aggregating the identification result set, and reasoning and matching the own labels of the users through natural semantic processing according to the loaded user own label system database. For example, when a brand and model of a vehicle are obtained, the enterprise own database is loaded in a linkage way, all data of the existing vehicle types are loaded, and vehicle characteristic information such as power information and configuration information of the vehicle can be further obtained through NLP processing.
And the minimum granularity plug-in design is adopted, for example, the general extraction module and the custom extraction module are independent designs, and can be flexibly combined and configured to work independently. For example, in vehicle type recognition, the general tag recognition model may select vehicle recognition of hundred degrees cloud, vehicle brand recognition of ali cloud, and the custom tag recognition model may be an open source model Resnet, or may be EasyD l of hundred degrees, and Pa i of ali cloud. Because each platform interface is unified in the system access layer specification, an abstract layer is provided at the upper layer of the platform service, so that the docking of the mainstream platform model can be compatible.
S107, obtaining a final label of the material data to be identified based on the general label and the custom label reasoning;
specifically, the universal label and the customized label are input into a natural language processing model, and the user own label is deduced and matched from a user own label system database through the natural language processing model;
and obtaining a final label based on the user own label, the universal label and the custom label.
According to the method for intelligently generating the material labels, a certain amount of material data are obtained, and preprocessing is carried out on the material data to obtain preprocessed data; constructing a training set based on the preprocessing data; constructing a custom tag identification model, wherein the custom tag identification model is related to a general tag identification model based on identification task requirements; inputting the training set into the custom tag recognition model for training to obtain a trained custom tag recognition model; inputting the material data to be identified into the universal label identification model to obtain a universal label of the material data to be identified; inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified; and reasoning to obtain a final label of the material data to be identified based on the universal label and the custom label. The problem that in the prior art, material labels cannot be accurately generated based on label generation requirements is solved.
The method for intelligently generating the material tag reduces the labor cost: the universal label recognition model is combined with the customized label recognition model, so that the diversity of labels and the laminating capability of materials can be effectively improved, the manual intervention cost can be effectively reduced, and the accuracy is improved: the mixing capability of the custom tag recognition model enables the custom tag recognition model to be more accurate in the vertical subdivision scene extraction capability, such as: vehicle age model, commodity series, model, IP attribute, etc.;
The method for intelligently generating the material labels is flexible and has expandability: based on plug-in, users in different industries of the SaaS platform can customize plug-in, thereby meeting the user customization requirements in various industry scenes and having better retrieval and propagation effects; the labels extracted by the intelligent label service system are used, so that materials are more easily retrieved, and valuable labels are extracted to be more beneficial to marketing propagation.
FIG. 2 is a diagram of a system architecture for intelligently generating material tags in accordance with an embodiment of the present invention; as shown in fig. 2, a system for intelligently generating a material tag according to an embodiment of the present invention includes the following steps:
The preprocessing module 10 is used for acquiring a certain amount of material data, and preprocessing the material data to obtain preprocessed data;
The preprocessing module 10 is further configured to:
Judging whether the material data are video data or not, if so, dividing the video data into a plurality of sub-videos according to a lens;
determining at least one video frame from the sub-video as a key frame of the sub-video;
And constructing a training set based on the key frames.
A first construction module 20 for constructing a training set based on the pre-processed data;
A second building module 30, configured to build a custom tag identification model, where the custom tag identification model is related to a generic tag identification model based on an identification task requirement;
The second building block 30 is further configured to:
Classifying the universal label identification model based on an industry identifier;
determining an industry identifier and a platform identifier corresponding to the identification task;
and determining a universal label recognition model to be associated based on the industry identifier, the platform identifier and the recognition task, wherein the number of the universal label recognition models is at least one.
Acquiring url calling addresses corresponding to the universal label identification models to be associated;
And associating the customized tag identification model with the to-be-associated universal tag identification model based on the url call address.
The training module 40 is configured to input the training set into the custom tag identification model for training, so as to obtain a trained custom tag identification model;
The training module 40 is further configured to:
Dividing the preprocessing data into a training set, a verification set and a test set;
Training the custom tag recognition model based on the training set;
performing performance evaluation on the trained custom tag recognition model based on the verification set to obtain a custom tag recognition model meeting performance conditions;
And evaluating the identification result of the customized label identification model meeting the performance condition based on the test set to obtain an evaluation index corresponding to the customized label identification model.
Inputting the material data to be identified into the universal label identification model to obtain a universal label of the material data to be identified;
performing character recognition, scene recognition and style recognition on the material data to be recognized based on the universal tag recognition model, and obtaining a recognition result;
and extracting the identification result to obtain the universal label of the material data to be identified.
Inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified;
And the reasoning module 50 is used for reasoning and obtaining the final label of the material data to be identified based on the universal label and the custom label.
The reasoning module 50 is further configured to:
Inputting the universal label and the custom label into a natural language processing model, and reasoning and matching the user own label from a user own label system database through the natural language processing model;
and obtaining a final label based on the user own label, the universal label and the custom label.
According to the system for intelligently generating the material labels, a certain amount of material data is acquired through the preprocessing module 10, and the material data is preprocessed to obtain preprocessed data; building a training set based on the pre-processed data by a first building module 20; constructing a custom tag identification model by a second construction module 30, wherein the custom tag identification model is related to a general tag identification model based on identification task requirements; inputting the training set into the custom tag recognition model through a training module 40 for training to obtain a trained custom tag recognition model; inputting the material data to be identified into the universal label identification model to obtain a universal label of the material data to be identified; inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified; the final tag of the material data to be identified is obtained by inference based on the generic tag and the custom tag through an inference module 50. The method for intelligently generating the material labels solves the problem that the material labels cannot be accurately generated based on label generation requirements in the prior art.
Fig. 3 is a schematic diagram of an entity structure of an electronic device according to an embodiment of the present invention, as shown in fig. 3, an electronic device 60 includes: a processor 601 (processor), a memory 602 (memory), and a bus 603;
Wherein, the processor 601 and the memory 602 complete communication with each other through the bus 603;
The processor 601 is configured to invoke program instructions in the memory 602 to perform the methods provided by the method embodiments described above, including, for example: acquiring a certain amount of material data, and preprocessing the material data to obtain preprocessed data; constructing a training set based on the preprocessing data; constructing a custom tag identification model, wherein the custom tag identification model is related to a general tag identification model based on identification task requirements; inputting the training set into the custom tag recognition model for training to obtain a trained custom tag recognition model; inputting the material data to be identified into the universal label identification model to obtain a universal label of the material data to be identified; inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified; and reasoning to obtain a final label of the material data to be identified based on the universal label and the custom label.
The present embodiment provides a non-transitory computer readable storage medium storing computer instructions that cause a computer to perform the methods provided by the above-described method embodiments, for example, including: acquiring a certain amount of material data, and preprocessing the material data to obtain preprocessed data; constructing a training set based on the preprocessing data; constructing a custom tag identification model, wherein the custom tag identification model is related to a general tag identification model based on identification task requirements; inputting the training set into the custom tag recognition model for training to obtain a trained custom tag recognition model; inputting the material data to be identified into the universal label identification model to obtain a universal label of the material data to be identified; inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified; and reasoning to obtain a final label of the material data to be identified based on the universal label and the custom label.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware associated with program instructions, where the foregoing program may be stored in a computer readable storage medium, and when executed, the program performs steps including the above method embodiments; and the aforementioned storage medium includes: various storage media such as ROM, RAM, magnetic or optical disks may store program code.
The apparatus embodiments described above are merely illustrative, wherein elements illustrated as separate elements may or may not be physically separate, and elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on such understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the embodiments or the methods of some parts of the embodiments.
While the invention has been described in detail in the foregoing general description and specific examples, it will be apparent to those skilled in the art that modifications and improvements can be made thereto. Accordingly, such modifications or improvements may be made without departing from the spirit of the invention and are intended to be within the scope of the invention as claimed.
Claims (7)
1. A method for intelligently generating material tags, the method comprising:
Acquiring a certain amount of material data, and preprocessing the material data to obtain preprocessed data, wherein the material data comprises video data;
constructing a training set based on the preprocessing data;
Constructing a custom tag identification model, wherein the custom tag identification model is associated with a general tag identification model based on identification task requirements, and comprises the following steps:
Classifying the universal label identification model based on an industry identifier;
determining an industry identifier and a platform identifier corresponding to the identification task;
determining a universal label recognition model to be associated based on the industry identifier, the platform identifier and the recognition task, wherein the number of the universal label recognition models is at least one;
Acquiring url calling addresses corresponding to the universal label identification models to be associated;
Associating the custom tag identification model with the universal tag identification model to be associated based on the url call address;
inputting the training set into the custom tag recognition model for training to obtain a trained custom tag recognition model;
inputting the material data to be identified into the universal label identification model to obtain a universal label of the material data to be identified;
Inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified;
And reasoning the final label of the material data to be identified based on the universal label and the custom label, wherein the final label comprises the following steps:
Inputting the universal label and the custom label into a natural language processing model, and reasoning and matching the user own label from a user own label system database through the natural language processing model;
and obtaining a final label based on the user own label, the universal label and the custom label.
2. The method for intelligently generating material labels according to claim 1, wherein the obtaining a certain amount of material data and preprocessing the material data to obtain preprocessed data comprises:
Judging whether the material data are video data or not, if so, dividing the video data into a plurality of sub-videos according to a lens;
determining at least one video frame from the sub-video as a key frame of the sub-video;
And constructing a training set based on the key frames.
3. The method for intelligently generating material labels according to claim 1, wherein said inputting the training set into the custom label recognition model for training to obtain a trained custom label recognition model comprises:
Dividing the preprocessing data into a training set, a verification set and a test set;
Training the custom tag recognition model based on the training set;
performing performance evaluation on the trained custom tag recognition model based on the verification set to obtain a custom tag recognition model meeting performance conditions;
And evaluating the identification result of the customized label identification model meeting the performance condition based on the test set to obtain an evaluation index corresponding to the customized label identification model.
4. The method for intelligently generating the material tag according to claim 1, wherein the step of inputting the material data to be identified into the universal tag identification model to obtain the universal tag of the material data to be identified comprises the steps of:
performing character recognition, scene recognition and style recognition on the material data to be recognized based on the universal tag recognition model, and obtaining a recognition result;
and extracting the identification result to obtain the universal label of the material data to be identified.
5. A system for intelligently generating material tags, comprising:
The preprocessing module is used for acquiring a certain amount of material data, preprocessing the material data to obtain preprocessed data, wherein the material data comprises video data;
A first construction module for constructing a training set based on the preprocessing data;
the second construction module is configured to construct a custom tag identification model, where the custom tag identification model associates a generic tag identification model based on an identification task requirement, and includes:
Classifying the universal label identification model based on an industry identifier;
determining an industry identifier and a platform identifier corresponding to the identification task;
determining a universal label recognition model to be associated based on the industry identifier, the platform identifier and the recognition task, wherein the number of the universal label recognition models is at least one;
Acquiring url calling addresses corresponding to the universal label identification models to be associated;
Associating the custom tag identification model with the universal tag identification model to be associated based on the url call address;
the training module is used for inputting the training set into the customized label recognition model for training to obtain a trained customized label recognition model;
inputting the material data to be identified into the universal label identification model to obtain a universal label of the material data to be identified;
Inputting the material data to be identified into a trained custom tag identification model to obtain custom tags of the material data to be identified;
The reasoning module is used for reasoning the final label of the material data to be identified based on the universal label and the custom label, and comprises the following steps:
Inputting the universal label and the custom label into a natural language processing model, and reasoning and matching the user own label from a user own label system database through the natural language processing model;
and obtaining a final label based on the user own label, the universal label and the custom label.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 4 when the computer program is executed.
7. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311454740.8A CN117456416B (en) | 2023-11-03 | Method and system for intelligently generating material labels |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311454740.8A CN117456416B (en) | 2023-11-03 | Method and system for intelligently generating material labels |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117456416A CN117456416A (en) | 2024-01-26 |
| CN117456416B true CN117456416B (en) | 2024-06-07 |
Family
ID=
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104091333A (en) * | 2014-07-01 | 2014-10-08 | 黄河科技学院 | Multi-class unsupervised color texture image segmentation method based on credible regional integration |
| CN104408153A (en) * | 2014-12-03 | 2015-03-11 | 中国科学院自动化研究所 | Short text hash learning method based on multi-granularity topic models |
| CN111104514A (en) * | 2019-12-23 | 2020-05-05 | 北京百度网讯科技有限公司 | Method and device for training document label model |
| CN111783861A (en) * | 2020-06-22 | 2020-10-16 | 北京百度网讯科技有限公司 | Data classification method, model training device and electronic equipment |
| US10891539B1 (en) * | 2017-10-31 | 2021-01-12 | STA Group, Inc. | Evaluating content on social media networks |
| CN113836304A (en) * | 2021-09-27 | 2021-12-24 | 山东亿云信息技术有限公司 | Intelligent labeling method and system based on natural language processing |
| CN113919357A (en) * | 2021-10-29 | 2022-01-11 | 平安普惠企业管理有限公司 | Method, device and equipment for training address entity recognition model and storage medium |
| CN114529888A (en) * | 2022-01-12 | 2022-05-24 | 盛视科技股份有限公司 | Non-motor vehicle driving recognition method, device, computer and readable storage medium |
| CN114764920A (en) * | 2022-04-01 | 2022-07-19 | 中山大学 | Pedestrian identification method, system, device and storage medium |
| CN115547311A (en) * | 2022-10-13 | 2022-12-30 | 北京有竹居网络技术有限公司 | Voice recognition method and device, electronic equipment and storage medium |
| CN116152750A (en) * | 2022-11-27 | 2023-05-23 | 天翼云科技有限公司 | Vehicle feature recognition method based on monitoring image |
| CN116541740A (en) * | 2022-01-24 | 2023-08-04 | 北京华航无线电测量研究所 | Radar target identification method based on multi-layer classification model |
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104091333A (en) * | 2014-07-01 | 2014-10-08 | 黄河科技学院 | Multi-class unsupervised color texture image segmentation method based on credible regional integration |
| CN104408153A (en) * | 2014-12-03 | 2015-03-11 | 中国科学院自动化研究所 | Short text hash learning method based on multi-granularity topic models |
| US10891539B1 (en) * | 2017-10-31 | 2021-01-12 | STA Group, Inc. | Evaluating content on social media networks |
| CN111104514A (en) * | 2019-12-23 | 2020-05-05 | 北京百度网讯科技有限公司 | Method and device for training document label model |
| CN111783861A (en) * | 2020-06-22 | 2020-10-16 | 北京百度网讯科技有限公司 | Data classification method, model training device and electronic equipment |
| CN113836304A (en) * | 2021-09-27 | 2021-12-24 | 山东亿云信息技术有限公司 | Intelligent labeling method and system based on natural language processing |
| CN113919357A (en) * | 2021-10-29 | 2022-01-11 | 平安普惠企业管理有限公司 | Method, device and equipment for training address entity recognition model and storage medium |
| CN114529888A (en) * | 2022-01-12 | 2022-05-24 | 盛视科技股份有限公司 | Non-motor vehicle driving recognition method, device, computer and readable storage medium |
| CN116541740A (en) * | 2022-01-24 | 2023-08-04 | 北京华航无线电测量研究所 | Radar target identification method based on multi-layer classification model |
| CN114764920A (en) * | 2022-04-01 | 2022-07-19 | 中山大学 | Pedestrian identification method, system, device and storage medium |
| CN115547311A (en) * | 2022-10-13 | 2022-12-30 | 北京有竹居网络技术有限公司 | Voice recognition method and device, electronic equipment and storage medium |
| CN116152750A (en) * | 2022-11-27 | 2023-05-23 | 天翼云科技有限公司 | Vehicle feature recognition method based on monitoring image |
Non-Patent Citations (3)
| Title |
|---|
| 基于BP神经网络的夏玉米多生育期叶面积指数反演研究;刘俊;孟庆岩;葛小三;刘顺喜;陈旭;孙云晓;;遥感技术与应用;20200220(第01期);论文全文 * |
| 基于灰色关联理论与经验模态分解的光伏出力超短期预测;索春梅;孙健;张宗峰;王贤宗;;中国电力;20171205(第12期);论文全文 * |
| 孤养对单次延长应激建立的创伤后应激障碍模型大鼠行为学的影响;方锦颖;陈光耀;李龙梅;郑思思;毛萌;;中国比较医学杂志;20180223(第03期);论文全文 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102253936B (en) | Method for recording access of user to merchandise information, search method and server | |
| CN107844634A (en) | Polynary universal model platform modeling method, electronic equipment and computer-readable recording medium | |
| CA3052527C (en) | Target document template generation | |
| CN108416003A (en) | A kind of picture classification method and device, terminal, storage medium | |
| US11176464B1 (en) | Machine learning-based recommendation system for root cause analysis of service issues | |
| CN110782318A (en) | Marketing method and device based on audio interaction and storage medium | |
| CN111209374B (en) | Data query display method, device, computer system and readable storage medium | |
| CN109542956A (en) | Report form generation method, device, computer equipment and storage medium | |
| KR101472451B1 (en) | System and Method for Managing Digital Contents | |
| CN115018549A (en) | Method for generating advertisement file, device, equipment, medium and product thereof | |
| CN105164672A (en) | Content classification | |
| CN110766460A (en) | User portrait drawing method and device, storage medium and computer equipment | |
| CN117456416B (en) | Method and system for intelligently generating material labels | |
| CN117131222A (en) | Semi-automatic labeling method and device based on open world large model | |
| CN110998542B (en) | Data exchange system, data exchange method, and data exchange program | |
| US20230137639A1 (en) | Data processing system and method for operating an enterprise application | |
| CN117456416A (en) | Method and system for intelligently generating material labels | |
| CN112182413B (en) | Intelligent recommendation method and server based on big teaching data | |
| CN115565042A (en) | Commodity image feature representation method and device, equipment, medium and product thereof | |
| CN115017385A (en) | Article searching method, device, equipment and storage medium | |
| CN110019809A (en) | A kind of classification determines method, apparatus and the network equipment | |
| US20210064652A1 (en) | Camera input as an automated filter mechanism for video search | |
| CN113806537A (en) | Commodity category classification method and device, equipment, medium and product thereof | |
| CN113449808A (en) | Multi-source image-text information classification method and corresponding device, equipment and medium | |
| CN112418260A (en) | Model training method, information prompting method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |