CN117171413B - Data processing system and method for digital collection management - Google Patents
Data processing system and method for digital collection management Download PDFInfo
- Publication number
- CN117171413B CN117171413B CN202311150900.XA CN202311150900A CN117171413B CN 117171413 B CN117171413 B CN 117171413B CN 202311150900 A CN202311150900 A CN 202311150900A CN 117171413 B CN117171413 B CN 117171413B
- Authority
- CN
- China
- Prior art keywords
- search query
- feature vector
- digital collection
- semantic
- alternative digital
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 title abstract description 16
- 239000013598 vector Substances 0.000 claims abstract description 155
- 238000004458 analytical method Methods 0.000 claims abstract description 23
- 238000005457 optimization Methods 0.000 claims description 20
- 238000003672 processing method Methods 0.000 claims description 15
- 230000003993 interaction Effects 0.000 claims description 14
- 230000007246 mechanism Effects 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000012790 confirmation Methods 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000007477 logistic regression Methods 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
A data processing system for digital collection management and a method thereof are disclosed. Firstly, acquiring a digital collection search query input by a user, then extracting text description of an alternative digital collection from a database, then carrying out joint semantic analysis on the digital collection search query and the text description of the alternative digital collection to obtain a search query-alternative digital collection semantic matching feature vector, and finally, determining whether to return the alternative digital collection based on the search query-alternative digital collection semantic matching feature vector. In this way, semantic matching between the digital collection search query and the alternative digital collection text description can be intelligently achieved.
Description
Technical Field
The present disclosure relates to the field of data processing, and more particularly, to a data processing system for digital collection management and a method thereof.
Background
The digital collection refers to cultural heritage, artwork, historical documents and the like which are stored and displayed in a digital form, and has important cultural value and social significance. The management and the search of the digital collection are key links of the protection and the utilization of the digital collection, and relate to a plurality of aspects of collection, arrangement, storage, display, sharing and the like of the digital collection. In order to improve the efficiency and accuracy of digital collection management and retrieval, effective data processing is required to be performed on the digital collections so as to realize quick retrieval and accurate matching of the digital collections.
Currently, search queries for digital collections are typically performed based on manually set rules. However, this method requires a large number of manual writing, and the written rules may be difficult to cover all cases, and it is difficult to apply digital stock data in different fields and different languages, resulting in low retrieval efficiency and limited retrieval range.
Thus, an optimized data processing scheme for digital collection management is desired.
Disclosure of Invention
In view of this, the present disclosure proposes a data processing system for digital collection management and a method thereof, which can intelligently implement semantic matching between a digital collection search query and an alternative digital collection text description.
According to an aspect of the present disclosure, there is provided a data processing method for digital collection management, including:
acquiring digital collection retrieval query input by a user;
extracting a text description of the alternative digital collection from the database;
carrying out joint semantic analysis on the digital collection search query and the text description of the alternative digital collection to obtain a search query-alternative digital collection semantic matching feature vector; and
and determining whether to return the alternative digital collection based on the search query-alternative digital collection semantic matching feature vector.
According to another aspect of the present disclosure, there is provided a data processing system for digital collection management, comprising:
the input acquisition module is used for acquiring digital collection retrieval query input by a user;
the text description extraction module is used for extracting the text description of the alternative digital collection from the database;
the joint semantic analysis module is used for carrying out joint semantic analysis on the digital collection search query and the text description of the alternative digital collection so as to obtain a search query-alternative digital collection semantic matching feature vector; and
and the return confirmation module is used for determining whether to return the alternative digital collection based on the search query-alternative digital collection semantic matching feature vector.
According to the embodiment of the disclosure, firstly, a digital collection search query input by a user is acquired, then, a text description of an alternative digital collection is extracted from a database, then, joint semantic analysis is carried out on the digital collection search query and the text description of the alternative digital collection to obtain a search query-alternative digital collection semantic matching feature vector, and finally, whether the alternative digital collection is returned or not is determined based on the search query-alternative digital collection semantic matching feature vector. In this way, semantic matching between the digital collection search query and the alternative digital collection text description can be intelligently achieved.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the present disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 illustrates a flow chart of a data processing method for digital collection management according to an embodiment of the present disclosure.
Fig. 2 shows an architectural diagram of a data processing method for digital collection management according to an embodiment of the present disclosure.
Fig. 3 shows a flowchart of sub-step S130 of a data processing method for digital collection management according to an embodiment of the present disclosure.
Fig. 4 shows a flowchart of sub-step S140 of a data processing method for digital collection management according to an embodiment of the present disclosure.
FIG. 5 illustrates a block diagram of a data processing system for digital collection management, according to an embodiment of the present disclosure.
Fig. 6 illustrates an application scenario diagram of a data processing method for digital collection management according to an embodiment of the present disclosure.
Detailed Description
The following description of the embodiments of the present disclosure will be made clearly and fully with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some, but not all embodiments of the disclosure. All other embodiments, which can be made by one of ordinary skill in the art without undue burden based on the embodiments of the present disclosure, are also within the scope of the present disclosure.
As used in this disclosure and in the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
In addition, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.
Aiming at the technical problems, the technical concept of the present disclosure is to extract semantic features from digital collection search query and alternative digital collection text description by adopting a natural language processing technology based on deep learning, so as to intelligently realize semantic matching between the digital collection search query and the alternative digital collection text description.
Based on this, fig. 1 shows a flowchart of a data processing method for digital collection management according to an embodiment of the present disclosure. Fig. 2 shows an architectural diagram of a data processing method for digital collection management according to an embodiment of the present disclosure. As shown in fig. 1 and 2, a data processing method for digital collection management according to an embodiment of the present disclosure includes the steps of: s110, acquiring digital collection retrieval query input by a user; s120, extracting text description of the alternative digital collection from the database; s130, carrying out joint semantic analysis on the digital collection search query and the text description of the alternative digital collection to obtain a search query-alternative digital collection semantic matching feature vector; and S140, determining whether to return the alternative digital collection based on the search query-alternative digital collection semantic matching feature vector.
Specifically, in the technical scheme of the present disclosure, firstly, a digital collection search query input by a user is obtained; at the same time, a textual description of the alternative digital collection is extracted from the database.
In digital collection management systems, the search query entered by the user is typically in the form of natural language, and the textual description of the alternative digital collection also contains rich semantic information. Merely by keyword matching or simple text similarity calculation, the query intent of the user and the content features of the digital collection often cannot be sufficiently captured. Therefore, in the technical solution of the present disclosure, it is expected that the digital collection search query and the text description of the alternative digital collection are subjected to joint semantic analysis to obtain a search query-alternative digital collection semantic matching feature vector. In this way, the model is enabled to better understand the semantic relationships between them for deep understanding and matching.
In a specific example of the present disclosure, the encoding process for performing joint semantic analysis on the digital collection search query and the text description of the candidate digital collection to obtain a search query-candidate digital collection semantic matching feature vector includes: firstly, carrying out semantic coding on the digital collection search query to obtain a digital collection search query semantic coding feature vector; simultaneously, carrying out semantic coding on the text description of the alternative digital stock to obtain a semantic coding feature vector of the text description of the alternative digital stock; and then using an attention layer between features to perform feature interaction based on an attention mechanism on the alternative digital collection text description semantic coding feature vector and the digital collection search query semantic coding feature vector so as to obtain a search query-alternative digital collection semantic matching feature vector.
It is worth mentioning that the goal of the traditional attention mechanism is to learn an attention weight matrix, applied to the individual neural nodes of the current layer, giving them greater weight for those important nodes and less weight for those secondary nodes. Because each neural node contains certain characteristic information, the neural network can select information which is more critical to the current task target from a plurality of characteristic information through the operation. The attention layers among the features are different, and the dependency relationship among the feature information is focused more.
Accordingly, as shown in fig. 3, performing joint semantic analysis on the digital collection search query and the text description of the alternative digital collection to obtain a search query-alternative digital collection semantic matching feature vector, including: s131, carrying out semantic coding on the digital collection search query to obtain a digital collection search query semantic coding feature vector; s132, carrying out semantic coding on the text description of the alternative digital stock to obtain a semantic coding feature vector of the text description of the alternative digital stock; and S133, fusing the description semantic coding feature vector of the alternative digital collection text and the digital collection search query semantic coding feature vector to obtain the search query-alternative digital collection semantic matching feature vector. It should be appreciated that this section involves three steps S131, S132 and S133 for performing joint semantic analysis to obtain semantically matched feature vectors for search query-alternative digital collections. The purpose of step S131 is to convert the search query into semantic codes so as to be able to represent the semantic meaning of the query, the semantically encoded feature vectors may capture key semantic information of the query, such as the subject matter, intent, and features of the query. The purpose of step S132 is to convert the text description of the alternative digital collection into semantic codes so as to be able to represent the semantic meaning of the text description, and the semantic code feature vectors can capture key semantic information of the text description, such as the features, content and style of the description. The purpose of step S133 is to fuse the semantic coding feature vector of the query with the semantic coding feature vector of the alternative digital collection to obtain a comprehensive feature vector, reflecting the degree of semantic matching between the query and the alternative digital collection, where the feature vector may be used to calculate the degree of similarity or matching between the query and the alternative digital collection, so as to perform retrieval and recommendation of the digital collection. Through the combined semantic analysis of the three steps, the semantic information of the query and the alternative digital collection can be encoded and matched, so that more accurate digital collection retrieval and recommendation are realized.
More specifically, in step S133, fusing the candidate digital stock text description semantic coding feature vector and the digital stock search query semantic coding feature vector to obtain the search query-candidate digital stock semantic matching feature vector, including: and performing feature interaction based on an attention mechanism on the alternative digital collection text description semantic coding feature vector and the digital collection search query semantic coding feature vector by using an attention layer among features to obtain the search query-alternative digital collection semantic matching feature vector. It should be appreciated that the role of step S133 is to capture important semantic interaction information between the query and the alternative digital stock through a attentive mechanism to better gauge the degree of matching between them. The inter-feature attention layer can automatically learn and assign different attention weights according to the similarity and the association degree between the semantically encoded feature vectors of the query and the alternative digital collection. In this way, important semantic interaction information can get more attention, and unimportant information can get less attention, and in this way, the expression capability and the discrimination of the semantic matching feature vectors can be improved. The combination of the feature interaction and the attention mechanism can better capture the semantic relation between the query and the alternative digital collection, thereby improving the accuracy of the search query and the accuracy of recommendation. Through the semantic matching feature vector, the similarity or the matching degree between the alternative digital collection and the query can be better evaluated, and further more accurate digital collection recommendation and retrieval are realized.
It is worth mentioning that the inter-feature attention layer is a layer in a neural network for learning the relevance and importance between different features. It dynamically assigns weights between different features through an attention mechanism to focus more on important features during feature interactions. In the inter-feature attention layer, the input is typically a set of feature vectors, which may be features from different sources or representing different aspects. Each feature vector is associated with a weight that represents the importance of the feature in the feature interaction. These weights may be calculated and adjusted by the attention mechanism. The attention mechanism typically calculates the weights based on similarity or correlation between feature vectors. It may use different methods such as dot product attention, additive attention, or multi-headed attention to calculate the degree of association between features and generate corresponding weights. These weights may be used to weight a linear combination of features to achieve interaction between features. Through the inter-feature attention layer, the network can automatically learn the association and importance between features and adjust the weights of the features according to the needs of the task. This may improve the modeling capabilities of the model for interactions and associations between different features, thereby improving the expressive power and performance of the model. In the semantic matching task, the inter-feature attention layer can help capture important semantic interaction information between the query and the alternative digital collection, so that matching accuracy is improved.
And then, passing the search query-alternative digital collection semantic matching feature vector through a classifier to obtain a classification result, wherein the classification result is used for indicating whether to return the alternative digital collection.
Accordingly, as shown in fig. 4, based on the search query-alternative digital collection semantic matching feature vector, determining whether to return the alternative digital collection includes: s141, performing feature distribution optimization on the search query-alternative digital stock semantic matching feature vector to obtain an optimized search query-alternative digital stock semantic matching feature vector; and S142, enabling the optimized search query-alternative digital collection semantic matching feature vector to pass through a classifier to obtain a classification result, wherein the classification result is used for indicating whether to return the alternative digital collection. It should be understood that the step S141 aims to improve the expressive power and the discrimination of feature vectors by adjusting the distribution of features. Through feature distribution optimization, semantic matching information between the query and the alternative digital collection can be captured better, and matching accuracy is improved. In step S142, the semantic matching feature vector of the optimized search query-alternative digital stock is passed through a classifier to obtain a classification result, where the classification result is used to indicate whether to return the alternative digital stock. The classifier may be a classifier for determining whether to return the alternative digital collection or a multi-classifier for determining the category or ordering of the return alternative digital collection. The alternative digital collections can be screened and ordered according to the semantic matching degree of the feature vectors through the classifier, so that the most relevant and proper alternative digital collections are provided. The purpose of these two steps is to further optimize and decide on the basis of the semantically matched feature vectors to determine whether to return an alternative digital stock. The feature distribution optimization can improve the representation capability of the feature vector, so that the feature vector is more suitable for classification decision. And the classifier makes a specific decision according to the matching degree of the feature vectors to determine whether to return the alternative digital stock. The goal of these steps is to improve the accuracy of the retrieval and recommendation of digital collections, enabling users to obtain alternative digital collections that better meet their needs.
In the technical scheme of the disclosure, the candidate digital collection text description semantic coding feature vector and the digital collection search query semantic coding feature vector respectively express text semantic features of the digital collection search query and the text description of the candidate digital collection, so that when feature interaction based on an attention mechanism is performed by using an inter-feature attention layer, the search query-candidate digital collection semantic matching feature vector can express text semantic dependency relationship features of the text semantic features of the text description of the digital collection search query and the candidate digital collection, and when the text semantic features of the digital collection search query and the text description of the candidate digital collection are taken as foreground object features, the inter-feature attention layer can also introduce background distribution noise related to feature distribution interference of source text semantic features, and the search query-candidate digital collection semantic matching feature vector also further has single sample semantic feature and sample inter-sample hierarchical space features, so that the search query-candidate digital collection semantic matching feature vector expresses the desired feature matching effect based on the expression of the search query-candidate digital collection.
Accordingly, the applicant of the present disclosure performs a probability density feature imitation paradigm-based distribution gain on the search query-alternative digital stock semantic matching feature vectors.
Accordingly, in one particular example, the search query-alternative numberPerforming feature distribution optimization on the word stock semantic matching feature vector to obtain an optimized search query-alternative digital stock semantic matching feature vector, including: performing feature distribution optimization on the search query-alternative digital stock semantic matching feature vector by using the following optimization formula to obtain the optimized search query-alternative digital stock semantic matching feature vector; wherein, the optimization formula is:wherein (1)>Is the semantic matching feature vector of the search query-alternative digital collection +.>Is the length of the semantic matching feature vector of the search query-alternative digital collection, +.>Is the +.f. of the search query-alternative digital stock semantic matching feature vector>Characteristic value of individual position->Square of two norms representing the semantic matching feature vector of the search query-alternative digital collection, and +.>Is a weighted superparameter,/->An exponential operation representing a numerical value, the exponential operation representing the calculation of a natural exponential function value that is a power of the numerical value,a +.f. representing the semantic matching feature vector of the optimized search query-alternative digital stock>Characteristic values of the individual positions.
Here, based on the characteristic simulation paradigm of the standard cauchy distribution on the probability density for the natural gaussian distribution, the distribution gain based on the probability density characteristic simulation paradigm can use the characteristic scale as a simulation mask to distinguish foreground object characteristics and background distribution noise in a high-dimensional characteristic space, so that the unconstrained distribution gain of the high-dimensional characteristic distribution is obtained by carrying out the associated semantic cognition distribution soft matching of the characteristic space mapping on the high-dimensional space based on the semantic space grading of the high-dimensional characteristics, the expression effect of the search query-candidate digital collection semantic matching characteristic vector based on the characteristic distribution characteristic is improved, and the accuracy of the classification result obtained by the classifier of the search query-candidate digital collection semantic matching characteristic vector is also improved.
More specifically, in step S142, the optimized search query-alternative digital collection semantic matching feature vector is passed through a classifier to obtain a classification result, where the classification result is used to indicate whether to return the alternative digital collection, and the method includes: using the full connection layer of the classifier to carry out full connection coding on the optimized search query-alternative digital stock semantic matching feature vector so as to obtain a coding classification feature vector; and inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
That is, in the technical solution of the present disclosure, the labels of the classifier include returning the alternative digital stock (first label) and not returning the alternative digital stock (second label), where the classifier determines to which classification label the optimized search query-alternative digital stock semantic matching feature vector belongs through a soft maximum function. It is noted that the first tag p1 and the second tag p2 do not contain a manually set concept, and in fact, during the training process, the computer model does not have a concept of "whether to return an alternative digital stock", which is simply two kinds of classification tags and outputs the probability that the feature is under the two classification tags, i.e., the sum of p1 and p2 is one. Therefore, the classification result of whether to return the alternative digital collection is actually converted into the classified probability distribution conforming to the natural rule through classifying the labels, and the physical meaning of the natural probability distribution of the labels is essentially used instead of the language text meaning of whether to return the alternative digital collection.
It should be appreciated that the role of the classifier is to learn the classification rules and classifier using a given class, known training data, and then classify (or predict) the unknown data. Logistic regression (logistics), SVM, etc. are commonly used to solve the classification problem, and for multi-classification problems (multi-class classification), logistic regression or SVM can be used as well, but multiple bi-classifications are required to compose multiple classifications, but this is error-prone and inefficient, and the commonly used multi-classification method is the Softmax classification function. It should be noted that the fully-connected encoding refers to a process of inputting the semantic matching feature vector of the optimized search query-alternative digital stock into the fully-connected layer for encoding. Fully connected layers are a common layer type in neural networks, where each node is connected to all nodes of the previous layer. In the full-join encoding process, each node performs linear transformation and nonlinear activation operations on the input feature vectors to generate encoded classification feature vectors. The purpose of full-join encoding is to map the input feature vector to a higher dimensional feature space through multi-layer nonlinear transformation, and extract a richer and abstract feature representation. These encoded classification feature vectors may better capture semantic and structural information of the input features, thereby improving the performance and decision making capabilities of the classifier. Full-concatenated coding may be implemented by stacking multiple full-concatenated layers, each of which performs linear transformation and nonlinear activation operations on the input feature vector. Parameters between these layers are learned and optimized during training by back propagation algorithms to enable the encoded classification feature vectors to better represent important information of the input features. The encoded classification feature vector is input into the Softmax classification function of the classifier, which can be converted into a classification result. The Softmax function maps the encoded classification feature vector to a probability distribution, one probability value for each class. These probability values represent the likelihood that the input feature vector belongs to the respective category and can be used for decision and prediction in multi-classification tasks. Through the full-connection coding and the Softmax classification function, the semantic matching feature vector of the optimized search query-alternative digital collection can be converted into the coding classification feature vector with higher expression capability and decision capability, and the classification result is represented through probability distribution. This helps to improve the performance of the classifier, enabling it to more accurately determine whether to return an alternative digital stock.
In summary, according to the data processing method for digital collection management disclosed by the embodiment of the disclosure, semantic matching between digital collection search query and alternative digital collection text description can be intelligently realized.
FIG. 5 illustrates a block diagram of a data processing system 100 for digital collection management, according to an embodiment of the present disclosure. As shown in fig. 5, a data processing system 100 for digital collection management according to an embodiment of the present disclosure includes: an input obtaining module 110, configured to obtain a digital collection search query input by a user; a text description extraction module 120 for extracting a text description of the alternative digital collection from the database; the joint semantic analysis module 130 is configured to perform joint semantic analysis on the digital collection search query and the text description of the alternative digital collection to obtain a search query-alternative digital collection semantic matching feature vector; and a return confirmation module 140, configured to determine whether to return the alternative digital collection based on the search query-alternative digital collection semantic matching feature vector.
In one possible implementation, the joint semantic analysis module 130 includes: the search query semantic coding unit is used for carrying out semantic coding on the digital collection search query to obtain a digital collection search query semantic coding feature vector; the text description semantic coding unit is used for carrying out semantic coding on the text description of the alternative digital stock to obtain an alternative digital stock text description semantic coding feature vector; and the fusion unit is used for fusing the alternative digital stock text description semantic coding feature vector and the digital stock search query semantic coding feature vector to obtain the search query-alternative digital stock semantic matching feature vector.
In a possible implementation manner, the fusion unit is configured to: and performing feature interaction based on an attention mechanism on the alternative digital collection text description semantic coding feature vector and the digital collection search query semantic coding feature vector by using an attention layer among features to obtain the search query-alternative digital collection semantic matching feature vector.
In one possible implementation, the return confirmation module 140 includes: the feature distribution optimizing unit is used for carrying out feature distribution optimization on the search query-alternative digital stock semantic matching feature vector so as to obtain an optimized search query-alternative digital stock semantic matching feature vector; and the classification unit is used for enabling the optimized search query-alternative digital collection semantic matching feature vector to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether to return the alternative digital collection.
In a possible implementation manner, the feature distribution optimizing unit is configured to: performing feature distribution optimization on the search query-alternative digital stock semantic matching feature vector by using the following optimization formula to obtain the optimized search query-alternative digital stock semantic matching feature vector; wherein, the optimization formula is:wherein (1)>Is the semantic matching feature vector of the search query-alternative digital collection +.>Is the length of the semantic matching feature vector of the search query-alternative digital collection, +.>Is the +.f. of the search query-alternative digital stock semantic matching feature vector>Characteristic value of individual position->Square of two norms representing the semantic matching feature vector of the search query-alternative digital collection, and +.>Is a weighted superparameter,/->An exponential operation representing a numerical value, the exponential operation representing the calculation of a natural exponential function value that is a power of the numerical value,a +.f. representing the semantic matching feature vector of the optimized search query-alternative digital stock>Characteristic values of the individual positions.
Here, it will be understood by those skilled in the art that the specific functions and operations of the respective units and modules in the above-described data processing system 100 for digital collection management have been described in detail in the above description of the data processing method for digital collection management with reference to fig. 1 to 4, and thus, repetitive descriptions thereof will be omitted.
As described above, the data processing system 100 for digital collection management according to the embodiment of the present disclosure may be implemented in various wireless terminals, such as a server or the like having a data processing algorithm for digital collection management. In one possible implementation, data processing system 100 for digital collection management according to embodiments of the present disclosure may be integrated into a wireless terminal as one software module and/or hardware module. For example, the data processing system 100 for digital collection management may be a software module in the operating system of the wireless terminal, or may be an application developed for the wireless terminal; of course, the data processing system 100 for digital collection management may also be one of the many hardware modules of the wireless terminal.
Alternatively, in another example, the data processing system 100 for digital collection management and the wireless terminal may be separate devices, and the data processing system 100 for digital collection management may be connected to the wireless terminal through a wired and/or wireless network and transmit interactive information in an agreed data format.
Fig. 6 illustrates an application scenario diagram of a data processing method for digital collection management according to an embodiment of the present disclosure. As shown in fig. 6, in this application scenario, first, a digital collection search query (e.g., D1 illustrated in fig. 6) input by a user and a text description of an alternative digital collection (e.g., D2 illustrated in fig. 6) are acquired from a database, and then the digital collection search query and the text description of the alternative digital collection are input into a server (e.g., S illustrated in fig. 6) in which a data processing algorithm for digital collection management is deployed, wherein the server is capable of processing the digital collection search query and the text description of the alternative digital collection using the data processing algorithm for digital collection management to obtain a classification result for indicating whether to return the alternative digital collection.
Further, it is worth mentioning that the present disclosure relates to a data processing system for digital collection management, which is capable of efficient storage, retrieval, presentation and analysis of digital collections. The system comprises the following components: a digital collection database for storing metadata and content of the digital collection, and other information related to the digital collection, such as author, source, copyright, history, classification, etc.; the digital collection acquisition module is used for acquiring digital collections from different data sources, converting the digital collections into a uniform format such as XML, JSON and the like, and storing the digital collections into the digital collection database; the digital collection retrieval module is used for retrieving the digital collection meeting the conditions from the digital collection database according to the query conditions of the user and returning the digital collection to the user; a digital collection display module for displaying digital collections in different manners, such as lists, grids, slides, maps, etc., according to the user's selection, and providing additional information related to the digital collections, such as comments, links, comments, etc.; and the digital collection analysis module is used for carrying out various analyses such as statistics, clustering, classification, association, recommendation and the like on the digital collection, and displaying analysis results to a user in the forms of charts, reports and the like. It should be appreciated that a digital blind box is a digital collection based on blockchain technology that is scarce, unique, and non-tamperable.
The invention has the advantages of realizing comprehensive management of the digital collection and improving the utilization value and social influence of the digital collection.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (6)
1. A data processing method for digital collection management, comprising:
acquiring digital collection retrieval query input by a user;
extracting a text description of the alternative digital collection from the database;
carrying out joint semantic analysis on the digital collection search query and the text description of the alternative digital collection to obtain a search query-alternative digital collection semantic matching feature vector; and
determining whether to return the alternative digital collection based on the search query-alternative digital collection semantic matching feature vector;
wherein determining whether to return the alternative digital collection based on the search query-alternative digital collection semantic matching feature vector comprises:
performing feature distribution optimization on the search query-alternative digital stock semantic matching feature vector to obtain an optimized search query-alternative digital stock semantic matching feature vector; and
the optimized search query-alternative digital collection semantic matching feature vector is passed through a classifier to obtain a classification result, wherein the classification result is used for indicating whether to return the alternative digital collection;
the feature distribution optimization is performed on the search query-alternative digital stock semantic matching feature vector to obtain an optimized search query-alternative digital stock semantic matching feature vector, which comprises the following steps:
performing feature distribution optimization on the search query-alternative digital stock semantic matching feature vector by using the following optimization formula to obtain the optimized search query-alternative digital stock semantic matching feature vector;
wherein, the optimization formula is:
wherein V is the semantic matching feature vector of the search query-alternative digital stock, L is the length of the semantic matching feature vector of the search query-alternative digital stock, V i Is the feature value of the ith position of the search query-alternative digital stock semantic matching feature vector,representing the square of the two norms of the semantic matching feature vector of the search query-alternative digital collection, and alpha is a weighted hyper-parameter, exp (·) representing the exponential operation of a value representing the calculation of a natural exponential function value raised to the power of the value, v' i And the feature value of the ith position of the semantic matching feature vector of the optimized search query-alternative digital collection is represented.
2. The data processing method for digital collection management according to claim 1, wherein performing joint semantic analysis on the digital collection search query and the text description of the alternative digital collection to obtain a search query-alternative digital collection semantic matching feature vector, comprises:
carrying out semantic coding on the digital collection search query to obtain a digital collection search query semantic coding feature vector;
carrying out semantic coding on the text description of the alternative digital stock to obtain a semantic coding feature vector of the text description of the alternative digital stock; and
and merging the description semantic coding feature vector of the alternative digital stock text and the semantic coding feature vector of the digital stock search query to obtain the search query-alternative digital stock semantic matching feature vector.
3. The data processing method for digital collection management according to claim 2, wherein fusing the candidate digital collection text description semantic coding feature vector and the digital collection search query semantic coding feature vector to obtain the search query-candidate digital collection semantic matching feature vector, comprises:
and performing feature interaction based on an attention mechanism on the alternative digital collection text description semantic coding feature vector and the digital collection search query semantic coding feature vector by using an attention layer among features to obtain the search query-alternative digital collection semantic matching feature vector.
4. A data processing system for digital collection management, comprising:
the input acquisition module is used for acquiring digital collection retrieval query input by a user;
the text description extraction module is used for extracting the text description of the alternative digital collection from the database;
the joint semantic analysis module is used for carrying out joint semantic analysis on the digital collection search query and the text description of the alternative digital collection so as to obtain a search query-alternative digital collection semantic matching feature vector; and
the return confirmation module is used for determining whether to return the alternative digital collection based on the search query-alternative digital collection semantic matching feature vector;
wherein, the return confirmation module comprises:
the feature distribution optimizing unit is used for carrying out feature distribution optimization on the search query-alternative digital stock semantic matching feature vector so as to obtain an optimized search query-alternative digital stock semantic matching feature vector; and
the classification unit is used for enabling the optimized search query-alternative digital collection semantic matching feature vector to pass through a classifier to obtain a classification result, wherein the classification result is used for indicating whether to return the alternative digital collection;
wherein the feature distribution optimizing unit is configured to:
performing feature distribution optimization on the search query-alternative digital stock semantic matching feature vector by using the following optimization formula to obtain the optimized search query-alternative digital stock semantic matching feature vector;
wherein, the optimization formula is:
wherein V is the semantic matching feature vector of the search query-alternative digital stock, L is the length of the semantic matching feature vector of the search query-alternative digital stock, V i Is the feature value of the ith position of the search query-alternative digital stock semantic matching feature vector,representing the square of the two norms of the semantic matching feature vector of the search query-alternative digital collection, and alpha is a weighted hyper-parameter, exp (·) representing the exponential operation of a value representing the calculation of a natural exponential function value raised to the power of the value, v' i And the feature value of the ith position of the semantic matching feature vector of the optimized search query-alternative digital collection is represented.
5. The data processing system for digital collection management of claim 4, wherein the joint semantic analysis module comprises:
the search query semantic coding unit is used for carrying out semantic coding on the digital collection search query to obtain a digital collection search query semantic coding feature vector;
the text description semantic coding unit is used for carrying out semantic coding on the text description of the alternative digital stock to obtain an alternative digital stock text description semantic coding feature vector; and
and the fusion unit is used for fusing the description semantic coding feature vector of the alternative digital stock text and the digital stock search query semantic coding feature vector to obtain the search query-alternative digital stock semantic matching feature vector.
6. The data processing system for digital collection management of claim 5, wherein the fusion unit is configured to:
and performing feature interaction based on an attention mechanism on the alternative digital collection text description semantic coding feature vector and the digital collection search query semantic coding feature vector by using an attention layer among features to obtain the search query-alternative digital collection semantic matching feature vector.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311150900.XA CN117171413B (en) | 2023-09-07 | 2023-09-07 | Data processing system and method for digital collection management |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311150900.XA CN117171413B (en) | 2023-09-07 | 2023-09-07 | Data processing system and method for digital collection management |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117171413A CN117171413A (en) | 2023-12-05 |
| CN117171413B true CN117171413B (en) | 2024-03-08 |
Family
ID=88931449
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311150900.XA Active CN117171413B (en) | 2023-09-07 | 2023-09-07 | Data processing system and method for digital collection management |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117171413B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111583965A (en) * | 2020-04-28 | 2020-08-25 | 北京慧闻科技(集团)有限公司 | Voice emotion recognition method, device, equipment and storage medium |
| CN115017279A (en) * | 2022-05-12 | 2022-09-06 | 华南理工大学 | Stack Overflow related question and answer retrieval method based on text semantic matching |
| CN115329001A (en) * | 2022-08-12 | 2022-11-11 | 王兆成 | Alliance-chain-based digital collection system and digital collection query method |
| CN115344733A (en) * | 2022-06-10 | 2022-11-15 | 广东技术师范大学 | Digital collection retrieval method and system based on deep hash |
| CN116010713A (en) * | 2023-03-27 | 2023-04-25 | 日照职业技术学院 | Innovative entrepreneur platform service data processing method and system based on cloud computing |
| CN116010487A (en) * | 2022-12-16 | 2023-04-25 | 上海渡渡网络科技有限公司 | Digital collection display method, device, equipment and application thereof |
| CN116189865A (en) * | 2023-03-30 | 2023-05-30 | 浙江大学 | Hospital reservation registration management system |
| CN116596347A (en) * | 2023-07-17 | 2023-08-15 | 泰山职业技术学院 | Multi-disciplinary interaction teaching system and teaching method based on cloud platform |
| CN116629275A (en) * | 2023-07-21 | 2023-08-22 | 北京无极慧通科技有限公司 | Intelligent decision support system and method based on big data |
| CN116662582A (en) * | 2023-08-01 | 2023-08-29 | 成都信通信息技术有限公司 | Specific domain business knowledge retrieval method and retrieval device based on natural language |
-
2023
- 2023-09-07 CN CN202311150900.XA patent/CN117171413B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111583965A (en) * | 2020-04-28 | 2020-08-25 | 北京慧闻科技(集团)有限公司 | Voice emotion recognition method, device, equipment and storage medium |
| CN115017279A (en) * | 2022-05-12 | 2022-09-06 | 华南理工大学 | Stack Overflow related question and answer retrieval method based on text semantic matching |
| CN115344733A (en) * | 2022-06-10 | 2022-11-15 | 广东技术师范大学 | Digital collection retrieval method and system based on deep hash |
| CN115329001A (en) * | 2022-08-12 | 2022-11-11 | 王兆成 | Alliance-chain-based digital collection system and digital collection query method |
| CN116010487A (en) * | 2022-12-16 | 2023-04-25 | 上海渡渡网络科技有限公司 | Digital collection display method, device, equipment and application thereof |
| CN116010713A (en) * | 2023-03-27 | 2023-04-25 | 日照职业技术学院 | Innovative entrepreneur platform service data processing method and system based on cloud computing |
| CN116189865A (en) * | 2023-03-30 | 2023-05-30 | 浙江大学 | Hospital reservation registration management system |
| CN116596347A (en) * | 2023-07-17 | 2023-08-15 | 泰山职业技术学院 | Multi-disciplinary interaction teaching system and teaching method based on cloud platform |
| CN116629275A (en) * | 2023-07-21 | 2023-08-22 | 北京无极慧通科技有限公司 | Intelligent decision support system and method based on big data |
| CN116662582A (en) * | 2023-08-01 | 2023-08-29 | 成都信通信息技术有限公司 | Specific domain business knowledge retrieval method and retrieval device based on natural language |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117171413A (en) | 2023-12-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111694924B (en) | Event extraction method and system | |
| CN106649561B (en) | Intelligent question-answering system for tax consultation service | |
| CN113822494A (en) | Risk prediction method, device, equipment and storage medium | |
| CN109783635A (en) | Use machine learning and fuzzy matching AUTOMATIC ZONING classifying documents and identification metadata | |
| CN105393265A (en) | Active featuring in computer-human interactive learning | |
| CN107688870B (en) | Text stream input-based hierarchical factor visualization analysis method and device for deep neural network | |
| CN113254659A (en) | File studying and judging method and system based on knowledge graph technology | |
| CN110795525A (en) | Text structuring method and device, electronic equipment and computer readable storage medium | |
| CN114896388A (en) | Hierarchical multi-label text classification method based on mixed attention | |
| CN116610803B (en) | Industrial chain excellent enterprise information management method and system based on big data | |
| CN111080551B (en) | Multi-label image complement method based on depth convolution feature and semantic neighbor | |
| CN115878904A (en) | Intellectual property personalized recommendation method, system and medium based on deep learning | |
| CN114461890A (en) | Hierarchical multi-modal intellectual property search engine method and system | |
| CN116127084A (en) | Knowledge graph-based micro-grid scheduling strategy intelligent retrieval system and method | |
| CN114611491A (en) | Intelligent government affair public opinion analysis research method based on text mining technology | |
| CN114357151A (en) | Processing method, device and equipment of text category identification model and storage medium | |
| Başarslan et al. | Sentiment analysis with ensemble and machine learning methods in multi-domain datasets | |
| CN117494051A (en) | Classification processing method, model training method and related device | |
| CN112839102A (en) | Business processing method applied to big data image pushing and machine learning server | |
| CN116956228A (en) | Text mining method for technical transaction platform | |
| CN117171413B (en) | Data processing system and method for digital collection management | |
| CN114254622A (en) | Intention identification method and device | |
| CN112270189A (en) | Question type analysis node generation method, question type analysis node generation system and storage medium | |
| Porwal et al. | Citation Classification Prediction Implying Text Features Using Natural Language Processing and Supervised Machine Learning Algorithms | |
| CN115795037B (en) | Multi-label text classification method based on label perception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |