CN114547357B - Furniture model retrieval method based on voice and sketch - Google Patents
Furniture model retrieval method based on voice and sketch Download PDFInfo
- Publication number
- CN114547357B CN114547357B CN202210043225.XA CN202210043225A CN114547357B CN 114547357 B CN114547357 B CN 114547357B CN 202210043225 A CN202210043225 A CN 202210043225A CN 114547357 B CN114547357 B CN 114547357B
- Authority
- CN
- China
- Prior art keywords
- sketch
- model
- dimensional
- voice
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000013528 artificial neural network Methods 0.000 claims abstract description 30
- 239000013598 vector Substances 0.000 claims description 30
- 238000004364 calculation method Methods 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 abstract description 5
- 238000011960 computer-aided design Methods 0.000 abstract description 4
- 238000010276 construction Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
- G06F16/532—Query formulation, e.g. graphical querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/216—Parsing using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Library & Information Science (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention belongs to the technical field of furniture model retrieval in computer aided design, and particularly relates to a furniture model retrieval method based on voice and sketch, which comprises the following steps: s1, constructing a database based on a neural network, standard keyword description and model description semantic features; s2, inputting a two-dimensional sketch into a database to obtain a sketch retrieval result; s3, inputting voice into a database to obtain a voice retrieval result; s4, obtaining a final search result through a voting mode based on the sketch search result and the voice search result. The quick and accurate furniture model retrieval method disclosed by the invention saves a great deal of design cost and time expenditure, so that the utilization rate and the use value of the furniture model are improved.
Description
Technical Field
The invention belongs to the technical field of furniture model retrieval in computer aided design, and particularly relates to a furniture model retrieval method based on voice and sketch.
Background
With the development of computer-aided designs, furniture designs increasingly rely on computer-aided designs. In the design of home scenes, the retrieval of furniture models is a critical issue. Due to the wide variety and number of furniture models, home designers need quick and accurate methods to retrieve furniture models. The furniture retrieval task mainly comprises two aspects of extracting model features and establishing a relationship between input and the model features.
The following search methods are generally used: text-based retrieval and sketch-based retrieval. The method based on text retrieval needs to carry out a large number of manual labels on the model, stores the model characteristics in a manual label mode, and compares the model characteristics with input text to obtain a retrieval result. And the sketch-based retrieval is performed by drawing a sketch of a model to be retrieved by a user, extracting the sketch features by the system, and calculating the similarity between the sketch features and the model features to obtain a retrieval result. The method has the advantages of no need of a large amount of manual labeling, and lower accuracy compared with the traditional text retrieval.
In text-based retrieval, due to the ambiguity of the input, a complete match with the model tag key is typically not possible, requiring text features to be expressed in a semantic-based manner. Word vector embedding is a semantic expression technique, and there are two types of word vector embedding methods: fixed representation and dynamic representation. The fixed representation method comprises the following steps: word2Vec, gloVe, fastText, etc., the dynamic representation method is: ELMo, bRT, etc. In sketch-based retrieval, two general categories can be distinguished: and extracting three-dimensional model features based on projection, and directly extracting three-dimensional model global features.
For example, the Chinese patent application number is: CN201810597066.1 discloses a three-dimensional model retrieval method based on sketch, which processes the three-dimensional model into a form of a plurality of view angle screenshots, then uses different contour extraction operators to process the three-dimensional model into sketches with different styles, and obtains a sketch dataset, and the dataset is used in training and testing processes after being provided with labels; the invention relates to a method for searching a furniture model based on voice and sketch, which is characterized in that a hierarchical network method is added on the basis of a convolutional neural network, 1 large classification network is used for training large classifications, the input sketch is classified into 40 large classes, 40 small classification networks respectively train specific differences in each class of models, the input sketch is classified into one specific three-dimensional model in a certain large class, the three-dimensional model can be searched with high accuracy, the information redundancy is small, an angle matrix calculated by using sampling contour points is used as the input of the convolutional neural network, the characteristics of the sketch are more consistent, and the searching accuracy is high, but a furniture model searching method based on voice and sketch is not proposed.
Disclosure of Invention
In order to solve the problems, the invention provides a furniture model retrieval method based on voice and sketch;
S1, constructing a database based on a neural network, standard keyword description and model description semantic features;
S2, inputting a two-dimensional sketch into a database to obtain a sketch retrieval result;
s3, inputting voice into a database to obtain a voice retrieval result;
s4, obtaining a final search result through a voting mode based on the sketch search result and the voice search result.
Further, the constructing the database based on the neural network, the standard keyword description and the model description semantic features in S1 includes:
s101, constructing a pseudo-twin neural network for calculating similarity calculation of two-dimensional sketch features and three-dimensional model features, taking the three-dimensional model features and the two-dimensional sketch features as input values of the pseudo-twin neural network, and extracting features of three-dimensional models in a database by adopting an LD-SIFT descriptor mode to obtain three-dimensional model features as input values of the pseudo-twin neural network;
S102, constructing standard keyword descriptions for standardizing keyword descriptions of input search texts, acquiring texts corresponding to all three-dimensional models in advance, inputting the texts into a word2Vec pre-training model for word vector embedding, and obtaining vector representations of all model description keywords as all standard keyword descriptions;
S103, constructing all model description semantic features corresponding to all standard keyword descriptions, acquiring standard keyword descriptions corresponding to each three-dimensional model, and calculating IDF weights corresponding to each three-dimensional model by adopting IDF through the standard keyword descriptions to serve as all model description semantic features;
S104, constructing a database by constructing a pseudo twin neural network, standard keyword description and model description semantic features.
Further, the step S2 of inputting the two-dimensional sketch into the database to obtain a sketch retrieval result includes:
s201, drawing a two-dimensional sketch of a model to be searched by a user, inputting the two-dimensional sketch into a trained VGG-19 model, and acquiring a 4096-dimensional vector as an input sketch characteristic through an fc-7 layer of the trained VGG-19 model;
s202, inputting the input sketch features and the three-dimensional model features into a pseudo-twin neural network, and obtaining the similarity of the input sketch features and the three-dimensional model features as sketch retrieval results.
Further, in S202, the similarity between the input sketch feature and the three-dimensional model feature is calculated as the following formula (1):
In the method, in the process of the invention, Representing input sketch features and three-dimensional model feature similarities,A contrast loss function is represented and is used,Represented as a sample feature 1,The characteristic is expressed as a sample characteristic 2, W is a neural network parameter, Y is expressed as whether samples are matched, and when Y is 1, the samples are matched; at 0, the samples do not match.
Further, in S202, the contrast loss function is calculated as follows (2):
Where N is the number of samples, D W is the euclidean distance between sample X 1 and sample X 2, m is the set upper distance threshold, and max (m-D w, 0) is the maximum value of m-D w, 0.
Further, in S202, the euclidean distance is calculated as follows (3):
In the method, in the process of the invention, the term "is expressed as the modulo length of the vector, X 1 represents sample 1 and X 2 represents sample 2.
Further, in S3, the step of inputting the voice into a pre-constructed database to obtain a voice search result includes:
s301, converting input voice into text data by adopting a voice recognition algorithm, performing word segmentation and word removal operation on the text data to obtain a keyword description sequence, and inputting the keyword description sequence into a word2Vec pre-training model to perform word vector embedding to obtain a keyword vector;
S302, carrying out similarity calculation on the keyword vector and standard keyword description corresponding to the database, converting the keyword description sequence into a standard keyword sequence through an optimal similarity principle, and carrying out IDF feature extraction on the standard keyword sequence to obtain IDF weight corresponding to the standard keyword sequence;
s303, determining a three-dimensional model based on the corresponding standard keyword description, and selecting corresponding model description semantic features through the three-dimensional model to serve as IDF weights corresponding to the three-dimensional model in a database;
S304, obtaining an inner product corresponding to the three-dimensional model through the IDF weight corresponding to the standard keyword sequence and the IDF weight corresponding to the three-dimensional model in the database, and taking the inner product as the similarity between the output voice and the three-dimensional model in the pre-built database.
Further, in S303, the IDF weight is calculated as follows (4):
Where W tag represents the weight, N model represents the number of three-dimensional models, and N tag represents the number of labels corresponding to the standard keyword sequence.
Further, in S4, the obtaining the final search result based on the sketch search result and the voice search result through the voting method includes:
S401, respectively acquiring the similarity of all three-dimensional models according to the input two-dimensional sketch and the input voice, respectively sequencing all the three-dimensional models from large to small according to the similarity, and distributing weights;
s402, voting the two search results according to the weights to obtain total weights, and sorting the two search results according to the total weights from high to low to obtain a final search result.
Compared with the prior art, the invention has the following beneficial effects:
1. The furniture model retrieval method based on the voice and the sketch overcomes the defects brought by a single retrieval method, combines the advantages of the sketch retrieval method and the voice retrieval method, achieves higher retrieval accuracy under the condition of insufficient labeling of text keywords, and meets higher retrieval quality requirements.
2. The furniture model retrieval method based on the voice and the sketch designs a quick and accurate furniture model retrieval method, and can save a great deal of design cost and time expenditure, thereby improving the utilization rate and the use value of the furniture model.
3. According to the furniture model retrieval method based on the voice and the sketch, the voice information and the sketch information are effectively extracted, the similarity comparison of heterogeneous data is realized through fusion, in the sketch retrieval, the sketch features and the model features are extracted by using VGG19 and LD-SIFT respectively, the registration of the features is carried out in a pseudo twin neural network, the two features are mapped into the same vector space, and the retrieval is carried out through calculating the similarity; in the retrieval based on voice, a Word2vec model is trained by using furniture related corpus, a semantic relation library of furniture keywords is established, the research is carried out by converting voice and text, extracting the furniture description keywords in the text, constructing keyword semantic relation, carrying out standard keyword conversion, giving weight to each keyword by using inverse document frequency, converting each furniture model description and query into vectors, realizing retrieval by similarity calculation of the vectors, and finally, merging two independent retrieval results by using a voting method, thereby realizing a multi-mode combined retrieval mode.
Drawings
FIG. 1 is a block flow diagram of a furniture model retrieval method based on voice and sketch according to the invention;
fig. 2 is a schematic flow chart of the furniture model searching method based on voice and sketch according to the invention, wherein the final searching result is obtained by voting based on the sketch searching result and the voice searching result.
Detailed Description
The invention is described in further detail below with reference to the drawings and the detailed description.
The invention provides a joint search method based on the existing model search method based on voice and the model search method based on sketch, and simultaneously refers to the results of the two search modes of the sketch search method and the voice search method, so that the search result is more accurate and the search mode is more efficient. As shown in fig. 1 and 2, the furniture model retrieval method based on voice and sketch includes:
s1, constructing a database based on a neural network, standard keyword description and model description semantic features:
s101, constructing a pseudo-twin neural network for calculating similarity calculation of two-dimensional sketch features and three-dimensional model features, taking the three-dimensional model features and the two-dimensional sketch features as input values of the pseudo-twin neural network, and extracting features of three-dimensional models in a database by adopting an LD-SIFT descriptor mode to obtain three-dimensional model features as input values of the pseudo-twin neural network;
S102, constructing standard keyword descriptions for standardizing keyword descriptions of input search texts, acquiring texts corresponding to all three-dimensional models in advance, inputting the texts into a word2Vec pre-training model for word vector embedding, and obtaining vector representations of all model description keywords as all standard keyword descriptions;
S103, constructing all model description semantic features corresponding to all standard keyword descriptions, acquiring standard keyword descriptions corresponding to each three-dimensional model, and calculating IDF weights corresponding to each three-dimensional model by adopting IDF through the standard keyword descriptions to serve as all model description semantic features;
S104, constructing a database by constructing a pseudo twin neural network, standard keyword description and model description semantic features.
The construction of the database mainly comprises 3 parts, namely, the construction of a neural network for calculating the similarity between a two-dimensional sketch and a three-dimensional model, the construction of standard keyword description and the construction of model description semantic features:
1) And constructing a neural network for calculating the similarity between the two-dimensional sketch and the three-dimensional model, and constructing the neural network for calculating the similarity between the two, wherein the characteristics of the two are extracted firstly. Extracting the features of the two-dimensional sketch by adopting VGG-19, extracting the fc-7 layer of a pre-trained VGG-19 model as the features of the input two-dimensional sketch, extracting the features of the three-dimensional model by adopting LD-SIFT descriptors, and constructing a pseudo-twin neural network on feature similarity calculation to train the features of the three-dimensional model and the features of the two-dimensional sketch so as to obtain a pseudo-twin neural network capable of calculating the similarity of the sketch and the model, wherein a loss function adopts a contrast loss function as an index of similarity measurement;
2) Constructing standard keyword descriptions, namely embedding word vectors into original model feature descriptions in a database through a word2Vec pre-training model to obtain vector representations of all model description keywords, wherein the vector representations are used for normalizing the keyword descriptions of the input retrieval text items;
3) Constructing semantic features of model description, constructing semantic feature description of each model by adopting IDF (inverse document frequency) for keyword description of all models in a database, calculating inverse document frequency of each keyword, and constructing the document frequency of the keyword which corresponds to each model as a vector which is the semantic feature of the model;
s2, inputting a two-dimensional sketch into a database to obtain a sketch retrieval result:
s201, drawing a two-dimensional sketch of a model to be searched by a user, inputting the two-dimensional sketch into a trained VGG-19 model, and acquiring a 4096-dimensional vector as an input sketch characteristic through an fc-7 layer of the trained VGG-19 model;
S202, inputting the input sketch features and the three-dimensional model features into a pseudo-twin neural network, and obtaining the similarity of the input sketch features and the three-dimensional model features as sketch retrieval results:
The similarity between the input sketch features and the three-dimensional model features is calculated as follows (1):
In the method, in the process of the invention, Representing input sketch features and three-dimensional model feature similarities,A contrast loss function is represented and is used,Represented as a sample feature 1,The characteristic is expressed as a sample characteristic 2, W is a neural network parameter, Y is expressed as whether samples are matched, and when Y is 1, the samples are matched; at 0, the samples do not match;
the contrast loss function is calculated according to the following formula (2):
Wherein N is the number of samples, D W is the Euclidean distance between the sample X 1 and the sample X 2, m is the set distance upper threshold, and max (m-D w, 0) is the maximum value of m-D w, 0;
the euclidean distance is calculated according to the following formula (3):
In the method, in the process of the invention, the term "is expressed as the modulo length of the vector, X 1 represents sample 1, and X 2 represents sample 2;
s3, inputting voice into a database to obtain a voice retrieval result:
s301, converting input voice into text data by adopting a voice recognition algorithm, performing word segmentation and word removal operation on the text data to obtain a keyword description sequence, and inputting the keyword description sequence into a word2Vec pre-training model to perform word vector embedding to obtain a keyword vector;
S302, carrying out similarity calculation on the keyword vector and standard keyword description corresponding to the database, converting the keyword description sequence into a standard keyword sequence through an optimal similarity principle, and carrying out IDF feature extraction on the standard keyword sequence to obtain IDF weight corresponding to the standard keyword sequence;
s303, determining a three-dimensional model based on the corresponding standard keyword description, and selecting corresponding model description semantic features through the three-dimensional model to serve as IDF weights corresponding to the three-dimensional model in a database;
S304, obtaining an inner product corresponding to the three-dimensional model through the IDF weight corresponding to the standard keyword sequence and the IDF weight corresponding to the three-dimensional model in the database, and taking the inner product as the similarity between the output voice and the three-dimensional model in the pre-built database:
The IDF weight is calculated as follows (4):
Wherein W tag represents weight, N model represents three-dimensional model number, and N tag represents label number corresponding to standard keyword sequence;
s4, obtaining a final search result through a voting mode based on the sketch search result and the voice search result:
S401, respectively acquiring the similarity of all three-dimensional models according to the input two-dimensional sketch and the input voice, respectively sequencing all the three-dimensional models from large to small according to the similarity, and distributing weights;
s402, voting the two search results according to the weights to obtain total weights, and sorting the two search results according to the total weights from high to low to obtain a final search result.
In step 4, the two search modes are combined by voting, different voting weights are allocated to different models according to the ranking of the search results, weights are respectively allocated from the first to the last of the search results, the two search results are voted according to the weights to obtain total weights, and finally the final search results are ranked according to the total weights from high to low.
The above description is only specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions easily come within the scope of the present invention as those skilled in the art can easily come within the scope of the present invention defined by the appended claims.
Claims (8)
1. A furniture model retrieval method based on voice and sketch is characterized by comprising the following steps:
S1, constructing a database based on a neural network, standard keyword description and model description semantic features;
s1, constructing a database based on a neural network, standard keyword description and model description semantic features, wherein the database comprises the following steps:
s101, constructing a pseudo-twin neural network for calculating similarity calculation of two-dimensional sketch features and three-dimensional model features, taking the three-dimensional model features and the two-dimensional sketch features as input values of the pseudo-twin neural network, and extracting features of three-dimensional models in a database by adopting an LD-SIFT descriptor mode to obtain three-dimensional model features as input values of the pseudo-twin neural network;
S102, constructing standard keyword descriptions for standardizing keyword descriptions of input search texts, acquiring texts corresponding to all three-dimensional models in advance, inputting the texts into a word2Vec pre-training model for word vector embedding, and obtaining vector representations of all model description keywords as all standard keyword descriptions;
S103, constructing all model description semantic features corresponding to all standard keyword descriptions, acquiring standard keyword descriptions corresponding to each three-dimensional model, and calculating IDF weights corresponding to each three-dimensional model by adopting IDF through the standard keyword descriptions to serve as all model description semantic features;
S104, constructing a database by constructing a pseudo twin neural network, standard keyword description and model description semantic features;
S2, inputting a two-dimensional sketch into a database to obtain a sketch retrieval result;
s3, inputting voice into a database to obtain a voice retrieval result;
s4, obtaining a final search result through a voting mode based on the sketch search result and the voice search result.
2. The method for retrieving a furniture model based on voice and sketch according to claim 1, wherein S2 of inputting the two-dimensional sketch into a database to obtain a sketch retrieval result comprises:
s201, drawing a two-dimensional sketch of a model to be searched by a user, inputting the two-dimensional sketch into a trained VGG-19 model, and acquiring a 4096-dimensional vector as an input sketch characteristic through an fc-7 layer of the trained VGG-19 model;
s202, inputting the input sketch features and the three-dimensional model features into a pseudo-twin neural network, and obtaining the similarity of the input sketch features and the three-dimensional model features as sketch retrieval results.
3. The furniture model retrieval method based on voice and sketch according to claim 2, wherein the calculation of the similarity between the input sketch features and the three-dimensional model features in S202 is as follows formula (1):
In the method, in the process of the invention, Representing input sketch features and three-dimensional model feature similarities,A contrast loss function is represented and is used,Represented as a sample feature 1,The characteristic is expressed as a sample characteristic 2, W is a neural network parameter, Y is expressed as whether samples are matched, and when Y is 1, the samples are matched; at 0, the samples do not match.
4. A furniture model retrieval method based on speech and sketch according to claim 3, characterized in that the calculation of the contrast loss function is as follows:
Where N is the number of samples, D W is the euclidean distance between sample X 1 and sample X 2, m is the set upper distance threshold, and max (m-D w, 0) is the maximum value of m-D w, 0.
5. The furniture model retrieval method based on voice and sketch according to claim 4, wherein the calculation of the euclidean distance is as follows:
In the formula, X 1 represents sample 1, and X 2 represents sample 2, expressed as a modulus of vector.
6. The furniture model retrieval method based on voice and sketch according to claim 1, wherein S3, obtaining a voice retrieval result by inputting voice into a pre-built database includes:
s301, converting input voice into text data by adopting a voice recognition algorithm, performing word segmentation and word removal operation on the text data to obtain a keyword description sequence, and inputting the keyword description sequence into a word2Vec pre-training model to perform word vector embedding to obtain a keyword vector;
S302, carrying out similarity calculation on the keyword vector and standard keyword description corresponding to the database, converting the keyword description sequence into a standard keyword sequence through an optimal similarity principle, and carrying out IDF feature extraction on the standard keyword sequence to obtain IDF weight corresponding to the standard keyword sequence;
s303, determining a three-dimensional model based on the corresponding standard keyword description, and selecting corresponding model description semantic features through the three-dimensional model to serve as IDF weights corresponding to the three-dimensional model in a database;
S304, obtaining an inner product corresponding to the three-dimensional model through the IDF weight corresponding to the standard keyword sequence and the IDF weight corresponding to the three-dimensional model in the database, and taking the inner product as the similarity between the output voice and the three-dimensional model in the pre-built database.
7. The method for retrieving a furniture model based on speech and sketch according to claim 6, wherein the calculation of the IDF weight in S304 is as follows:
Where W tag represents the weight, N model represents the number of three-dimensional models, and N tag represents the number of labels corresponding to the standard keyword sequence.
8. The furniture model retrieval method based on voice and sketch according to claim 1, wherein S4 the obtaining the final retrieval result based on the sketch retrieval result and the voice retrieval result through voting comprises:
S401, respectively acquiring the similarity of all three-dimensional models according to the input two-dimensional sketch and the input voice, respectively sequencing all the three-dimensional models from large to small according to the similarity, and distributing weights;
s402, voting the two search results according to the weights to obtain total weights, and sorting the two search results according to the total weights from high to low to obtain a final search result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210043225.XA CN114547357B (en) | 2022-01-14 | 2022-01-14 | Furniture model retrieval method based on voice and sketch |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210043225.XA CN114547357B (en) | 2022-01-14 | 2022-01-14 | Furniture model retrieval method based on voice and sketch |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114547357A CN114547357A (en) | 2022-05-27 |
| CN114547357B true CN114547357B (en) | 2024-08-16 |
Family
ID=81672138
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210043225.XA Active CN114547357B (en) | 2022-01-14 | 2022-01-14 | Furniture model retrieval method based on voice and sketch |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114547357B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101004748A (en) * | 2006-10-27 | 2007-07-25 | 北京航空航天大学 | Method for searching 3D model based on 2D sketch |
| CN102955848A (en) * | 2012-10-29 | 2013-03-06 | 北京工商大学 | Semantic-based three-dimensional model retrieval system and method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130059040A (en) * | 2011-11-28 | 2013-06-05 | 연세대학교 산학협력단 | Article retrival system and method using sketch in store |
-
2022
- 2022-01-14 CN CN202210043225.XA patent/CN114547357B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101004748A (en) * | 2006-10-27 | 2007-07-25 | 北京航空航天大学 | Method for searching 3D model based on 2D sketch |
| CN102955848A (en) * | 2012-10-29 | 2013-03-06 | 北京工商大学 | Semantic-based three-dimensional model retrieval system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114547357A (en) | 2022-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109829104B (en) | Semantic similarity based pseudo-correlation feedback model information retrieval method and system | |
| CN112905822B (en) | Deep supervision cross-modal counterwork learning method based on attention mechanism | |
| CN104376406B (en) | A kind of enterprise innovation resource management and analysis method based on big data | |
| CN117290489B (en) | Method and system for quickly constructing industry question-answer knowledge base | |
| CN102207945B (en) | Knowledge network-based text indexing system and method | |
| CN110188197B (en) | Active learning method and device for labeling platform | |
| CN104199965A (en) | Semantic information retrieval method | |
| CN111061939B (en) | Scientific research academic news keyword matching recommendation method based on deep learning | |
| CN112559684A (en) | Keyword extraction and information retrieval method | |
| CN114139533A (en) | Text content auditing method for Chinese novel field | |
| CN112307182A (en) | Question-answering system-based pseudo-correlation feedback extended query method | |
| CN110990597A (en) | Cross-modal data retrieval system based on text semantic mapping and retrieval method thereof | |
| CN115203421A (en) | Method, device and equipment for generating label of long text and storage medium | |
| CN118069812B (en) | Navigation method based on large model | |
| CN111104437A (en) | Test data unified retrieval method and system based on object model | |
| CN118245564B (en) | Method and device for constructing feature comparison library supporting semantic review and repayment | |
| CN116450883A (en) | Video moment retrieval method based on video content fine granularity information | |
| CN117993393A (en) | Method, device and system for checking online labeling policy terms based on word and sentence vectors | |
| CN114493783A (en) | Commodity matching method based on double retrieval mechanism | |
| CN117454898A (en) | Method and device for realizing legal entity standardized output according to input text | |
| CN105117735A (en) | Image detection method in big data environment | |
| Zhang et al. | mSHINE: A multiple-meta-paths simultaneous learning framework for heterogeneous information network embedding | |
| CN114547357B (en) | Furniture model retrieval method based on voice and sketch | |
| CN114298020B (en) | Keyword vectorization method based on topic semantic information and application thereof | |
| CN110688559A (en) | Retrieval method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |