CN110750656A - Multimedia detection method based on knowledge graph - Google Patents
Multimedia detection method based on knowledge graph Download PDFInfo
- Publication number
- CN110750656A CN110750656A CN201911036867.1A CN201911036867A CN110750656A CN 110750656 A CN110750656 A CN 110750656A CN 201911036867 A CN201911036867 A CN 201911036867A CN 110750656 A CN110750656 A CN 110750656A
- Authority
- CN
- China
- Prior art keywords
- file
- multimedia
- portrait
- user
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/41—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/435—Filtering based on additional data, e.g. user or group profiles
- G06F16/436—Filtering based on additional data, e.g. user or group profiles using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/45—Clustering; Classification
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Animal Behavior & Ethology (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Molecular Biology (AREA)
- Physiology (AREA)
- Processing Or Creating Images (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a multimedia detection method based on a knowledge graph, which comprises the following steps: each user uploads a shared multimedia file and a custom tag to a multimedia database; aiming at picture files and video files containing the portrait in the multimedia files, processing through face detection, face recognition and face comparison to construct a portrait library; processing the picture file and the video file with the portrait removed in the multimedia file through image classification and target detection to identify a scene and a real object; classifying the text files in the multimedia files through two classifiers, namely a general classifier and an education classifier, and marking category labels according to classification results; composing a document map for each user; constructing a character relationship network; and based on the file map, the character library and the character relation network, the user searches. And multimedia attribute association retrieval is carried out on the basis of the knowledge graph, so that the use by a user is facilitated.
Description
Technical Field
The invention relates to the technical field of information retrieval, in particular to a multimedia detection method based on a knowledge graph.
Background
At present, image and text retrieval is mainly focused on single-mode retrieval, and the contents of a query and a candidate set belong to the same mode. And Cross-modal retrieval (Cross-modal retrieval) completes the expression and transformation of information in different morphological spaces by establishing an information mapping relation among multiple modalities, and finally realizes the retrieval of crossing information resource morphological differences. Along with the development and enrichment of multimedia technology, the demand for retrieval among multiple modes is stronger and stronger. The biggest problem faced by the current cross-modal retrieval is how to better realize mutual recognition and retrieval among multiple modes such as texts, images and the like, which is the target and meaning of the cross-modal retrieval. After the knowledge graph is introduced into the multimedia search system, the method is beneficial to obtaining situation data under different searches, better supports a user to express search intention in natural language, can discover characteristics under different situations through further reasoning, and realizes more accurate user query semantic analysis and search.
With the development of artificial intelligence and the dramatic increase in knowledge demand, knowledge maps have received a great deal of attention in the industry and academia. The knowledge graph is proposed by Google corporation in 2012, and is particularly used for a knowledge base for improving the performance of a search engine; the generalized knowledge graph generally refers to various knowledge base items. The knowledge graph aggregates various information, data and link relations into knowledge, and is an effective organization method of knowledge in a big data environment. Various large-scale knowledge maps play an important role in the fields of intelligent search, intelligent question answering, intelligent recommendation, information analysis, anti-fraud, user input disambiguation, social networking, finance, medical treatment, e-commerce, education and scientific research and the like. With the rapid growth in scale, understanding, analysis, and utilization of large-scale knowledge maps remains a challenge. The visualization maps abstract data into graphic elements and is assisted with a human-computer interaction means to help a user to effectively perceive and analyze the data. Therefore, the introduction of the knowledge graph into the multimedia retrieval plays an important role in improving the retrieval effect.
Disclosure of Invention
The invention aims to provide a knowledge graph-based multimedia detection method, which is used for performing multimedia attribute association retrieval on the basis of a knowledge graph and is convenient for users to use.
The technical scheme for realizing the purpose is as follows:
a multimedia detection method based on knowledge graph includes:
step S1, a multimedia database is built, and each user uploads a shared multimedia file and a custom label to the multimedia database;
step S2, aiming at the picture files and video files containing the portrait in the multimedia files, processing through face detection, face recognition and face comparison to construct a portrait library; processing the picture file and the video file with the portrait removed in the multimedia file through image classification and target detection to identify a scene and a real object; classifying the text files in the multimedia files through two classifiers, namely a general classifier and an education classifier, and marking category labels according to classification results;
step S3, for each user, analyzing the uploaded multimedia file through step S2 to obtain portrait data, scene data, real object data and text file classification data, on one hand, associating the portrait data, the scene data, the real object data and the text file classification data with each multimedia file, on the other hand, associating the user with the uploaded multimedia file, the custom label and the portrait data to form a file map;
step S4, determining the relationship type between two persons according to the number of group photos and the number of people on the group photos existing between each two persons in the constructed portrait base, and constructing a person relationship network;
in step S5, the user searches based on the document map, the person library, and the person relationship network.
Preferably, the step S2 includes:
when a human image is detected in a key frame of a picture file or a video file, a side face and a fuzzy face are filtered through face detection, then human image information is vectorized and expressed through face identification, then the vectorized human image information is compared with data in a human image library through the face, the similarity is calculated and is arranged in a descending order, if the similarity is the highest and is greater than a preset value, the matching is considered to be successful, new human image information is added into the human image library, otherwise, when the matching is failed, data are newly added into the human image library, and the human image library is constructed in the mode;
and aiming at the key frames of the picture file or the video file with the portrait removed, carrying out image classification and target detection by using a target detection model so as to identify related scenes or real objects, and then classifying the real objects by using a deep convolutional network mode.
Preferably, the face detection means: calculating the distance ratio from the canthus to the tail of the eyes of the portrait by the distance of the characteristic points, judging that the face is in the lateral direction when the distance ratio is greater than a preset multiple, and filtering the face; and performing edge detection by using a Laplace operator, and when the Laplace operator value is smaller than a preset value, considering the portrait image as a fuzzy face and filtering the fuzzy face.
Preferably, the face detection and the face recognition are performed by using a face recognition model based on angles.
Preferably, the character relationship in step S4 includes:
when two people appear in the same photo in the form of a group photo, the two people are considered to be in the same frame relationship;
when the number of the group photos of the two people is larger than a first preset value and not larger than a third preset value, and the number of people in each photo is smaller than a second preset value, the two people are considered to be in an artificial recognition relationship;
and when the number of the group photos of the two people is larger than a third preset value and the number of people in each photo is smaller than the second preset value, the two people are considered to be in close relationship.
Preferably, in step S3, the document map records operation behaviors of other users on the data in the document map, and associates the operation behaviors with each other.
Preferably, the general class of text categories includes sports, finance, real estate, home, education, science and technology, fashion, games, entertainment, lottery, stock, society, and constellation; education text classification includes instrument, party administration, capital construction, field, teaching, scientific research, administration, financial accounting.
Preferably, the step S5 includes:
the user searches in a picture mode: representing the human image information in a vectorization way through human face detection and human face recognition, and/or processing through image classification and target detection to recognize scenes and real objects; then retrieving and matching the vectorized portrait information with the file map, the character library and the portrait in the character relation network according to the similarity, and/or retrieving and matching the identified scene and the identified real object with the scene and the real object in the file map according to the similarity; or
The user searches keywords in natural language: performing text error correction and correction on the keywords by combining a word bank and a shortest editing distance method, and performing retrieval matching from a file map, a character bank and a character relation network;
and the user searches and matches a corresponding result, preferentially displays related picture, video, text and homepage links corresponding to the result, and gives an option taking a search intention as a general content search.
Preferably, in the step S5,
when the matched picture or the keyword information is the scene name, preferentially displaying the related picture of the same scene to form a user file network map taking the scene name as the center, and providing an option entry taking the scene name as the general content retrieval;
when the matched picture or the keyword information is the name of the article, preferentially displaying the related picture containing the article, forming a user file network map taking the article as the center, and providing an option entry taking the article name as a general content retrieval;
when the matched keyword information is a file category name, preferentially displaying related documents under the category name to form a user file network map taking the file category as a center, and providing an option entry taking the file category name as a general content retrieval;
when the matched picture information is the name of the portrait, the related picture and video of the portrait are preferentially displayed to form a user file network map taking the portrait as the center, and an option entry taking the name of the portrait as a general content retrieval is provided;
when the matched keyword information is a user name, preferentially displaying the homepage link of the user to form a user file network map taking the user as the center, and providing an option entry taking the user name as general content retrieval;
when the matched keyword information is a custom tag, preferentially displaying a related file containing the custom tag, forming a user file network map taking the related file of the custom tag as a center, and providing an option entry taking the custom tag as a general content retrieval;
and when the matching is not successful, the picture or the keyword information is used as the general content, the general content is searched, and the relevant corresponding result is displayed.
The invention has the beneficial effects that: the invention is based on the knowledge graph, combines the multimedia file analysis technology, namely picture, image recognition (including face detection, filtration, recognition, comparison, object and scene recognition) and text classification (using two classification methods of a general classifier and a special education to carry out double-layer classification), and the like, obtains retrieval data of various dimensions which are consistent with the query content of the current user, and visualizes the retrieval data in the form of the knowledge graph, thereby facilitating the better understanding of the current user. The development of the retrieval technology towards the multi-mode retrieval and intelligent retrieval is promoted, and the method has very high practical application value and wide application prospect.
Drawings
FIG. 1 is a schematic flow diagram of a method for knowledgegraph-based multimedia detection in accordance with the present invention;
FIG. 2 is a schematic diagram of a process for constructing a character library according to the present invention;
FIG. 3 is a schematic diagram of a person relationship definition in the present invention;
fig. 4 is a schematic diagram of the search process in the present invention.
Detailed Description
The invention will be further explained with reference to the drawings.
Referring to fig. 1, the method for detecting multimedia based on knowledge-graph of the present invention includes the following steps:
and step S1, constructing a multimedia database, and uploading the shared multimedia file and the custom label to the multimedia database by each user. Because the related knowledge of the current dissemination entity is no longer represented by only a single medium, the information is often disseminated in a multimedia multi-channel manner. The entity information of various media forms (pictures, videos, texts, users and the like) is analyzed by utilizing technologies such as image classification, target detection, text classification and the like. The multimedia file comprises natural attributes (such as the uploading place, time, size, file type and the like) and social attributes (such as the classified category of the file, the portrait, the real object, the scene and the like identified in the file).
And step S2, aiming at the picture files and video files containing the portrait in the multimedia files, processing through face detection, face recognition and face comparison to construct a portrait library. As shown in fig. 2, specifically:
firstly, face detection and face recognition are carried out by utilizing an open-source face recognition model based on angles. When a human image is detected in a key frame of a picture file or a video file, a side face and a fuzzy face are filtered through human face detection, the distance ratio from the canthus to the tail of eyes of two eyes of the human image is calculated mainly through the distance of characteristic points, the human face is determined to be in a side direction when the distance ratio is larger than a preset multiple (for example, 3), and the side face and the fuzzy face are filtered. And performing edge detection by using the Laplacian in the open-source image processing library. The laplacian operator determines a slowly changing edge line by using a zero-crossing point between a second-order differential positive peak and a negative peak, and highlights isolated points, isolated lines and line end points in the image. When the laplacian operator value is less than a predetermined value (e.g., 12), the human image is considered as a blurred face and is filtered, and a clear image with operator value greater than 12 is retained. The laplacian is the simplest isotropic differential operator, with rotational invariance. The laplace transform of a two-dimensional image function is the isotropic second derivative, defined as:
And then, expressing the human image information in a vectorization mode through face recognition, namely, utilizing the reciprocal second-layer network output vector of the face recognition model as the human face information vectorization mode, then carrying out face comparison on the vectorized human image information and data in a human image library, calculating the similarity and carrying out descending arrangement, judging that the matching is successful if the similarity is highest and is greater than a preset value (for example, 70 percent), adding new human image information into the human image library, otherwise, adding new data in the human image library when the matching is failed, and constructing the human image library in such a way.
And aiming at the key frames of the picture file or the video file with the portrait removed, carrying out image classification and target detection by using a target detection model so as to identify related scenes or real objects, and then classifying the real objects by using a deep convolutional network mode.
The text files in the multimedia files are classified through two classifiers, namely a general classifier and an education classifier, and category labels are marked according to classification results. The general class is trained and classified by using a Chinese text classification training set (Thunews) of a Natural Language Processing (NLP) laboratory of Qinghua university, and an algorithm (TextCNN) classification algorithm for classifying texts by using a convolutional neural network is trained to classify texts into 14 classes of 'sports', 'finance', 'real estate', 'home', 'education', 'science', 'fashion', 'political', 'game', 'entertainment', 'lottery', 'stock', 'society', 'constellation', and the like, and the accuracy is 98.7 percent higher. The education category is formed by forming a test set through articles of various categories of colleges and universities in the mobile phone of the education website, text classification training is carried out through TextCNN, the text is divided into 8 categories of 'instrument equipment', 'party administration', 'capital construction', 'field affair', 'teaching', 'scientific research', 'administrative', 'financial accounting', and the like, and the accuracy rate is 93%. A user uploads a text file, obtains the file name and the detailed content of the text file, performs word removal, word segmentation and feature vectorization processing, firstly classifies the text data by using a general classifier, and if the text is classified into other 13 types which are not education types, marks corresponding type labels on the text; and if the text is classified into education, continuously classifying the text by using the education special classifier, and marking a corresponding class label according to a classification result.
Step S3, for each user, analyzing the uploaded multimedia file in step S2 to obtain portrait data, scene data, real object data, and text file classification data, on one hand, associating the portrait data, the scene data, the real object data, and the text file classification data with each multimedia file, on the other hand, associating the user with the uploaded multimedia file, the custom tag, and the portrait data, respectively, to initially form a network structure represented by a knowledge graph, i.e., a file graph. And then, the behaviors of other users operating the data in the file map on the platform can be recorded (such as previewing, downloading, uploading and the like) in association relation, and the data are updated into the map context, so that the map information is continuously enriched and displayed.
And step S4, determining the relationship type between two persons according to the number of the group photos and the number of people on the group photos existing between every two persons in the constructed person photo library, and constructing a person relationship network. As shown in fig. 3, specifically, the human relationships include same frame, recognition and intimacy, as follows:
when two people appear in the same photo in the form of a group photo, the two people are considered to be in the same frame relationship;
when the number of the group photos of the two people is larger than a first preset value (for example, 3) and not larger than a third preset value (for example, 8), and the number of people in each photo is smaller than a second preset value (for example, 5), the two people are considered as the cognitive relationship;
when the number of group photos of two people is larger than a third preset value (for example, 8) and the number of people in each photo is smaller than a second preset value (for example, 5), the two people are considered to be in close relationship.
In step S5, the user searches based on the document map, the person library, and the person relationship network. When the user inquires, the user can search in a multimedia mode (natural language such as a person name, a file name, a keyword, a file such as a picture and the like), and a file map of a search word or a person relation map of the person name and other basic results such as file information, map information, person information and the like can be obtained. For the query content input by the user, the text and other multimedia content are respectively analyzed and combined, so that the query intention of the user can be better analyzed, and a desired result can be fed back. As shown in fig. 4, in particular,
the user searches in a picture mode: representing the human image information in a vectorization way through human face detection and human face recognition, and/or processing through image classification and target detection to recognize scenes and real objects; then retrieving and matching the vectorized portrait information with the file map, the character library and the portrait in the character relation network according to the similarity, and/or retrieving and matching the identified scene and the identified real object with the scene and the real object in the file map according to the similarity; or
The user searches keywords in natural language: performing text error correction and correction on the keywords by combining a word bank and a shortest editing distance method, and performing retrieval matching from a file map, a character bank and a character relation network;
and the user searches and matches a corresponding result, preferentially displays related picture, video, text and homepage links corresponding to the result, and gives an option taking a search intention as a general content search. The following is divided:
when the matched picture or the keyword information is the scene name, preferentially displaying the related picture of the same scene to form a user file network map taking the scene name as the center, and providing an option entry taking the scene name as the general content retrieval;
when the matched picture or the keyword information is the name of the article, preferentially displaying the related picture containing the article, forming a user file network map taking the article as the center, and providing an option entry taking the article name as a general content retrieval;
when the matched keyword information is a file category name, preferentially displaying related documents under the category name to form a user file network map taking the file category as a center, and providing an option entry taking the file category name as a general content retrieval;
when the matched picture information is the name of the portrait, the related picture and video of the portrait are preferentially displayed to form a user file network map taking the portrait as the center, and an option entry taking the name of the portrait as a general content retrieval is provided;
when the matched keyword information is a user name, preferentially displaying the homepage link of the user to form a user file network map taking the user as the center, and providing an option entry taking the user name as general content retrieval;
when the matched keyword information is a custom tag, preferentially displaying a related file containing the custom tag, forming a user file network map taking the related file of the custom tag as a center, and providing an option entry taking the custom tag as a general content retrieval;
and when the matching is not successful, the picture or the keyword information is used as the general content, the general content is searched, and the relevant corresponding result is displayed.
Based on the incidence relation and standardized information of the document map such as the real object social attribute and the natural attribute, the corresponding information in the document map is integrated to construct related applications such as character relation, portrait retrieval, keyword search and map search, so that a multimedia omnibearing multi-angle retrieval system based on the document map technology is realized, a relatively complete knowledge system is constructed, the search breadth and depth are improved, and the search result is better displayed.
The above embodiments are provided only for illustrating the present invention and not for limiting the present invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and therefore all equivalent technical solutions should also fall within the scope of the present invention, and should be defined by the claims.
Claims (9)
1. A multimedia detection method based on knowledge graph is characterized by comprising the following steps:
step S1, a multimedia database is built, and each user uploads a shared multimedia file and a custom label to the multimedia database;
step S2, aiming at the picture files and video files containing the portrait in the multimedia files, processing through face detection, face recognition and face comparison to construct a portrait library; processing the picture file and the video file with the portrait removed in the multimedia file through image classification and target detection to identify a scene and a real object; classifying the text files in the multimedia files through two classifiers, namely a general classifier and an education classifier, and marking category labels according to classification results;
step S3, for each user, analyzing the uploaded multimedia file through step S2 to obtain portrait data, scene data, real object data and text file classification data, on one hand, associating the portrait data, the scene data, the real object data and the text file classification data with each multimedia file, on the other hand, associating the user with the uploaded multimedia file, the custom label and the portrait data to form a file map;
step S4, determining the relationship type between two persons according to the number of group photos and the number of people on the group photos existing between each two persons in the constructed portrait base, and constructing a person relationship network;
in step S5, the user searches based on the document map, the person library, and the person relationship network.
2. The method for detecting multimedia based on knowledge-graph according to claim 1, wherein the step S2 comprises:
when a human image is detected in a key frame of a picture file or a video file, a side face and a fuzzy face are filtered through face detection, then human image information is vectorized and expressed through face identification, then the vectorized human image information is compared with data in a human image library through the face, the similarity is calculated and is arranged in a descending order, if the similarity is the highest and is greater than a preset value, the matching is considered to be successful, new human image information is added into the human image library, otherwise, when the matching is failed, data are newly added into the human image library, and the human image library is constructed in the mode;
and aiming at the key frames of the picture file or the video file with the portrait removed, carrying out image classification and target detection by using a target detection model so as to identify related scenes or real objects, and then classifying the real objects by using a deep convolutional network mode.
3. The method of knowledge-graph-based multimedia detection according to claim 2, wherein the face detection means: calculating the distance ratio from the canthus to the tail of the eyes of the portrait by the distance of the characteristic points, judging that the face is in the lateral direction when the distance ratio is greater than a preset multiple, and filtering the face; and performing edge detection by using a Laplace operator, and when the Laplace operator value is smaller than a preset value, considering the portrait image as a fuzzy face and filtering the fuzzy face.
4. The method of claim 2, wherein the face detection and face recognition are performed using an angle-based face recognition model.
5. The method for detecting multimedia based on knowledge-graph according to claim 1, wherein the human relationship in step S4 includes:
when two people appear in the same photo in the form of a group photo, the two people are considered to be in the same frame relationship;
when the number of the group photos of the two people is larger than a first preset value and not larger than a third preset value, and the number of people in each photo is smaller than a second preset value, the two people are considered to be in an artificial recognition relationship;
and when the number of the group photos of the two people is larger than a third preset value and the number of people in each photo is smaller than the second preset value, the two people are considered to be in close relationship.
6. The method for detecting multimedia based on knowledge-graph as claimed in claim 1, wherein in step S3, the document graph records the operation behavior of other users on the data in the document graph and associates the operation behavior with the data.
7. The method of claim 1, wherein the generic text classes comprise sports, finance, real estate, home, education, science and technology, fashion, games, entertainment, lottery, stock, social, constellation; education text classification includes instrument, party administration, capital construction, field, teaching, scientific research, administration, financial accounting.
8. The method for detecting multimedia based on knowledge-graph according to claim 1, wherein the step S5 comprises:
the user searches in a picture mode: representing the human image information in a vectorization way through human face detection and human face recognition, and/or processing through image classification and target detection to recognize scenes and real objects; then retrieving and matching the vectorized portrait information with the file map, the character library and the portrait in the character relation network according to the similarity, and/or retrieving and matching the identified scene and the identified real object with the scene and the real object in the file map according to the similarity; or
The user searches keywords in natural language: performing text error correction and correction on the keywords by combining a word bank and a shortest editing distance method, and performing retrieval matching from a file map, a character bank and a character relation network;
and the user searches and matches a corresponding result, preferentially displays related picture, video, text and homepage links corresponding to the result, and gives an option taking a search intention as a general content search.
9. The method for knowledge-graph based multimedia detection according to claim 8, wherein in step S5,
when the matched picture or the keyword information is the scene name, preferentially displaying the related picture of the same scene to form a user file network map taking the scene name as the center, and providing an option entry taking the scene name as the general content retrieval;
when the matched picture or the keyword information is the name of the article, preferentially displaying the related picture containing the article, forming a user file network map taking the article as the center, and providing an option entry taking the article name as a general content retrieval;
when the matched keyword information is a file category name, preferentially displaying related documents under the category name to form a user file network map taking the file category as a center, and providing an option entry taking the file category name as a general content retrieval;
when the matched picture information is the name of the portrait, the related picture and video of the portrait are preferentially displayed to form a user file network map taking the portrait as the center, and an option entry taking the name of the portrait as a general content retrieval is provided;
when the matched keyword information is a user name, preferentially displaying the homepage link of the user to form a user file network map taking the user as the center, and providing an option entry taking the user name as general content retrieval;
when the matched keyword information is a custom tag, preferentially displaying a related file containing the custom tag, forming a user file network map taking the related file of the custom tag as a center, and providing an option entry taking the custom tag as a general content retrieval;
and when the matching is not successful, the picture or the keyword information is used as the general content, the general content is searched, and the relevant corresponding result is displayed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911036867.1A CN110750656B (en) | 2019-10-29 | 2019-10-29 | Multimedia detection method based on knowledge graph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911036867.1A CN110750656B (en) | 2019-10-29 | 2019-10-29 | Multimedia detection method based on knowledge graph |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110750656A true CN110750656A (en) | 2020-02-04 |
| CN110750656B CN110750656B (en) | 2023-03-14 |
Family
ID=69280772
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911036867.1A Active CN110750656B (en) | 2019-10-29 | 2019-10-29 | Multimedia detection method based on knowledge graph |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110750656B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111324819A (en) * | 2020-03-24 | 2020-06-23 | 北京字节跳动网络技术有限公司 | Method and device for searching media content, computer equipment and storage medium |
| CN111428044A (en) * | 2020-03-06 | 2020-07-17 | 中国平安人寿保险股份有限公司 | Method, device, equipment and storage medium for obtaining supervision identification result in multiple modes |
| CN111460119A (en) * | 2020-03-27 | 2020-07-28 | 海信集团有限公司 | Intelligent question and answer method and system for economic knowledge and intelligent equipment |
| CN111680202A (en) * | 2020-04-24 | 2020-09-18 | 烽火通信科技股份有限公司 | Body-based face image data collection method and device |
| CN111694965A (en) * | 2020-05-29 | 2020-09-22 | 中国科学院上海微系统与信息技术研究所 | Image scene retrieval system and method based on multi-mode knowledge graph |
| CN112069331A (en) * | 2020-08-31 | 2020-12-11 | 深圳市商汤科技有限公司 | Data processing method, data retrieval method, data processing device, data retrieval device, data processing equipment and storage medium |
| CN112069326A (en) * | 2020-09-03 | 2020-12-11 | Oppo广东移动通信有限公司 | Knowledge graph construction method and device, electronic equipment and storage medium |
| CN112905812A (en) * | 2021-02-01 | 2021-06-04 | 上海德拓信息技术股份有限公司 | Media file auditing method and system |
| CN113157956A (en) * | 2021-04-23 | 2021-07-23 | 雅马哈发动机（厦门）信息系统有限公司 | Picture searching method, system, mobile terminal and storage medium |
| CN113360459A (en) * | 2021-07-08 | 2021-09-07 | 国网能源研究院有限公司 | Method, system and device for semi-automatically marking and storing files |
| WO2021237731A1 (en) * | 2020-05-29 | 2021-12-02 | 西门子股份公司 | Target detection method and device, and computer readable medium |
| CN114090777A (en) * | 2021-11-26 | 2022-02-25 | 北京金山数字娱乐科技有限公司 | Text data processing method and device |
| WO2023141900A1 (en) * | 2022-01-27 | 2023-08-03 | 基建通(三亚)国际科技有限公司 | Method and apparatus for establishing news image-text data knowledge graph, and device and medium |
| CN116881472A (en) * | 2023-07-14 | 2023-10-13 | 郑州华商科技有限公司 | Funds penetration and character relation analysis method based on graph database technology |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070130112A1 (en) * | 2005-06-30 | 2007-06-07 | Intelligentek Corp. | Multimedia conceptual search system and associated search method |
| US20090157625A1 (en) * | 2007-12-13 | 2009-06-18 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Methods and systems for identifying an avatar-linked population cohort |
| CN105550190A (en) * | 2015-06-26 | 2016-05-04 | 许昌学院 | Knowledge graph-oriented cross-media retrieval system |
| WO2018055646A1 (en) * | 2016-09-22 | 2018-03-29 | Dogma Srl. | Method and system for searching, publishing and managing the life cycle of multimedia contents related to public events and the user experience |
-
2019
- 2019-10-29 CN CN201911036867.1A patent/CN110750656B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070130112A1 (en) * | 2005-06-30 | 2007-06-07 | Intelligentek Corp. | Multimedia conceptual search system and associated search method |
| US20090157625A1 (en) * | 2007-12-13 | 2009-06-18 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Methods and systems for identifying an avatar-linked population cohort |
| CN105550190A (en) * | 2015-06-26 | 2016-05-04 | 许昌学院 | Knowledge graph-oriented cross-media retrieval system |
| WO2018055646A1 (en) * | 2016-09-22 | 2018-03-29 | Dogma Srl. | Method and system for searching, publishing and managing the life cycle of multimedia contents related to public events and the user experience |
Non-Patent Citations (1)
| Title |
|---|
| 李夏风;: "人脸识别技术在智慧城市的深度应用" * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111428044A (en) * | 2020-03-06 | 2020-07-17 | 中国平安人寿保险股份有限公司 | Method, device, equipment and storage medium for obtaining supervision identification result in multiple modes |
| CN111428044B (en) * | 2020-03-06 | 2024-04-05 | 中国平安人寿保险股份有限公司 | Method, device, equipment and storage medium for acquiring supervision and identification results in multiple modes |
| CN111324819A (en) * | 2020-03-24 | 2020-06-23 | 北京字节跳动网络技术有限公司 | Method and device for searching media content, computer equipment and storage medium |
| CN111460119A (en) * | 2020-03-27 | 2020-07-28 | 海信集团有限公司 | Intelligent question and answer method and system for economic knowledge and intelligent equipment |
| CN111460119B (en) * | 2020-03-27 | 2024-04-12 | 海信集团有限公司 | Intelligent question-answering method and system for economic knowledge and intelligent equipment |
| CN111680202A (en) * | 2020-04-24 | 2020-09-18 | 烽火通信科技股份有限公司 | Body-based face image data collection method and device |
| CN111680202B (en) * | 2020-04-24 | 2022-04-26 | 烽火通信科技股份有限公司 | Body-based face image data collection method and device |
| WO2021237731A1 (en) * | 2020-05-29 | 2021-12-02 | 西门子股份公司 | Target detection method and device, and computer readable medium |
| CN111694965A (en) * | 2020-05-29 | 2020-09-22 | 中国科学院上海微系统与信息技术研究所 | Image scene retrieval system and method based on multi-mode knowledge graph |
| CN111694965B (en) * | 2020-05-29 | 2023-06-13 | 中国科学院上海微系统与信息技术研究所 | Image scene retrieval system and method based on multi-mode knowledge graph |
| CN112069331A (en) * | 2020-08-31 | 2020-12-11 | 深圳市商汤科技有限公司 | Data processing method, data retrieval method, data processing device, data retrieval device, data processing equipment and storage medium |
| CN112069331B (en) * | 2020-08-31 | 2024-06-11 | 深圳市商汤科技有限公司 | Data processing and searching method, device, equipment and storage medium |
| CN112069326A (en) * | 2020-09-03 | 2020-12-11 | Oppo广东移动通信有限公司 | Knowledge graph construction method and device, electronic equipment and storage medium |
| CN112905812B (en) * | 2021-02-01 | 2023-07-11 | 上海德拓信息技术股份有限公司 | Media file auditing method and system |
| CN112905812A (en) * | 2021-02-01 | 2021-06-04 | 上海德拓信息技术股份有限公司 | Media file auditing method and system |
| CN113157956A (en) * | 2021-04-23 | 2021-07-23 | 雅马哈发动机（厦门）信息系统有限公司 | Picture searching method, system, mobile terminal and storage medium |
| CN113360459A (en) * | 2021-07-08 | 2021-09-07 | 国网能源研究院有限公司 | Method, system and device for semi-automatically marking and storing files |
| CN114090777A (en) * | 2021-11-26 | 2022-02-25 | 北京金山数字娱乐科技有限公司 | Text data processing method and device |
| WO2023141900A1 (en) * | 2022-01-27 | 2023-08-03 | 基建通(三亚)国际科技有限公司 | Method and apparatus for establishing news image-text data knowledge graph, and device and medium |
| CN116881472A (en) * | 2023-07-14 | 2023-10-13 | 郑州华商科技有限公司 | Funds penetration and character relation analysis method based on graph database technology |
| CN116881472B (en) * | 2023-07-14 | 2024-04-30 | 郑州华商科技有限公司 | Funds penetration and character relation analysis method based on graph database technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110750656B (en) | 2023-03-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110750656B (en) | Multimedia detection method based on knowledge graph | |
| CN111858954B (en) | Task-oriented text-generated image network model | |
| Strezoski et al. | Omniart: a large-scale artistic benchmark | |
| CN104063683B (en) | Expression input method and device based on face identification | |
| CN113158023B (en) | Public digital life accurate classification service method based on mixed recommendation algorithm | |
| CN110532379B (en) | Electronic information recommendation method based on LSTM (least Square TM) user comment sentiment analysis | |
| Zhou et al. | Conceptlearner: Discovering visual concepts from weakly labeled image collections | |
| CN107562742A (en) | A kind of image processing method and device | |
| Lee et al. | Style classification and visualization of art painting’s genre using self-organizing maps | |
| CN107515934A (en) | A kind of film semanteme personalized labels optimization method based on big data | |
| CN110750995A (en) | File management method based on user-defined map | |
| Roy et al. | Automated detection of substance use-related social media posts based on image and text analysis | |
| CN115131698B (en) | Video attribute determining method, device, equipment and storage medium | |
| Yang et al. | Explaining deep convolutional neural networks via latent visual-semantic filter attention | |
| JP5433396B2 (en) | Manga image analysis device, program, search device and method for extracting text from manga image | |
| CN107908749B (en) | Character retrieval system and method based on search engine | |
| CN114925198B (en) | Knowledge-driven text classification method integrating character information | |
| Chen et al. | Exploiting aesthetic features in visual contents for movie recommendation | |
| Fouad et al. | Adaptive visual sentiment prediction model based on event concepts and object detection techniques in social media | |
| CN116955707A (en) | Content tag determination method, device, equipment, medium and program product | |
| Wieczorek et al. | Semantic Image-Based Profiling of Users' Interests with Neural Networks | |
| CN109062995B (en) | Personalized recommendation algorithm for drawing Board (Board) cover on social strategy exhibition network | |
| CN111125387B (en) | Multimedia list generation and naming method and device, electronic equipment and storage medium | |
| RU2710942C1 (en) | Simultaneous recognition of person attributes and identification of person in organizing photo albums | |
| Vadivukarassi et al. | A framework of keyword based image retrieval using proposed Hog_Sift feature extraction method from Twitter Dataset |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |