CN105550190A - Knowledge graph-oriented cross-media retrieval system - Google Patents

Knowledge graph-oriented cross-media retrieval system Download PDF

Info

Publication number
CN105550190A
CN105550190A CN201510358374.5A CN201510358374A CN105550190A CN 105550190 A CN105550190 A CN 105550190A CN 201510358374 A CN201510358374 A CN 201510358374A CN 105550190 A CN105550190 A CN 105550190A
Authority
CN
China
Prior art keywords
semantic
media
data
user
knowledge
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201510358374.5A
Other languages
Chinese (zh)
Other versions
CN105550190B (en
Inventor
杨月华
张铃丽
平源
王亚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Huchuan Technology Co.,Ltd.
Original Assignee
Xuchang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xuchang University filed Critical Xuchang University
Priority to CN201510358374.5A priority Critical patent/CN105550190B/en
Publication of CN105550190A publication Critical patent/CN105550190A/en
Application granted granted Critical
Publication of CN105550190B publication Critical patent/CN105550190B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

In order to meet the needs of cross-media semantic description and knowledge acquisition and effectively utilize cross-media attributes and various correlations covered in a knowledge graph, the invention proposes to establish a cross-media attribute perception model for performing perceptive and correlative analysis on natural attributes and social attributes contained in cross-media data, and establish a unified cross-media data correlation description mechanism for performing unified quantitative expression on different correlative relationships; a method for mapping data in different forms contained in the knowledge graph to a same semantic space is proposed, thereby realizing semantic knowledge consistency expression; for a query request expressed by user through a natural language, a multimedia example or combination of different media data, a method for performing semantic analysis on user query by virtue of various correlations contained in the knowledge graph to understand a user retrieval intention so as to retrieve related results meeting user query demands is proposed; and a cross-media retrieval system framework introducing the knowledge graph and a realization method are proposed.

Description

Towards the cross-media retrieval system of knowledge mapping
Technical field
The invention belongs to information retrieval technique category, be specially the cross-media retrieval system towards knowledge mapping.The context data that knowledge mapping contributes to obtaining various dimension is introduced in cross-media retrieval, even find the feature under different situation by further reasoning, thus the semanteme of user's query contents can be understood better, return the result for retrieval of more meeting consumers' demand.
Background technology
Current, the development of global network and universally reached unprecedented scale, people have got used to searching various information on the internet, and search engine has become the center of internet.Domestic each internet giant is just sparing no effort to improve oneself search engine, the important development direction supported during " New-generation search engines and browser " is also classified as " 12 " by country's " core Gao Ji " scientific and technological key special subjects.But the information on internet is exponentially level growth, and type is various, complicated association is there is between the information of various media format, these cross correlations make internet data present across media characteristic, and thisly have higher requirement to internet information analysis and retrieval across media characteristic.After knowledge mapping is introduced cross-media retrieval system, contribute to the context data obtaining various dimension, support that user is with natural language, multimedia sample or dissimilar sets of media data incompatible expression retrieval intention better, the feature under different situation can also be found by further reasoning, realize user's query semantics more accurately and analyze and retrieval.Therefore, the present invention gives the implementation of cross-media retrieval system from the angle of knowledge mapping.
Knowledge mapping has developed after Open database connectivity company Metaweb purchased in 2010 in Google.Metaweb was mainly absorbed at that time and different literals statement was coupled together with same entity, and explored these entity attributes (age of such as star) and contact each other, finally provided a kind of search form newly.Although can not substitute keyword search completely, index, the searching method of Metaweb are more efficient when processing the inquiry of natural language.Equally, in cross-media retrieval, by knowledge mapping, also can understand the inquiry request of user better and sum up relevant content semantic to query demand, for user finds out the relevant information more accurately and more having the degree of depth.In addition, knowledge mapping also can help user to understand relation between things.When the inquiry request that user expresses with the combination of natural language, multimedia sample or dissimilar media data, such a inquiry request may represent multiple implication, knowledge mapping can understand difference wherein, and that implication Search Results range shorter can wanted to user.Moreover, because knowledge mapping constructs a complete knowledge hierarchy relevant to Search Results, merge a lot of subject, the knowledge hierarchy relevant to user's query semantics is showed user systematically, so user may recognize certain new fact or new contact when retrieving, impel it to carry out a series of brand-new search inquiry, allow search more have depth & wideth.Therefore, knowledge mapping is introduced in cross-media retrieval, for improvement retrieval performance, there is vital role.
Therefore, the present invention is with the cross-media retrieval gordian technique towards knowledge mapping for research object, and the sensor model proposed across medium property is expressed, across the consistance expression of media knowledge and the implementation of user's query semantics analytical approach of knowledge based collection of illustrative plates and the cross-media retrieval system towards knowledge mapping with the multiple unified quantization that associates.In information retrieval field, development from Present Domestic, becomes inexorable trend towards knowledge mapping with across media, and therefore the present invention has very large actual application value and wide application prospect.
Summary of the invention
The object of the present invention is to provide one in cross-media retrieval, to introduce knowledge mapping across media information retrieval instrument, what knowledge based collection of illustrative plates was contained carries out semantic analysis and reasoning across media semantic association and knowledge, realizes cross-media retrieval.Specifically, content of the present invention comprise following some.
(1) for complicated across media data on internet, set up across medium property sensor model and analyze the incidence relation wherein contained, what propose a kind of unification describes mechanism across media data association.Pass through text resolution, entity extracts, metadata analysis, the technology such as semantic tagger and user behavior analysis obtains across the natural quality of media data and social property, then association modeling is carried out to across the complex relationship in media data between natural quality and social property, consider across the relevance existed between media data (same mode) in modeling process, semantic association (different modalities), sequential correlation, the multiple association such as structure connection, according to the link between the webpage of multimedia object place, based on probability graph model to the modeling analysis carrying out randomization across media content and link, thus unified quantization expression is carried out to dissimilar incidence relation.
(2) in order to meet the needs across media semantic description and knowledge acquisition, proposing the method for the data-mapping of different shape to same semantic label space, realizing semantic consistency and expressing.Work as text, when the media modalities co expression one of the isomery complementations such as image is semantic, by learning certain mapping relations, these isomery modal informations are mapped to a semantic label space, thus directly similarity measurement is carried out to isomeric data under expressing framework at one, and according to semantic similarity, semantic coverage and semantic space calibration set up evaluation function, the alternative of semantic label is evaluated, semantic label information is utilized to be respectively each shape up exercise sorter, and using the result of classification as sharing feature, make the data of different shape also can be mapped to same semantic label space, thus realize semantic consistency expression.
(3) propose when user to carry out the method for semantic analysis and reasoning to it with the association contained in conjunction with knowledge mapping during natural language, multimedia sample or dissimilar media data combination expression inquiry request.For the query contents of user's input, respectively the content of text and multimedia inquiry is carried out separately and Conjoint Analysis, carry out analyzing user queries intention from semantic level.Therefore first gather from internet and enough set up semantic model respectively across media information and for the data of different media types, realize across the feature interpretation of media data on same semantic space.Then the semantic distributional analysis of composite image data and text data and the semanteme of identification user inquiry, and carry out the semantic excavation of further association in conjunction with knowledge mapping.Knowledge based collection of illustrative plates contain data semantic association, sequential correlation and structure connection etc., obtain the context data of the various dimensions relevant to user's query contents, and find the feature under different situation by reasoning, thus obtain more perfect query semantics.
(4) cross-media retrieval system architecture and the implementation method of introducing knowledge mapping are proposed.System, except possessing the elements such as user's query analysis, index, retrieval and sequence, also will create knowledge mapping knowledge base of certain scale and be integrated in system.In user's query analysis part, support that user is with natural language, query contents across the input of the form such as media sample, different media types data.When carrying out query semantics and analyzing, except semantic analysis will be carried out respectively to the various media type data of user's input, also to carry out combination semantic analysis and further reasoning, to understand user's query intention better according to context knowledge such as the time on knowledge mapping, place, entity and social relationships thereof in conjunction with knowledge mapping to it.More existing algorithms are mainly being called across media hash index and sort sections.
Accompanying drawing explanation
Fig. 1 is across medium property perception and association analysis;
Fig. 2 is user's query semantics analysis of knowledge based collection of illustrative plates;
Fig. 3 is the cross-media retrieval system architecture towards knowledge mapping.
Embodiment
For making object of the present invention, technical scheme and advantage clearly understand, below in conjunction with Figure of description, the present invention is described in further detail.
1. across medium property perception and association analysis
The mode that current knowledge is propagated more and more has the characteristic across media, the relevant knowledge of same entity and information are often from multiple support channels, with media form coordinate expression, and contain multiple natural quality and social property, in order to utilize across the association knowledge contained in media data and use it in cross-media retrieval, in the process building knowledge mapping, except the semantic relation of inter-entity will be considered, also to consider the perception across medium property to entity, set up across medium property sensor model and association analysis is carried out to it.In order to carry out unified quantization expression to dissimilar incidence relation, and effectively predicting potential association, making mutually utilize between different incidence relations, that sets up a kind of unification describes mechanism across media data association.
(microblogging is comprised for from multiple support channels, micro-letter, forum, news website, professional website etc.), with media form (text, sound, image, video) coordinate expression, and contain the multiple natural quality (time, place, personage, apparent information etc.) and social property (as temperature, evaluate and preference etc.) entity relevant information, based on and text accompanying information between complementary information extract the high-level semantic of other media type data, then text resolution is passed through, entity extracts, metadata analysis, the technology such as semantic tagger and user behavior analysis obtains across the natural quality of media data and social property, by one group of support vector machine classifier, new data is classified again, thus from noisy network image concentrate automatically extract and identify generic target, or by analyzing the network user, modeling is carried out to the attention rate etc. of forwarding behavior to real-world user across media data, behavior is forwarded by the data contents such as analysis microblogging, micro-letter, social networks and user, build forwarding tree model, and utilize frequent subtree to find repeatability and the tendentiousness rule of user behavior, thus community interest degree is followed the tracks of more accurately and predicts.Next association modeling is carried out to across the complex relationship in media data between natural quality and social property, consider across the relevance existed between media data (same mode) in modeling process, semantic association (different modalities), sequential correlation, the multiple association such as structure connection, according to the link between the webpage of multimedia object place, based on probability graph model to the modeling analysis carrying out randomization across media content and link, thus unified quantization expression is carried out to dissimilar incidence relation, and the interaction prediction realized further across media data, as shown in Figure 1.
2. the consistance across media knowledge is expressed
Because existing knowledge representation mode and knowledge base sources are also confined to the state of single mode substantially, the needs across media semantic description and knowledge acquisition cannot be met, therefore contain after across medium property knowledge in the knowledge mapping built, use it in cross-media retrieval, on the basis analyzing single modal data semantic knowledge expression rule, propose the method for the data-mapping of different shape to same semantic label space, thus realize semantic consistency expression.In order to from single Modal Expansion to across media representation of knowledge aspect, propose the method that the content of medium type various in knowledge mapping is calculated and measured, be mapped to certain space to carry out structure analysis, fusion and reasoning etc. by unified for the structural information of media data theoretically.
Obtain enough across medium property knowledge and incidence relation after, in order to use it in cross-media retrieval, different data granularities, different knowledge hierarchy be set up and represent mechanism across media knowledge consistance.When the media modalities co expression one of the isomery such as text, image complementation is semantic, by learning certain mapping relations, these isomery modal informations are mapped to a shared subspace, under just can expressing framework at one, directly similarity measurement are carried out to isomeric data.For content and semantically have correlativity across media data, adopt probability generation model that the data of different media types are transformed into unified latent variables space to be described, using across the distribution of media data in each hidden variable as its semantic label, and set up evaluation function according to semantic similarity, semantic coverage and semantic space calibration, the alternative of semantic label is evaluated, and sets up set of semantics.Utilize the semantic label information of set of semantics, same modal data in different multimedia document is extracted respectively, the semantic label of utilization group is respectively each shape up exercise sorter, and using the result of classification as sharing feature, make the data of different shape also can be mapped to same semantic label space, thus realize semantic consistency expression.
The key that semantic label is selected calculates it and the semantic dependency across media content, namely the coupling between semantic label and semantic model, in order to can directly semantic label and semantic model be compared, the mode that semantic label distributes with semanteme is represented, uses the Semantic Similarity between KL distance computing semantic label and semantic model.In order to obtain semanteme distribution { p (w|l) } of semantic label l, by carrying out approximate evaluation { p (w|l, D) } across media data collection D.So just can use the Semantic Similarity between KL distance computing semantic label { p (w|l) } and semantic model { p (w| θ) }:
S ( l , θ ) = - d ( θ | | l ) = - Σ w p ( w | θ ) log p ( w | θ ) p ( w | l ) - - - ( 1 )
In order to ensure that semantic label has higher coverage to the semantic content across media data, the neology word selected can cover other semantic component, instead of the content that existing semantic word has been contained, adopting maximal margin correlation technique, obtaining maximum correlation and the semantic word of otherness by maximizing maximal margin correlativity:
l ^ = arg max l ∈ L - S [ λS ( l , θ ) - ( 1 - λ ) max Sim ( l ' , l ) l ∈ L - S ] - - - ( 2 )
Sim ( l ' , l ) = - d ( l ' | | l ) = - Σ w p ( w | l ' ) log p ( w | l ' ) p ( w | l ) - - - ( 3 )
Wherein, S is the semantic word selected.
In addition, when marking multiple semantic content, in order to ensure that a semantic word can not have the higher degree of correlation with multiple semantic content, also to consider the differentiation between different semantic content, i.e. semantic space calibration, in this case, need to adopt the Semantic Similarity computing method considering discrimination:
S’(l,θ i)=S(l,θ i)-αS(l,θ -i)(4)
S (l, θ -i)=-d (θ -i‖ l) (5) wherein, θ -1represent except semantic feature θ 1outside other k-1 semantic feature, i.e. θ 1 ... i-1i+1 ... k, k is semantic feature number.Pass through S ' (l, θ i) calculate and to go forward side by side line ordering across the semantic similarity of semantic feature, thus can be correlated with for multiple semantic content generative semantics and there is the semantic word of certain coverage and discrimination.
3. user's query semantics of knowledge based collection of illustrative plates is analyzed
For the query contents of user's input, need to carry out separately and Conjoint Analysis the content of text and multimedia inquiry respectively, carry out analyzing user queries intention from semantic level.Therefore, first gather from internet and enough set up semantic model respectively across media information and for the data of different media types, as shown in Figure 2: the text semantic model described with text word and the vision semantic model described with visual word; Then utilize these two models that the text data in document to be analyzed and view data are all transformed into identical semantic space, and be described in the mode of semantic probability distribution.The Semantic mapping of different media types data is realized afterwards by semantic study.In order to be associated between the data of different media types, excavate the shared subspace existed between relevance dissimilar medium data, for have semantic dependency across media data, vision data as relevant to text semantic in image, video etc., text data is adopted to carry out the study of vision semanteme, describe text semantic with the form of visual word, set up the mapping relations between text semantic and vision semanteme, thus realize across the feature interpretation of media data on same semantic space.
After obtaining the semantic characteristics description across media data, the semantic distributional analysis of composite image data and text data and the semanteme of identification user inquiry, and carry out the semantic excavation of further association in conjunction with knowledge mapping.Knowledge based collection of illustrative plates contain data semantic association, sequential correlation and structure connection etc., obtain the context data of the various dimensions relevant to user's query contents, as time, place, entity and social relationships thereof etc., and find the feature under different situation by reasoning, thus obtain more perfect query semantics.What relate to due to reasoning is across media data, so first to realize carrying out formalization representation across media to the conversion of Text Mode based on technology such as moving object action recognition in image labeling, video before reasoning, then text based inference technology realizes reasoning.Need in semantic layer process across media data in transfer process, can realize across media semantic model based on set up.
4. towards the cross-media retrieval system of knowledge mapping
In order to realize a cross-media retrieval system towards knowledge mapping, first the cross-media retrieval system architecture introducing knowledge mapping is proposed, as shown in Figure 3.System, except possessing the elements such as user's query analysis, index, retrieval, sequence, adds and expresses several part across medium property perception and association analysis and consistance.First enough multi-medium datas are gathered from internet, based on the natural quality obtained respectively across medium property sensor model across media data and social property, then association analysis and description are carried out to the semantic association, sequential correlation, structure connection etc. of the entity object association of wherein containing, various media type data.Build on this basis to be formed afterwards and reach the knowledge mapping of certain scale, in order to utilize contain in knowledge mapping across media knowledge, express framework based on proposed consistance and it represented.
In user's query analysis part, support that user is with natural language, query contents across the input of the form such as media sample, different media types data.When carrying out query semantics and analyzing, except semantic analysis will be carried out respectively to the various media type data of user's input, also to carry out combination semantic analysis and further reasoning, to understand user's query intention better according to context knowledge such as the time on knowledge mapping, place, entity and social relationships thereof in conjunction with knowledge mapping to it.More existing algorithms are mainly being called across media hash index and sort sections.

Claims (5)

1. towards the cross-media retrieval system of knowledge mapping, it is characterized in that, this system covers the content of following aspect:
Across medium property perception and association analysis;
Consistance across media knowledge is expressed;
User's query semantics of knowledge based collection of illustrative plates is analyzed;
Towards cross-media retrieval system architecture and the realization of knowledge mapping.
2. system according to claim 1, is characterized in that, sets up across medium property sensor model and analyzes the incidence relation wherein contained, and what propose a kind of unification describes mechanism across media data association.Pass through text resolution, entity extracts, metadata analysis, the technology such as semantic tagger and user behavior analysis obtains across the natural quality of media data and social property, then association modeling is carried out to across the complex relationship in media data between natural quality and social property, consider across the relevance existed between media data (same mode) in modeling process, semantic association (different modalities), sequential correlation, the multiple association such as structure connection, according to the link between the webpage of multimedia object place, based on probability graph model to the modeling analysis carrying out randomization across media content and link, thus unified quantization expression is carried out to dissimilar incidence relation.
3. system according to claim 1, is characterized in that, in order to meet the needs across media semantic description and knowledge acquisition, proposes the method for the data-mapping of different shape to same semantic label space, realizes semantic consistency and expresses.Work as text, when the media modalities co expression one of the isomery complementations such as image is semantic, by learning certain mapping relations, these isomery modal informations are mapped to a semantic label space, thus directly similarity measurement is carried out to isomeric data under expressing framework at one, and according to semantic similarity, semantic coverage and semantic space calibration set up evaluation function, the alternative of semantic label is evaluated, semantic label information is utilized to be respectively each shape up exercise sorter, and using the result of classification as sharing feature, make the data of different shape also can be mapped to same semantic label space, thus realize semantic consistency expression.
4. system according to claim 1, is characterized in that, proposes when user to carry out the method for semantic analysis and reasoning to it with the association contained in conjunction with knowledge mapping during natural language, multimedia sample or dissimilar media data combination expression inquiry request.For the query contents of user's input, respectively the content of text and multimedia inquiry is carried out separately and Conjoint Analysis, carry out analyzing user queries intention from semantic level.Therefore first gather from internet and enough set up semantic model respectively across media information and for the data of different media types, realize across the feature interpretation of media data on same semantic space.Then the semantic distributional analysis of composite image data and text data and the semanteme of identification user inquiry, and carry out the semantic excavation of further association in conjunction with knowledge mapping.Knowledge based collection of illustrative plates contain data semantic association, sequential correlation and structure connection etc., obtain the context data of the various dimensions relevant to user's query contents, and find the feature under different situation by reasoning, thus obtain more perfect query semantics.
5. system according to claim 1, is characterized in that, system, except possessing the elements such as user's query analysis, index, retrieval and sequence, also will create knowledge mapping knowledge base of certain scale and be integrated in system.In user's query analysis part, support that user is with natural language, query contents across the input of the form such as media sample, different media types data.When carrying out query semantics and analyzing, except semantic analysis will be carried out respectively to the various media type data of user's input, also to carry out combination semantic analysis and further reasoning, to understand user's query intention better according to context knowledge such as the time on knowledge mapping, place, entity and social relationships thereof in conjunction with knowledge mapping to it.
CN201510358374.5A 2015-06-26 2015-06-26 Cross-media retrieval system towards knowledge mapping Active CN105550190B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510358374.5A CN105550190B (en) 2015-06-26 2015-06-26 Cross-media retrieval system towards knowledge mapping

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510358374.5A CN105550190B (en) 2015-06-26 2015-06-26 Cross-media retrieval system towards knowledge mapping

Publications (2)

Publication Number Publication Date
CN105550190A true CN105550190A (en) 2016-05-04
CN105550190B CN105550190B (en) 2019-03-29

Family

ID=55829379

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510358374.5A Active CN105550190B (en) 2015-06-26 2015-06-26 Cross-media retrieval system towards knowledge mapping

Country Status (1)

Country Link
CN (1) CN105550190B (en)

Cited By (57)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106156365A (en) * 2016-08-03 2016-11-23 北京智能管家科技有限公司 A kind of generation method and device of knowledge mapping
CN106776370A (en) * 2016-12-05 2017-05-31 哈尔滨工业大学(威海) Cloud storage method and device based on the assessment of object relevance
CN106776564A (en) * 2016-12-21 2017-05-31 张永成 The method for recognizing semantics and system of a kind of knowledge based collection of illustrative plates
CN106844334A (en) * 2016-12-20 2017-06-13 网易(杭州)网络有限公司 Method and apparatus for evaluating and testing session robotic intelligence
CN107038261A (en) * 2017-05-28 2017-08-11 海南大学 A kind of processing framework resource based on data collection of illustrative plates, Information Atlas and knowledge mapping can Dynamic and Abstract Semantic Modeling Method
CN107247739A (en) * 2017-05-10 2017-10-13 浙江大学 A kind of financial publication text knowledge extracting method based on factor graph
CN107256271A (en) * 2017-06-27 2017-10-17 鲁东大学 Cross-module state Hash search method based on mapping dictionary learning
CN107273418A (en) * 2017-05-11 2017-10-20 浙江大学 A kind of across Noumenon property chain inference method based on cloud platform
CN107291828A (en) * 2017-05-27 2017-10-24 北京百度网讯科技有限公司 Spoken inquiry analytic method, device and storage medium based on artificial intelligence
CN107330520A (en) * 2017-06-09 2017-11-07 上海电力学院 The object Affording acquisition inference method that a kind of knowledge based storehouse is represented
CN107369098A (en) * 2016-05-11 2017-11-21 华为技术有限公司 The treating method and apparatus of data in social networks
CN107423820A (en) * 2016-05-24 2017-12-01 清华大学 The knowledge mapping of binding entity stratigraphic classification represents learning method
CN107748754A (en) * 2017-09-15 2018-03-02 广州唯品会研究院有限公司 A kind of knowledge mapping improving method and device
CN107783973A (en) * 2016-08-24 2018-03-09 慧科讯业有限公司 The methods, devices and systems being monitored based on domain knowledge spectrum data storehouse to the Internet media event
CN107967267A (en) * 2016-10-18 2018-04-27 中兴通讯股份有限公司 A kind of knowledge mapping construction method, apparatus and system
CN108009182A (en) * 2016-10-28 2018-05-08 京东方科技集团股份有限公司 A kind of information extracting method and device
CN108090167A (en) * 2017-12-14 2018-05-29 畅捷通信息技术股份有限公司 Method, system, computing device and the storage medium of data retrieval
CN108491502A (en) * 2018-03-21 2018-09-04 腾讯科技(深圳)有限公司 A kind of method, terminal, server and the storage medium of news tracking
CN108549667A (en) * 2018-03-23 2018-09-18 绍兴诺雷智信息科技有限公司 A kind of semantic retrieving method of structuring engineering design knowledge
CN108959328A (en) * 2017-05-27 2018-12-07 株式会社理光 Processing method, device and the electronic equipment of knowledge mapping
CN109522465A (en) * 2018-10-22 2019-03-26 国家电网公司 The semantic searching method and device of knowledge based map
CN109697233A (en) * 2018-12-03 2019-04-30 中电科大数据研究院有限公司 A kind of knowledge mapping system building method
CN109716286A (en) * 2016-08-16 2019-05-03 电子湾有限公司 Determine the item with confirmed feature
CN109710776A (en) * 2018-12-29 2019-05-03 中国科学技术大学 The construction method of the knowledge mapping of photograph album
CN109710923A (en) * 2018-12-06 2019-05-03 浙江大学 Based on across the entity language matching process across media information
CN110275898A (en) * 2018-03-16 2019-09-24 埃森哲环球解决方案有限公司 Use the integrated monitoring and communication system of the explanatory equipment management of knowledge based figure
CN110457502A (en) * 2019-08-21 2019-11-15 京东方科技集团股份有限公司 Construct knowledge mapping method, man-machine interaction method, electronic equipment and storage medium
CN110489565A (en) * 2019-08-15 2019-11-22 广州拓尔思大数据有限公司 Based on the object root type design method and system in domain knowledge map ontology
CN110532341A (en) * 2019-09-03 2019-12-03 华东师范大学 Spatial information space-time big data constraint expression method
CN110532404A (en) * 2019-09-03 2019-12-03 北京百度网讯科技有限公司 One provenance multimedia determines method, apparatus, equipment and storage medium
CN110597992A (en) * 2019-09-10 2019-12-20 腾讯科技(深圳)有限公司 Semantic reasoning method and device based on knowledge graph and electronic equipment
CN110647804A (en) * 2019-08-09 2020-01-03 中国传媒大学 Violent video identification method, computer system and storage medium
CN110689033A (en) * 2018-07-05 2020-01-14 第四范式(北京)技术有限公司 Data acquisition method, device and equipment for model training and storage medium
CN110741389A (en) * 2017-11-21 2020-01-31 谷歌有限责任公司 Improved data communication of entities
CN110750656A (en) * 2019-10-29 2020-02-04 上海德拓信息技术股份有限公司 Multimedia detection method based on knowledge graph
CN110928961A (en) * 2019-11-14 2020-03-27 出门问问(苏州)信息科技有限公司 Multi-mode entity linking method, equipment and computer readable storage medium
CN110970112A (en) * 2018-09-29 2020-04-07 九阳股份有限公司 Method and system for constructing knowledge graph for nutrition and health
CN110968776A (en) * 2018-09-30 2020-04-07 北京国双科技有限公司 Policy knowledge recommendation method, device storage medium and processor
CN111221984A (en) * 2020-01-15 2020-06-02 北京百度网讯科技有限公司 Multimodal content processing method, device, equipment and storage medium
CN111460169A (en) * 2020-03-27 2020-07-28 科大讯飞股份有限公司 Semantic expression generation method, device and equipment
CN111666425A (en) * 2020-06-10 2020-09-15 深圳开思时代科技有限公司 Automobile accessory searching method based on semantic knowledge
CN111680173A (en) * 2020-05-31 2020-09-18 西南电子技术研究所(中国电子科技集团公司第十研究所) CMR model for uniformly retrieving cross-media information
CN111680207A (en) * 2020-03-11 2020-09-18 华中科技大学鄂州工业技术研究院 Method and device for determining search intention of user
CN111708745A (en) * 2020-06-18 2020-09-25 全球能源互联网研究院有限公司 Cross-media data sharing representation method and user behavior analysis method and system
CN112084339A (en) * 2020-08-11 2020-12-15 同济大学 Traffic knowledge graph construction method based on cross-media data
CN112115270A (en) * 2019-06-20 2020-12-22 国电南瑞科技股份有限公司 Method for constructing transformer knowledge graph ontology relation model
CN112732969A (en) * 2021-01-14 2021-04-30 珠海格力电器股份有限公司 Image semantic analysis method and device, storage medium and electronic equipment
CN112749289A (en) * 2020-12-31 2021-05-04 重庆空间视创科技有限公司 Multi-mode-based knowledge graph retrieval system and method
CN112836060A (en) * 2019-11-25 2021-05-25 中国科学技术信息研究所 Map construction method and device for scientific and technological innovation data
CN112948547A (en) * 2021-01-26 2021-06-11 中国石油大学(北京) Logging knowledge graph construction query method, device, equipment and storage medium
CN113010701A (en) * 2021-02-25 2021-06-22 北京四达时代软件技术股份有限公司 Video-centered fused media content recommendation method and device
CN113111161A (en) * 2021-04-09 2021-07-13 北京语言大学 Cross-media association analysis method
CN113157882A (en) * 2021-03-31 2021-07-23 山东大学 Knowledge graph path retrieval method and device with user semantics as center
CN113254678A (en) * 2021-07-14 2021-08-13 北京邮电大学 Training method of cross-media retrieval model, cross-media retrieval method and equipment thereof
CN114781400A (en) * 2022-06-17 2022-07-22 之江实验室 Cross-media knowledge semantic expression method and device
CN115374765A (en) * 2022-10-27 2022-11-22 浪潮通信信息系统有限公司 Computing power network 5G data analysis system and method based on natural language processing
CN117150031A (en) * 2023-07-24 2023-12-01 青海师范大学 Multi-mode data-oriented processing method and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130246328A1 (en) * 2010-06-22 2013-09-19 Peter Joseph Sweeney Methods and devices for customizing knowledge representation systems
CN103488713A (en) * 2013-09-10 2014-01-01 浙江大学 Cross-modal search method capable of directly measuring similarity of different modal data
CN103593792A (en) * 2013-11-13 2014-02-19 复旦大学 Individual recommendation method and system based on Chinese knowledge mapping
CN104035917A (en) * 2014-06-10 2014-09-10 复旦大学 Knowledge graph management method and system based on semantic space mapping
CN104166684A (en) * 2014-07-24 2014-11-26 北京大学 Cross-media retrieval method based on uniform sparse representation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130246328A1 (en) * 2010-06-22 2013-09-19 Peter Joseph Sweeney Methods and devices for customizing knowledge representation systems
CN103488713A (en) * 2013-09-10 2014-01-01 浙江大学 Cross-modal search method capable of directly measuring similarity of different modal data
CN103593792A (en) * 2013-11-13 2014-02-19 复旦大学 Individual recommendation method and system based on Chinese knowledge mapping
CN104035917A (en) * 2014-06-10 2014-09-10 复旦大学 Knowledge graph management method and system based on semantic space mapping
CN104166684A (en) * 2014-07-24 2014-11-26 北京大学 Cross-media retrieval method based on uniform sparse representation

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
蔡思: "基于概率的跨媒体检索方法研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 *
赵耀 等: "跨媒体时代的知识表达---感知、关联及一致性表示", 《中国计算机学会通讯》 *

Cited By (93)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107369098A (en) * 2016-05-11 2017-11-21 华为技术有限公司 The treating method and apparatus of data in social networks
CN107423820A (en) * 2016-05-24 2017-12-01 清华大学 The knowledge mapping of binding entity stratigraphic classification represents learning method
CN106156365B (en) * 2016-08-03 2019-06-18 北京儒博科技有限公司 A kind of generation method and device of knowledge mapping
CN106156365A (en) * 2016-08-03 2016-11-23 北京智能管家科技有限公司 A kind of generation method and device of knowledge mapping
CN109716286A (en) * 2016-08-16 2019-05-03 电子湾有限公司 Determine the item with confirmed feature
CN107783973A (en) * 2016-08-24 2018-03-09 慧科讯业有限公司 The methods, devices and systems being monitored based on domain knowledge spectrum data storehouse to the Internet media event
CN107783973B (en) * 2016-08-24 2022-02-25 慧科讯业有限公司 Method, device and system for monitoring internet media event based on industry knowledge map database
CN107967267A (en) * 2016-10-18 2018-04-27 中兴通讯股份有限公司 A kind of knowledge mapping construction method, apparatus and system
CN108009182B (en) * 2016-10-28 2020-03-10 京东方科技集团股份有限公司 Information extraction method and device
CN108009182A (en) * 2016-10-28 2018-05-08 京东方科技集团股份有限公司 A kind of information extracting method and device
US10657330B2 (en) 2016-10-28 2020-05-19 Boe Technology Group Co., Ltd. Information extraction method and apparatus
CN106776370A (en) * 2016-12-05 2017-05-31 哈尔滨工业大学(威海) Cloud storage method and device based on the assessment of object relevance
CN106844334B (en) * 2016-12-20 2022-07-15 网易(杭州)网络有限公司 Method and equipment for evaluating conversation robot intelligence
CN106844334A (en) * 2016-12-20 2017-06-13 网易(杭州)网络有限公司 Method and apparatus for evaluating and testing session robotic intelligence
CN106776564A (en) * 2016-12-21 2017-05-31 张永成 The method for recognizing semantics and system of a kind of knowledge based collection of illustrative plates
CN107247739B (en) * 2017-05-10 2019-11-01 浙江大学 A kind of financial bulletin text knowledge extracting method based on factor graph
CN107247739A (en) * 2017-05-10 2017-10-13 浙江大学 A kind of financial publication text knowledge extracting method based on factor graph
CN107273418A (en) * 2017-05-11 2017-10-20 浙江大学 A kind of across Noumenon property chain inference method based on cloud platform
CN107291828A (en) * 2017-05-27 2017-10-24 北京百度网讯科技有限公司 Spoken inquiry analytic method, device and storage medium based on artificial intelligence
US10698932B2 (en) 2017-05-27 2020-06-30 Beijing Baidu Netcom Science And Technology Co., Ltd. Method and apparatus for parsing query based on artificial intelligence, and storage medium
CN108959328A (en) * 2017-05-27 2018-12-07 株式会社理光 Processing method, device and the electronic equipment of knowledge mapping
CN108959328B (en) * 2017-05-27 2021-12-21 株式会社理光 Knowledge graph processing method and device and electronic equipment
CN107038261A (en) * 2017-05-28 2017-08-11 海南大学 A kind of processing framework resource based on data collection of illustrative plates, Information Atlas and knowledge mapping can Dynamic and Abstract Semantic Modeling Method
CN107330520A (en) * 2017-06-09 2017-11-07 上海电力学院 The object Affording acquisition inference method that a kind of knowledge based storehouse is represented
CN107256271B (en) * 2017-06-27 2020-04-03 鲁东大学 Cross-modal Hash retrieval method based on mapping dictionary learning
CN107256271A (en) * 2017-06-27 2017-10-17 鲁东大学 Cross-module state Hash search method based on mapping dictionary learning
CN107748754A (en) * 2017-09-15 2018-03-02 广州唯品会研究院有限公司 A kind of knowledge mapping improving method and device
US11769064B2 (en) 2017-11-21 2023-09-26 Google Llc Onboarding of entity data
CN110741389A (en) * 2017-11-21 2020-01-31 谷歌有限责任公司 Improved data communication of entities
CN108090167B (en) * 2017-12-14 2020-11-10 畅捷通信息技术股份有限公司 Data retrieval method, system, computing device and storage medium
CN108090167A (en) * 2017-12-14 2018-05-29 畅捷通信息技术股份有限公司 Method, system, computing device and the storage medium of data retrieval
CN110275898A (en) * 2018-03-16 2019-09-24 埃森哲环球解决方案有限公司 Use the integrated monitoring and communication system of the explanatory equipment management of knowledge based figure
CN110275898B (en) * 2018-03-16 2023-07-21 埃森哲环球解决方案有限公司 Integrated monitoring and communication system using knowledge graph-based interpretive device management
CN108491502A (en) * 2018-03-21 2018-09-04 腾讯科技(深圳)有限公司 A kind of method, terminal, server and the storage medium of news tracking
CN108549667B (en) * 2018-03-23 2022-04-08 绍兴诺雷智信息科技有限公司 Semantic retrieval method for structural engineering design knowledge
CN108549667A (en) * 2018-03-23 2018-09-18 绍兴诺雷智信息科技有限公司 A kind of semantic retrieving method of structuring engineering design knowledge
CN110689033A (en) * 2018-07-05 2020-01-14 第四范式(北京)技术有限公司 Data acquisition method, device and equipment for model training and storage medium
CN110970112B (en) * 2018-09-29 2024-03-12 九阳股份有限公司 Knowledge graph construction method and system for nutrition and health
CN110970112A (en) * 2018-09-29 2020-04-07 九阳股份有限公司 Method and system for constructing knowledge graph for nutrition and health
CN110968776A (en) * 2018-09-30 2020-04-07 北京国双科技有限公司 Policy knowledge recommendation method, device storage medium and processor
CN109522465A (en) * 2018-10-22 2019-03-26 国家电网公司 The semantic searching method and device of knowledge based map
CN109697233B (en) * 2018-12-03 2023-06-20 中电科大数据研究院有限公司 Knowledge graph system construction method
CN109697233A (en) * 2018-12-03 2019-04-30 中电科大数据研究院有限公司 A kind of knowledge mapping system building method
CN109710923A (en) * 2018-12-06 2019-05-03 浙江大学 Based on across the entity language matching process across media information
CN109710923B (en) * 2018-12-06 2020-09-01 浙江大学 Cross-language entity matching method based on cross-media information
CN109710776B (en) * 2018-12-29 2022-10-28 中国科学技术大学 Construction method of knowledge graph of photo album
CN109710776A (en) * 2018-12-29 2019-05-03 中国科学技术大学 The construction method of the knowledge mapping of photograph album
CN112115270A (en) * 2019-06-20 2020-12-22 国电南瑞科技股份有限公司 Method for constructing transformer knowledge graph ontology relation model
CN112115270B (en) * 2019-06-20 2022-09-16 北京南瑞数字技术有限公司 Method for constructing transformer knowledge graph ontology relation model
CN110647804A (en) * 2019-08-09 2020-01-03 中国传媒大学 Violent video identification method, computer system and storage medium
CN110489565A (en) * 2019-08-15 2019-11-22 广州拓尔思大数据有限公司 Based on the object root type design method and system in domain knowledge map ontology
CN110489565B (en) * 2019-08-15 2023-05-16 广州拓尔思大数据有限公司 Method and system for designing object root type in domain knowledge graph body
CN110457502A (en) * 2019-08-21 2019-11-15 京东方科技集团股份有限公司 Construct knowledge mapping method, man-machine interaction method, electronic equipment and storage medium
CN110532404A (en) * 2019-09-03 2019-12-03 北京百度网讯科技有限公司 One provenance multimedia determines method, apparatus, equipment and storage medium
CN110532404B (en) * 2019-09-03 2023-08-04 北京百度网讯科技有限公司 Source multimedia determining method, device, equipment and storage medium
CN110532341A (en) * 2019-09-03 2019-12-03 华东师范大学 Spatial information space-time big data constraint expression method
CN110597992A (en) * 2019-09-10 2019-12-20 腾讯科技(深圳)有限公司 Semantic reasoning method and device based on knowledge graph and electronic equipment
CN110597992B (en) * 2019-09-10 2023-08-29 腾讯科技(深圳)有限公司 Semantic reasoning method and device based on knowledge graph and electronic equipment
CN110750656A (en) * 2019-10-29 2020-02-04 上海德拓信息技术股份有限公司 Multimedia detection method based on knowledge graph
CN110928961A (en) * 2019-11-14 2020-03-27 出门问问(苏州)信息科技有限公司 Multi-mode entity linking method, equipment and computer readable storage medium
CN110928961B (en) * 2019-11-14 2023-04-28 出门问问(苏州)信息科技有限公司 Multi-mode entity linking method, equipment and computer readable storage medium
CN112836060A (en) * 2019-11-25 2021-05-25 中国科学技术信息研究所 Map construction method and device for scientific and technological innovation data
CN112836060B (en) * 2019-11-25 2023-11-24 中国科学技术信息研究所 Atlas construction method and apparatus for technological innovation data
CN111221984B (en) * 2020-01-15 2024-03-01 北京百度网讯科技有限公司 Multi-mode content processing method, device, equipment and storage medium
CN111221984A (en) * 2020-01-15 2020-06-02 北京百度网讯科技有限公司 Multimodal content processing method, device, equipment and storage medium
CN111680207A (en) * 2020-03-11 2020-09-18 华中科技大学鄂州工业技术研究院 Method and device for determining search intention of user
CN111680207B (en) * 2020-03-11 2023-08-04 华中科技大学鄂州工业技术研究院 Method and device for determining search intention of user
CN111460169A (en) * 2020-03-27 2020-07-28 科大讯飞股份有限公司 Semantic expression generation method, device and equipment
CN111460169B (en) * 2020-03-27 2023-06-02 科大讯飞股份有限公司 Semantic expression generation method, device and equipment
CN111680173A (en) * 2020-05-31 2020-09-18 西南电子技术研究所(中国电子科技集团公司第十研究所) CMR model for uniformly retrieving cross-media information
CN111680173B (en) * 2020-05-31 2024-02-23 西南电子技术研究所(中国电子科技集团公司第十研究所) CMR model for unified searching cross-media information
CN111666425A (en) * 2020-06-10 2020-09-15 深圳开思时代科技有限公司 Automobile accessory searching method based on semantic knowledge
CN111666425B (en) * 2020-06-10 2023-04-18 深圳开思时代科技有限公司 Automobile accessory searching method based on semantic knowledge
CN111708745A (en) * 2020-06-18 2020-09-25 全球能源互联网研究院有限公司 Cross-media data sharing representation method and user behavior analysis method and system
CN111708745B (en) * 2020-06-18 2023-04-21 全球能源互联网研究院有限公司 Cross-media data sharing representation method and user behavior analysis method and system
CN112084339B (en) * 2020-08-11 2023-11-24 同济大学 Traffic knowledge graph construction method based on cross-media data
CN112084339A (en) * 2020-08-11 2020-12-15 同济大学 Traffic knowledge graph construction method based on cross-media data
CN112749289A (en) * 2020-12-31 2021-05-04 重庆空间视创科技有限公司 Multi-mode-based knowledge graph retrieval system and method
CN112732969A (en) * 2021-01-14 2021-04-30 珠海格力电器股份有限公司 Image semantic analysis method and device, storage medium and electronic equipment
CN112948547B (en) * 2021-01-26 2024-04-09 中国石油大学(北京) Logging knowledge graph construction query method, device, equipment and storage medium
CN112948547A (en) * 2021-01-26 2021-06-11 中国石油大学(北京) Logging knowledge graph construction query method, device, equipment and storage medium
CN113010701A (en) * 2021-02-25 2021-06-22 北京四达时代软件技术股份有限公司 Video-centered fused media content recommendation method and device
CN113157882A (en) * 2021-03-31 2021-07-23 山东大学 Knowledge graph path retrieval method and device with user semantics as center
CN113157882B (en) * 2021-03-31 2022-05-31 山东大学 Knowledge graph path retrieval method and device with user semantics as center
CN113111161B (en) * 2021-04-09 2023-09-08 北京语言大学 Cross-media association analysis method
CN113111161A (en) * 2021-04-09 2021-07-13 北京语言大学 Cross-media association analysis method
CN113254678B (en) * 2021-07-14 2021-10-01 北京邮电大学 Training method of cross-media retrieval model, cross-media retrieval method and equipment thereof
CN113254678A (en) * 2021-07-14 2021-08-13 北京邮电大学 Training method of cross-media retrieval model, cross-media retrieval method and equipment thereof
CN114781400A (en) * 2022-06-17 2022-07-22 之江实验室 Cross-media knowledge semantic expression method and device
CN114781400B (en) * 2022-06-17 2022-09-09 之江实验室 Cross-media knowledge semantic expression method and device
CN115374765A (en) * 2022-10-27 2022-11-22 浪潮通信信息系统有限公司 Computing power network 5G data analysis system and method based on natural language processing
CN115374765B (en) * 2022-10-27 2023-06-02 浪潮通信信息系统有限公司 Computing power network 5G data analysis system and method based on natural language processing
CN117150031A (en) * 2023-07-24 2023-12-01 青海师范大学 Multi-mode data-oriented processing method and system

Also Published As

Publication number Publication date
CN105550190B (en) 2019-03-29

Similar Documents

Publication Publication Date Title
CN105550190A (en) Knowledge graph-oriented cross-media retrieval system
Kristiadi et al. Incorporating literals into knowledge graph embeddings
Ballatore et al. Geographic knowledge extraction and semantic similarity in OpenStreetMap
CN110362660A (en) A kind of Quality of electronic products automatic testing method of knowledge based map
Li et al. Embeddingvis: A visual analytics approach to comparative network embedding inspection
Li et al. Towards geospatial semantic search: Exploiting latent semantic relations in geospatial data
CN111680173A (en) CMR model for uniformly retrieving cross-media information
Hu et al. Metadata topic harmonization and semantic search for linked‐data‐driven geoportals: A case study using ArcGIS Online
Li et al. Semantic-based web service discovery and chaining for building an Arctic spatial data infrastructure
US20170228459A1 (en) Method and device for mobile searching based on artificial intelligence
Di Noia et al. Using ontology-based data summarization to develop semantics-aware recommender systems
Kalantari et al. Geospatial metadata 2.0–an approach for volunteered geographic information
JP5992497B2 (en) Realization method of situation awareness ontology for providing user interest information service based on context awareness
Sagi et al. Artificial intelligence for ocean science data integration: current state, gaps, and way forward
Ramshankar et al. A novel recommendation system enabled by adaptive fuzzy aided sentiment classification for E-commerce sector using black hole-based grey wolf optimization
Wu et al. Boosting climate analysis with semantically uplifted knowledge graphs
Nie et al. Cross-domain semantic transfer from large-scale social media
Petrovich et al. Exploring knowledge dynamics in the humanities. Two science mapping experiments
Berlanga et al. Quality indicators for social business intelligence
Cousseau et al. Linking place records using multi-view encoders
Mo et al. Research on application effective evaluation of artificial intelligence technology in marketing communication
US20220230053A1 (en) Graph neural network for signal processing
Vo An integrated network embedding with reinforcement learning for explainable recommendation
Berlanga et al. Context-Aware Business Intelligence
ElGindy et al. Enriching user profiles using geo-social place semantics in geo-folksonomies

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20201225

Address after: 210012 18 / F, A1 North building, 32 Fengzhan Road, Yuhuatai District, Nanjing City, Jiangsu Province

Patentee after: Jiangsu Huchuan Technology Co.,Ltd.

Address before: 461000 No. 88 Bayi Road, Henan, Xuchang

Patentee before: XUCHANG University

PE01 Entry into force of the registration of the contract for pledge of patent right
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: A Cross Media Retrieval System for Knowledge Graph

Effective date of registration: 20231130

Granted publication date: 20190329

Pledgee: Nanjing Bank Co.,Ltd. Nanjing Financial City Branch

Pledgor: Jiangsu Huchuan Technology Co.,Ltd.

Registration number: Y2023980067998