CN108170684B - 文本相似度计算方法及系统、数据查询系统和计算机产品 - Google Patents

文本相似度计算方法及系统、数据查询系统和计算机产品 Download PDF

Info

Publication number
CN108170684B
CN108170684B CN201810060942.7A CN201810060942A CN108170684B CN 108170684 B CN108170684 B CN 108170684B CN 201810060942 A CN201810060942 A CN 201810060942A CN 108170684 B CN108170684 B CN 108170684B
Authority
CN
China
Prior art keywords
text
similarity
similar
similarity calculation
calculating
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810060942.7A
Other languages
English (en)
Chinese (zh)
Other versions
CN108170684A (zh
Inventor
张振中
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BOE Technology Group Co Ltd
Original Assignee
BOE Technology Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BOE Technology Group Co Ltd filed Critical BOE Technology Group Co Ltd
Priority to CN201810060942.7A priority Critical patent/CN108170684B/zh
Publication of CN108170684A publication Critical patent/CN108170684A/zh
Priority to US16/330,545 priority patent/US11281861B2/en
Priority to JP2019569437A priority patent/JP7513396B2/ja
Priority to EP18901142.2A priority patent/EP3743831A4/en
Priority to PCT/CN2018/091822 priority patent/WO2019140863A1/en
Priority to KR1020207014877A priority patent/KR102411921B1/ko
Application granted granted Critical
Publication of CN108170684B publication Critical patent/CN108170684B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/3332—Query translation
    • G06F16/3334—Selection or weighting of terms from queries, including natural language queries
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3344—Query execution using natural language analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3347—Query execution using vector based model
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/338—Presentation of query results
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/194—Calculation of difference between files
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
CN201810060942.7A 2018-01-22 2018-01-22 文本相似度计算方法及系统、数据查询系统和计算机产品 Active CN108170684B (zh)

Priority Applications (6)

Application Number Priority Date Filing Date Title
CN201810060942.7A CN108170684B (zh) 2018-01-22 2018-01-22 文本相似度计算方法及系统、数据查询系统和计算机产品
US16/330,545 US11281861B2 (en) 2018-01-22 2018-06-19 Method of calculating relevancy, apparatus for calculating relevancy, data query apparatus, and non-transitory computer-readable storage medium
JP2019569437A JP7513396B2 (ja) 2018-01-22 2018-06-19 関連性を算出する方法、関連性を算出する装置、データクエリ装置及び非一時的なコンピュータ可読記録媒体
EP18901142.2A EP3743831A4 (en) 2018-01-22 2018-06-19 PROCESS FOR CALCULATION OF RELEVANCE, DEVICE FOR CALCULATION OF RELEVANCE, DEVICE FOR DATA REQUEST AND NON-TRANSITORY COMPUTER-READABLE STORAGE MEDIUM
PCT/CN2018/091822 WO2019140863A1 (en) 2018-01-22 2018-06-19 Method of calculating relevancy, apparatus for calculating relevancy, data query apparatus, and non-transitory computer-readable storage medium
KR1020207014877A KR102411921B1 (ko) 2018-01-22 2018-06-19 관련성을 계산하는 방법, 관련성을 계산하기 위한 장치, 데이터 쿼리 장치, 및 비-일시적 컴퓨터-판독가능 저장 매체

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810060942.7A CN108170684B (zh) 2018-01-22 2018-01-22 文本相似度计算方法及系统、数据查询系统和计算机产品

Publications (2)

Publication Number Publication Date
CN108170684A CN108170684A (zh) 2018-06-15
CN108170684B true CN108170684B (zh) 2020-06-05

Family

ID=62515240

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810060942.7A Active CN108170684B (zh) 2018-01-22 2018-01-22 文本相似度计算方法及系统、数据查询系统和计算机产品

Country Status (6)

Country Link
US (1) US11281861B2 (enExample)
EP (1) EP3743831A4 (enExample)
JP (1) JP7513396B2 (enExample)
KR (1) KR102411921B1 (enExample)
CN (1) CN108170684B (enExample)
WO (1) WO2019140863A1 (enExample)

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108170684B (zh) * 2018-01-22 2020-06-05 京东方科技集团股份有限公司 文本相似度计算方法及系统、数据查询系统和计算机产品
CN110688119A (zh) * 2018-07-04 2020-01-14 阿里巴巴集团控股有限公司 业务功能和单元的生成、实现方法和服务器、设备、介质
CN109145085B (zh) * 2018-07-18 2020-11-27 北京市农林科学院 语义相似度的计算方法及系统
CN110852056B (zh) * 2018-07-25 2024-09-24 中兴通讯股份有限公司 一种获取文本相似度的方法、装置、设备及可读存储介质
CN110852057A (zh) * 2018-07-27 2020-02-28 北京京东尚科信息技术有限公司 一种计算文本相似度的方法和装置
CN111382246B (zh) * 2018-12-29 2023-03-14 深圳市优必选科技有限公司 文本的匹配方法、匹配装置、终端及计算机可读存储介质
CN110147429B (zh) * 2019-04-15 2023-08-15 平安科技（深圳）有限公司 文本比较方法、装置、计算机设备和存储介质
CN110265104B (zh) * 2019-05-08 2022-04-29 长沙市中心医院 诊断报告符合度检测方法、装置、计算机设备和存储介质
CN110176315B (zh) * 2019-06-05 2022-06-28 京东方科技集团股份有限公司 医疗问答方法及系统、电子设备、计算机可读介质
CN110750977B (zh) * 2019-10-23 2023-06-02 支付宝(杭州)信息技术有限公司 一种文本相似度计算方法及系统
CN110969006B (zh) * 2019-12-02 2023-03-21 支付宝(杭州)信息技术有限公司 一种文本排序模型的训练方法和系统
CN111159343A (zh) * 2019-12-26 2020-05-15 上海科技发展有限公司 基于文本嵌入的文本相似性搜索方法、装置、设备和介质
CN111160028B (zh) * 2019-12-31 2023-05-16 东软集团股份有限公司 判断两个文本语义相似度的方法、装置、存储介质及设备
WO2021191938A1 (ja) * 2020-03-23 2021-09-30 日本電信電話株式会社 要約生成装置、要約生成方法及びプログラム
DE102020109953A1 (de) * 2020-04-09 2021-10-14 Contexon GmbH Verfahren und Computersystem zur Bestimmung der Relevanz eines Textes
CN112819893A (zh) * 2021-02-08 2021-05-18 北京航空航天大学 三维语义地图的构建方法和装置
CN114973219B (zh) * 2021-02-25 2025-12-12 阿里巴巴集团控股有限公司 一种文字内容提取方法和装置
CN113761868B (zh) * 2021-04-20 2023-06-09 腾讯科技（深圳）有限公司 文本处理方法、装置、电子设备及可读存储介质
US12259912B2 (en) 2021-06-23 2025-03-25 Servicenow Canada Inc. Method and system for interactive searching based on semantic similarity of semantic representations of text objects
CN114168837A (zh) * 2021-11-18 2022-03-11 深圳市梦网科技发展有限公司 一种Chatbot的搜索方法、设备及存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105138647A (zh) * 2015-08-26 2015-12-09 陕西师范大学 一种基于Simhash算法的旅游网络社区划分方法
CN107436864A (zh) * 2017-08-04 2017-12-05 逸途（北京）科技有限公司 一种基于Word2Vec的中文问答语义相似度计算方法

Family Cites Families (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6658626B1 (en) * 1998-07-31 2003-12-02 The Regents Of The University Of California User interface for displaying document comparison information
US6990628B1 (en) * 1999-06-14 2006-01-24 Yahoo! Inc. Method and apparatus for measuring similarity among electronic documents
US6560620B1 (en) * 1999-08-03 2003-05-06 Aplix Research, Inc. Hierarchical document comparison system and method
US6766316B2 (en) * 2001-01-18 2004-07-20 Science Applications International Corporation Method and system of ranking and clustering for document indexing and retrieval
US20040093564A1 (en) * 2002-11-07 2004-05-13 International Business Machines Corporation Method and apparatus for visualizing changes in data
JP4391157B2 (ja) 2003-08-15 2009-12-24 株式会社リコー 文書処理装置、文書処理方法、文書処理プログラム、および記録媒体
EP1866738A4 (en) * 2005-03-18 2010-09-15 Search Engine Technologies Llc SEARCH ENGINE THAT APPLIES USER REPORTS TO IMPROVE SEARCH RESULTS
US9189482B2 (en) * 2012-10-10 2015-11-17 Abbyy Infopoisk Llc Similar document search
US9892111B2 (en) * 2006-10-10 2018-02-13 Abbyy Production Llc Method and device to estimate similarity between documents having multiple segments
US20080114750A1 (en) * 2006-11-14 2008-05-15 Microsoft Corporation Retrieval and ranking of items utilizing similarity
US8082233B2 (en) * 2007-03-29 2011-12-20 Microsoft Corporation Comparing data sets through identification of matching blocks
US8122032B2 (en) * 2007-07-20 2012-02-21 Google Inc. Identifying and linking similar passages in a digital text corpus
US7945525B2 (en) * 2007-11-09 2011-05-17 International Business Machines Corporation Methods for obtaining improved text similarity measures which replace similar characters with a string pattern representation by using a semantic data tree
US20090240498A1 (en) * 2008-03-19 2009-09-24 Microsoft Corporation Similiarity measures for short segments of text
JP5458880B2 (ja) * 2009-03-02 2014-04-02 富士通株式会社 文書検査装置、コンピュータ読み取り可能な記録媒体および文書検査方法
WO2011029474A1 (en) * 2009-09-09 2011-03-17 Universität Bremen Document comparison
JP2013149061A (ja) * 2012-01-19 2013-08-01 Nec Corp 文書類似性評価システム、文書類似性評価方法およびコンピュータ・プログラム
US9720979B2 (en) * 2012-06-22 2017-08-01 Krishna Kishore Dhara Method and system of identifying relevant content snippets that include additional information
US20140249799A1 (en) * 2013-03-04 2014-09-04 Microsoft Corporation Relational similarity measurement
US20150212993A1 (en) * 2014-01-30 2015-07-30 International Business Machines Corporation Impact Coverage
US9477654B2 (en) * 2014-04-01 2016-10-25 Microsoft Corporation Convolutional latent semantic models and their applications
US20160196342A1 (en) * 2015-01-06 2016-07-07 Inha-Industry Partnership Plagiarism Document Detection System Based on Synonym Dictionary and Automatic Reference Citation Mark Attaching System
JP2016197289A (ja) * 2015-04-02 2016-11-24 日本電信電話株式会社 パラメタ学習装置、類似度算出装置、方法、及びプログラム
JP2017010249A (ja) * 2015-06-22 2017-01-12 日本電信電話株式会社 パラメタ学習装置、文類似度算出装置、方法、及びプログラム
JP6555810B2 (ja) * 2015-07-15 2019-08-07 日本放送協会 類似度算出装置、類似検索装置、および類似度算出プログラム
US9836671B2 (en) 2015-08-28 2017-12-05 Microsoft Technology Licensing, Llc Discovery of semantic similarities between images and text
CN106485507B (zh) * 2015-09-01 2019-10-18 阿里巴巴集团控股有限公司 一种软件推广作弊的检测方法、装置及系统
CN105893410A (zh) 2015-11-18 2016-08-24 乐视网信息技术（北京）股份有限公司 一种关键词提取方法和装置
CN107220220A (zh) * 2016-03-22 2017-09-29 索尼公司 用于文本处理的电子设备和方法
US10102191B2 (en) * 2016-06-16 2018-10-16 Adobe Systems Incorporated Propagation of changes in master content to variant content
US10769522B2 (en) * 2017-02-17 2020-09-08 Wipro Limited Method and system for determining classification of text
CN107133202A (zh) * 2017-06-01 2017-09-05 北京百度网讯科技有限公司 基于人工智能的文本校验方法和装置
CN107491547B (zh) * 2017-08-28 2020-11-10 北京百度网讯科技有限公司 基于人工智能的搜索方法和装置
CN108170684B (zh) * 2018-01-22 2020-06-05 京东方科技集团股份有限公司 文本相似度计算方法及系统、数据查询系统和计算机产品

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105138647A (zh) * 2015-08-26 2015-12-09 陕西师范大学 一种基于Simhash算法的旅游网络社区划分方法
CN107436864A (zh) * 2017-08-04 2017-12-05 逸途（北京）科技有限公司 一种基于Word2Vec的中文问答语义相似度计算方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
文本相似度计算方法研究综述;陈二静等;《数据分析与知识发现》;20170630(第06期);第1-11页 *

Also Published As

Publication number Publication date
JP7513396B2 (ja) 2024-07-09
EP3743831A4 (en) 2021-09-29
WO2019140863A1 (en) 2019-07-25
US20210334464A1 (en) 2021-10-28
JP2021510858A (ja) 2021-04-30
US11281861B2 (en) 2022-03-22
EP3743831A1 (en) 2020-12-02
CN108170684A (zh) 2018-06-15
KR20200078576A (ko) 2020-07-01
KR102411921B1 (ko) 2022-06-23

Similar Documents

Publication Publication Date Title
CN108170684A (zh) 文本相似度计算方法及系统、数据查询系统和计算机产品
Kanwal et al. A review of text-based recommendation systems
US12282504B1 (en) Systems and methods for graph-based dynamic information retrieval and synthesis
CN112231569B (zh) 新闻推荐方法、装置、计算机设备及存储介质
CN114138985B (zh) 文本数据处理的方法、装置、计算机设备以及存储介质
CN107491547B (zh) 基于人工智能的搜索方法和装置
US20200042597A1 (en) Generating question-answer pairs for automated chatting
CN106156023B (zh) 语义匹配的方法、装置和系统
CN112800205B (zh) 基于语义变化流形分析获取问答相关段落的方法、装置
CN111767375A (zh) 语义召回方法、装置、计算机设备及存储介质
CN112966053B (zh) 一种基于知识图谱的海洋领域专家库构建方法及装置
CN111782769A (zh) 基于关系预测的知识图谱智能问答方法
CN112463914A (zh) 一种用于互联网服务的实体链接方法、装置及存储介质
CN117556024B (zh) 知识问答方法以及相关设备
CN114036322A (zh) 用于搜索系统的训练方法、电子设备和存储介质
WO2021082086A1 (zh) 机器阅读方法、系统、装置及存储介质
Dewi et al. XAI for Image Captioning using SHAP.
CN115795007A (zh) 智能问答方法、智能问答装置、电子设备及存储介质
Fernández-Reyes et al. CV Retrieval System based on job description matching using hybrid word embeddings
JP7849447B2 (ja) インタラクション情報の特定方法、装置、電子機器、記憶媒体及びコンピュータプログラム
Chai Design and implementation of English intelligent communication platform based on similarity algorithm
CN113704420A (zh) 文本中的角色识别方法、装置、电子设备及存储介质
Zhang et al. Exploring coevolution of emotional contagion and behavior for microblog sentiment analysis: a deep learning architecture
Wang et al. Crisis event summary generative model based on hierarchical multimodal fusion
CN114741587A (zh) 一种物品推荐方法、装置、介质及设备

Legal Events

Date Code Title Description
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant