WO2024016695A1 - Multiview learning-based teaching knowledge graph construction and retrieval method and system - Google Patents

Multiview learning-based teaching knowledge graph construction and retrieval method and system Download PDF

Info

Publication number
WO2024016695A1
WO2024016695A1 PCT/CN2023/082103 CN2023082103W WO2024016695A1 WO 2024016695 A1 WO2024016695 A1 WO 2024016695A1 CN 2023082103 W CN2023082103 W CN 2023082103W WO 2024016695 A1 WO2024016695 A1 WO 2024016695A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
teaching
knowledge
retrieval
feature extraction
Prior art date
Application number
PCT/CN2023/082103
Other languages
French (fr)
Chinese (zh)
Inventor
孙善宝
Original Assignee
山东浪潮科学研究院有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 山东浪潮科学研究院有限公司 filed Critical 山东浪潮科学研究院有限公司
Publication of WO2024016695A1 publication Critical patent/WO2024016695A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/335—Filtering based on additional data, e.g. user or group profiles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education
    • G06Q50/205—Education administration or guidance

Definitions

  • the present invention relates to the technical field of teaching resource recommendation, specifically to a teaching knowledge graph construction and retrieval method and system based on multi-view learning.
  • Knowledge Graph is a large-scale semantic network based on data. As a knowledge representation form, it describes domain entities, concepts and various semantic relationships between them. Google proposed the "Google Knowledge Graph” in 2012, and the knowledge graph began to attract widespread attention from academia and industry. After continuous development in recent years, it has been used in many fields such as search optimization, e-commerce, intelligent recommendations, and social media. It has been applied in practice and has gradually become a necessary way to manage massive information.
  • the technical task of the present invention is to address the above shortcomings and provide a teaching knowledge graph construction and retrieval method and system based on multi-view learning to solve how to use deep learning technology, combined with multi-view learning and user portrait technology, to effectively utilize massive teaching resources and automatically Technical issues to construct a more accurate and reasonable teaching knowledge graph and realize personalized knowledge point retrieval and recommendation.
  • a teaching knowledge graph construction and retrieval method based on multi-view learning of the present invention method including the following steps:
  • Construct a multi-view feature extractor which is used to perform feature extraction and feature fusion on teaching resource data in multiple view modes to obtain the knowledge point ontology structure and associated knowledge points;
  • a teaching resource portrait model is constructed based on a convolutional neural network.
  • the teaching resource portrait model takes teaching resource data and teaching resource Internet-related extended data as inputs to profile teaching resources and output teaching resource attributes.
  • the teaching resource attributes include teaching resources. Basic attributes and extended attributes of teaching resources;
  • the knowledge graph construction model uses the ontology structure of knowledge points, associated knowledge points and teaching resource attributes as input to generate a knowledge graph of teaching knowledge points;
  • Construct a user portrait model which takes user information as input, profiles the user, and outputs user attributes
  • the retrieval recommendation model is used to build an index based on the knowledge graph of teaching knowledge points and provide retrieval recommendation services. It is used to output multiple sets of knowledge points and recommended resources for users through the input retrieval content and combined with user attributes. choose;
  • teaching resource portrait model knowledge graph construction model, user portrait model and retrieval recommendation model, a graph construction and retrieval model is constructed, and the multi-view feature extractor, teaching resource portrait model, knowledge graph are constructed in sequence.
  • a knowledge graph of teaching knowledge points is constructed through the trained graph construction and retrieval model, an index is constructed based on the knowledge graph of teaching knowledge points and retrieval recommendation services are provided, and multiple sets of knowledge points and recommended resources are output to users for selection.
  • the data sources of the teaching resources include teaching materials and books, lesson plans, teaching videos, teaching voices and speech scripts.
  • the basic attributes of the teaching resources include knowledge points, content structure, processes, principles, concepts, and tools;
  • the extended attributes of the teaching resources include Internet evaluation information and presentation forms;
  • the view form of the retrieval content includes text, voice, video and image;
  • the knowledge map of teaching knowledge points includes semantic network, basic data of teaching resources and teaching resources. Extended attributes, the semantic network is formed based on the ontology structure of knowledge points and the association of knowledge points;
  • the user information includes basic information and learning status
  • the user attributes include knowledge point preferences, mastery level and comprehensive ability.
  • the multi-view feature extractor includes:
  • the video feature extraction model is a network model built based on a three-dimensional CNN convolutional neural network and is used to extract semantic features of knowledge points from video teaching resource data;
  • Audio feature extraction model is a network model built based on a CNN convolutional neural network and is used to extract semantic features of knowledge points from audio-based teaching resource data;
  • the image feature extraction model is a network model built based on a convolutional neural network and is used to extract semantic features of knowledge points from image-based teaching resource data;
  • Text feature extraction model is a language model based on BERT, which is used to extract semantic features of knowledge points from text-based teaching resource data;
  • the feature fusion model is used to fuse the semantic features of knowledge points output by the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model to obtain the knowledge point ontology structure and associated knowledge point;
  • the knowledge graph construction model includes:
  • Feature encoder which is used to encode knowledge points so that the vector calculation distance between similar knowledge points is small and used to provide resource index queries;
  • model training is performed on the multi-view feature extractor, teaching resource portrait model, knowledge graph construction model, user portrait model and retrieval recommendation model in sequence, including the following steps:
  • model parameters of the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor are fixed, and the feature fusion model is type for model training;
  • the video feature extraction model, audio feature extraction model, image feature extraction model, text feature extraction model and feature fusion model in the multi-view feature extractor are combined, and the entire multi-view feature extractor is processed through the teaching resource data. Carry out model training and fine-tune parameters of each model in the multi-view feature extractor;
  • the retrieval recommendation model is lightweight and tailored based on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor, and the search condition extraction model in the retrieval recommendation model is trained. ;
  • model training is performed on the retrieval recommendation model.
  • building a knowledge graph of teaching knowledge points through the trained graph construction and retrieval model includes the following steps:
  • Collect the Internet-related extended data of teaching resources and based on the Internet-related extended data of teaching resources and the teaching resource data, profile the teaching resources through the trained teaching resource portrait model to obtain the teaching resource attributes;
  • the knowledge points are encoded through the trained knowledge graph construction model, so that the vector calculation distance between similar knowledge point resources is small, which can be used for resource index query and generate teaching Knowledge point knowledge map;
  • the view mode of the search content includes text, voice, image and video;
  • knowledge points are extracted from the retrieval content input by users, combined with user attributes to form knowledge point feature vectors, and knowledge point queries and feature vector calculations are performed to output multiple sets of knowledge points and recommended resources for users make a choice;
  • the collected teaching resource data, Internet-related extended data, user information and search content, as well as the output sets of knowledge points and recommended resources, are fed back to the map construction and retrieval model, and model training is performed on the map construction and retrieval model. , to continuously optimize the map construction and retrieval model.
  • a teaching knowledge graph construction and retrieval system based on multi-view learning of the present invention is used to construct and retrieve a teaching knowledge graph based on multi-view learning as described in any one of the first aspects.
  • Users provide knowledge points and teaching resource recommendation services.
  • the system includes:
  • a model building module which is used to build a graph construction and retrieval model.
  • the graph construction and retrieval model includes a multi-view feature extractor, a teaching resource portrait model, a knowledge graph construction model, a user portrait model and a retrieval recommendation model,
  • the multi-feature view extractor is used to perform feature extraction and feature fusion on teaching resource data in multiple view modes to obtain the knowledge point ontology structure and associated knowledge points;
  • the teaching resource portrait model is a network model based on a convolutional neural network. , using teaching resource data and teaching resource Internet-related extended data as input, the teaching resources are profiled, and the teaching resource attributes are output.
  • the teaching resource attributes include basic teaching resource attributes and teaching resource extended attributes;
  • the knowledge graph construction model is based on The ontology structure of knowledge points, associated knowledge points and teaching resource attributes are used as input to generate a knowledge graph of teaching knowledge points;
  • the user portrait model uses user information as input to profile the user and output user attributes;
  • the retrieval recommendation model is used based on The knowledge graph of teaching knowledge points constructs an index and provides retrieval recommendation services, which is used to output multiple sets of knowledge points and recommended resources for users to choose based on the input retrieval content and combined with user attributes;
  • a model training module which is used to extract the multi-view features in sequence model, teaching resource portrait model, knowledge graph construction model, user portrait model and retrieval recommendation model for model training, and the trained graph construction and retrieval model is obtained;
  • the retrieval recommendation module is used to construct a knowledge graph of teaching knowledge points through the trained graph construction and retrieval model, build an index based on the knowledge graph of teaching knowledge points and provide retrieval recommendation services, and output multiple sets of knowledge points for users. and recommend resources for users to choose from.
  • the data sources of the teaching resources include teaching materials and books, lesson plans, teaching videos, teaching voices and speech scripts.
  • the basic attributes of the teaching resources include knowledge points, content structure, processes, principles, concepts, and tools;
  • the extended attributes of the teaching resources include Internet evaluation information and presentation forms;
  • the view form of the retrieval content includes text, voice, video and image;
  • the knowledge map of teaching knowledge points includes a semantic network, basic data of teaching resources and extended attributes of teaching resources.
  • the semantic network is formed based on the ontology structure of knowledge points and the association of knowledge points;
  • the user information includes basic information and learning status
  • the user attributes include knowledge point preferences, mastery level and comprehensive ability.
  • the multi-view feature extractor includes:
  • the video feature extraction model is a network model built based on a three-dimensional CNN convolutional neural network and is used to extract semantic features of knowledge points from video teaching resource data;
  • Audio feature extraction model is a network model built based on a CNN convolutional neural network and is used to extract semantic features of knowledge points from audio-based teaching resource data;
  • the image feature extraction model is a network model built based on a convolutional neural network and is used to extract semantic features of knowledge points from image-based teaching resource data;
  • Text feature extraction model is a language model based on BERT, which is used to extract semantic features of knowledge points from text-based teaching resource data;
  • the feature fusion model is used to fuse the semantic features of knowledge points output by the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model to obtain the knowledge point ontology structure and associated knowledge point;
  • the knowledge graph construction model includes:
  • Feature encoder which is used to encode knowledge points so that the vector calculation distance between similar knowledge points is small and used to provide resource index queries;
  • the model sequence module is used to perform model training on the multi-view feature extractor, teaching resource portrait model, knowledge graph construction model, user portrait model and retrieval recommendation model in sequence through the following steps:
  • the video feature extraction model, audio feature extraction model, image feature extraction model, text feature extraction model and feature fusion model in the multi-view feature extractor are combined, and the entire multi-view feature extractor is processed through the teaching resource data. Carry out model training and fine-tune parameters of each model in the multi-view feature extractor;
  • the retrieval recommendation model is lightweight and tailored based on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor, and the search condition extraction model in the retrieval recommendation model is trained. ;
  • model training is performed on the retrieval recommendation model.
  • the retrieval and recommendation module is used to construct a knowledge graph of teaching knowledge points through the following steps, and output multiple sets of knowledge points and recommended resources for the user to select:
  • Collect the Internet-related extended data of teaching resources and based on the Internet-related extended data of teaching resources and the teaching resource data, profile the teaching resources through the trained teaching resource portrait model to obtain the teaching resource attributes;
  • the knowledge points are encoded through the trained knowledge graph construction model, so that the vector calculation distance between similar knowledge point resources is small, which can be used for resource index query and generate teaching Knowledge point knowledge map;
  • the view mode of the search content includes text, voice, image and video;
  • knowledge points are extracted from the retrieval content input by users, combined with user attributes to form knowledge point feature vectors, and knowledge point queries and feature vector calculations are performed to output multiple sets of knowledge points and recommended resources for users make a choice;
  • the collected teaching resource data, Internet-related extended data, user information and search content, as well as the output sets of knowledge points and recommended resources, are fed back to the map construction and retrieval model, and model training is performed on the map construction and retrieval model. , to continuously optimize the map construction and retrieval model.
  • Figure 1 is a block diagram of the working principle of the teaching knowledge graph construction and retrieval method based on multi-view learning in Embodiment 1.
  • Embodiments of the present invention provide teaching knowledge graph construction and retrieval methods and systems based on multi-view learning, which are used to solve how to use deep learning technology, combined with multi-view learning and user portrait technology, to effectively utilize massive teaching resources and automatically build more accurate and reasonable Technical issues of teaching knowledge graph and realizing personalized knowledge point retrieval and recommendation.
  • the present invention is a teaching knowledge graph construction and retrieval method based on multi-view learning, which includes the following steps:
  • the teaching resource profiling model CR-Profiler takes the teaching resource data CR and the teaching resource Internet-related extended data CR-WWW as inputs to profile the teaching resources and output Teaching resource attributes, which include teaching resource basic attributes CR-Basic and teaching resource extended attributes CR-Ext;
  • the user profile model Stu-Profiler takes user information Stu-Info as input, profiles the user, and outputs user attributes Stu-Prop;
  • the retrieval recommendation model CR-Recommend is used to build an index based on the teaching knowledge point knowledge graph EP-KG and provide retrieval recommendation services. It is used to use the input retrieval content and combine it with user attributes Stu -Prop, outputs multiple sets of knowledge points and recommended resources for users to choose;
  • the method of this embodiment is based on massive teaching resource data CR, effectively utilizes deep learning feature extraction technology, fully explores the connections between multiple views such as teaching materials, teaching plans, teaching videos, teaching voices, and speech drafts, and combines it with Internet resource evaluation information , forming more reasonable attributes of knowledge points through teaching resource portraits, and building a more accurate and reasonable teaching knowledge map.
  • a retrieval recommendation model CR-Recommend based on the teaching knowledge map is formed to provide knowledge point learning and teaching resources for learners' personalized learning and in line with their own characteristics, to achieve Teach students in accordance with their aptitude.
  • the data sources of teaching resources collected in this embodiment include teaching materials and books, lesson plans, teaching videos, teaching voices and speech drafts, which are mainly divided into four types: knowledge point video V, knowledge point audio A, knowledge point film P and knowledge point teaching material L view mode.
  • the multi-view feature extractor CR-Mutiview-Fet is responsible for extracting knowledge point features, outputting the ontology structure of knowledge points and associated knowledge points EP-OR.
  • Four different feature extraction models are used for feature extraction and feature fusion for the above four views. , including video feature extraction module FV, audio feature extraction module FA, image feature extraction module FP, text feature extraction module FL and feature fusion module FF.
  • the core of the video feature extraction module FV is a three-dimensional CNN convolutional neural network, which is responsible for extracting the semantic features of video knowledge points;
  • the core of the audio feature extraction module FA is a CNN convolutional neural network, which is responsible for extracting the semantic features of knowledge points in audio;
  • the core of the image feature extraction module FP is a convolutional neural network, which is responsible for extracting the semantic features of knowledge points in images;
  • the core of the text feature extraction module FL is It is a pre-trained language model based on BERT, responsible for extracting the semantic features of text knowledge points;
  • the feature fusion module FF will be based on the video feature extraction module FV, audio feature extraction module FA, image feature extraction module FP and text feature extraction module FL.
  • Feature vectors are fused to obtain the knowledge point ontology structure and associated knowledge point EP-OR.
  • the core of the teaching resource profiling model CR-Profiler is the CNN convolutional neural network model, which is responsible for profiling the teaching resources based on the extended data CR-WWW and teaching resource data CR related to the Internet teaching resources, and obtaining the basic attributes of the teaching resources CR-Basic (which Including knowledge points, content structure, processes, principles, concepts, tools, etc.) and teaching resource extension attributes CR-Ext (which includes Internet evaluation information, presentation forms, etc.).
  • the core of the knowledge graph construction model KG-Gen is a neural network model, including the feature encoder Enc and the generative network model GN.
  • the feature encoder is used to encode knowledge points so that the vector calculation distance between similar knowledge points is small and used to provide Resource index query;
  • the generative network model is used to generate the knowledge graph EP-KG of teaching knowledge points. That is, the knowledge graph construction model KG-Gen is based on the knowledge point ontology structure formed by knowledge point extraction and associated knowledge points EP-OR, and combined with the educational resource attributes obtained by the teaching resource portrait CR-Profiler to generate the teaching knowledge point knowledge graph EP-KG .
  • the user portrait model Stu-Profiler is based on the user's user information Stu-Info (that is, the learner's personal information and learning situation) to profile the learner to form the user attribute Stu-Prop (that is, the knowledge point preferences, mastery and Comprehensive ability and other labels.)
  • the retrieval recommendation model CR-Recommend is based on the constructed knowledge graph EP-KG of teaching resource knowledge points to build an index and provide retrieval recommendation functions. Through the input retrieval content (including text, voice, video, image, etc.), combined with user portraits The user attribute Stu-Prop generated by the model Stu-Profiler outputs multiple sets of knowledge points and recommended teaching resources, which users as learners can choose from.
  • Step S200 combines and connects the multi-view feature extractor CR-Mutiview-Fet, the teaching resource profile model CR-Profiler, the knowledge graph construction model KG-Gen, the user profile model Stu-Profiler and the retrieval recommendation model CR-Recommend to form graph construction and retrieval. model, and perform model training on the multi-view feature extractor CR-Mutiview-Fet, teaching resource profile model CR-Profiler, knowledge graph construction model KG-Gen, user profile model Stu-Profiler and retrieval recommendation model CR-Recommend in sequence , obtain the trained graph construction and retrieval model.
  • the retrieval recommendation model CR-Recommend is lightweight and tailored.
  • the search condition extraction model in the retrieval recommendation model CR-Recommend is trained;
  • Step S300 constructs a knowledge graph EP-KG of teaching knowledge points through the trained graph construction and retrieval model, including the following steps:
  • the model KG-Gen is constructed through the trained knowledge graph to encode the knowledge points, so that the vector calculation distance between similar knowledge point resources is small, using Query the resource index and generate a knowledge graph EP-KG of teaching knowledge points.
  • the viewing mode of the retrieval content includes text, voice, image and video;
  • S380 based on indexing and retrieval recommendation services, extract knowledge points from the retrieval content input by the user, combine it with user attributes Stu-Prop to form a knowledge point feature vector, perform knowledge point query and feature vector calculation, and output multiple sets of knowledge points and recommendations. resources for users to choose;
  • the method of this embodiment is based on massive teaching resource data CR, effectively utilizes deep learning feature extraction technology, fully considers the characteristics of Internet online learning, and explores the connections between multiple views such as textbooks, lesson plans, teaching videos, teaching voices, and speech scripts. , and combined with Internet resource evaluation information, more reasonable knowledge point attributes are formed through teaching resource portraits, and a more accurate and reasonable teaching knowledge map is constructed. Compared with traditional knowledge graph construction and recommendation methods, multi-view learning and deep learning are used to construct query methods.
  • the neural network model is designed based on multi-view data formed by different presentation methods of resources, which can better consider diversity and explore Potential connections within the knowledge points enable a more accurate and reasonable extraction of knowledge points; teaching resource portraits and learner user portraits are added to the method, which makes the knowledge point extraction and retrieval process more targeted and focused, and the recommended resources are more effective Conform to learners’ learning habits and satisfy learners personalized needs, and the recommendation results contain multiple sets of data, which increases the accuracy and fault tolerance of recommendations. In addition, learners learn based on recommended learning resources and provide timely feedback to continuously optimize the recommendation model.
  • the present invention is a teaching knowledge graph construction and retrieval system based on multi-view learning, which includes a model construction module, a model training module and a retrieval recommendation module.
  • the system provides users with knowledge points and teaching resource recommendation services based on the method disclosed in the embodiment.
  • the model building module is used to build a graph construction and retrieval model, which includes a multi-view feature extractor CR-Mutiview-Fet, a teaching resource profile model CR-Profiler, a knowledge graph construction model KG-Gen, and a user profile model.
  • a graph construction and retrieval model which includes a multi-view feature extractor CR-Mutiview-Fet, a teaching resource profile model CR-Profiler, a knowledge graph construction model KG-Gen, and a user profile model.
  • Stu-Profiler and retrieval recommendation model CR-Recommend a graph construction and retrieval model
  • the data sources of teaching resources collected in this embodiment include teaching materials and books, lesson plans, teaching videos, teaching voices and speech drafts, which are mainly divided into four types: knowledge point video V, knowledge point audio A, knowledge point film P and knowledge point teaching material L view mode.
  • the multi-view feature extractor CR-Mutiview-Fet is responsible for extracting knowledge point features, outputting the knowledge point ontology structure and associated knowledge point EP-OR.
  • Four different feature extraction models are used for feature extraction and feature fusion for the above four views. It includes video feature extraction module FV, audio feature extraction module FA, image feature extraction module FP, text feature extraction module FL and feature fusion module FF.
  • the core of the video feature extraction module FV is a three-dimensional CNN convolutional neural network, which is responsible for extracting the semantic features of knowledge points in the video;
  • the core of the audio feature extraction module FA is a CNN convolutional neural network, which is responsible for extracting the semantic features of the knowledge points in the audio;
  • the FP core is a convolutional neural network, responsible for extracting the semantic features of knowledge points in images;
  • the text feature extraction module FL core is a pre-trained language model based on BERT, responsible for extracting the semantic features of knowledge points in text;
  • the feature fusion module FF will extract the semantic features of knowledge points from the video.
  • the feature vectors of the feature extraction module FV, audio feature extraction module FA, image feature extraction module FP and text feature extraction module FL are fused to obtain the knowledge point ontology structure and associated knowledge point EP-OR.
  • the core of the teaching resource profiling model CR-Profiler is the CNN convolutional neural network model, which is responsible for profiling the teaching resources based on the extended data CR-WWW and teaching resource data CR related to the Internet teaching resources, and obtaining the basic attributes of the teaching resources CR-Basic (which Including knowledge points, content structure, processes, principles, concepts, tools, etc.) and teaching resource extension attributes CR-Ext (which includes Internet evaluation information, presentation forms, etc.).
  • the core of the knowledge graph construction model KG-Gen is a neural network model, including the feature encoder Enc and the generative network model GN.
  • the feature encoder is used to encode knowledge points so that the vector calculation distance between similar knowledge points is small and used to provide Resource index query;
  • generative network model is used to generate teaching knowledge Point Knowledge Graph EP-KG. That is, the knowledge graph construction model KG-Gen is based on the knowledge point ontology structure and associated knowledge points EP-OR formed by knowledge point extraction, and combined with the educational resource attributes obtained by the teaching resource portrait CR-Profiler to generate the teaching knowledge point knowledge graph EP-KG .
  • the user portrait model Stu-Profiler is based on the user's user information Stu-Info (that is, the learner's personal information and learning situation) to profile the learner to form the user attribute Stu-Prop (that is, the knowledge point preferences, mastery and Comprehensive ability and other labels.)
  • the retrieval recommendation model CR-Recommend is based on the constructed knowledge graph EP-KG of teaching resource knowledge points to build an index and provide retrieval recommendation functions. Through the input retrieval content (including text, voice, video, image, etc.), combined with user portraits The user attribute Stu-Prop generated by the model Stu-Profiler outputs multiple sets of knowledge points and recommended teaching resources, which users as learners can choose from.
  • the model training module is used to sequentially train the multi-view feature extractor CR-Mutiview-Fet, the teaching resource profile model CR-Profiler, the knowledge graph construction model KG-Gen, the user profile model Stu-Profiler and the retrieval recommendation model CR-Recommend. Model training to obtain the trained map construction and retrieval model.
  • model training module is used to train through the following steps:
  • the retrieval recommendation model CR-Recommend is lightweight and tailored based on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor CR-Mutiview-Fet.
  • the search condition extraction model in the above-mentioned retrieval recommendation model CR-Recommend is trained;
  • the retrieval and recommendation module is used to construct a knowledge graph EP-KG of teaching knowledge points through the trained graph construction and retrieval model, build an index based on the knowledge graph EP-KG of teaching knowledge points and provide retrieval recommendation services, and output multiple sets of knowledge points for users. and recommend resources for users to choose from.
  • the retrieval and recommendation module is used to construct the knowledge graph EP-KG of teaching knowledge points through the following steps, and output multiple sets of knowledge points and recommended resources for users to choose:
  • the trained knowledge graph is used to construct the model KG-Gen to encode the knowledge points, so that the vector calculation distance between similar knowledge point resources is small. Used for resource index query and generation of knowledge graph EP-KG of teaching knowledge points;
  • the view mode of the search content includes text, voice, image and video;

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Databases & Information Systems (AREA)
  • Strategic Management (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Tourism & Hospitality (AREA)
  • Human Resources & Organizations (AREA)
  • Biomedical Technology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Human Computer Interaction (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • General Business, Economics & Management (AREA)
  • Economics (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A multiview learning-based teaching knowledge graph construction and retrieval method and system. The method comprises the following steps: constructing a graph construction and retrieval model on the basis of a multiview feature extractor, a teaching resource profile model, a knowledge graph construction model, a user profile model and a retrieval recommendation model, and performing model training on the multiview feature extractor, the teaching resource profile model, the knowledge graph construction model, the user profile model and the retrieval recommendation model in sequence; and constructing a teaching knowledge point knowledge graph by means of the trained graph construction and retrieval model, constructing an index on the basis of the teaching knowledge point knowledge graph, and providing a retrieval recommendation service.

Description

基于多视图学习的教学知识图谱构建及检索方法及系统Teaching knowledge graph construction and retrieval method and system based on multi-view learning 技术领域Technical field
本发明涉及教学资源推荐技术领域，具体地说是基于多视图学习的教学知识图谱构建及检索方法及系统。The present invention relates to the technical field of teaching resource recommendation, specifically to a teaching knowledge graph construction and retrieval method and system based on multi-view learning.
背景技术Background technique
知识图谱(Knowledge Graph)是一种基于数据的大规模语义网络，作为一种知识表示形式，其描述了领域实体、概念及其之间的各种语义关系。谷歌公司于2012年提出了“Google Knowledge Graph”，知识图谱开始引起学术界和工业界的广泛关注，经过近年来不断地发展，在搜索优化、电子商务、智能推荐、社交媒体等多个领域已经得到了应用实践，逐渐成为一种管理海量信息的必选方式。Knowledge Graph is a large-scale semantic network based on data. As a knowledge representation form, it describes domain entities, concepts and various semantic relationships between them. Google proposed the "Google Knowledge Graph" in 2012, and the knowledge graph began to attract widespread attention from academia and industry. After continuous development in recent years, it has been used in many fields such as search optimization, e-commerce, intelligent recommendations, and social media. It has been applied in practice and has gradually become a necessary way to manage massive information.
近年来，人工智能技术发展迅速，其商业化速度超出预期，人工智能将会给整个社会带来颠覆性的变化，已经成为未来各国重要的发展战略。特别是以深度学习为核心的算法演进，其超强的进化能力，在大数据的支持下，通过训练构建得到类似人脑结构的大规模神经网络，已经可以解决各类问题。In recent years, artificial intelligence technology has developed rapidly, and its commercialization speed has exceeded expectations. Artificial intelligence will bring disruptive changes to the entire society and has become an important development strategy for various countries in the future. In particular, the evolution of algorithms with deep learning as the core has strong evolutionary capabilities. With the support of big data, large-scale neural networks similar to the human brain structure can be constructed through training, which can already solve various problems.
随着互联网技术的快速发展，传统的教育行业也迎来了“互联网+”的新模式，海量的在线教学资源改变了传统的教学方式，从互联网到移动互联网，创造了跨时空的生活、工作和学习方式，对于知识的获取和探索方式发生了根本变化。海量的教学资源呈现出多样性的特点，存在教材书籍、教案、教学视频、教学语音、演讲稿等多种知识点展现形式，同时知识点之间的关联关系也更加复杂，对于不同的学习者，也有其学习个性化的需求。在这种情况下，如何利用深度学习技术，结合多视图学习和用户画像技术，有效利用海量教学资源，自动构建更加准确合理的教学知识图谱，并实现个性化的知识点检索推荐成为亟需解决的技术问题。With the rapid development of Internet technology, the traditional education industry has also ushered in the new model of "Internet +". Massive online teaching resources have changed the traditional teaching methods, from the Internet to the mobile Internet, creating a life and work that spans time and space. and learning styles, the way knowledge is acquired and explored has undergone fundamental changes. A large number of teaching resources are characterized by diversity, including textbooks, lesson plans, teaching videos, teaching voices, speeches and other forms of presentation of knowledge points. At the same time, the relationships between knowledge points are also more complex, and for different learners , also has its own needs for personalized learning. In this case, how to use deep learning technology, combined with multi-view learning and user portrait technology, to effectively utilize massive teaching resources, automatically build a more accurate and reasonable teaching knowledge map, and achieve personalized knowledge point retrieval and recommendation has become an urgent solution technical issues.
发明内容Contents of the invention
本发明的技术任务是针对以上不足，提供基于多视图学习的教学知识图谱构建及检索方法及系统，来解决如何利用深度学习技术，结合多视图学习和用户画像技术，有效利用海量教学资源，自动构建更加准确合理的教学知识图谱，并实现个性化的知识点检索推荐的技术问题。The technical task of the present invention is to address the above shortcomings and provide a teaching knowledge graph construction and retrieval method and system based on multi-view learning to solve how to use deep learning technology, combined with multi-view learning and user portrait technology, to effectively utilize massive teaching resources and automatically Technical issues to construct a more accurate and reasonable teaching knowledge graph and realize personalized knowledge point retrieval and recommendation.
第一方面，本发明的一种基于多视图学习的教学知识图谱构建及检索方 法，包括如下步骤：In the first aspect, a teaching knowledge graph construction and retrieval method based on multi-view learning of the present invention method, including the following steps:
构建多视图特征提取器，所述多特征视图提取器用于对多种视图方式的教学资源数据进行特征提取和特征融合，得到知识点本体结构及关联知识点；Construct a multi-view feature extractor, which is used to perform feature extraction and feature fusion on teaching resource data in multiple view modes to obtain the knowledge point ontology structure and associated knowledge points;
基于卷积神经网络构建教学资源画像模型，所述教学资源画像模型以教学资源数据以及教学资源互联网相关扩展数据为输入，对教学资源进行画像，输出教学资源属性，所述教学资源属性包括教学资源基础属性和教学资源扩展属性；A teaching resource portrait model is constructed based on a convolutional neural network. The teaching resource portrait model takes teaching resource data and teaching resource Internet-related extended data as inputs to profile teaching resources and output teaching resource attributes. The teaching resource attributes include teaching resources. Basic attributes and extended attributes of teaching resources;
基于神经网络构建知识图谱构建模型，所述知识图谱构建模型以基于知识点本体结构及关联知识点以及教学资源属性为输入，生成教学知识点知识图谱；Build a knowledge graph construction model based on a neural network. The knowledge graph construction model uses the ontology structure of knowledge points, associated knowledge points and teaching resource attributes as input to generate a knowledge graph of teaching knowledge points;
构建用户画像模型，所述用户画像模型以用户信息为输入，对用户进行画像，输出用户属性；Construct a user portrait model, which takes user information as input, profiles the user, and outputs user attributes;
构建检索推荐模型，所述检索推荐模型用于基于教学知识点知识图谱构建索引并提供检索推荐服务，用于通过输入的检索内容、并结合用户属性，输出多组知识点及推荐资源以供用户选择；Construct a retrieval recommendation model. The retrieval recommendation model is used to build an index based on the knowledge graph of teaching knowledge points and provide retrieval recommendation services. It is used to output multiple sets of knowledge points and recommended resources for users through the input retrieval content and combined with user attributes. choose;
基于所述多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型构建图谱构建及检索模型，并依次对所述多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型进行模型训练，得到训练后的图谱构建及检索模型；Based on the multi-view feature extractor, teaching resource portrait model, knowledge graph construction model, user portrait model and retrieval recommendation model, a graph construction and retrieval model is constructed, and the multi-view feature extractor, teaching resource portrait model, knowledge graph are constructed in sequence. Carry out model training on the graph construction model, user portrait model and retrieval recommendation model, and obtain the trained graph construction and retrieval model;
通过所述训练后的图谱构建及检索模型构建教学知识点知识图谱，基于教学知识点知识图谱构建索引并提供检索推荐服务，为用户输出多组知识点及推荐资源以供用户选择。A knowledge graph of teaching knowledge points is constructed through the trained graph construction and retrieval model, an index is constructed based on the knowledge graph of teaching knowledge points and retrieval recommendation services are provided, and multiple sets of knowledge points and recommended resources are output to users for selection.
作为优选，所述教学资源的数据源包括教材书籍、教案、教学视频、教学语音以及演讲稿，所述教学资源的视图方式共四种，分别为视频、音频、图像和文字；Preferably, the data sources of the teaching resources include teaching materials and books, lesson plans, teaching videos, teaching voices and speech scripts. There are four viewing modes of the teaching resources, namely video, audio, image and text;
所述教学资源基础属性包括知识点、内容结构、过程、原理、概念、工具；The basic attributes of the teaching resources include knowledge points, content structure, processes, principles, concepts, and tools;
所述教学资源扩展属性包括互联网评价信息和展现形式；The extended attributes of the teaching resources include Internet evaluation information and presentation forms;
所述检索内容的视图形式包括文字、语音、视频和图像；The view form of the retrieval content includes text, voice, video and image;
所述教学知识点知识图谱包括语义网络、教学资源基础数据和教学资源 扩展属性，所述语义网络为基于知识点本体结构及知识点关联形成的；The knowledge map of teaching knowledge points includes semantic network, basic data of teaching resources and teaching resources. Extended attributes, the semantic network is formed based on the ontology structure of knowledge points and the association of knowledge points;
所述用户信息包括基础信息和学习情况；The user information includes basic information and learning status;
所述用户属性包括知识点喜好、掌握程度以及综合能力。The user attributes include knowledge point preferences, mastery level and comprehensive ability.
作为优选，所述多视图特征提取器包括：Preferably, the multi-view feature extractor includes:
视频特征提取模型，所述视频特征提取模型为基于三维CNN卷积神经网络构建的网络模型，用于从视频方式的教学资源数据提取知识点语义特征；A video feature extraction model. The video feature extraction model is a network model built based on a three-dimensional CNN convolutional neural network and is used to extract semantic features of knowledge points from video teaching resource data;
音频特征提取模型，所述音频特征提取模型为基于CNN卷积神经网络构建的网络模型，用于从音频方式的教学资源数据中提取知识点语义特征；Audio feature extraction model. The audio feature extraction model is a network model built based on a CNN convolutional neural network and is used to extract semantic features of knowledge points from audio-based teaching resource data;
图像特征提取模型，所述图像特征提取模型为基于卷积神经网络构建的网路模型，用于从图像方式的教学资源数据中提取知识点语义特征；Image feature extraction model. The image feature extraction model is a network model built based on a convolutional neural network and is used to extract semantic features of knowledge points from image-based teaching resource data;
文字特征提取模型，所述文字特征提取模型为基于BERT的语言模型，用于从文字方式的教学资源数据中提取知识点语义特征；Text feature extraction model. The text feature extraction model is a language model based on BERT, which is used to extract semantic features of knowledge points from text-based teaching resource data;
特征融合模型，所述特征融合模型用于将所述视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型输出的知识点语义特征进行融合，得到知识点本体结构及关联知识点；Feature fusion model, the feature fusion model is used to fuse the semantic features of knowledge points output by the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model to obtain the knowledge point ontology structure and associated knowledge point;
所述知识图谱构建模型包括：The knowledge graph construction model includes:
特征编码器，所述特征编码器用于对知识点进行编码，使得相似知识点之间的向量计算距离小，用于提供资源索引查询；Feature encoder, which is used to encode knowledge points so that the vector calculation distance between similar knowledge points is small and used to provide resource index queries;
生成网络模型，所述生成网络模型用于生成教学知识点知识图谱。Generate a network model, which is used to generate a knowledge graph of teaching knowledge points.
作为优选，依次对所述多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型进行模型训练，包括如下步骤：Preferably, model training is performed on the multi-view feature extractor, teaching resource portrait model, knowledge graph construction model, user portrait model and retrieval recommendation model in sequence, including the following steps:
从多个数据源获取教学资源数据，并根据其素材进行数据标注；Obtain teaching resource data from multiple data sources and annotate the data based on its materials;
基于所述教学资源数据，分别对所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型进行模型预训练；Based on the teaching resource data, perform model pre-training on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor;
将所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型的模型参数固定，对所述特征融合模 型进行模型训练；The model parameters of the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor are fixed, and the feature fusion model is type for model training;
将所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型、文字特征提取模型以及特征融合模型进行组合，通过所述教学资源数据对所述多视图特征提取器整体进行模型训练，对所述多视图特征提取器中每个模型进行参数微调；The video feature extraction model, audio feature extraction model, image feature extraction model, text feature extraction model and feature fusion model in the multi-view feature extractor are combined, and the entire multi-view feature extractor is processed through the teaching resource data. Carry out model training and fine-tune parameters of each model in the multi-view feature extractor;
收集教学资源互联网相关扩展数据，并进行标签标注；Collect Internet-related extended data on teaching resources and label them;
基于所述教学资源数据、教学资源互联网相关扩展数据以及标签，对所述教学资源画像模型进行模型训练；Carry out model training on the teaching resource portrait model based on the teaching resource data, teaching resource Internet-related extended data and tags;
基于所述多视图特征提取器输出的知识点本体结构及关联知识点，以及所述教学资源画像模型输出的教学资源基础属性和教学资源扩展属性，对所述知识图谱构建模型进行模型训练；Based on the knowledge point ontology structure and associated knowledge points output by the multi-view feature extractor, as well as the teaching resource basic attributes and teaching resource extended attributes output by the teaching resource portrait model, perform model training on the knowledge graph construction model;
收集用户信息，并进行标签标注；Collect user information and label it;
基于用户信息和标签，对所述用户画像模型进行模型训练；Based on user information and tags, perform model training on the user portrait model;
基于多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型对所述检索推荐模型进行轻量化剪裁，对所述检索推荐模型中搜索条件提取模型进行训练；The retrieval recommendation model is lightweight and tailored based on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor, and the search condition extraction model in the retrieval recommendation model is trained. ;
基于所述用户画像模型输出的用户属性以及所述知识图谱构建模型输出的教学知识点知识图谱，对所述检索推荐模型进行模型训练。Based on the user attributes output by the user portrait model and the teaching knowledge point knowledge graph output by the knowledge graph construction model, model training is performed on the retrieval recommendation model.
作为优选，通过所述训练后的图谱构建及检索模型构建教学知识点知识图谱，包括如下步骤：Preferably, building a knowledge graph of teaching knowledge points through the trained graph construction and retrieval model includes the following steps:
选定领域，收集选定领域的教学资源数据；Select a field and collect teaching resource data in the selected field;
通过训练后的多视图特征提取器对所述教学资源进行特征提取和特征融合，得到知识点本体结构及关联知识点；Use the trained multi-view feature extractor to perform feature extraction and feature fusion on the teaching resources to obtain the knowledge point ontology structure and associated knowledge points;
收集教学资源互联网相关扩展数据，基于所述教学资源互联网相关扩展数据以及教学资源数据，通过训练后的教学资源画像模型对教学资源进行画像，得到教学资源属性；Collect the Internet-related extended data of teaching resources, and based on the Internet-related extended data of teaching resources and the teaching resource data, profile the teaching resources through the trained teaching resource portrait model to obtain the teaching resource attributes;
基于知识点本体结构及关联知识点、教学资源属性，通过训练后的知识图谱构建模型对知识点进行编码，使得相似知识点资源之间的向量计算距离小，用于资源索引查询，并生成教学知识点知识图谱； Based on the knowledge point ontology structure, associated knowledge points, and teaching resource attributes, the knowledge points are encoded through the trained knowledge graph construction model, so that the vector calculation distance between similar knowledge point resources is small, which can be used for resource index query and generate teaching Knowledge point knowledge map;
基于教学知识点知识图谱构建索引并提供检索推荐服务，为用户输出多组知识点及推荐资源以供用户选择，包括如下步骤：Build an index based on the knowledge graph of teaching knowledge points and provide retrieval and recommendation services, and output multiple sets of knowledge points and recommended resources for users to choose, including the following steps:
输入检索内容，所述检索内容的视图方式包括文字、语音、图像和视频；Enter the search content, and the view mode of the search content includes text, voice, image and video;
获取用户的用户信息，通过训练后的用户画像模型对用户进行画像，生成用户属性；Obtain the user's user information, profile the user through the trained user portrait model, and generate user attributes;
基于教学知识点知识图谱、通过训练后的检索推荐模型构建索引并提供检索推荐服务；Based on the knowledge graph of teaching knowledge points, build an index through the trained retrieval recommendation model and provide retrieval recommendation services;
基于索引以及检索推荐服务，对用户输入的检索内容进行知识点提取、并结合用户属性形成知识点特征向量，并进行知识点查询和特征向量计算，输出多组知识点和推荐资源，以供用户进行选择；Based on indexing and retrieval recommendation services, knowledge points are extracted from the retrieval content input by users, combined with user attributes to form knowledge point feature vectors, and knowledge point queries and feature vector calculations are performed to output multiple sets of knowledge points and recommended resources for users make a choice;
将收集的教学资源数据、互联网相关扩展数据、用户信息以及检索内容，以及输出的多组知识点和推荐资源，反馈至所述图谱构建及检索模型，对所述图谱构建及检索模型进行模型训练，以持续优化所述图谱构建及检索模型。The collected teaching resource data, Internet-related extended data, user information and search content, as well as the output sets of knowledge points and recommended resources, are fed back to the map construction and retrieval model, and model training is performed on the map construction and retrieval model. , to continuously optimize the map construction and retrieval model.
第二方面，本发明的一种基于多视图学习的教学知识图谱构建及检索系统，用于通过如第一方面任一项所述的一种基于多视图学习的教学知识图谱构建及检索方法为用户提供知识点和教学资源推荐服务，所述系统包括：In the second aspect, a teaching knowledge graph construction and retrieval system based on multi-view learning of the present invention is used to construct and retrieve a teaching knowledge graph based on multi-view learning as described in any one of the first aspects. Users provide knowledge points and teaching resource recommendation services. The system includes:
模型构建模块，所述模型构建模块用于构建图谱构建及检索模型，所述图谱构建及检索模型包括多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型，所述多特征视图提取器用于对多种视图方式的教学资源数据进行特征提取和特征融合，得到知识点本体结构及关联知识点；所述教学资源画像模型为基于卷积神经网络构建的网络模型，以教学资源数据以及教学资源互联网相关扩展数据为输入，对教学资源进行画像，输出教学资源属性，所述教学资源属性包括教学资源基础属性和教学资源扩展属性；所述知识图谱构建模型以基于知识点本体结构及关联知识点以及教学资源属性为输入，生成教学知识点知识图谱；所述用户画像模型以用户信息为输入，对用户进行画像，输出用户属性；所述检索推荐模型用于基于教学知识点知识图谱构建索引并提供检索推荐服务，用于通过输入的检索内容、并结合用户属性，输出多组知识点及推荐资源以供用户选择；A model building module, which is used to build a graph construction and retrieval model. The graph construction and retrieval model includes a multi-view feature extractor, a teaching resource portrait model, a knowledge graph construction model, a user portrait model and a retrieval recommendation model, The multi-feature view extractor is used to perform feature extraction and feature fusion on teaching resource data in multiple view modes to obtain the knowledge point ontology structure and associated knowledge points; the teaching resource portrait model is a network model based on a convolutional neural network. , using teaching resource data and teaching resource Internet-related extended data as input, the teaching resources are profiled, and the teaching resource attributes are output. The teaching resource attributes include basic teaching resource attributes and teaching resource extended attributes; the knowledge graph construction model is based on The ontology structure of knowledge points, associated knowledge points and teaching resource attributes are used as input to generate a knowledge graph of teaching knowledge points; the user portrait model uses user information as input to profile the user and output user attributes; the retrieval recommendation model is used based on The knowledge graph of teaching knowledge points constructs an index and provides retrieval recommendation services, which is used to output multiple sets of knowledge points and recommended resources for users to choose based on the input retrieval content and combined with user attributes;
模型训练模块，所述模型训练模块用于依次对所述多视图特征提取 器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型进行模型训练，得到训练后的图谱构建及检索模型；a model training module, which is used to extract the multi-view features in sequence model, teaching resource portrait model, knowledge graph construction model, user portrait model and retrieval recommendation model for model training, and the trained graph construction and retrieval model is obtained;
检索推荐模块，所述检索推荐模块用于通过所述训练后的图谱构建及检索模型构建教学知识点知识图谱，基于教学知识点知识图谱构建索引并提供检索推荐服务，为用户输出多组知识点及推荐资源以供用户选择。A retrieval recommendation module. The retrieval recommendation module is used to construct a knowledge graph of teaching knowledge points through the trained graph construction and retrieval model, build an index based on the knowledge graph of teaching knowledge points and provide retrieval recommendation services, and output multiple sets of knowledge points for users. and recommend resources for users to choose from.
作为优选，所述教学资源的数据源包括教材书籍、教案、教学视频、教学语音以及演讲稿，所述教学资源的视图方式共四种，分别为视频、音频、图像和文字；Preferably, the data sources of the teaching resources include teaching materials and books, lesson plans, teaching videos, teaching voices and speech scripts. There are four viewing modes of the teaching resources, namely video, audio, image and text;
所述教学资源基础属性包括知识点、内容结构、过程、原理、概念、工具；The basic attributes of the teaching resources include knowledge points, content structure, processes, principles, concepts, and tools;
所述教学资源扩展属性包括互联网评价信息和展现形式；The extended attributes of the teaching resources include Internet evaluation information and presentation forms;
所述检索内容的视图形式包括文字、语音、视频和图像；The view form of the retrieval content includes text, voice, video and image;
所述教学知识点知识图谱包括语义网络、教学资源基础数据和教学资源扩展属性，所述语义网络为基于知识点本体结构及知识点关联形成的；The knowledge map of teaching knowledge points includes a semantic network, basic data of teaching resources and extended attributes of teaching resources. The semantic network is formed based on the ontology structure of knowledge points and the association of knowledge points;
所述用户信息包括基础信息和学习情况；The user information includes basic information and learning status;
所述用户属性包括知识点喜好、掌握程度以及综合能力。The user attributes include knowledge point preferences, mastery level and comprehensive ability.
作为优选，所述多视图特征提取器包括：Preferably, the multi-view feature extractor includes:
视频特征提取模型，所述视频特征提取模型为基于三维CNN卷积神经网络构建的网络模型，用于从视频方式的教学资源数据提取知识点语义特征；A video feature extraction model. The video feature extraction model is a network model built based on a three-dimensional CNN convolutional neural network and is used to extract semantic features of knowledge points from video teaching resource data;
音频特征提取模型，所述音频特征提取模型为基于CNN卷积神经网络构建的网络模型，用于从音频方式的教学资源数据中提取知识点语义特征；Audio feature extraction model. The audio feature extraction model is a network model built based on a CNN convolutional neural network and is used to extract semantic features of knowledge points from audio-based teaching resource data;
图像特征提取模型，所述图像特征提取模型为基于卷积神经网络构建的网路模型，用于从图像方式的教学资源数据中提取知识点语义特征；Image feature extraction model. The image feature extraction model is a network model built based on a convolutional neural network and is used to extract semantic features of knowledge points from image-based teaching resource data;
文字特征提取模型，所述文字特征提取模型为基于BERT的语言模型，用于从文字方式的教学资源数据中提取知识点语义特征；Text feature extraction model. The text feature extraction model is a language model based on BERT, which is used to extract semantic features of knowledge points from text-based teaching resource data;
特征融合模型，所述特征融合模型用于将所述视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型输出的知识点语义特征进行融合，得到知识点本体结构及关联知识点； Feature fusion model, the feature fusion model is used to fuse the semantic features of knowledge points output by the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model to obtain the knowledge point ontology structure and associated knowledge point;
所述知识图谱构建模型包括：The knowledge graph construction model includes:
特征编码器，所述特征编码器用于对知识点进行编码，使得相似知识点之间的向量计算距离小，用于提供资源索引查询；Feature encoder, which is used to encode knowledge points so that the vector calculation distance between similar knowledge points is small and used to provide resource index queries;
生成网络模型，所述生成网络模型用于生成教学知识点知识图谱。Generate a network model, which is used to generate a knowledge graph of teaching knowledge points.
作为优选，所述模型序列模块用于通过如下步骤依次对所述多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型进行模型训练：Preferably, the model sequence module is used to perform model training on the multi-view feature extractor, teaching resource portrait model, knowledge graph construction model, user portrait model and retrieval recommendation model in sequence through the following steps:
从多个数据源获取教学资源数据，并根据其素材进行数据标注；Obtain teaching resource data from multiple data sources and annotate the data based on its materials;
基于所述教学资源数据，分别对所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型进行模型预训练；Based on the teaching resource data, perform model pre-training on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor;
将所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型的模型参数固定，对所述特征融合模型进行模型训练；Fix the model parameters of the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor, and perform model training on the feature fusion model;
将所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型、文字特征提取模型以及特征融合模型进行组合，通过所述教学资源数据对所述多视图特征提取器整体进行模型训练，对所述多视图特征提取器中每个模型进行参数微调；The video feature extraction model, audio feature extraction model, image feature extraction model, text feature extraction model and feature fusion model in the multi-view feature extractor are combined, and the entire multi-view feature extractor is processed through the teaching resource data. Carry out model training and fine-tune parameters of each model in the multi-view feature extractor;
收集教学资源互联网相关扩展数据，并进行标签标注；Collect Internet-related extended data on teaching resources and label them;
基于所述教学资源数据、教学资源互联网相关扩展数据以及标签，对所述教学资源画像模型进行模型训练；Carry out model training on the teaching resource portrait model based on the teaching resource data, teaching resource Internet-related extended data and tags;
基于所述多视图特征提取器输出的知识点本体结构及关联知识点，以及所述教学资源画像模型输出的教学资源基础属性和教学资源扩展属性，对所述知识图谱构建模型进行模型训练；Based on the knowledge point ontology structure and associated knowledge points output by the multi-view feature extractor, as well as the teaching resource basic attributes and teaching resource extended attributes output by the teaching resource portrait model, perform model training on the knowledge graph construction model;
收集用户信息，并进行标签标注；Collect user information and label it;
基于用户信息和标签，对所述用户画像模型进行模型训练；Based on user information and tags, perform model training on the user portrait model;
基于多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型对所述检索推荐模型进行轻量化剪裁，对所述检索推荐模型中搜索条件提取模型进行训练；The retrieval recommendation model is lightweight and tailored based on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor, and the search condition extraction model in the retrieval recommendation model is trained. ;
基于所述用户画像模型输出的用户属性以及所述知识图谱构建模型输出的教学知识点知识图谱，对所述检索推荐模型进行模型训练。 Based on the user attributes output by the user portrait model and the teaching knowledge point knowledge graph output by the knowledge graph construction model, model training is performed on the retrieval recommendation model.
作为优选，所述检索推荐模块用于通过如下步骤构建教学知识点知识图谱，并为用户输出多组知识点及推荐资源以供用户选择：Preferably, the retrieval and recommendation module is used to construct a knowledge graph of teaching knowledge points through the following steps, and output multiple sets of knowledge points and recommended resources for the user to select:
选定领域，收集选定领域的教学资源数据；Select a field and collect teaching resource data in the selected field;
通过训练后的多视图特征提取器对所述教学资源进行特征提取和特征融合，得到知识点本体结构及关联知识点；Use the trained multi-view feature extractor to perform feature extraction and feature fusion on the teaching resources to obtain the knowledge point ontology structure and associated knowledge points;
收集教学资源互联网相关扩展数据，基于所述教学资源互联网相关扩展数据以及教学资源数据，通过训练后的教学资源画像模型对教学资源进行画像，得到教学资源属性；Collect the Internet-related extended data of teaching resources, and based on the Internet-related extended data of teaching resources and the teaching resource data, profile the teaching resources through the trained teaching resource portrait model to obtain the teaching resource attributes;
基于知识点本体结构及关联知识点、教学资源属性，通过训练后的知识图谱构建模型对知识点进行编码，使得相似知识点资源之间的向量计算距离小，用于资源索引查询，并生成教学知识点知识图谱；Based on the knowledge point ontology structure, associated knowledge points, and teaching resource attributes, the knowledge points are encoded through the trained knowledge graph construction model, so that the vector calculation distance between similar knowledge point resources is small, which can be used for resource index query and generate teaching Knowledge point knowledge map;
输入检索内容，所述检索内容的视图方式包括文字、语音、图像和视频；Enter the search content, and the view mode of the search content includes text, voice, image and video;
获取用户的用户信息，通过训练后的用户画像模型对用户进行画像，生成用户属性；Obtain the user's user information, profile the user through the trained user portrait model, and generate user attributes;
基于教学知识点知识图谱、通过训练后的检索推荐模型构建索引并提供检索推荐服务；Based on the knowledge graph of teaching knowledge points, build an index through the trained retrieval recommendation model and provide retrieval recommendation services;
基于索引以及检索推荐服务，对用户输入的检索内容进行知识点提取、并结合用户属性形成知识点特征向量，并进行知识点查询和特征向量计算，输出多组知识点和推荐资源，以供用户进行选择；Based on indexing and retrieval recommendation services, knowledge points are extracted from the retrieval content input by users, combined with user attributes to form knowledge point feature vectors, and knowledge point queries and feature vector calculations are performed to output multiple sets of knowledge points and recommended resources for users make a choice;
将收集的教学资源数据、互联网相关扩展数据、用户信息以及检索内容，以及输出的多组知识点和推荐资源，反馈至所述图谱构建及检索模型，对所述图谱构建及检索模型进行模型训练，以持续优化所述图谱构建及检索模型。The collected teaching resource data, Internet-related extended data, user information and search content, as well as the output sets of knowledge points and recommended resources, are fed back to the map construction and retrieval model, and model training is performed on the map construction and retrieval model. , to continuously optimize the map construction and retrieval model.
本发明的基于多视图学习的教学知识图谱构建及检索方法及系统具有以下优点：The teaching knowledge graph construction and retrieval method and system based on multi-view learning of the present invention have the following advantages:
1、基于海量的教学资源数据，有效利用深度学习特征提取技术，充分考虑互联网在线学习的特点，发掘教材书籍、教案、教学视频、教学语音、演讲稿等多视图之间的联系，并结合互联网资源评价信息，通过教学资源画像形成更加合理的知识点属性，构建更加准确合理的教学知识图谱，与传统知识图谱构建和推荐方式相比，采用了多视图学习及深度学习的构建查询方式，根据资源不同的展现方式形成的多视图数据来设计神经网络模型，能够更好的考虑多样性，发掘知识点内部的潜在联系，更加准确合理的提取知识点； 1. Based on massive teaching resource data, effectively utilize deep learning feature extraction technology, fully consider the characteristics of Internet online learning, explore the connections between multiple views such as textbooks, lesson plans, teaching videos, teaching voices, and speech drafts, and combine it with the Internet Resource evaluation information is used to form more reasonable knowledge point attributes through teaching resource portraits, and a more accurate and reasonable teaching knowledge graph is constructed. Compared with the traditional knowledge graph construction and recommendation methods, multi-view learning and deep learning are used to construct query methods. According to Designing neural network models using multi-view data formed by different presentation methods of resources can better consider diversity, explore potential connections within knowledge points, and extract knowledge points more accurately and reasonably;
2、加入了教学资源画像和用户画像，在知识点提取和检索过程中，更加有针对性和侧重点，推荐的资源更能符合学习者的学习习惯，满足学习者的个性化的需求，同时推荐结果包含多组数据，增加了推荐的准确性和容错性；2. Teaching resource portraits and user portraits have been added to make the knowledge point extraction and retrieval process more targeted and focused. The recommended resources are more in line with learners' learning habits and meet learners' personalized needs. The recommendation results contain multiple sets of data, which increases the accuracy and fault tolerance of recommendations;
3、学习者根据推荐的学习资源进行学习并及时进行反馈，持续优化推荐模型。3. Learners learn based on recommended learning resources and provide timely feedback to continuously optimize the recommendation model.
附图说明Description of drawings
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings in the following description are only illustrative of the present invention. For some embodiments, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.
下面结合附图对本发明进一步说明。The present invention will be further described below in conjunction with the accompanying drawings.
图1为实施例1基于多视图学习的教学知识图谱构建及检索方法的工作原理框图。Figure 1 is a block diagram of the working principle of the teaching knowledge graph construction and retrieval method based on multi-view learning in Embodiment 1.
具体实施方式Detailed ways
下面结合附图和具体实施例对本发明作进一步说明，以使本领域的技术人员可以更好地理解本发明并能予以实施，但所举实施例不作为对本发明的限定，在不冲突的情况下，本发明实施例以及实施例中的技术特征可以相互结合。The present invention will be further described below in conjunction with the accompanying drawings and specific examples, so that those skilled in the art can better understand the present invention and implement it. However, the illustrated embodiments are not intended to limit the present invention. In the absence of conflict, Below, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.
本发明实施例提供基于多视图学习的教学知识图谱构建及检索方法及系统，用于解决如何利用深度学习技术，结合多视图学习和用户画像技术，有效利用海量教学资源，自动构建更加准确合理的教学知识图谱，并实现个性化的知识点检索推荐的技术问题。Embodiments of the present invention provide teaching knowledge graph construction and retrieval methods and systems based on multi-view learning, which are used to solve how to use deep learning technology, combined with multi-view learning and user portrait technology, to effectively utilize massive teaching resources and automatically build more accurate and reasonable Technical issues of teaching knowledge graph and realizing personalized knowledge point retrieval and recommendation.
实施例1：Example 1:
本发明一种基于多视图学习的教学知识图谱构建及检索方法，包括如下步骤：The present invention is a teaching knowledge graph construction and retrieval method based on multi-view learning, which includes the following steps:
S100、构建多视图特征提取器CR-Mutiview-Fet，所述多特征视图提取器CR-Mutiview-Fet用于对多种视图方式的教学资源数据进行特征提取和特征融合，得到知识点本体结构及关联知识点EP-OR；S100. Construct a multi-view feature extractor CR-Mutiview-Fet. The multi-feature view extractor CR-Mutiview-Fet is used to perform feature extraction and feature fusion on teaching resource data in multiple views to obtain the knowledge point ontology structure and Related knowledge points EP-OR;
S200、基于卷积神经网络构建教学资源画像模型CR-Profiler，所述教学资源画像模型CR-Profiler以教学资源数据CR以及教学资源互联网相关扩展数据CR-WWW为输入，对教学资源进行画像，输出教学资源属性，所述教学资源属性包括教学资源基础属性CR-Basic和教学资源扩展属性CR-Ext； S200. Construct a teaching resource profiling model CR-Profiler based on the convolutional neural network. The teaching resource profiling model CR-Profiler takes the teaching resource data CR and the teaching resource Internet-related extended data CR-WWW as inputs to profile the teaching resources and output Teaching resource attributes, which include teaching resource basic attributes CR-Basic and teaching resource extended attributes CR-Ext;
S300、基于神经网络构建知识图谱构建模型KG-Gen，所述知识图谱构建模型KG-Gen以基于知识点本体结构及关联知识点EP-OR以及教学资源属性为输入，生成教学知识点知识图谱EP-KG；S300. Construct a knowledge graph construction model KG-Gen based on the neural network. The knowledge graph construction model KG-Gen uses the ontology structure of knowledge points, associated knowledge points EP-OR and teaching resource attributes as inputs to generate a knowledge graph EP of teaching knowledge points. -KG;
S400、构建用户画像模型Stu-Profiler，所述用户画像模型Stu-Profiler以用户信息Stu-Info为输入，对用户进行画像，输出用户属性Stu-Prop；S400. Construct a user profile model Stu-Profiler. The user profile model Stu-Profiler takes user information Stu-Info as input, profiles the user, and outputs user attributes Stu-Prop;
S500、构建检索推荐模型CR-Recommend，所述检索推荐模型CR-Recommend用于基于教学知识点知识图谱EP-KG构建索引并提供检索推荐服务，用于通过输入的检索内容、并结合用户属性Stu-Prop，输出多组知识点及推荐资源以供用户选择；S500. Construct a retrieval recommendation model CR-Recommend. The retrieval recommendation model CR-Recommend is used to build an index based on the teaching knowledge point knowledge graph EP-KG and provide retrieval recommendation services. It is used to use the input retrieval content and combine it with user attributes Stu -Prop, outputs multiple sets of knowledge points and recommended resources for users to choose;
S600、基于所述多视图特征提取器CR-Mutiview-Fet、教学资源画像模型CR-Profiler、知识图谱构建模型KG-Gen、用户画像模型Stu-Profiler以及检索推荐模型CR-Recommend构建图谱构建及检索模型，并依次对所述多视图特征提取器CR-Mutiview-Fet、教学资源画像模型CR-Profiler、知识图谱构建模型KG-Gen、用户画像模型Stu-Profiler以及检索推荐模型CR-Recommend进行模型训练，得到训练后的图谱构建及检索模型；S600. Graph construction and retrieval based on the multi-view feature extractor CR-Mutiview-Fet, teaching resource profile model CR-Profiler, knowledge graph construction model KG-Gen, user profile model Stu-Profiler and retrieval recommendation model CR-Recommend model, and perform model training on the multi-view feature extractor CR-Mutiview-Fet, teaching resource profile model CR-Profiler, knowledge graph construction model KG-Gen, user profile model Stu-Profiler and retrieval recommendation model CR-Recommend in sequence , obtain the trained graph construction and retrieval model;
S700、通过所述训练后的图谱构建及检索模型构建教学知识点知识图谱EP-KG，基于教学知识点知识图谱EP-KG构建索引并提供检索推荐服务，为用户输出多组知识点及推荐资源以供用户选择。S700. Construct a knowledge graph EP-KG of teaching knowledge points through the trained graph construction and retrieval model, build an index based on the knowledge graph EP-KG of teaching knowledge points and provide retrieval recommendation services, and output multiple sets of knowledge points and recommended resources for users. for users to choose.
本实施例的方法基于海量的教学资源数据CR，有效利用深度学习特征提取技术，充分发掘教材书籍、教案、教学视频、教学语音、演讲稿等多视图之间的联系，并结合互联网资源评价信息，通过教学资源画像形成更加合理的知识点属性，构建更加准确合理的教学知识图谱。利用形成的知识点语义网络和教学资源图谱，结合学生用户画像，形成基于教学知识图谱的检索推荐模型CR-Recommend，提供针对学习者个性化学习、符合其自身特点的知识点学习教学资源，达到因材施教。The method of this embodiment is based on massive teaching resource data CR, effectively utilizes deep learning feature extraction technology, fully explores the connections between multiple views such as teaching materials, teaching plans, teaching videos, teaching voices, and speech drafts, and combines it with Internet resource evaluation information , forming more reasonable attributes of knowledge points through teaching resource portraits, and building a more accurate and reasonable teaching knowledge map. Utilizing the formed knowledge point semantic network and teaching resource map, combined with student user portraits, a retrieval recommendation model CR-Recommend based on the teaching knowledge map is formed to provide knowledge point learning and teaching resources for learners' personalized learning and in line with their own characteristics, to achieve Teach students in accordance with their aptitude.
本实施例中采集的教学资源的数据源包括教材书籍、教案、教学视频、教学语音以及演讲稿，主要分为知识点视频V、知识点音频A、知识点胶片P和知识点教材L四种视图方式。The data sources of teaching resources collected in this embodiment include teaching materials and books, lesson plans, teaching videos, teaching voices and speech drafts, which are mainly divided into four types: knowledge point video V, knowledge point audio A, knowledge point film P and knowledge point teaching material L view mode.
即多视图特征提取器CR-Mutiview-Fet负责知识点特征的提取，输出知识点本体结构及关联知识点EP-OR，针对上述四种视图采用四个不同的特征提取模型进行特征提取和特征融合，包括视频特征提取模块FV、音频特征提取模块FA、图像特征提取模块FP、文字特征提取模块FL和特征融合模块FF。视频特征提取模块FV核心是三维CNN卷积神经网络，负责提取视频的知识点语义特征； 音频特征提取模块FA核心是CNN卷积神经网络，负责提取音频中的知识点语义特征；图像特征提取模块FP核心是卷积神经网络，负责提取图像的知识点语义特征；文字特征提取模块FL核心是基于BERT的预训练的语言模型，负责提取文字的知识点语义特征；特征融合模块FF将根据来自视频特征提取模块FV、音频特征提取模块FA、图像特征提取模块FP和文字特征提取模块FL的特征向量进行融合，获取知识点本体结构及关联知识点EP-OR。That is, the multi-view feature extractor CR-Mutiview-Fet is responsible for extracting knowledge point features, outputting the ontology structure of knowledge points and associated knowledge points EP-OR. Four different feature extraction models are used for feature extraction and feature fusion for the above four views. , including video feature extraction module FV, audio feature extraction module FA, image feature extraction module FP, text feature extraction module FL and feature fusion module FF. The core of the video feature extraction module FV is a three-dimensional CNN convolutional neural network, which is responsible for extracting the semantic features of video knowledge points; The core of the audio feature extraction module FA is a CNN convolutional neural network, which is responsible for extracting the semantic features of knowledge points in audio; the core of the image feature extraction module FP is a convolutional neural network, which is responsible for extracting the semantic features of knowledge points in images; the core of the text feature extraction module FL is It is a pre-trained language model based on BERT, responsible for extracting the semantic features of text knowledge points; the feature fusion module FF will be based on the video feature extraction module FV, audio feature extraction module FA, image feature extraction module FP and text feature extraction module FL. Feature vectors are fused to obtain the knowledge point ontology structure and associated knowledge point EP-OR.
教学资源画像模型CR-Profiler核心是CNN卷积神经网络模型，负责基于自互联网教学资源相关扩展数据CR-WWW和教学资源数据CR，为教学资源进行画像，得到教学资源基础属性CR-Basic(其包括知识点、内容结构、过程、原理、概念、工具等)和教学资源扩展属性CR-Ext(其包括互联网评价信息、展现形式等)。The core of the teaching resource profiling model CR-Profiler is the CNN convolutional neural network model, which is responsible for profiling the teaching resources based on the extended data CR-WWW and teaching resource data CR related to the Internet teaching resources, and obtaining the basic attributes of the teaching resources CR-Basic (which Including knowledge points, content structure, processes, principles, concepts, tools, etc.) and teaching resource extension attributes CR-Ext (which includes Internet evaluation information, presentation forms, etc.).
知识图谱构建模型KG-Gen其核心是神经网络模型，包括特征编码器Enc和生成网络模型GN，特征编码器用于对知识点进行编码，使得相似知识点之间的向量计算距离小，用于提供资源索引查询；生成网络模型用于生成教学知识点知识图谱EP-KG。即该知识图谱构建模型KG-Gen基于知识点提取形成的知识点本体结构及关联知识点EP-OR，并结合教学资源画像CR-Profiler获取的教育资源属性，生成教学知识点知识图谱EP-KG。The core of the knowledge graph construction model KG-Gen is a neural network model, including the feature encoder Enc and the generative network model GN. The feature encoder is used to encode knowledge points so that the vector calculation distance between similar knowledge points is small and used to provide Resource index query; the generative network model is used to generate the knowledge graph EP-KG of teaching knowledge points. That is, the knowledge graph construction model KG-Gen is based on the knowledge point ontology structure formed by knowledge point extraction and associated knowledge points EP-OR, and combined with the educational resource attributes obtained by the teaching resource portrait CR-Profiler to generate the teaching knowledge point knowledge graph EP-KG .
用户画像模型Stu-Profiler是根据用户的用户信息Stu-Info(即学习者的个人信息及学习情况)为学习者进行画像，形成用户属性Stu-Prop(即其学习的知识点喜好、掌握程度以及综合能力等标签。)The user portrait model Stu-Profiler is based on the user's user information Stu-Info (that is, the learner's personal information and learning situation) to profile the learner to form the user attribute Stu-Prop (that is, the knowledge point preferences, mastery and Comprehensive ability and other labels.)
检索推荐模型CR-Recommend是基于构建的教学资源知识点知识图谱EP-KG构建索引并提供检索推荐功能，通过输入的检索内容(包括文字、语音、视频、图像等多种形式)，结合用户画像模型Stu-Profiler生成的用户属性Stu-Prop，输出多组知识点及推荐教学资源，作为学习者的用户可以进行选择。The retrieval recommendation model CR-Recommend is based on the constructed knowledge graph EP-KG of teaching resource knowledge points to build an index and provide retrieval recommendation functions. Through the input retrieval content (including text, voice, video, image, etc.), combined with user portraits The user attribute Stu-Prop generated by the model Stu-Profiler outputs multiple sets of knowledge points and recommended teaching resources, which users as learners can choose from.
步骤S200将多视图特征提取器CR-Mutiview-Fet、教学资源画像模型CR-Profiler、知识图谱构建模型KG-Gen、用户画像模型Stu-Profiler以及检索推荐模型CR-Recommend组合连接形成图谱构建及检索模型，并依次对所述多视图特征提取器CR-Mutiview-Fet、教学资源画像模型CR-Profiler、知识图谱构建模型KG-Gen、用户画像模型Stu-Profiler以及检索推荐模型CR-Recommend进行模型训练，得到训练后的图谱构建及检索模型。Step S200 combines and connects the multi-view feature extractor CR-Mutiview-Fet, the teaching resource profile model CR-Profiler, the knowledge graph construction model KG-Gen, the user profile model Stu-Profiler and the retrieval recommendation model CR-Recommend to form graph construction and retrieval. model, and perform model training on the multi-view feature extractor CR-Mutiview-Fet, teaching resource profile model CR-Profiler, knowledge graph construction model KG-Gen, user profile model Stu-Profiler and retrieval recommendation model CR-Recommend in sequence , obtain the trained graph construction and retrieval model.
在该步骤中，通过如下步骤进行训练：In this step, training is performed through the following steps:
S210、从多个数据源获取教学资源数据CR，并根据其素材进行数据标注； S210. Obtain teaching resource data CR from multiple data sources, and perform data annotation based on its materials;
S220、基于所述教学资源数据CR，分别对所述多视图特征提取器CR-Mutiview-Fet中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型进行模型预训练；S220. Based on the teaching resource data CR, perform model pre-training on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor CR-Mutiview-Fet respectively;
S230、将所述多视图特征提取器CR-Mutiview-Fet中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型的模型参数固定，对所述特征融合模型进行模型训练；S230. Fix the model parameters of the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor CR-Mutiview-Fet, and perform model training on the feature fusion model;
S240、将所述多视图特征提取器CR-Mutiview-Fet中视频特征提取模型、音频特征提取模型、图像特征提取模型、文字特征提取模型以及特征融合模型进行组合，通过所述教学资源数据CR对所述多视图特征提取器CR-Mutiview-Fet整体进行模型训练，对所述多视图特征提取器CR-Mutiview-Fet中每个模型进行参数微调；S240. Combine the video feature extraction model, audio feature extraction model, image feature extraction model, text feature extraction model and feature fusion model in the multi-view feature extractor CR-Mutiview-Fet, and use the teaching resource data CR to The multi-view feature extractor CR-Mutiview-Fet performs model training as a whole, and parameters are fine-tuned for each model in the multi-view feature extractor CR-Mutiview-Fet;
S250、收集教学资源互联网相关扩展数据CR-WWW，并进行标签标注；S250. Collect the Internet-related extended data CR-WWW of teaching resources and perform labeling;
S260、基于所述教学资源数据CR、教学资源互联网相关扩展数据CR-WWW以及标签，对所述教学资源画像模型CR-Profiler进行模型训练；S260. Perform model training on the teaching resource profile model CR-Profiler based on the teaching resource data CR, teaching resource Internet-related extended data CR-WWW and tags;
S270、基于所述多视图特征提取器CR-Mutiview-Fet输出的知识点本体结构及关联知识点EP-OR，以及所述教学资源画像模型CR-Profiler输出的教学资源基础属性CR-Basic和教学资源扩展属性CR-Ext，对所述知识图谱构建模型KG-Gen进行模型训练；S270. Based on the knowledge point ontology structure and associated knowledge points EP-OR output by the multi-view feature extractor CR-Mutiview-Fet, and the teaching resource basic attributes CR-Basic and teaching output by the teaching resource profile model CR-Profiler. Resource extension attribute CR-Ext, perform model training on the knowledge graph construction model KG-Gen;
S280、收集用户信息Stu-Info，并进行标签标注；S280. Collect user information Stu-Info and label it;
S290、基于用户信息Stu-Info和标签，对所述用户画像模型Stu-Profiler进行模型训练；S290. Based on the user information Stu-Info and tags, perform model training on the user profile model Stu-Profiler;
S2A0、基于多视图特征提取器CR-Mutiview-Fet中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型对所述检索推荐模型CR-Recommend进行轻量化剪裁，对所述检索推荐模型CR-Recommend中搜索条件提取模型进行训练；S2A0. Based on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor CR-Mutiview-Fet, the retrieval recommendation model CR-Recommend is lightweight and tailored. The search condition extraction model in the retrieval recommendation model CR-Recommend is trained;
S2B0、基于所述用户画像模型Stu-Profiler输出的用户属性Stu-Prop以及所述知识图谱构建模型KG-Gen输出的教学知识点知识图谱EP-KG，对所述检索推荐模型CR-Recommend进行模型训练。S2B0. Model the retrieval recommendation model CR-Recommend based on the user attribute Stu-Prop output by the user portrait model Stu-Profiler and the teaching knowledge point knowledge graph EP-KG output by the knowledge graph construction model KG-Gen. train.
步骤S300通过所述训练后的图谱构建及检索模型构建教学知识点知识图谱EP-KG，包括如下步骤：Step S300 constructs a knowledge graph EP-KG of teaching knowledge points through the trained graph construction and retrieval model, including the following steps:
S310、选定领域，收集选定领域的教学资源数据CR； S310. Select a field and collect teaching resource data CR in the selected field;
S320、通过训练后的多视图特征提取器CR-Mutiview-Fet对所述教学资源进行特征提取和特征融合，得到知识点本体结构及关联知识点EP-OR；S320. Use the trained multi-view feature extractor CR-Mutiview-Fet to perform feature extraction and feature fusion on the teaching resources to obtain the knowledge point ontology structure and associated knowledge point EP-OR;
S330、收集教学资源互联网相关扩展数据CR-WWW，基于所述教学资源互联网相关扩展数据CR-WWW以及教学资源数据CR，通过训练后的教学资源画像模型CR-Profiler对教学资源进行画像，得到教学资源属性；S330. Collect the Internet-related extended data CR-WWW of teaching resources. Based on the Internet-related extended data CR-WWW of teaching resources and the teaching resource data CR, profile the teaching resources through the trained teaching resource profile model CR-Profiler to obtain the teaching resources. Resource attributes;
S340、基于知识点本体结构及关联知识点EP-OR、教学资源属性，通过训练后的知识图谱构建模型KG-Gen对知识点进行编码，使得相似知识点资源之间的向量计算距离小，用于资源索引查询，并生成教学知识点知识图谱EP-KG。S340. Based on the knowledge point ontology structure, associated knowledge point EP-OR, and teaching resource attributes, the model KG-Gen is constructed through the trained knowledge graph to encode the knowledge points, so that the vector calculation distance between similar knowledge point resources is small, using Query the resource index and generate a knowledge graph EP-KG of teaching knowledge points.
基于教学知识点知识图谱EP-KG构建索引并提供检索推荐服务，为用户输出多组知识点及推荐资源以供用户选择，包括如下步骤：Build an index based on the teaching knowledge point knowledge graph EP-KG and provide search recommendation services, output multiple sets of knowledge points and recommended resources for users to choose, including the following steps:
S350、输入检索内容，所述检索内容的视图方式包括文字、语音、图像和视频；S350. Input the retrieval content. The viewing mode of the retrieval content includes text, voice, image and video;
S360、获取用户的用户信息Stu-Info，通过训练后的用户画像模型Stu-Profiler对用户进行画像，生成用户属性Stu-Prop；S360: Obtain the user's user information Stu-Info, profile the user through the trained user profile model Stu-Profiler, and generate the user attribute Stu-Prop;
S370、基于教学知识点知识图谱EP-KG、通过训练后的检索推荐模型CR-Recommend构建索引并提供检索推荐服务；S370. Based on the teaching knowledge point knowledge graph EP-KG, build an index through the trained retrieval recommendation model CR-Recommend and provide retrieval recommendation services;
S380、基于索引以及检索推荐服务，对用户输入的检索内容进行知识点提取、并结合用户属性Stu-Prop形成知识点特征向量，并进行知识点查询和特征向量计算，输出多组知识点和推荐资源，以供用户进行选择；S380, based on indexing and retrieval recommendation services, extract knowledge points from the retrieval content input by the user, combine it with user attributes Stu-Prop to form a knowledge point feature vector, perform knowledge point query and feature vector calculation, and output multiple sets of knowledge points and recommendations. resources for users to choose;
S390、将收集的教学资源数据CR、互联网相关扩展数据、用户信息Stu-Info以及检索内容，以及输出的多组知识点和推荐资源，反馈至所述图谱构建及检索模型，对所述图谱构建及检索模型进行模型训练，以持续优化所述图谱构建及检索模型。S390. Feed back the collected teaching resource data CR, Internet-related extended data, user information Stu-Info and search content, as well as the output sets of knowledge points and recommended resources to the map construction and retrieval model, and build the map and retrieval models for model training to continuously optimize the map construction and retrieval models.
本实施例的方法基于海量的教学资源数据CR，有效利用深度学习特征提取技术，充分考虑互联网在线学习的特点，发掘教材书籍、教案、教学视频、教学语音、演讲稿等多视图之间的联系，并结合互联网资源评价信息，通过教学资源画像形成更加合理的知识点属性，构建更加准确合理的教学知识图谱。与传统知识图谱构建和推荐方式相比，采用了多视图学习及深度学习的构建查询方式，根据资源不同的展现方式形成的多视图数据来设计神经网络模型，能够更好的考虑多样性，发掘知识点内部的潜在联系，更加准确合理的提取知识点；方法中加入了教学资源画像和学习者用户画像，在知识点提取和检索过程中，更加有针对性和侧重点，推荐的资源更能符合学习者的学习习惯，满足学习者 的个性化的需求，同时推荐结果包含多组数据，增加了推荐的准确性和容错性。另外，学习者根据推荐的学习资源进行学习并及时进行反馈，持续优化推荐模型。The method of this embodiment is based on massive teaching resource data CR, effectively utilizes deep learning feature extraction technology, fully considers the characteristics of Internet online learning, and explores the connections between multiple views such as textbooks, lesson plans, teaching videos, teaching voices, and speech scripts. , and combined with Internet resource evaluation information, more reasonable knowledge point attributes are formed through teaching resource portraits, and a more accurate and reasonable teaching knowledge map is constructed. Compared with traditional knowledge graph construction and recommendation methods, multi-view learning and deep learning are used to construct query methods. The neural network model is designed based on multi-view data formed by different presentation methods of resources, which can better consider diversity and explore Potential connections within the knowledge points enable a more accurate and reasonable extraction of knowledge points; teaching resource portraits and learner user portraits are added to the method, which makes the knowledge point extraction and retrieval process more targeted and focused, and the recommended resources are more effective Conform to learners’ learning habits and satisfy learners personalized needs, and the recommendation results contain multiple sets of data, which increases the accuracy and fault tolerance of recommendations. In addition, learners learn based on recommended learning resources and provide timely feedback to continuously optimize the recommendation model.
实施例2：Example 2:
本发明一种基于多视图学习的教学知识图谱构建及检索系统，包括模型构建模块、模型训练模块以及检索推荐模块，该系统基于实施例公开的方法为用户提供知识点和教学资源推荐服务。The present invention is a teaching knowledge graph construction and retrieval system based on multi-view learning, which includes a model construction module, a model training module and a retrieval recommendation module. The system provides users with knowledge points and teaching resource recommendation services based on the method disclosed in the embodiment.
模型构建模块用于构建图谱构建及检索模型，所述图谱构建及检索模型包括多视图特征提取器CR-Mutiview-Fet、教学资源画像模型CR-Profiler、知识图谱构建模型KG-Gen、用户画像模型Stu-Profiler以及检索推荐模型CR-Recommend。The model building module is used to build a graph construction and retrieval model, which includes a multi-view feature extractor CR-Mutiview-Fet, a teaching resource profile model CR-Profiler, a knowledge graph construction model KG-Gen, and a user profile model. Stu-Profiler and retrieval recommendation model CR-Recommend.
本实施例中采集的教学资源的数据源包括教材书籍、教案、教学视频、教学语音以及演讲稿，主要分为知识点视频V、知识点音频A、知识点胶片P和知识点教材L四种视图方式。The data sources of teaching resources collected in this embodiment include teaching materials and books, lesson plans, teaching videos, teaching voices and speech drafts, which are mainly divided into four types: knowledge point video V, knowledge point audio A, knowledge point film P and knowledge point teaching material L view mode.
多视图特征提取器CR-Mutiview-Fet负责知识点特征的提取，输出知识点本体结构及关联知识点EP-OR，针对上述四种视图采用四个不同的特征提取模型进行特征提取和特征融合，包括视频特征提取模块FV、音频特征提取模块FA、图像特征提取模块FP、文字特征提取模块FL和特征融合模块FF。视频特征提取模块FV核心是三维CNN卷积神经网络，负责提取视频的知识点语义特征；音频特征提取模块FA核心是CNN卷积神经网络，负责提取音频中的知识点语义特征；图像特征提取模块FP核心是卷积神经网络，负责提取图像的知识点语义特征；文字特征提取模块FL核心是基于BERT的预训练的语言模型，负责提取文字的知识点语义特征；特征融合模块FF将根据来自视频特征提取模块FV、音频特征提取模块FA、图像特征提取模块FP和文字特征提取模块FL的特征向量进行融合，获取知识点本体结构及关联知识点EP-OR。The multi-view feature extractor CR-Mutiview-Fet is responsible for extracting knowledge point features, outputting the knowledge point ontology structure and associated knowledge point EP-OR. Four different feature extraction models are used for feature extraction and feature fusion for the above four views. It includes video feature extraction module FV, audio feature extraction module FA, image feature extraction module FP, text feature extraction module FL and feature fusion module FF. The core of the video feature extraction module FV is a three-dimensional CNN convolutional neural network, which is responsible for extracting the semantic features of knowledge points in the video; the core of the audio feature extraction module FA is a CNN convolutional neural network, which is responsible for extracting the semantic features of the knowledge points in the audio; the image feature extraction module The FP core is a convolutional neural network, responsible for extracting the semantic features of knowledge points in images; the text feature extraction module FL core is a pre-trained language model based on BERT, responsible for extracting the semantic features of knowledge points in text; the feature fusion module FF will extract the semantic features of knowledge points from the video. The feature vectors of the feature extraction module FV, audio feature extraction module FA, image feature extraction module FP and text feature extraction module FL are fused to obtain the knowledge point ontology structure and associated knowledge point EP-OR.
教学资源画像模型CR-Profiler核心是CNN卷积神经网络模型，负责基于自互联网教学资源相关扩展数据CR-WWW和教学资源数据CR，为教学资源进行画像，得到教学资源基础属性CR-Basic(其包括知识点、内容结构、过程、原理、概念、工具等)和教学资源扩展属性CR-Ext(其包括互联网评价信息、展现形式等)。The core of the teaching resource profiling model CR-Profiler is the CNN convolutional neural network model, which is responsible for profiling the teaching resources based on the extended data CR-WWW and teaching resource data CR related to the Internet teaching resources, and obtaining the basic attributes of the teaching resources CR-Basic (which Including knowledge points, content structure, processes, principles, concepts, tools, etc.) and teaching resource extension attributes CR-Ext (which includes Internet evaluation information, presentation forms, etc.).
知识图谱构建模型KG-Gen其核心是神经网络模型，包括特征编码器Enc和生成网络模型GN，特征编码器用于对知识点进行编码，使得相似知识点之间的向量计算距离小，用于提供资源索引查询；生成网络模型用于生成教学知识 点知识图谱EP-KG。即该知识图谱构建模型KG-Gen基于知识点提取形成的知识点本体结构及关联知识点EP-OR，并结合教学资源画像CR-Profiler获取的教育资源属性，生成教学知识点知识图谱EP-KG。The core of the knowledge graph construction model KG-Gen is a neural network model, including the feature encoder Enc and the generative network model GN. The feature encoder is used to encode knowledge points so that the vector calculation distance between similar knowledge points is small and used to provide Resource index query; generative network model is used to generate teaching knowledge Point Knowledge Graph EP-KG. That is, the knowledge graph construction model KG-Gen is based on the knowledge point ontology structure and associated knowledge points EP-OR formed by knowledge point extraction, and combined with the educational resource attributes obtained by the teaching resource portrait CR-Profiler to generate the teaching knowledge point knowledge graph EP-KG .
用户画像模型Stu-Profiler是根据用户的用户信息Stu-Info(即学习者的个人信息及学习情况)为学习者进行画像，形成用户属性Stu-Prop(即其学习的知识点喜好、掌握程度以及综合能力等标签。)The user portrait model Stu-Profiler is based on the user's user information Stu-Info (that is, the learner's personal information and learning situation) to profile the learner to form the user attribute Stu-Prop (that is, the knowledge point preferences, mastery and Comprehensive ability and other labels.)
检索推荐模型CR-Recommend是基于构建的教学资源知识点知识图谱EP-KG构建索引并提供检索推荐功能，通过输入的检索内容(包括文字、语音、视频、图像等多种形式)，结合用户画像模型Stu-Profiler生成的用户属性Stu-Prop，输出多组知识点及推荐教学资源，作为学习者的用户可以进行选择。The retrieval recommendation model CR-Recommend is based on the constructed knowledge graph EP-KG of teaching resource knowledge points to build an index and provide retrieval recommendation functions. Through the input retrieval content (including text, voice, video, image, etc.), combined with user portraits The user attribute Stu-Prop generated by the model Stu-Profiler outputs multiple sets of knowledge points and recommended teaching resources, which users as learners can choose from.
模型训练模块用于依次对所述多视图特征提取器CR-Mutiview-Fet、教学资源画像模型CR-Profiler、知识图谱构建模型KG-Gen、用户画像模型Stu-Profiler以及检索推荐模型CR-Recommend进行模型训练，得到训练后的图谱构建及检索模型。The model training module is used to sequentially train the multi-view feature extractor CR-Mutiview-Fet, the teaching resource profile model CR-Profiler, the knowledge graph construction model KG-Gen, the user profile model Stu-Profiler and the retrieval recommendation model CR-Recommend. Model training to obtain the trained map construction and retrieval model.
作为具体实施，该模型训练模块用于通过如下步骤进行训练：As a specific implementation, the model training module is used to train through the following steps:
(1)从多个数据源获取教学资源数据CR，并根据其素材进行数据标注；(1) Obtain teaching resource data CR from multiple data sources, and perform data annotation based on its materials;
(2)基于所述教学资源数据CR，分别对所述多视图特征提取器CR-Mutiview-Fet中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型进行模型预训练；(2) Based on the teaching resource data CR, perform model pre-training on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor CR-Mutiview-Fet respectively;
(3)将所述多视图特征提取器CR-Mutiview-Fet中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型的模型参数固定，对所述特征融合模型进行模型训练；(3) Fix the model parameters of the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor CR-Mutiview-Fet, and perform model training on the feature fusion model ;
(4)将所述多视图特征提取器CR-Mutiview-Fet中视频特征提取模型、音频特征提取模型、图像特征提取模型、文字特征提取模型以及特征融合模型进行组合，通过所述教学资源数据CR对所述多视图特征提取器CR-Mutiview-Fet整体进行模型训练，对所述多视图特征提取器CR-Mutiview-Fet中每个模型进行参数微调；(4) Combine the video feature extraction model, audio feature extraction model, image feature extraction model, text feature extraction model and feature fusion model in the multi-view feature extractor CR-Mutiview-Fet, and use the teaching resource data CR Perform model training on the multi-view feature extractor CR-Mutiview-Fet as a whole, and perform parameter fine-tuning on each model in the multi-view feature extractor CR-Mutiview-Fet;
(5)收集教学资源互联网相关扩展数据CR-WWW，并进行标签标注；(5) Collect the Internet-related extended data CR-WWW of teaching resources and label them;
(6)基于所述教学资源数据CR、教学资源互联网相关扩展数据CR-WWW以及标签，对所述教学资源画像模型CR-Profiler进行模型训练；(6) Carry out model training on the teaching resource profile model CR-Profiler based on the teaching resource data CR, teaching resource Internet-related extended data CR-WWW and tags;
(7)基于所述多视图特征提取器CR-Mutiview-Fet输出的知识点本体结构及关联知识点EP-OR，以及所述教学资源画像模型CR-Profiler输出的教学资源 基础属性CR-Basic和教学资源扩展属性CR-Ext，对所述知识图谱构建模型KG-Gen进行模型训练；(7) Based on the ontology structure of knowledge points and associated knowledge points EP-OR output by the multi-view feature extractor CR-Mutiview-Fet, and the teaching resources output by the teaching resource profile model CR-Profiler The basic attribute CR-Basic and the teaching resource extended attribute CR-Ext are used to perform model training on the knowledge graph construction model KG-Gen;
(8)收集用户信息Stu-Info，并进行标签标注；(8) Collect user information Stu-Info and label it;
(9)基于用户信息Stu-Info和标签，对所述用户画像模型Stu-Profiler进行模型训练；(9) Based on the user information Stu-Info and tags, perform model training on the user profile model Stu-Profiler;
(10)基于多视图特征提取器CR-Mutiview-Fet中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型对所述检索推荐模型CR-Recommend进行轻量化剪裁，对所述检索推荐模型CR-Recommend中搜索条件提取模型进行训练；(10) The retrieval recommendation model CR-Recommend is lightweight and tailored based on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor CR-Mutiview-Fet. The search condition extraction model in the above-mentioned retrieval recommendation model CR-Recommend is trained;
(11)基于所述用户画像模型Stu-Profiler输出的用户属性Stu-Prop以及所述知识图谱构建模型KG-Gen输出的教学知识点知识图谱EP-KG，对所述检索推荐模型CR-Recommend进行模型训练。(11) Based on the user attribute Stu-Prop output by the user portrait model Stu-Profiler and the teaching knowledge point knowledge graph EP-KG output by the knowledge graph construction model KG-Gen, perform the retrieval recommendation model CR-Recommend Model training.
检索推荐模块用于通过所述训练后的图谱构建及检索模型构建教学知识点知识图谱EP-KG，基于教学知识点知识图谱EP-KG构建索引并提供检索推荐服务，为用户输出多组知识点及推荐资源以供用户选择。The retrieval and recommendation module is used to construct a knowledge graph EP-KG of teaching knowledge points through the trained graph construction and retrieval model, build an index based on the knowledge graph EP-KG of teaching knowledge points and provide retrieval recommendation services, and output multiple sets of knowledge points for users. and recommend resources for users to choose from.
作为具体实施，检索推荐模块用于通过如下步骤构建教学知识点知识图谱EP-KG，并为用户输出多组知识点及推荐资源以供用户选择：As a specific implementation, the retrieval and recommendation module is used to construct the knowledge graph EP-KG of teaching knowledge points through the following steps, and output multiple sets of knowledge points and recommended resources for users to choose:
(1)选定领域，收集选定领域的教学资源数据CR；(1) Select a field and collect teaching resource data CR in the selected field;
(2)通过训练后的多视图特征提取器CR-Mutiview-Fet对所述教学资源进行特征提取和特征融合，得到知识点本体结构及关联知识点EP-OR；(2) Use the trained multi-view feature extractor CR-Mutiview-Fet to perform feature extraction and feature fusion on the teaching resources to obtain the knowledge point ontology structure and associated knowledge point EP-OR;
(3)收集教学资源互联网相关扩展数据CR-WWW，基于所述教学资源互联网相关扩展数据CR-WWW以及教学资源数据CR，通过训练后的教学资源画像模型CR-Profiler对教学资源进行画像，得到教学资源属性；(3) Collect the Internet-related extended data CR-WWW of teaching resources. Based on the Internet-related extended data CR-WWW of teaching resources and the teaching resource data CR, profile the teaching resources through the trained teaching resource profile model CR-Profiler, and obtain Teaching resource attributes;
(4)基于知识点本体结构及关联知识点EP-OR、教学资源属性，通过训练后的知识图谱构建模型KG-Gen对知识点进行编码，使得相似知识点资源之间的向量计算距离小，用于资源索引查询，并生成教学知识点知识图谱EP-KG；(4) Based on the knowledge point ontology structure and associated knowledge point EP-OR and teaching resource attributes, the trained knowledge graph is used to construct the model KG-Gen to encode the knowledge points, so that the vector calculation distance between similar knowledge point resources is small. Used for resource index query and generation of knowledge graph EP-KG of teaching knowledge points;
(5)输入检索内容，所述检索内容的视图方式包括文字、语音、图像和视频；(5) Enter the search content, and the view mode of the search content includes text, voice, image and video;
(6)获取用户的用户信息Stu-Info，通过训练后的用户画像模型Stu-Profiler对用户进行画像，生成用户属性Stu-Prop； (6) Obtain the user's user information Stu-Info, profile the user through the trained user profile model Stu-Profiler, and generate user attributes Stu-Prop;
(7)基于教学知识点知识图谱EP-KG、通过训练后的检索推荐模型CR-Recommend构建索引并提供检索推荐服务；(7) Build an index based on the teaching knowledge point knowledge graph EP-KG and the trained retrieval recommendation model CR-Recommend and provide retrieval recommendation services;
(8)基于索引以及检索推荐服务，对用户输入的检索内容进行知识点提取、并结合用户属性Stu-Prop形成知识点特征向量，并进行知识点查询和特征向量计算，输出多组知识点和推荐资源，以供用户进行选择；(8) Based on indexing and retrieval recommendation services, extract knowledge points from the retrieval content input by the user, form a knowledge point feature vector based on the user attribute Stu-Prop, perform knowledge point query and feature vector calculation, and output multiple sets of knowledge point sums Recommend resources for users to choose from;
(9)将收集的教学资源数据CR、互联网相关扩展数据、用户信息Stu-Info以及检索内容，以及输出的多组知识点和推荐资源，反馈至所述图谱构建及检索模型，对所述图谱构建及检索模型进行模型训练，以持续优化所述图谱构建及检索模型。(9) Feed back the collected teaching resource data CR, Internet-related extended data, user information Stu-Info and search content, as well as the output sets of knowledge points and recommended resources to the map construction and retrieval model, and analyze the map Build and retrieve models and conduct model training to continuously optimize the map construction and retrieval models.
上文通过附图和优选实施例对本发明进行了详细展示和说明，然而本发明不限于这些已揭示的实施例，基与上述多个实施例本领域技术人员可以知晓，可以组合上述不同实施例中的代码审核手段得到本发明更多的实施例，这些实施例也在本发明的保护范围之内。 The present invention has been shown and described in detail through the drawings and preferred embodiments above. However, the present invention is not limited to these disclosed embodiments. Based on the above-mentioned multiple embodiments, those skilled in the art will know that the above-mentioned different embodiments can be combined. The code review means in the method can lead to more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims (10)

  1. 一种基于多视图学习的教学知识图谱构建及检索方法，其特征在于包括如下步骤：A teaching knowledge graph construction and retrieval method based on multi-view learning, which is characterized by including the following steps:
    构建多视图特征提取器，所述多特征视图提取器用于对多种视图方式的教学资源数据进行特征提取和特征融合，得到知识点本体结构及关联知识点；Construct a multi-view feature extractor, which is used to perform feature extraction and feature fusion on teaching resource data in multiple view modes to obtain the knowledge point ontology structure and associated knowledge points;
    基于卷积神经网络构建教学资源画像模型，所述教学资源画像模型以教学资源数据以及教学资源互联网相关扩展数据为输入，对教学资源进行画像，输出教学资源属性，所述教学资源属性包括教学资源基础属性和教学资源扩展属性；A teaching resource portrait model is constructed based on the convolutional neural network. The teaching resource portrait model takes teaching resource data and teaching resource Internet-related extended data as inputs to profile the teaching resources and outputs teaching resource attributes. The teaching resource attributes include teaching resources. Basic attributes and extended attributes of teaching resources;
    基于神经网络构建知识图谱构建模型，所述知识图谱构建模型以基于知识点本体结构及关联知识点以及教学资源属性为输入，生成教学知识点知识图谱；Build a knowledge graph construction model based on a neural network. The knowledge graph construction model uses the ontology structure of knowledge points, associated knowledge points and teaching resource attributes as input to generate a knowledge graph of teaching knowledge points;
    构建用户画像模型，所述用户画像模型以用户信息为输入，对用户进行画像，输出用户属性；Construct a user portrait model, which takes user information as input, profiles the user, and outputs user attributes;
    构建检索推荐模型，所述检索推荐模型用于基于教学知识点知识图谱构建索引并提供检索推荐服务，用于通过输入的检索内容、并结合用户属性，输出多组知识点及推荐资源以供用户选择；Construct a retrieval recommendation model. The retrieval recommendation model is used to build an index based on the knowledge graph of teaching knowledge points and provide retrieval recommendation services. It is used to output multiple sets of knowledge points and recommended resources for users through the input retrieval content and combined with user attributes. choose;
    基于所述多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型构建图谱构建及检索模型，并依次对所述多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型进行模型训练，得到训练后的图谱构建及检索模型；Based on the multi-view feature extractor, teaching resource portrait model, knowledge graph construction model, user portrait model and retrieval recommendation model, a graph construction and retrieval model is constructed, and the multi-view feature extractor, teaching resource portrait model, knowledge graph are constructed in sequence. Carry out model training on the graph construction model, user portrait model and retrieval recommendation model, and obtain the trained graph construction and retrieval model;
    通过所述训练后的图谱构建及检索模型构建教学知识点知识图谱，基于教学知识点知识图谱构建索引并提供检索推荐服务，为用户输出多组知识点及推荐资源以供用户选择。A knowledge graph of teaching knowledge points is constructed through the trained graph construction and retrieval model, an index is constructed based on the knowledge graph of teaching knowledge points and retrieval recommendation services are provided, and multiple sets of knowledge points and recommended resources are output to users for selection.
  2. 根据权利要求1所述的基于多视图学习的教学知识图谱构建及检索方法，其特征在于所述教学资源的数据源包括教材书籍、教案、教学视频、教学语音以及演讲稿，所述教学资源的视图方式共四种，分别为视频、音频、图像和文字；The teaching knowledge graph construction and retrieval method based on multi-view learning according to claim 1, characterized in that the data sources of the teaching resources include teaching materials and books, lesson plans, teaching videos, teaching voices and speech scripts, and the data sources of the teaching resources include There are four view modes, namely video, audio, image and text;
    所述教学资源基础属性包括知识点、内容结构、过程、原理、概念、工具；The basic attributes of the teaching resources include knowledge points, content structure, processes, principles, concepts, and tools;
    所述教学资源扩展属性包括互联网评价信息和展现形式； The extended attributes of the teaching resources include Internet evaluation information and presentation forms;
    所述检索内容的视图形式包括文字、语音、视频和图像；The view form of the retrieval content includes text, voice, video and image;
    所述教学知识点知识图谱包括语义网络、教学资源基础数据和教学资源扩展属性，所述语义网络为基于知识点本体结构及知识点关联形成的；The knowledge map of teaching knowledge points includes a semantic network, basic data of teaching resources and extended attributes of teaching resources. The semantic network is formed based on the ontology structure of knowledge points and the association of knowledge points;
    所述用户信息包括基础信息和学习情况；The user information includes basic information and learning status;
    所述用户属性包括知识点喜好、掌握程度以及综合能力。The user attributes include knowledge point preferences, mastery level and comprehensive ability.
  3. 根据权利要求2所述的基于多视图学习的教学知识图谱构建及检索方法，其特征在于所述多视图特征提取器包括：The teaching knowledge graph construction and retrieval method based on multi-view learning according to claim 2, characterized in that the multi-view feature extractor includes:
    视频特征提取模型，所述视频特征提取模型为基于三维CNN卷积神经网络构建的网络模型，用于从视频方式的教学资源数据提取知识点语义特征；A video feature extraction model. The video feature extraction model is a network model built based on a three-dimensional CNN convolutional neural network and is used to extract semantic features of knowledge points from video teaching resource data;
    音频特征提取模型，所述音频特征提取模型为基于CNN卷积神经网络构建的网络模型，用于从音频方式的教学资源数据中提取知识点语义特征；Audio feature extraction model. The audio feature extraction model is a network model built based on a CNN convolutional neural network and is used to extract semantic features of knowledge points from audio-based teaching resource data;
    图像特征提取模型，所述图像特征提取模型为基于卷积神经网络构建的网路模型，用于从图像方式的教学资源数据中提取知识点语义特征；Image feature extraction model. The image feature extraction model is a network model built based on a convolutional neural network and is used to extract semantic features of knowledge points from image-based teaching resource data;
    文字特征提取模型，所述文字特征提取模型为基于BERT的语言模型，用于从文字方式的教学资源数据中提取知识点语义特征；Text feature extraction model. The text feature extraction model is a language model based on BERT, which is used to extract semantic features of knowledge points from text-based teaching resource data;
    特征融合模型，所述特征融合模型用于将所述视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型输出的知识点语义特征进行融合，得到知识点本体结构及关联知识点；Feature fusion model, the feature fusion model is used to fuse the semantic features of knowledge points output by the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model to obtain the knowledge point ontology structure and associated knowledge point;
    所述知识图谱构建模型包括：The knowledge graph construction model includes:
    特征编码器，所述特征编码器用于对知识点进行编码，使得相似知识点之间的向量计算距离小，用于提供资源索引查询；Feature encoder, which is used to encode knowledge points so that the vector calculation distance between similar knowledge points is small and used to provide resource index queries;
    生成网络模型，所述生成网络模型用于生成教学知识点知识图谱。Generate a network model, which is used to generate a knowledge graph of teaching knowledge points.
  4. 根据权利要求3所述的基于多视图学习的教学知识图谱构建及检索方法，其特征在于依次对所述多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型进行模型训练，包括如下步骤：The teaching knowledge graph construction and retrieval method based on multi-view learning according to claim 3, characterized in that the multi-view feature extractor, teaching resource portrait model, knowledge graph construction model, user portrait model and retrieval recommendation model are sequentially Carry out model training, including the following steps:
    从多个数据源获取教学资源数据，并根据其素材进行数据标注；Obtain teaching resource data from multiple data sources and annotate the data based on its materials;
    基于所述教学资源数据，分别对所述多视图特征提取器中视频特征提取 模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型进行模型预训练；Based on the teaching resource data, video features are extracted in the multi-view feature extractor model, audio feature extraction model, image feature extraction model and text feature extraction model for model pre-training;
    将所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型的模型参数固定，对所述特征融合模型进行模型训练；Fix the model parameters of the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor, and perform model training on the feature fusion model;
    将所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型、文字特征提取模型以及特征融合模型进行组合，通过所述教学资源数据对所述多视图特征提取器整体进行模型训练，对所述多视图特征提取器中每个模型进行参数微调；The video feature extraction model, audio feature extraction model, image feature extraction model, text feature extraction model and feature fusion model in the multi-view feature extractor are combined, and the entire multi-view feature extractor is processed through the teaching resource data. Carry out model training and fine-tune parameters of each model in the multi-view feature extractor;
    收集教学资源互联网相关扩展数据，并进行标签标注；Collect Internet-related extended data on teaching resources and label them;
    基于所述教学资源数据、教学资源互联网相关扩展数据以及标签，对所述教学资源画像模型进行模型训练；Carry out model training on the teaching resource portrait model based on the teaching resource data, teaching resource Internet-related extended data and tags;
    基于所述多视图特征提取器输出的知识点本体结构及关联知识点，以及所述教学资源画像模型输出的教学资源基础属性和教学资源扩展属性，对所述知识图谱构建模型进行模型训练；Based on the knowledge point ontology structure and associated knowledge points output by the multi-view feature extractor, as well as the teaching resource basic attributes and teaching resource extended attributes output by the teaching resource portrait model, perform model training on the knowledge graph construction model;
    收集用户信息，并进行标签标注；Collect user information and label it;
    基于用户信息和标签，对所述用户画像模型进行模型训练；Based on user information and tags, perform model training on the user portrait model;
    基于多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型对所述检索推荐模型进行轻量化剪裁，对所述检索推荐模型中搜索条件提取模型进行训练；The retrieval recommendation model is lightweight and tailored based on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor, and the search condition extraction model in the retrieval recommendation model is trained. ;
    基于所述用户画像模型输出的用户属性以及所述知识图谱构建模型输出的教学知识点知识图谱，对所述检索推荐模型进行模型训练。Based on the user attributes output by the user portrait model and the teaching knowledge point knowledge graph output by the knowledge graph construction model, model training is performed on the retrieval recommendation model.
  5. 根据权利要求3所述的基于多视图学习的教学知识图谱构建及检索方法，其特征在于通过所述训练后的图谱构建及检索模型构建教学知识点知识图谱，包括如下步骤：The teaching knowledge graph construction and retrieval method based on multi-view learning according to claim 3, characterized in that the teaching knowledge point knowledge graph is constructed through the trained graph construction and retrieval model, including the following steps:
    选定领域，收集选定领域的教学资源数据；Select a field and collect teaching resource data in the selected field;
    通过训练后的多视图特征提取器对所述教学资源进行特征提取和特征融合，得到知识点本体结构及关联知识点；Use the trained multi-view feature extractor to perform feature extraction and feature fusion on the teaching resources to obtain the knowledge point ontology structure and associated knowledge points;
    收集教学资源互联网相关扩展数据，基于所述教学资源互联网相关扩展数据以及教学资源数据，通过训练后的教学资源画像模型对教学资源进行画像，得到教学资源属性； Collect the Internet-related extended data of teaching resources, and based on the Internet-related extended data of teaching resources and the teaching resource data, profile the teaching resources through the trained teaching resource portrait model to obtain the teaching resource attributes;
    基于知识点本体结构及关联知识点、教学资源属性，通过训练后的知识图谱构建模型对知识点进行编码，使得相似知识点资源之间的向量计算距离小，用于资源索引查询，并生成教学知识点知识图谱；Based on the knowledge point ontology structure, associated knowledge points, and teaching resource attributes, the knowledge points are encoded through the trained knowledge graph construction model, so that the vector calculation distance between similar knowledge point resources is small, which can be used for resource index query and generate teaching Knowledge point knowledge map;
    基于教学知识点知识图谱构建索引并提供检索推荐服务，为用户输出多组知识点及推荐资源以供用户选择，包括如下步骤：Build an index based on the knowledge graph of teaching knowledge points and provide retrieval and recommendation services, and output multiple sets of knowledge points and recommended resources for users to choose, including the following steps:
    输入检索内容，所述检索内容的视图方式包括文字、语音、图像和视频；Enter the search content, and the view mode of the search content includes text, voice, image and video;
    获取用户的用户信息，通过训练后的用户画像模型对用户进行画像，生成用户属性；Obtain the user's user information, profile the user through the trained user portrait model, and generate user attributes;
    基于教学知识点知识图谱、通过训练后的检索推荐模型构建索引并提供检索推荐服务；Based on the knowledge graph of teaching knowledge points, build an index through the trained retrieval recommendation model and provide retrieval recommendation services;
    基于索引以及检索推荐服务，对用户输入的检索内容进行知识点提取、并结合用户属性形成知识点特征向量，并进行知识点查询和特征向量计算，输出多组知识点和推荐资源，以供用户进行选择；Based on indexing and retrieval recommendation services, knowledge points are extracted from the retrieval content input by users, combined with user attributes to form knowledge point feature vectors, and knowledge point queries and feature vector calculations are performed to output multiple sets of knowledge points and recommended resources for users make a choice;
    将收集的教学资源数据、互联网相关扩展数据、用户信息以及检索内容，以及输出的多组知识点和推荐资源，反馈至所述图谱构建及检索模型，对所述图谱构建及检索模型进行模型训练，以持续优化所述图谱构建及检索模型。The collected teaching resource data, Internet-related extended data, user information and search content, as well as the output sets of knowledge points and recommended resources, are fed back to the map construction and retrieval model, and model training is performed on the map construction and retrieval model. , to continuously optimize the map construction and retrieval model.
  6. 一种基于多视图学习的教学知识图谱构建及检索系统，其特征在于用于通过如权利要求1-5任一项所述的一种基于多视图学习的教学知识图谱构建及检索方法为用户提供知识点和教学资源推荐服务，所述系统包括：A teaching knowledge graph construction and retrieval system based on multi-view learning, which is characterized in that it is used to provide users with a teaching knowledge graph construction and retrieval method based on multi-view learning as described in any one of claims 1-5. Knowledge points and teaching resource recommendation services, the system includes:
    模型构建模块，所述模型构建模块用于构建图谱构建及检索模型，所述图谱构建及检索模型包括多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型，所述多特征视图提取器用于对多种视图方式的教学资源数据进行特征提取和特征融合，得到知识点本体结构及关联知识点；所述教学资源画像模型为基于卷积神经网络构建的网络模型，以教学资源数据以及教学资源互联网相关扩展数据为输入，对教学资源进行画像，输出教学资源属性，所述教学资源属性包括教学资源基础属性和教学资源扩展属性；所述知识图谱构建模型以基于知识点本体结构及关联知识点以及教学资源属性为输入，生成教学知识点知识图谱；所述用户画像模 型以用户信息为输入，对用户进行画像，输出用户属性；所述检索推荐模型用于基于教学知识点知识图谱构建索引并提供检索推荐服务，用于通过输入的检索内容、并结合用户属性，输出多组知识点及推荐资源以供用户选择；A model building module, which is used to build a graph construction and retrieval model. The graph construction and retrieval model includes a multi-view feature extractor, a teaching resource portrait model, a knowledge graph construction model, a user portrait model and a retrieval recommendation model, The multi-feature view extractor is used to perform feature extraction and feature fusion on teaching resource data in multiple view modes to obtain the knowledge point ontology structure and associated knowledge points; the teaching resource portrait model is a network model based on a convolutional neural network. , using teaching resource data and teaching resource Internet-related extended data as input, the teaching resources are profiled, and the teaching resource attributes are output. The teaching resource attributes include basic teaching resource attributes and teaching resource extended attributes; the knowledge graph construction model is based on The ontology structure of knowledge points, associated knowledge points and teaching resource attributes are used as input to generate a knowledge graph of teaching knowledge points; the user portrait model The model takes user information as input, profiles the user, and outputs user attributes; the retrieval recommendation model is used to build an index based on the knowledge graph of teaching knowledge points and provide retrieval recommendation services, and is used to combine the input retrieval content with user attributes. Output multiple sets of knowledge points and recommended resources for users to choose;
    模型训练模块，所述模型训练模块用于依次对所述多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型进行模型训练，得到训练后的图谱构建及检索模型；A model training module, which is used to sequentially perform model training on the multi-view feature extractor, teaching resource portrait model, knowledge graph construction model, user portrait model and retrieval recommendation model, and obtain the trained graph construction and retrieval Model;
    检索推荐模块，所述检索推荐模块用于通过所述训练后的图谱构建及检索模型构建教学知识点知识图谱，基于教学知识点知识图谱构建索引并提供检索推荐服务，为用户输出多组知识点及推荐资源以供用户选择。A retrieval recommendation module. The retrieval recommendation module is used to construct a knowledge graph of teaching knowledge points through the trained graph construction and retrieval model, build an index based on the knowledge graph of teaching knowledge points and provide retrieval recommendation services, and output multiple sets of knowledge points for users. and recommend resources for users to choose from.
  7. 根据权利要求6所述的基于多视图学习的教学知识图谱构建及检索系统，其特征在于所述教学资源的数据源包括教材书籍、教案、教学视频、教学语音以及演讲稿，所述教学资源的视图方式共四种，分别为视频、音频、图像和文字；The teaching knowledge graph construction and retrieval system based on multi-view learning according to claim 6, characterized in that the data sources of the teaching resources include teaching materials and books, lesson plans, teaching videos, teaching voices and speech scripts, and the data sources of the teaching resources include There are four view modes, namely video, audio, image and text;
    所述教学资源基础属性包括知识点、内容结构、过程、原理、概念、工具；The basic attributes of the teaching resources include knowledge points, content structure, processes, principles, concepts, and tools;
    所述教学资源扩展属性包括互联网评价信息和展现形式；The extended attributes of the teaching resources include Internet evaluation information and presentation forms;
    所述检索内容的视图形式包括文字、语音、视频和图像；The view form of the retrieval content includes text, voice, video and image;
    所述教学知识点知识图谱包括语义网络、教学资源基础数据和教学资源扩展属性，所述语义网络为基于知识点本体结构及知识点关联形成的；The knowledge map of teaching knowledge points includes a semantic network, basic data of teaching resources and extended attributes of teaching resources. The semantic network is formed based on the ontology structure of knowledge points and the association of knowledge points;
    所述用户信息包括基础信息和学习情况；The user information includes basic information and learning status;
    所述用户属性包括知识点喜好、掌握程度以及综合能力。The user attributes include knowledge point preferences, mastery level and comprehensive ability.
  8. 根据权利要求7所述的基于多视图学习的教学知识图谱构建及检索系统，其特征在于所述多视图特征提取器包括：The teaching knowledge graph construction and retrieval system based on multi-view learning according to claim 7, characterized in that the multi-view feature extractor includes:
    视频特征提取模型，所述视频特征提取模型为基于三维CNN卷积神经网络构建的网络模型，用于从视频方式的教学资源数据提取知识点语义特征；A video feature extraction model. The video feature extraction model is a network model built based on a three-dimensional CNN convolutional neural network and is used to extract semantic features of knowledge points from video teaching resource data;
    音频特征提取模型，所述音频特征提取模型为基于CNN卷积神经网络构建的网络模型，用于从音频方式的教学资源数据中提取知识点语义特征；Audio feature extraction model. The audio feature extraction model is a network model built based on a CNN convolutional neural network and is used to extract semantic features of knowledge points from audio-based teaching resource data;
    图像特征提取模型，所述图像特征提取模型为基于卷积神经网络构建的网路模型，用于从图像方式的教学资源数据中提取知识点语义特征；Image feature extraction model. The image feature extraction model is a network model built based on a convolutional neural network and is used to extract semantic features of knowledge points from image-based teaching resource data;
    文字特征提取模型，所述文字特征提取模型为基于BERT的语言模型， 用于从文字方式的教学资源数据中提取知识点语义特征；Text feature extraction model, the text feature extraction model is a language model based on BERT, Used to extract semantic features of knowledge points from text-based teaching resource data;
    特征融合模型，所述特征融合模型用于将所述视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型输出的知识点语义特征进行融合，得到知识点本体结构及关联知识点；Feature fusion model, the feature fusion model is used to fuse the semantic features of knowledge points output by the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model to obtain the knowledge point ontology structure and associated knowledge point;
    所述知识图谱构建模型包括：The knowledge graph construction model includes:
    特征编码器，所述特征编码器用于对知识点进行编码，使得相似知识点之间的向量计算距离小，用于提供资源索引查询；Feature encoder, which is used to encode knowledge points so that the vector calculation distance between similar knowledge points is small and used to provide resource index queries;
    生成网络模型，所述生成网络模型用于生成教学知识点知识图谱。Generate a network model, which is used to generate a knowledge graph of teaching knowledge points.
  9. 根据权利要求8所述的基于多视图学习的教学知识图谱构建及检索系统，其特征在于所述模型序列模块用于通过如下步骤依次对所述多视图特征提取器、教学资源画像模型、知识图谱构建模型、用户画像模型以及检索推荐模型进行模型训练：The teaching knowledge graph construction and retrieval system based on multi-view learning according to claim 8, characterized in that the model sequence module is used to sequentially perform the following steps on the multi-view feature extractor, the teaching resource portrait model, and the knowledge graph. Build models, user portrait models and retrieval recommendation models for model training:
    从多个数据源获取教学资源数据，并根据其素材进行数据标注；Obtain teaching resource data from multiple data sources and annotate the data based on its materials;
    基于所述教学资源数据，分别对所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型进行模型预训练；Based on the teaching resource data, perform model pre-training on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor;
    将所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型的模型参数固定，对所述特征融合模型进行模型训练；Fix the model parameters of the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor, and perform model training on the feature fusion model;
    将所述多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型、文字特征提取模型以及特征融合模型进行组合，通过所述教学资源数据对所述多视图特征提取器整体进行模型训练，对所述多视图特征提取器中每个模型进行参数微调；The video feature extraction model, audio feature extraction model, image feature extraction model, text feature extraction model and feature fusion model in the multi-view feature extractor are combined, and the entire multi-view feature extractor is processed through the teaching resource data. Carry out model training and fine-tune parameters of each model in the multi-view feature extractor;
    收集教学资源互联网相关扩展数据，并进行标签标注；Collect Internet-related extended data on teaching resources and label them;
    基于所述教学资源数据、教学资源互联网相关扩展数据以及标签，对所述教学资源画像模型进行模型训练；Carry out model training on the teaching resource portrait model based on the teaching resource data, teaching resource Internet-related extended data and tags;
    基于所述多视图特征提取器输出的知识点本体结构及关联知识点，以及所述教学资源画像模型输出的教学资源基础属性和教学资源扩展属性，对所述知识图谱构建模型进行模型训练；Based on the knowledge point ontology structure and associated knowledge points output by the multi-view feature extractor, as well as the teaching resource basic attributes and teaching resource extended attributes output by the teaching resource portrait model, perform model training on the knowledge graph construction model;
    收集用户信息，并进行标签标注； Collect user information and label it;
    基于用户信息和标签，对所述用户画像模型进行模型训练；Based on user information and tags, perform model training on the user portrait model;
    基于多视图特征提取器中视频特征提取模型、音频特征提取模型、图像特征提取模型以及文字特征提取模型对所述检索推荐模型进行轻量化剪裁，对所述检索推荐模型中搜索条件提取模型进行训练；The retrieval recommendation model is lightweight and tailored based on the video feature extraction model, audio feature extraction model, image feature extraction model and text feature extraction model in the multi-view feature extractor, and the search condition extraction model in the retrieval recommendation model is trained. ;
    基于所述用户画像模型输出的用户属性以及所述知识图谱构建模型输出的教学知识点知识图谱，对所述检索推荐模型进行模型训练。Based on the user attributes output by the user portrait model and the teaching knowledge point knowledge graph output by the knowledge graph construction model, model training is performed on the retrieval recommendation model.
  10. 根据权利要求9所述的基于多视图学习的教学知识图谱构建及检索系统，其特征在于所述检索推荐模块用于通过如下步骤构建教学知识点知识图谱，并为用户输出多组知识点及推荐资源以供用户选择：The teaching knowledge graph construction and retrieval system based on multi-view learning according to claim 9, characterized in that the retrieval recommendation module is used to construct a knowledge graph of teaching knowledge points through the following steps, and output multiple sets of knowledge points and recommendations for users Resources for users to choose from:
    选定领域，收集选定领域的教学资源数据；Select a field and collect teaching resource data in the selected field;
    通过训练后的多视图特征提取器对所述教学资源进行特征提取和特征融合，得到知识点本体结构及关联知识点；Use the trained multi-view feature extractor to perform feature extraction and feature fusion on the teaching resources to obtain the knowledge point ontology structure and associated knowledge points;
    收集教学资源互联网相关扩展数据，基于所述教学资源互联网相关扩展数据以及教学资源数据，通过训练后的教学资源画像模型对教学资源进行画像，得到教学资源属性；Collect the Internet-related extended data of teaching resources, and based on the Internet-related extended data of teaching resources and the teaching resource data, profile the teaching resources through the trained teaching resource portrait model to obtain the teaching resource attributes;
    基于知识点本体结构及关联知识点、教学资源属性，通过训练后的知识图谱构建模型对知识点进行编码，使得相似知识点资源之间的向量计算距离小，用于资源索引查询，并生成教学知识点知识图谱；Based on the knowledge point ontology structure, associated knowledge points, and teaching resource attributes, the knowledge points are encoded through the trained knowledge graph construction model, so that the vector calculation distance between similar knowledge point resources is small, which can be used for resource index query and generate teaching Knowledge point knowledge map;
    输入检索内容，所述检索内容的视图方式包括文字、语音、图像和视频；Enter the search content, and the view mode of the search content includes text, voice, image and video;
    获取用户的用户信息，通过训练后的用户画像模型对用户进行画像，生成用户属性；Obtain the user's user information, profile the user through the trained user portrait model, and generate user attributes;
    基于教学知识点知识图谱、通过训练后的检索推荐模型构建索引并提供检索推荐服务；Based on the knowledge graph of teaching knowledge points, build an index through the trained retrieval recommendation model and provide retrieval recommendation services;
    基于索引以及检索推荐服务，对用户输入的检索内容进行知识点提取、并结合用户属性形成知识点特征向量，并进行知识点查询和特征向量计算，输出多组知识点和推荐资源，以供用户进行选择；Based on indexing and retrieval recommendation services, knowledge points are extracted from the retrieval content input by the user, combined with user attributes to form knowledge point feature vectors, knowledge point queries and feature vector calculations are performed, and multiple sets of knowledge points and recommended resources are output for users make a choice;
    将收集的教学资源数据、互联网相关扩展数据、用户信息以及检索内容，以及输出的多组知识点和推荐资源，反馈至所述图谱构建及检索模型，对所述图谱构建及检索模型进行模型训练，以持续优化所述图谱构建及检索模型。 The collected teaching resource data, Internet-related extended data, user information and search content, as well as the output sets of knowledge points and recommended resources, are fed back to the map construction and retrieval model, and model training is performed on the map construction and retrieval model. , to continuously optimize the map construction and retrieval model.
PCT/CN2023/082103 2022-07-22 2023-03-17 Multiview learning-based teaching knowledge graph construction and retrieval method and system WO2024016695A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210873011.5A CN115292513A (en) 2022-07-22 2022-07-22 Teaching knowledge graph construction and retrieval method and system based on multi-view learning
CN202210873011.5 2022-07-22

Publications (1)

Publication Number Publication Date
WO2024016695A1 true WO2024016695A1 (en) 2024-01-25

Family

ID=83824281

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/082103 WO2024016695A1 (en) 2022-07-22 2023-03-17 Multiview learning-based teaching knowledge graph construction and retrieval method and system

Country Status (2)

Country Link
CN (1) CN115292513A (en)
WO (1) WO2024016695A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117633254A (en) * 2024-01-26 2024-03-01 武汉大学 Knowledge-graph-based map retrieval user portrait construction method and system
CN117672027A (en) * 2024-02-01 2024-03-08 青岛培诺教育科技股份有限公司 VR teaching method, device, equipment and medium
CN117787402A (en) * 2024-02-28 2024-03-29 徐州医科大学 Personalized learning path generation method and system based on multi-course knowledge graph fusion
CN117952072A (en) * 2024-03-22 2024-04-30 浙江蓝鸽科技有限公司 AI (advanced technology attachment) correction English teaching material editing method and system

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115292513A (en) * 2022-07-22 2022-11-04 山东浪潮科学研究院有限公司 Teaching knowledge graph construction and retrieval method and system based on multi-view learning
CN116010636B (en) * 2022-12-01 2023-08-11 广东工业大学 Retrieval pushing method based on art image label and application thereof

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107943998A (en) * 2017-12-05 2018-04-20 竹间智能科技（上海）有限公司 A kind of human-machine conversation control system and method for knowledge based collection of illustrative plates
CN109697233A (en) * 2018-12-03 2019-04-30 中电科大数据研究院有限公司 A kind of knowledge mapping system building method
CN111753098A (en) * 2020-06-23 2020-10-09 陕西师范大学 Teaching method and system based on cross-media dynamic knowledge graph
CN111831914A (en) * 2020-07-22 2020-10-27 上海掌学教育科技有限公司 Intelligent question pushing system for online education
CN112200317A (en) * 2020-09-28 2021-01-08 西南电子技术研究所（中国电子科技集团公司第十研究所） Multi-modal knowledge graph construction method
CN114372155A (en) * 2022-01-11 2022-04-19 湖南科技职业学院 Personalized learning platform based on self-expansion knowledge base and multi-mode portrait
CN114385821A (en) * 2020-10-21 2022-04-22 腾讯科技（深圳）有限公司 Resource retrieval method and device, storage medium and electronic equipment
CN115292513A (en) * 2022-07-22 2022-11-04 山东浪潮科学研究院有限公司 Teaching knowledge graph construction and retrieval method and system based on multi-view learning

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107943998A (en) * 2017-12-05 2018-04-20 竹间智能科技（上海）有限公司 A kind of human-machine conversation control system and method for knowledge based collection of illustrative plates
CN109697233A (en) * 2018-12-03 2019-04-30 中电科大数据研究院有限公司 A kind of knowledge mapping system building method
CN111753098A (en) * 2020-06-23 2020-10-09 陕西师范大学 Teaching method and system based on cross-media dynamic knowledge graph
CN111831914A (en) * 2020-07-22 2020-10-27 上海掌学教育科技有限公司 Intelligent question pushing system for online education
CN112200317A (en) * 2020-09-28 2021-01-08 西南电子技术研究所（中国电子科技集团公司第十研究所） Multi-modal knowledge graph construction method
CN114385821A (en) * 2020-10-21 2022-04-22 腾讯科技（深圳）有限公司 Resource retrieval method and device, storage medium and electronic equipment
CN114372155A (en) * 2022-01-11 2022-04-19 湖南科技职业学院 Personalized learning platform based on self-expansion knowledge base and multi-mode portrait
CN115292513A (en) * 2022-07-22 2022-11-04 山东浪潮科学研究院有限公司 Teaching knowledge graph construction and retrieval method and system based on multi-view learning

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117633254A (en) * 2024-01-26 2024-03-01 武汉大学 Knowledge-graph-based map retrieval user portrait construction method and system
CN117633254B (en) * 2024-01-26 2024-04-05 武汉大学 Knowledge-graph-based map retrieval user portrait construction method and system
CN117672027A (en) * 2024-02-01 2024-03-08 青岛培诺教育科技股份有限公司 VR teaching method, device, equipment and medium
CN117672027B (en) * 2024-02-01 2024-04-30 青岛培诺教育科技股份有限公司 VR teaching method, device, equipment and medium
CN117787402A (en) * 2024-02-28 2024-03-29 徐州医科大学 Personalized learning path generation method and system based on multi-course knowledge graph fusion
CN117787402B (en) * 2024-02-28 2024-05-07 徐州医科大学 Personalized learning path generation method and system based on multi-course knowledge graph fusion
CN117952072A (en) * 2024-03-22 2024-04-30 浙江蓝鸽科技有限公司 AI (advanced technology attachment) correction English teaching material editing method and system

Also Published As

Publication number Publication date
CN115292513A (en) 2022-11-04

Similar Documents

Publication Publication Date Title
WO2024016695A1 (en) Multiview learning-based teaching knowledge graph construction and retrieval method and system
CN112200317B (en) Multi-mode knowledge graph construction method
US11081018B2 (en) Personalized learning system and method for the automated generation of structured learning assets based on user data
CN110598770B (en) Multi-space fusion learning environment construction method and device
CN111753098A (en) Teaching method and system based on cross-media dynamic knowledge graph
CN111538835B (en) Social media emotion classification method and device based on knowledge graph
CN112380435B (en) Document recommendation method and system based on heterogeneous graph neural network
CN113792177B (en) Scene character visual question-answering method based on knowledge-guided deep attention network
Li et al. Progress, challenges and countermeasures of adaptive learning
Sharma et al. A survey of methods, datasets and evaluation metrics for visual question answering
CN114419642A (en) Method, device and system for extracting key value pair information in document image
CN113886567A (en) Teaching method and system based on knowledge graph
CN116860978B (en) Primary school Chinese personalized learning system based on knowledge graph and large model
CN115601772A (en) Multi-mode learning-based aesthetic quality evaluation model and method
CN115114421A (en) Question-answer model training method
CN115438176A (en) Method and equipment for generating downstream task model and executing task
CN115953569A (en) One-stage visual positioning model construction method based on multi-step reasoning
CN116975350A (en) Image-text retrieval method, device, equipment and storage medium
Liu The artistic design of user interaction experience for mobile systems based on context-awareness and machine learning
Bashmal et al. Visual question generation from remote sensing images
Liang et al. Generative AI-driven semantic communication networks: Architecture, technologies and applications
CN112445899B (en) Attribute matching method in knowledge base question and answer based on neural network
Sun et al. Teaching analysis for visual communication design with the perspective of digital technology
CN111222533B (en) Deep learning visual question-answering method and system based on dependency tree
CN116702784B (en) Entity linking method, entity linking device, computer equipment and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23841771

Country of ref document: EP

Kind code of ref document: A1