WO2021036117A1

WO2021036117A1 - Method and device for constructing multi-spatial fused learning environment

Info

Publication number: WO2021036117A1
Application number: PCT/CN2019/126856
Authority: WO
Inventors: 杨宗凯; 刘三女牙; 周东波; 王泰; 刘智; 张�浩; 孙建文
Original assignee: 华中师范大学
Priority date: 2019-08-30
Filing date: 2019-12-20
Publication date: 2021-03-04
Also published as: AU2020101287A4; CN110598770B; CN110598770A

Abstract

A method and a device for constructing a multi-spatial fused learning environment. The method comprises: pre-defining a plurality of spaces included in a multi-spatial fused learning environment and construction parameters of each space learning environment; constructing each said spatial semantic network model; acquiring a learning subject association event and a learning environment association event, to construct a learning subject-oriented multi-spatial data fusion model having consistent semantic levels; constructing a learning scene-based integrated embodied model; and setting the learning environment construction parameters for the learning subject according to the integrated embodied model. The present invention is able to dynamically construct, in a customized manner, a multi-spatial fused learning environment for a learning subject.

Description

Method and device for constructing multi-space fusion learning environment

[Technical Field]

The present invention relates to the field of information technology, in particular to a method and system for constructing a multi-space fusion learning environment.

[Background technique]

The development of information technology is very rapid, information technology is widely used in education, and the traditional teaching environment is gradually developing in the direction of digitization and intelligence. While studying in offline physical spaces such as classrooms, students can also use smart devices such as interactive whiteboards and portable tablets for online learning. Therefore, the student's learning environment integrates offline physical space and digital classrooms, various intelligent terminal equipment and teaching appliances and other online spaces.

However, in the prior art, offline physical space and online space are independent of each other, seamless interaction between online and offline cannot be achieved, and multi-space integrated learning cannot be personalized according to students' behavior in any space. surroundings. For example, offline educational resources and online educational resources are independent of each other. After students learn a certain content offline, the online resources cannot be adjusted to match the appropriate online learning resources according to the offline learning content.

[Summary of the Invention]

In view of the above defects or improvement needs of the prior art, this application proposes a method that can personalize the construction of a multi-space fusion learning environment based on the behavior of the learning subject.

According to one aspect of the present application, a method for constructing a multi-space integrated learning environment of the present application includes:

S1, a plurality of spaces included in a pre-defined multi-space fusion learning environment and the construction parameters of each space learning environment, the plurality of spaces include at least two of a physical space, a network space, a resource space, and a social space;

S2, predefine the service content of the space, and construct each spatial semantic network model according to the service content;

S3, collect the relevant events of the learning subject and the relevant events of the learning environment, and construct a data fusion model of multi-space semantic level consistency for the learning subject;

S4. Construct an integrated embodied model based on a learning scenario, where the integrated embodied model is used to describe the relationship between the learning subject and the learning environment in the learning scenario and the construction parameter association relationship, and the integrated embodied model can be based on any space Dynamic adjustment of newly collected learning subject related events or learning environment related events;

S5, setting the learning environment construction parameters for the learning subject according to the integrated embodied model.

As a further improvement of this application, the step S2 specifically includes:

S21: Predefine the service content of the space according to the learning scenario, perform semantic calibration on the service content of each space, establish the semantic main unit of the service content, and determine the semantic unit of the service content;

S22: Analyze the semantic relationship of each of the spatial service contents, clarify the sequence relationship, hierarchical relationship, and containment relationship among the semantic subject units, and construct a semantic relationship table;

S23: For each semantic unit of the spatial service content, construct a semantic web organization structure according to the semantic relationship table.

As a further improvement of this application, the step S3 specifically includes:

S31: Create instantiated objects of the same learning subject in different spaces, and determine the consistent expression of the same learning subject in different spaces;

S32, collecting learning subject-related events and learning environment-related events, and converting the learning subject-related events and learning environment-related events into data according to the semantic network model to form a learning behavior data pool;

S33: Construct a multi-space semantic level consistency data fusion model based on the learning behavior data pool.

As a further improvement of this application, the step S33 is specifically:

A long- and short-term memory model and a convolutional neural network including three channels of space, local time domain and global time domain are used to extract and classify the learning behavior data to construct a data fusion model with multi-space semantic level consistency.

As a further improvement of this application, the step S4 specifically includes:

S41: Convert the associated events of the learning subject and the associated events of the learning environment into standard format data according to the semantic network model and the data fusion model;

S42: Based on the standard format data, generalize the association relationship between the learning environment related events and the learning subject related events into a graph network model, use the graph network model vertices to express the spatial environment, and use the graph network model edges to express differences The embodied relationship of the space, the graph neural network model is used to construct an integrated embodied model based on the learning scene, and the integrated embodied model is used to describe the relationship between the learning subject and the learning environment to construct the parameter association and can be based on any new space. The collected learning subject-related events or learning environment-related events are adjusted.

As a further improvement of the present application, the learning environment construction parameters are one or more of the driving parameters of hardware or software in the physical space, the cyberspace, or the social space, or the resource acquisition parameters of the resource space.

According to another aspect of the present invention, a device for constructing a multi-space fusion learning environment of the present invention includes:

The space pre-defined module is used to pre-define the multiple spaces included in the multi-space fusion learning environment and the construction parameters of each of the spatial learning environment, the multiple spaces include at least physical space, cyber space, resource space, and social space Two of

A semantic network model building module for predefining the service content of the space, and constructing a semantic network model for each space according to the service content;

The data fusion model building module is used to collect the related events of the learning subject and the learning environment in different spaces, and construct a data fusion model of multi-space semantic level consistency for the learning subject;

The integrated embodied model building module is used to construct an integrated embodied model based on a learning scenario. The integrated embodied model is used to describe the relationship between the learning subject and the learning environment in the learning scenario and build parameter associations based on the newly collected Dynamic adjustment of learning subject related events or learning environment related events;

The learning environment construction module is used to set the learning environment construction parameters for the learning subject according to the integrated embodied model.

Compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:

1) The technical solution of the present invention starts from the space where education and teaching are developed, analyzes the physical space, network space, resource space, social space, etc. involved in the educational context, and explores a semantic analysis method based on the content of education and teaching services. Realize the multi-space unified expression and data fusion method that the subject is consistent, and the multi-space fusion learning environment can be personalized according to the student's behavior in any space.

2) The technical scheme of the present invention introduces the method of multi-level data fusion into the application of educational scenes, making it possible to construct a consistent multi-space learning environment with learners as the main body.

3) The technical scheme of the present invention proposes a learning space fusion method in which learners are embodied, and considers expression methods such as semantic network models and traditional independent space expressions to be unified and organized, filling the application gap.

4) The technical scheme of the present invention constructs a personalized multi-space learning environment for the subject of the learner, and designs a multi-space fusion system, which provides a basis for supporting a personalized learning environment that solves the indifferent experience under different physical conditions .

[Explanation of drawings]

Fig. 1 is a flowchart of a method for constructing a multi-space fusion learning environment provided by an embodiment of the present application;

FIG. 2 is an application schematic diagram of a method for constructing a multi-space fusion learning environment according to an embodiment of the present application;

Figures 3 and 4 are schematic diagrams of a semantic network model provided by an embodiment of the present application;

FIG. 5 is an application schematic diagram of a device for constructing a multi-space fusion learning environment provided by an embodiment of the present application;

Fig. 6 is a schematic structural diagram of a device for constructing a multi-space fusion learning environment provided by an embodiment of the present application.

[detailed description]

In order to make the purpose, technical solutions, and advantages of this application clearer, the following further describes this application in detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described here are only used to explain the present application, and are not used to limit the present application. In addition, the technical features involved in the various embodiments of the application described below can be combined with each other as long as they do not conflict with each other.

The solutions provided by the embodiments of this application can be applied to the informatization of education. While studying in offline physical spaces such as classrooms, students can also use interactive whiteboards, portable tablets and other smart devices for online learning. The students’ learning environment is integrated The offline physical space and digital classrooms, various intelligent terminal equipment and teaching equipment and other cyberspace, resource space and social space. A multi-space integrated learning environment can be personalized according to students' behavior in any space. For example, offline educational resources and online educational resources are interrelated. After students learn a certain content offline, the online resources can be adjusted to match the appropriate online learning resources according to the offline learning content.

As shown in FIG. 1, a method for constructing a multi-space fusion learning environment according to an embodiment of the present invention includes the following steps:

S1, a plurality of spaces included in a pre-defined multi-space fusion learning environment and construction parameters of each space learning environment, the plurality of spaces include at least two of a physical space, a network space, a resource space, and a social space.

The learning environment construction parameters are one or more of the driving parameters of hardware or software in the physical space, the cyberspace or the social space, and the resource acquisition parameters of the resource space.

Through the analysis of the educational learning situation, the different situations and learning spaces involved in the education and learning scenarios are clarified. It can be physical space, cyber space, resource space, social space, etc. Define the specific medium and presentation form of each space; clarify the typical learning scenes contained in each space and the learning situations in that scene.

As shown in Figure 2, step S1 may further include the following steps:

S11, define the new learning environment, clarify the spatial dimensions included in the new learning environment; its spatial dimensions include but are not limited to: physical space, cyber space, resource space, and social space. Among them, physical space is a description of the real world, cyber space (or cloud space, cyber space) is a virtual space formed by the Internet; and resource space and social space are other virtual spaces formed by knowledge content and interpersonal social interaction.

S12: Define the physical space in the new learning situation, clarify the meaning of the physical space, the content involved, and the content and conditions of the learning environment contained in the space. In this embodiment, it is preferable to define a physical space for the environment of education and learning, and describe the physical world in which students study; the physical space is the physical existence of the objective world. According to the difference of learning content, the physical space where learning takes place is also quite different, that is, it can be expressed as classrooms, learning activity places, etc., such as classrooms, libraries, reading rooms, sports fields, dormitories, lecture halls, etc.

S13: Define cyberspace in the new learning context, clarify the meaning of cyberspace, including content and the content of the learning environment supported in the space; cyberspace is the Internet connected through wired or wireless means, and connected to the Internet through corresponding equipment Services, in essence, are service partitions and organizations at different levels of the Internet, providing a network place for learning; more expressed as the use of the network for learning; processing capabilities, such as network computing capabilities, network storage capabilities, and network service capabilities; At the same time, cyberspace is reflected in the description of the network used for learning, such as network connection bandwidth and speed, network computing and processing capabilities, network storage capacity and reading and writing speed, network service content and service quality, etc.

S14. Define the resource space in the new learning context, clarify the meaning of the resource space, determine the characteristics of the resource space and its supporting elements for the construction of the learning environment; the resource space is defined as the content space formed by the learning content of the students, according to different Learning content, which is demarcated by the meaning of the subject or knowledge, is reflected in the unique resource organization structure space formed by the content of each subject, such as the knowledge map space of the subject. The resource space is more embodied as the knowledge space formed by the semantics of the resource; Resource space, in essence, is the semantic space expression of different knowledge content, and its manifestation is a knowledge map. At present, because there is no unified knowledge map that can express everything, it is necessary to construct its knowledge map according to different disciplines and research contents. In this embodiment, in an application scenario, the resource space is expressed as a knowledge graph of a specific content. In particular, this embodiment does not innovate and research the generation of knowledge graphs. Therefore, only the currently widely accepted, Public knowledge graph of known content.

S15. Define the social space in the new learning situation, clarify the meaning of the social space, determine the characteristics of the social space and its elements of the learning environment; social space, describe the virtual space for students to communicate and communicate with others in the learning process, reflecting students’ The virtual space of communication and exchange activities in the learning process, including social activities between students and students, between students and teachers, and other related social activities. It can be peer-to-peer communication formed by learning network forums, social network tools, Group communication, etc. The social space is a space for socializing through forums, instant messaging and other tools. The space records social content, such as text posted, text chatted, voice, or pictures and videos posted in the circle of friends, etc., and more express learning The individual’s personal expression of learning content or with other learners.

S16. Define other dimensional spaces in the new learning context, clarify the meaning of the space, the basis for setting the space, and the elements of the space for the construction of the learning environment. In a preferred embodiment, the learning space includes at least the physical space, network space, resource space, and social space described above, but is not limited to the above space, and may also include other undefined spaces, which can also support the learning process.

S17: Define the presentation form and presentation medium of different spaces in a specific learning scenario. In particular, in the claims, the physical space specifies a place where information equipment and conditions are equipped. Such as smart classrooms, equipped with acoustic, video and biosensing sensors, lighting, projection and other output devices, as well as traditional blackboards or whiteboards, etc.; network space expresses cloud storage scenes and cloud service scenes, where cloud storage scenes can include learning content Storage space context, learning activity process storage space context, etc.; cloud service scenarios can include online course autonomous learning scenarios, online classroom learning scenarios, and online tutoring teaching scenarios; resource space learning scenarios include online learning scenarios, learning content construction scenarios, and learning content Management scene. The online learning situation is expressed as the learner's online learning process of a certain subject or a certain course. The construction of learning content is expressed as the context of the construction of specific subject knowledge. The learning content management context is for users to manage the operation and processing of resources in the learning process. Social space. In this embodiment, the social space can be divided into different learning scenarios, which mainly include: course forum discussion scenarios, social networking tools and individual learners, teachers, or class chat scenarios, and also include discussions in the physical space.

S2, predefine the service content of the space, and construct a semantic network model for each space according to the service content.

As shown in Fig. 3, step S2 further includes the following steps:

S21: Perform semantic calibration on the service content of each space according to the service content of the predefined space of the learning scene, establish the semantic main unit of the service content, and determine the semantic unit of the service content;

S22: Analyze the semantic relationship of each spatial service content, clarify the sequence relationship, hierarchical relationship, and containment relationship among the semantic subject units, and construct a semantic relationship table;

S23, for the semantic units of each spatial service content, construct a semantic web organization structure according to the semantic relationship table. It can realize the expression and storage of the semantic structure.

The method of constructing the semantic model of the physical space: define the physical space in the learning context, clarify the service content provided by the physical space when providing education and learning services; clarify the equipment and hardware elements involved in the provision of services, analyze the content semantics of the physical space , Build a semantic network model; classroom teaching situation perception will mainly use intelligent sensing, electronic tags, image/voice collection, video surveillance and other technologies to achieve intelligent perception of classroom teaching environment parameters and automatic identification of collected target information, providing different perspectives Classroom actual video, teaching courseware video and customized synthesis video, real-time recording of teaching content, learning methods, teaching methods and other contextual information. Multi-scenario online learning process perception will use "activity flow" to track learners' effective behavior information collection in all activities in real time, realize the unified processing, quantification and recording of structured, semi-structured and unstructured behavior data, and form a feasible A panoramic learning behavior data pool with reusability and computability.

Classroom teaching behavior perception mainly supports the perception function of typical classroom teaching behavior information such as teacher or student classroom interaction, question answering, discussion, etc., and automatically extracts interactive behavior data such as interactive frequency, interactive subject and interactive content in different stages and different links. "Activity stream" generates formats, such as <subject, action, object, result, scene, timestamp, authority>, and transforms it into well-formed data suitable for modeling through semantic definition.

The method of constructing the semantic model of cyberspace: define the cyberspace in the learning context, clarify the service content provided by the cyberspace when providing education and learning services; clarify the hardware elements of cyberspace services and the service elements of cyberspace; the analysis is based on The content semantics of the service, and the construction of the semantic network model of the cyberspace. At present, personalized services in cyberspace mostly involve personalized learning diagnosis, personalized learning path planning, personalized resource recommendation, and visualization of learning status, but rarely involve personalized learning intervention. As far as the group is concerned, personalized learning in cyberspace focuses on improving the online experience of the group of netizens. It is mostly based on social computing, complex networks and other technologies to build a netizen group model. By analyzing group needs and interactions between groups, groups are classified according to different needs. , Provide customized learning path planning and other personalized service content for different groups on demand.

The method of constructing the semantic model of the resource space: define the resource space in the learning context, clarify the service content provided by the resource space when providing education and learning services; clarify the hardware elements and service elements of the resource space service, analyze the resources in the resource space Semantic graph relations to construct a semantic network model; according to the content of the course knowledge, including nodes (domain concepts, fragmented knowledge) and relations between nodes (relationships between domain concepts, relations between courses, knowledge fragments and domain concepts Etc.), construct a knowledge semantic network, use the existing single-sentence semantic type classification method to distinguish the text content sentence by sentence, and find the demonstrative sentences of various types of domain concepts; secondly, use the lexical chain method to carry out the adjacent sentences of the demonstrative Vocabulary chain analysis finds the vocabulary dependency relationship between sentences; finally, combined with the results of the lexical chain analysis, further use CRF (Conditional Random Fields), HMM (Hidden Model HMM), MEMM (Maximum Entropy Markov Model) and other sequence labeling models to label domain concepts From the beginning and ending points of the text, through the comparative analysis of the labeling results, a method for dividing the domain concept boundary is proposed. On this basis, based on the word frequency, word order, context and other characteristics of the text, term-to-competitive learning methods are used to identify core terms, and multi-class classifiers are used to identify the semantic types of domain concepts. In the process of tracking each learner's online learning, according to the user's click behavior, click resource types (text, images, exercises/examination questions of domain concepts), and knowledge sharing methods discussed in forums, combined with the metadata of learning resources ( (Including keywords), the degree of mastery of the knowledge concepts learned by the user, etc., using the conditionally constrained sequential pattern mining method to discover the click behavior pattern of the learner, analyze the cognitive strategy of the learner, and dig and discover the typical model of the massive learner Model, draw lessons from the Felder-Silverman learning style classification method, and summarize different learner styles.

The method of constructing the semantic model of the social space: define the social space in the learning context, clarify the content of the social space to provide education and learning services; clarify the hardware elements and service elements of the resource space service, analyze the individual and relationship semantics of the social space, and construct Semantic network model.

S3, collect the related events of the learning subject and the related events of the learning environment, and construct a data fusion model of multi-space semantic level consistency for the learning subject.

As shown in Fig. 4, step S3 further includes the following steps:

S31: Create instantiated objects of the same learning subject in different spaces, and determine the consistent expression of the same learning subject in different spaces.

The subject of the learner, the construction of the instantiation of the unified learning subject in different spaces, the analysis of the static characteristics of the individual (background information, pre-knowledge ability, learning style, etc.) and learning in multiple scenarios (online resource browsing, collaborative mutual evaluation, mutual assistance) Dynamic features (current knowledge ability, learning motivation, cognitive level, emotional attitude, interest preference, etc.) in question and answer, offline classroom interaction, outdoor learning, etc.), extract individual key learning features; based on content analysis and cognitive classification Theory, sentiment analysis, and deep learning and other theoretical methods dig out the key components of individual characteristics from multi-source learning activity data, analyze the key factors that affect the individual’s learning process, and dig out learners’ knowledge and cognition in different scenarios. The characteristics and commonalities of level, emotional attitude, etc., taking into account the temporality, contextuality and deep semantic characteristics of the learning state characteristics, to identify the learning state of the individual in a specific time and space. Finally, a dynamic unified model of learners is constructed for specific teaching design and ability evaluation.

S32: Collect learning subject related events and learning environment related events in different spaces, and convert the learning subject related events and learning environment related events into data according to the semantic network model to form a behavior learning behavior data pool.

The related events of the learning subject and the related events of the learning environment are monitoring parameters related to the learning environment, such as learner behaviors/actions collected by various sensors in the environment and network monitoring devices. For example, classroom teaching situation perception will mainly use technologies such as intelligent sensing, electronic tags, image/voice collection, and video surveillance to realize intelligent perception of classroom teaching environment parameters and automatic identification of collected target information, and provide classroom videos from different angles. Teaching courseware video and customized synthesis video, real-time recording of teaching content, learning methods, teaching methods and other contextual information. Multi-scenario online learning process perception will use "activity flow" to track learners' effective behavior information collection in all activities in real time, realize the unified processing, quantification and recording of structured, semi-structured and unstructured behavior data, and form a feasible A panoramic learning behavior data pool with reusability and computability. Classroom teaching behavior perception mainly supports the perception function of typical classroom teaching behavior information such as teacher or student classroom interaction, question answering, discussion, etc., and automatically extracts interactive behavior data such as interactive frequency, interactive subject and interactive content in different stages and different links. "Activity flow" generates formats, such as <subject, action, object, result, scene, time stamp, authority>, and transforms it into well-formed data suitable for modeling through semantic definition.

S33: Construct a multi-space semantic level consistency data fusion model based on the learning behavior data pool. Clarify the semantics of the object subject and learning behavior of the same learning subject in different spaces, analyze the semantics of different spaces describing the same event, the same behavior and activity, clarify the data entities of each semantic object, and construct multi-space data fusion standards; different classroom teaching Scenes, different network platforms, and social organization spaces to realize automatic extraction, intelligent recognition and automatic recording of classroom teaching situations, teaching subjects and teaching status, learning behaviors, and realize unified processing of structured, semi-structured and unstructured behavioral data , Quantify and record to form a panoramic learning behavior data pool with reusability and computability.

The long- and short-term memory model and the convolutional neural network containing three channels of space, local time domain and global time domain are used to extract and classify learning behavior data, and build a data fusion model with multi-space semantic level consistency.

A three-stream CNNs framework containing three channels in space, local time domain and global time domain is used to extract spatiotemporal features of learner behavior. The input of the Three-stream CNNs framework; The Three-stream CNNs framework contains 4 convolutional layers (Conv1-4), normalizes the 2 convolutional layers (Norm1 and Norm2), and connects to 2 pooling Layers (Pooling1 and Pooling2), after the three channels (spatial channel, local time domain channel, and global time domain channel) undergo convolution and pooling operations, deep features are obtained. The spatial channel CNNs perform depth on the learner's action images For learning, local time-domain channel CNNs conduct in-depth learning of optical flow features, and global time-domain channel CNNs conduct in-depth learning of the difference image product of the learner's behavior.

With the help of long and short-term memory model (LSTM) in dealing with time series problems, the LSTM model is introduced in the training of the classification module to recognize learners' actions. The features extracted by 3DCNN are input into the LSTM model for learning. Sequence learning can introduce time domain information and bring more accurate results to classification. Then, a spatial pyramid pooling layer (SPP) is added between the fully connected layer (Fully Connected, FC) and the LSTM model. After the feature maps of different sizes are calculated by the SPP layer, a fixed-length feature vector can be obtained. Then through the fully connected layer, 3DCNN+LSTM learns to classify the single type of features. Finally, through the classification results of the independent characteristics of the three channels, voting is performed to obtain the behavior action category, so that the corresponding behavior data is summarized and stored in the form of labeling. Through the two-way long and short-term memory neural network to learn the learner's data in different learning spaces, it can effectively learn the characteristics of a single form of the learning process data and the shared characteristics that show the correlation between different data forms, which can capture learners in different The relevance of time and different spaces to related learning content and learning behavior data, so that the data can be aggregated according to the corresponding "activity flow" specification. Using these learned multi-level features can help us automatically clarify the relationship between different data. On this basis, the Deep Boltzmann Machine (DBM) is used to fuse feature-level data from different learning spaces

S4. Construct an integrated embodied model based on a learning scenario, where the integrated embodied model is used to describe the relationship between the learning subject and the learning environment in the learning scenario and the construction parameter association relationship, and the integrated embodied model can be based on any space The newly collected learning subject related events and learning environment related events are adjusted.

The above step S4 may specifically include the steps:

S42. Based on the standard format data, generalize the association relationship between the multi-space learning environment related events and the learning subject related events into a graph network model, use the graph network model vertices to express the spatial environment, and use the graph network model edges to express differences The embodied relationship of the space, the graph neural network model is used to construct an integrated embodied model based on the learning scene, and the integrated embodied model is used to describe the relationship between the learning subject and the learning environment to construct the parameter association and can be based on any new space. The collected learning subject related events and learning environment related events are dynamically adjusted.

Specifically: construct an integrated embodied model based on the fusion of scene-based space and environment structure, resource and content semantics, learning activities and behavior; apply graph network to realize the data fusion of embodied learning environment, that is, the physics of the intelligent learning companion system Embodied.

The graph neural network is used to express the embodied learning environment and embodied relationship. Set G (term: graph) to be expressed as a collection of embodied environment and embodied relationship of multi-space fusion, where N (term: vertex) is expressed as the spatial collection of embodied learning environment, and E (term: edge) is expressed in each space The collection of embodied relationships.

G=(N,E)

Among them, ne[n] represents the adjacent space of a certain space n (represented by vertices), and co[n] relates to the relationship of space n (represented by edges). The corresponding attributes of space n, relation (n_1, n_2) are expressed as l_n ∈ R^(l_N) and (n_1, n_2) ∈ R^(l_E), where l represents a tensor formed by stacking all the attributes in the graph.

By generalizing the multi-space embodied learning environment and its relationship into a graph neural network model, that is, using vertices to express the spatial environment and edges to express the embodied relationship of different spaces, based on the calculation model of the graph neural network, construct a multi- The calculation rules of spatial embodied data perception and fusion realize the solution of personalized embodied environment.

Applying multi-modal interaction technology, through voice interaction, action interaction, holographic visual interaction, wearable sensor interaction and other technologies, build data perception of all elements of the environment and the whole process of behavior, and realize the multi-space embodied interaction of learners; through embodied The learning environment perceives the learner's cognitive results and constructs a feedback mechanism for the cognitive results to the embodied learning environment and intelligent services. Cognitive practice results are embodied and fed back to the learning environment to achieve personalized customization of the learning environment, and cognitive results are fed back to intelligent services to achieve personalized resource customization of intelligent services, and achieve the personality of "different from person to person and from time to time" Learning.

On the basis of constructing a multi-space embodied model, it includes data standards and specifications for the integration of spatial hierarchy, resource maps and learning interaction behaviors.

Aiming at the characteristics of unstructured, distributed, heterogeneous, and scattered sources of data, under the guidance of learning science and education and teaching related theories, a multi-spatial data fusion standard system is formed, which specifically includes main body standards, resource standards, and evaluation standards , Management standards, teaching process standards and other education-related standards, data processing and data quality standards, and data interoperability standards. Through the standardized unified conversion gateway of education big data, the standardized processing of heterogeneous education data is realized, including information extraction, data storage and retrieval of structured, unstructured and semi-structured data, as well as steps of data cleaning and data verification. , Provide usable and trustworthy data sources for data modeling, analysis and application. Multi-source data aggregation will establish a cross-scenario, cross-temporal and cross-temporal education big data information association model by acquiring the entities of multi-source data and their multi-level association relationships, so as to realize the information extraction and aggregation of multi-source heterogeneous education data, so that the processed Data can meet the needs of applications such as data analysis and modeling. Realize data exchange and sharing services based on application requirements, and support on-demand data exchange of multiple data sources through unified standard data interfaces and standard data formats, including data aggregation, data distribution, data update, data conversion, etc., to provide identity Support for verification, user authorization, transmission encryption, data integrity, data credibility, and data validity.

Teaching subject standard: Describe the basic information of the teaching subject, and realize the continuous recording and data association of the active subject across platforms and systems. The teaching subject is the subject of the implementation of teaching activities, including students, parents, teachers, teaching researchers, and teaching administrators. Teaching resource standards: Teaching resource standards include a series of standards for unified description, packaging and reorganization of teaching resources of different forms, different granularities, and different formats, such as courses, videos, exercises, etc. Teaching resource standards include not only the metadata description of the resource attributes, but also the semantic attributes of the resources, supporting automatic recognition and processing by machines to realize the personalized and intelligent push of resources. Teaching process standard: The teaching process standard is to describe the teaching process, the teaching subject and teaching content (such as courses, resources, etc.), the teaching environment (such as traditional classrooms, outdoor learning environments), and any activities carried out by participants in other teaching activities Interaction or related experience. The teaching process standards are oriented to traditional teaching environments (such as schools, classrooms) and non-traditional teaching environments (such as online learning environments, outdoor learning environments). The core includes how and when teaching activities occur, contextual information, and the results of the teaching process Wait. Data processing and data quality standards: Data processing standards mainly regulate data collection, preprocessing, analysis, visualization, and access. Data quality standards mainly put forward specific management requirements and corresponding index requirements for data quality, to ensure the quality of data in various links such as generation, storage, exchange, and use, and lay a good foundation for the application of big data in education. Educational data interoperability standards: This type of standard is mainly aimed at the heterogeneity of educational data, in order to achieve the interoperability requirements of the connection between massive data sets, the coupling, integration, migration, and information extraction of educational data.

(1) Establish a technology for automatically constructing learners' personalized multi-space embodied learning environment; apply multi-modal interaction technology, through voice interaction, action interaction, holographic visual interaction, wearable sensor interaction and other technologies, to build all elements and behaviors of the environment The whole process of data perception realizes the learner's multi-space embodied interaction; through the embodied learning environment, the learner's cognitive results are perceived, and the feedback mechanism of the cognitive results to the embodied learning environment and intelligent services is constructed. Cognitive practice results are embodied and fed back to the learning environment to achieve personalized customization of the learning environment, and cognitive results are fed back to smart services to achieve personalized resource customization for smart services.

Perceive the context of the learning space, learn the behavior of the subject, obtain environmental data, learning process data, learning behavior and other data, construct multi-space embodied entities, expressed as the vertices of the graph network, and form edges by using the spatial embodied relationship constraints Structure, describe the relationship between the vertices, and finally form the network structure of the multi-space learning environment graph, apply the topological calculation method of the graph, develop the multi-space data fusion algorithm, and realize the data fusion according to the subject.

Apply learner feature extraction algorithm, learner state recognition algorithm, learner profile technology and learning analysis technology to analyze the state and visualization of the learning process of the learning subject in multiple spaces. In the trajectory of the learner's learning process, conduct data mining and in-depth analysis to draw the learner's learning curve, so as to make a detailed diagnosis of the student's knowledge structure, find the blind spots of learning, and design more personalized learning for students' weak knowledge Plan, carry out precise positioning, and provide learners with personalized learning diagnosis.

(2) Develop and integrate the creation device of a multi-space fusion learning environment that meets the requirements of learners' embodied cognition; an intelligent learning companion system oriented to embodied cognition, through multi-space embodied cognition data perception and fusion module The perception and fusion of learning process data use the learner learning analysis module based on multi-source data to realize the recognition of the characteristics and status of the learning subject, accurately describe the learner and the feedback of learning behavior, and apply the intelligent guidance module to realize autonomous learning.

According to the individual needs of learners, apply data mining and semantic search to continuously improve the curriculum knowledge graph, combine the learning process, forgetting rules and success rate, dynamically update the node relationship in the curriculum knowledge graph, and apply genetic algorithm (GA) to achieve path optimization , Output a personalized learning path that adapts to learners. Through the optimized learning path and high-quality adapted learning resources, the universal resource recommendation method is applied to realize the personalized and intelligent guidance for learners.

A schematic diagram of the application of a device for constructing a multi-space fusion learning environment according to an embodiment of the present invention is shown in FIG. 5, the device for constructing a multi-space fusion learning environment and the video module, network module, and IoT module in the learning environment are connected through an interface. Modules, network modules, and IoT modules are used to collect learning subject-related events and learning environment related events in different spaces, and are controlled to configure the learning environment according to the learning environment construction parameters set by the multi-space fusion learning environment construction device.

The structure of the device for constructing a multi-space fusion learning environment is shown in Figure 6 and includes:

(1) The space pre-definition module is used to predefine multiple spaces included in the multi-space fusion learning environment and the construction parameters of each space learning environment according to the learning scene, the multiple spaces including at least physical space, network space, and resource space And two in the social space;

(2) The semantic network model building module is used to predefine the service content of the space according to the learning scenario, and to construct the semantic network model of each space according to the service content.

The building blocks of the semantic network model specifically include:

(2-1) The spatial content semantic calibration module is used to perform semantic calibration on the service content of each space according to the service content of the predefined space of the learning scene, establish the semantic main unit of the service content, and determine the semantic unit of the service content;

(2-2) The spatial content semantic analysis module is used to analyze the semantic relationship of the service content in each space, clarify the sequence relationship, hierarchical relationship and containment relationship among the semantic main units, and construct the semantic relationship table;

(2-3) The spatial content semantic web-building module is used to construct the semantic web organization structure based on the semantic relationship table for the semantic units of each spatial service content.

Semantic network building module, semantic calibration and hierarchical organization of different spatial contents, constructing semantic network model based on the semantic relationship between spatial contents, establishing content-based organization structure, establishing semantic consistency expression of multi-space services and content; integrating multi-space The connection relationship between learning environment, learning behavior and cognitive results (learning results) is generalized to a graph network model; the vertices of the graph network express the cognitive results (knowledge) in a certain space and express the learning behavior; a certain part of the graph network The subnet expresses a certain learning space (that is, the embodied learning environment), and the aggregation operation of nodes between subnets is the personalized solution of this learning space. The spanning tree after the multi-subnet personalized solution is expressed as a personalized multi-space model. The embodied interaction framework and feedback mechanism are the constraints of aggregate computing.

(3) The data fusion model building module is used to collect the related events of the learning subject and the learning environment in different spaces, and build a multi-space semantic level consistency data fusion model for the learning subject, and use the data fusion model to integrate Multi-space non-standard format data is converted to standard format data.

The data fusion model building module specifically includes:

(3-1) The consistency verification module of the learning subject is used to create instantiated objects of the same learning subject in different spaces and determine the consistent expression of the same learning subject in different spaces;

(3-2) The learning subject multi-space data collection module is used to collect learning subject related events and learning environment related events in different spaces, and convert the learning subject related events and learning environment related events into data according to the semantic network model, Form a behavioral learning behavior data pool;

(3-3) The data fusion module is used to construct a multi-space semantic level consistency data fusion model based on the learning behavior data pool.

The learner subject data fusion module is used to construct a data fusion method of learner subject and consistent semantic level of learning content, learning service and personality characteristics for multiple spaces, so as to realize the multi-space instantiated data expression of the learner subject based on the same semantics; Standardized unified conversion gateway to achieve standardized processing of heterogeneous educational data, including information extraction, data storage and retrieval of structured, unstructured and semi-structured data, as well as data cleaning and data verification steps, to build data Models, analysis and applications provide usable and trustworthy data sources. The preprocessing process includes steps such as data cleaning, data verification, and standardized processing to provide usable and trustworthy data sources for data modeling, analysis, and application. Eliminate erroneous data and redundant data in the data, verify the consistency and completeness of the original data, and use unified standards to process the data, such as text information format conversion, unity of measurement units, etc., to make the data set to be processed more complete , To transform heterogeneous and complex data into analyzable and applicable information.

(4) The integrated embodied model building module is used to construct an integrated embodied model based on a learning scenario. The integrated embodied model is used to describe the relationship between the learning subject and the learning environment in the learning scenario and can be based on The newly collected learning subject related events and learning environment related events are dynamically adjusted.

(5) The learning environment construction module is used to set the learning environment construction parameters for the learning subject according to the integrated embodied model.

The learning environment building module constructs a multi-space integrated learning environment with learners as the main body, realizes the integrated expression, display and data service of different spaces, and proposes a personalized learning environment.

Multi-space embodied environment creation: Calculate the construction parameters of different learning spaces according to the learner’s main situation, apply parameter-driven environment construction technology to realize the environment setting and customization of different learning spaces; learners learn in different spaces, and their learning experience It should be expressed as a unified whole, and its overall effect needs to be embodied in each space; in a certain space, the resources or capabilities of other spaces can be obtained through embodied objects, and activities in one space can be embodied through embodied objects. Feedback is applied to different spaces. For example, learning behaviors in cloud space can be fed back to learning activities in physical space, embodied in multiple spaces, and no longer a learning behavior or process in a single space;

Multi-space learning environment integration: Develop learning environment fusion devices to connect non-physical spaces in a certain physical space, and apply holographic technology and multi-modal interaction technology to realize the integrated presentation of multiple spaces and provide a personalized intelligent learning environment. According to the learner’s individual characteristics, customize the multi-space learning environment that matches the personality, such as the learning environment that conforms to the learner’s individual interaction mode (voice interaction and somatosensory interaction, etc.). Sense, touch, vision and other influences.

The method and system for constructing a multi-space fusion learning environment of the present invention have the following beneficial effects:

(1) Starting from the unfolded space of education and teaching, analyze the physical space, network space, resource space, social space, etc. involved in the educational context, and explore semantic analysis methods based on the content of education and teaching services to achieve consistent subjects The multi-space unified expression and data fusion method can personalize the construction of a multi-space fusion learning environment according to the student's behavior in any space. For example, offline educational resources and online educational resources are interrelated. After students learn a certain content offline, the online resources can be adjusted to match the appropriate online learning resources according to the offline learning content.

(2) The technical scheme of the present invention introduces the method of multi-level data fusion into the application of the education scene, making it possible to build a consistent multi-space learning environment with learners as the main body.

(3) The technical scheme of the present invention proposes a learning space fusion method in which learners are embodied, and considers expression methods such as semantic network models and traditional independent space expressions for unified organization, filling the application gap.

(4) The technical scheme of the present invention constructs a personalized multi-space learning environment for the subject of learners, and designs a multi-space fusion system to provide a personalized learning environment that supports the indifferent experience under different physical conditions basis.

It should be understood that although the various steps in the flowchart of FIG. 1 are displayed in sequence as indicated by the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order for the execution of these steps, and these steps can be executed in other orders.

Those skilled in the art can easily understand that the above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modification, equivalent replacement and improvement, etc. made within the spirit and principle of the present invention, All should be included in the protection scope of the present invention.

Claims

A method for constructing a multi-space fusion learning environment, which is characterized in that it includes:

S1, a plurality of spaces included in a pre-defined multi-space fusion learning environment and the construction parameters of each space learning environment, the plurality of spaces include at least two of a physical space, a network space, a resource space, and a social space;

S2, predefine the service content of the space, and construct each spatial semantic network model according to the service content;

S3, collect the relevant events of the learning subject and the relevant events of the learning environment, and construct a data fusion model of multi-space semantic level consistency for the learning subject;

S4. Construct an integrated embodied model based on a learning scenario, where the integrated embodied model is used to describe the relationship between the learning subject and the learning environment in the learning scenario and the construction parameter association relationship, and the integrated embodied model can be based on any space Dynamic adjustment of newly collected learning subject related events or learning environment related events;

S5, setting the learning environment construction parameters for the learning subject according to the integrated embodied model.
The method for constructing a multi-space fusion learning environment according to claim 1, wherein the step S2 specifically includes:

S21: Predefine the service content of the space according to the learning scenario, perform semantic calibration on the service content of each space, establish the semantic main unit of the service content, and determine the semantic unit of the service content;

S22: Analyze the semantic relationship of each of the spatial service contents, clarify the sequence relationship, hierarchical relationship, and containment relationship among the semantic subject units, and construct a semantic relationship table;

S23: For each semantic unit of the spatial service content, construct a semantic web organization structure according to the semantic relationship table.
A method for constructing a multi-space fusion learning environment according to claim 1 or 2, wherein the step S3 specifically includes:

S31: Create instantiated objects of the same learning subject in different spaces, and determine the consistent expression of the same learning subject in different spaces;

S32, collecting learning subject-related events and learning environment-related events, and converting the learning subject-related events and learning environment-related events into data according to the semantic network model to form a learning behavior data pool;

S33: Construct a multi-space semantic level consistency data fusion model based on the learning behavior data pool.
The method for constructing a multi-space fusion learning environment according to claim 3, wherein the step S33 is specifically:

A long- and short-term memory model and a convolutional neural network including three channels of space, local time domain and global time domain are used to extract and classify the learning behavior data to construct a data fusion model with multi-space semantic level consistency.
The method for constructing a multi-space fusion learning environment according to claim 1 or 2, wherein the step S4 specifically includes:

S41: Convert the associated events of the learning subject and the associated events of the learning environment into standard format data according to the semantic network model and the data fusion model;

S42: Based on the standard format data, generalize the association relationship between the learning environment related events and the learning subject related events into a graph network model, use the graph network model vertices to express the spatial environment, and use the graph network model edges to express differences The embodied relationship of the space, the graph neural network model is used to construct an integrated embodied model based on the learning scene, and the integrated embodied model is used to describe the relationship between the learning subject and the learning environment to construct the parameter association and can be based on any new space. The collected learning subject-related events or learning environment-related events are adjusted.
The method for constructing a multi-space fusion learning environment according to claim 1 or 2, wherein the learning environment construction parameters are hardware or software driving parameters in physical space, cyber space, or social space, or resources in resource space. Get one or more of the parameters.
A device for constructing a multi-space fusion learning environment, which is characterized in that it includes:

The space pre-defined module is used to pre-define the multiple spaces included in the multi-space fusion learning environment and the construction parameters of each of the spatial learning environment, the multiple spaces include at least physical space, cyber space, resource space, and social space Two of

A semantic network model building module for predefining the service content of the space, and constructing a semantic network model for each space according to the service content;

The data fusion model building module is used to collect the relevant events of the learning subject and the learning environment in different spaces, and construct a multi-space semantic level consistency data fusion model for the learning subject;

The integrated embodied model building module is used to construct an integrated embodied model based on a learning scenario. The integrated embodied model is used to describe the relationship between the learning subject and the learning environment in the learning scenario and build parameter associations based on the newly collected Dynamic adjustment of learning subject related events or learning environment related events;

The learning environment construction module is used to set the learning environment construction parameters for the learning subject according to the integrated embodied model.
The device for constructing a multi-space fusion learning environment according to claim 7, wherein the semantic network model construction module specifically comprises:

The spatial content semantic calibration module is used to pre-define the service content of the space according to the learning scene, perform semantic calibration on the service content of each space, establish the semantic main unit of the service content, and determine the semantic unit of the service content;

The spatial content semantic analysis module is used to analyze the semantic relationship of the service content in each space, clarify the sequence relationship, hierarchical relationship and containment relationship among the semantic main units, and construct the semantic relationship table;

The spatial content semantic web-building module is used to construct the semantic web organization structure based on the semantic relationship table for the semantic units of each spatial service content.
The device for constructing a multi-space fusion learning environment according to claim 7 or 8, wherein the data fusion model construction module specifically includes:

The consistency verification module of the learning subject is used to create instantiated objects of the same learning subject in different spaces, and determine the consistent expression of the same learning subject in different spaces;

The learning subject multi-space data collection module is used to collect learning subject related events and learning environment related events, and convert the learning subject related events and learning environment related events into data according to the semantic network model to form a learning behavior data pool;

The data fusion module is used to construct a multi-space semantic level consistency data fusion model based on the learning behavior data pool.
The device for constructing a multi-space fusion learning environment according to claim 7 or 8, wherein the learning environment construction parameters are specifically hardware or software driving parameters or resource space parameters in physical space, cyber space, or social space. Resource acquisition parameters.