CN116055436A

CN116055436A - Knowledge-graph-driven multi-user cognitive semantic communication system and method

Info

Publication number: CN116055436A
Application number: CN202211411520.2A
Authority: CN
Inventors: 李怡昊; 周福辉; 袁璐; 丁锐; 徐铭; 吴启晖; 董超
Original assignee: Nanjing University of Aeronautics and Astronautics
Current assignee: Nanjing University of Aeronautics and Astronautics
Priority date: 2022-11-11
Filing date: 2022-11-11
Publication date: 2023-05-02

Abstract

The invention discloses a knowledge graph driven multi-user cognitive semantic communication method, which belongs to the technical field of wireless communication, wherein a message input module inputs a message comprising semantic information into a multi-user cognitive semantic communication system; the semantic information acquisition module acquires semantic information in the message according to the received message comprising the semantic information and sends the semantic information to the semantic information processing module; the semantic information processing module processes the received semantic information to obtain processed semantic information, and sends the processed semantic information to the semantic information conversion module; the semantic information conversion module converts the received processed semantic information into a reconstructed message and sends the reconstructed message to the message output module; the message output module outputs the received reconstructed message for multi-user cognition, so that multi-user cognition semantic communication is realized, and the compression rate of the information source and the reliability of communication are greatly improved.

Description

Knowledge-graph-driven multi-user cognitive semantic communication system and method

Technical Field

The invention belongs to the technical field of wireless communication, and particularly relates to a knowledge-graph-driven multi-user cognitive semantic communication system and method.

Background

Wireless communication systems are rapidly evolving due to the increasing demand for mobile internet and wireless data services. However, these systems have gradually approached the limit of communication theory. For example, source coding techniques have been close to source entropy, while advanced channel coding techniques have been close to channel capacity. Due to the increasing demands for emerging ultra-wideband service services such as online gaming, augmented Reality (AR), virtual Reality (VR), etc., as well as the demands for high data rates and unprecedented popularity of mobile devices, people are faced with increasingly severe spectrum scarcity problems. Therefore, we have to make new breakthroughs and develop new communication modes to improve the spectral efficiency.

Semantic communication is regarded as a communication system which focuses more on the meaning behind the transmission of data rather than precisely transmitting messages, and is considered as a key for breaking through the technical bottleneck of the traditional communication system in the post shannon age. Once proposed, they have received a consistent attention from industry and academia. Current communication technology aims at accurately transmitting data or accurately transmitting signal waveforms, the meaning of which is not of particular interest. And the real purpose of communication is to let the receiver understand the meaning of the sender's information. Perhaps more bit quantity does not mean that more semantic information is contained in the data, but semantic communication aims at transmitting the semantic rather than carrying the semantic signals to the greatest extent, so that the semantic communication is hopeful to overcome the limitation of conditions such as bandwidth and the like, redundant information is removed to the greatest extent through semantic compression, and the purpose of communication is realized to the greatest extent while less data quantity is transmitted. Therefore, the method has important research value and practical research significance for research of semantic communication.

Currently, semantic communication is typically implemented in three ways. The first is an object-oriented communication mode that transmits important and necessary messages based on object-oriented metrics. In this manner, semantic compression is achieved by filtering out redundant messages that are less relevant to the communication objective. For example, P.Popovski, O.Simeone, F.Boccardi, D.Gunduz, and O.Sahin et al in their published papers "Semantic-Effectiveness Filtering and Control for Post-5G Wireless Connectivity" (IEEE J Indian Inst Sci, vol.100, pp.435-443, dec.2019) have introduced the concept of a Semantic effectiveness plane as a core part of future communication architecture in order to provide a platform for Semantic aware solutions. The semantically effective plane enhances the protocol stack by providing standardized interfaces that support information filtering and direct control of the functions of all layers of the protocol stack. The introduction of semantic planes may help replace the current traditional communication framework with a framework that is continually improving and expanding based on systems and standards. However, the method has the following defects: since semantic information theory is still an open problem, there is currently no unified index to evaluate the achievable performance. This results in the first approach being limited to certain specific wireless services or communication environments, with limitations in application.

The second way is to embed the original data into the low-dimensional vector space by using the deep neural network to compress the information source information and realize semantic communication. For example, H.Xie, Z.Qin, G.Y.Li, and b. -h.juang et al in its published paper "Deep learning enabled semantic communication systems," propose a deep learning based semantic communication system, called deep sc, for text transmission. Based on the transducer model in the NLP algorithm, deep sc aims to maximize system capacity and minimize semantic errors by recovering the meaning of sentences, not bit errors or symbol errors in traditional communications. Based on deep sc, xie H, qin Z, tao X et al propose a Multi-User Semantic communication system, MU-deep sc, in its published paper "Task-Oriented Multi-User security communication," for Multi-User communication scenarios for transmitting Multi-modal data. The defects of the method are that: deep learning algorithms typically require a large amount of high quality marker data. Furthermore, end-to-end operation is often considered a black box process, lacking in interpretability.

The last way is to implement semantic compression and semantic communication with the help of a knowledge base. In addition, knowledge in the knowledge base has a certain interpretation in the compression and error correction process due to its readability and existing inference rules. Chinese patent CN114461816a, for example, discloses an information supplementing semantic communication system based on knowledge-graph. The system not only reduces the data quantity required to be transmitted in the communication process, but also ensures that the system model has better robustness than the traditional communication model under the condition of low signal-to-noise ratio by utilizing the entity similarity error correction, knowledge association and text generation technology based on the knowledge graph. However, for the multi-user communication scene, the system model only realizes the semantic compression and recovery process from the perspective of effectiveness through the knowledge graph completion technology, and has no 'cognitive' feature, namely, no reasoning and error correction steps are involved, so that the improvement of the performance is limited. Therefore, from multiple viewpoints of effectiveness, reliability and the like, a new multi-user cognitive semantic communication system based on a knowledge graph is required to be developed by utilizing a knowledge graph reasoning technology in combination with the cognitive characteristic.

Disclosure of Invention

The invention aims to: in order to overcome the defects existing in the prior art, the existing semantic communication technology does not have a unified semantic measurement standard due to lack of a unified semantic information theory instruction, so that the problems of incapability of popularization of application scenes, lack of interpretability in use of a large-scale deep neural network, limitation in a multi-user communication scene and the like are caused. The invention provides a knowledge-graph-driven multi-user cognitive semantic communication system and a knowledge-graph-driven multi-user cognitive semantic communication method, which not only realize multi-user cognitive semantic communication transmission, but also greatly improve the compression rate of a source and the reliability of communication.

The technical scheme is as follows: in order to solve the technical problems, the knowledge graph driven multi-user cognitive semantic communication system comprises a message input module, a semantic information acquisition module, a semantic information processing module, a semantic information conversion module and a message output module which are connected in sequence;

the message input module is used for inputting a message comprising semantic information into the multi-user cognitive semantic communication system; the semantic information acquisition module is used for acquiring semantic information in the message according to the received message comprising the semantic information and sending the semantic information to the semantic information processing module; the semantic information processing module is used for processing the received semantic information to obtain processed semantic information, and sending the processed semantic information to the semantic information conversion module; the semantic information conversion module is used for converting the received processed semantic information into a reconstructed message and sending the reconstructed message to the message output module; the message output module is used for outputting the received reconstructed message for multi-user cognition, so as to realize the multi-user cognition semantic communication system.

Further, the semantic information acquisition module is used for acquiring semantic information in the received message according to the received message comprising the semantic information and sending the semantic information to the semantic information processing module; the semantic information acquisition module extracts semantic information in the message according to the formula (1):

s=f (m) formula (1)

In the above formula (1), m represents a message including semantic information sent by a source, f represents a semantic information extraction function, and s represents semantic information in the extracted message.

Further, the semantic information processing module comprises a semantic symbol reconstruction module and a semantic symbol user distinguishing module which are connected with each other, wherein the semantic symbol reconstruction module reconstructs semantic symbols according to received semantic symbols to obtain reconstructed semantic symbols, and sends the reconstructed semantic symbols to the semantic symbol user distinguishing module; the semantic symbol user distinguishing module distinguishes different user semantic symbols according to the received reconstructed semantic symbols and sends the different user semantic symbol information to the semantic information conversion module.

Further, the semantic information conversion module converts the semantic symbol information corresponding to different users into the reconstructed message corresponding to different users, specifically converts the semantic symbol information corresponding to different users into the reconstructed message corresponding to different users through the T5 model after fine tuning, and sends the reconstructed message corresponding to different users to the message output module.

A multi-user cognitive semantic communication method based on knowledge graph driving is based on the multi-user cognitive semantic communication system based on knowledge graph driving, and comprises the following steps:

step 1, inputting a message comprising semantic information into a multi-user cognitive semantic communication system by a message input module;

step 2, the semantic information acquisition module acquires semantic information in the message according to the received message comprising the semantic information and sends the semantic information to the semantic information processing module;

step 3, processing the received semantic information by a semantic information processing module to obtain processed semantic information, and sending the processed semantic information to a semantic information conversion module;

step 4, the semantic information conversion module converts the received processed semantic information into a reconstructed message and sends the reconstructed message to the message output module;

and step 5, outputting the received reconstructed message by the message output module for multi-user cognition to realize multi-user cognition semantic communication.

Further, in step 2, the method specifically includes the following steps:

step 21, aligning the message including semantic information sent by each source with the triplet in the knowledge graph, specifically, aligning the message including semantic information sent by each source with the triplet in the knowledge graph by using Text2KG alignment algorithm, and setting n sources, different messages are sent by the n sources, the messages all comprise semantic information, and the messages which are sent by the sources and comprise the semantic information are represented by a set M: m= (M ₁ ,m ₂ ,…,m _n ) The triples in the knowledge graph are expressed as (h, r, t);

step 22, inquiring the head entity and the tail entity of all triples (h, r, t) in the knowledge graph by using a synonym set WordNet to obtain all synonyms;

step 23, inquiring all words in the message m including the semantic information sent by the source, judging whether the head entity and the tail entity of one triplet (h, r, t) and all synonyms thereof are in the message m including the semantic information sent by the source, and if the head entity and the tail entity of one triplet (h, r, t) and all synonyms thereof are in the message including the semantic information sent by the source, determining that the alignment is successful;

step 24, performing semantic coding, and extracting semantic symbols according to formula (2):

s=text 2KG (m) formula (2)

In the above formula (2), m represents a message including semantic information sent by a source, s represents semantic information in the extracted message, and Text2KG represents a semantic symbol extraction function, i.e., a Text2KG alignment algorithm function.

Further, in step 3, the method specifically includes the following steps: step 31, obtaining a reconstructed semantic symbol according to the received semantic symbol by a semantic symbol reconstruction module in the semantic information processing module, and sending the reconstructed semantic symbol to a semantic symbol user distinguishing module; and step 32, distinguishing users to which the semantic symbols belong by a semantic symbol user distinguishing module in the semantic information processing module according to the received reconstructed semantic symbols, and sending the semantic symbol information of different users to a semantic information conversion module.

Further, the method comprises the steps of,

the step 31 specifically includes the following steps:

step 311, performing channel coding, specifically, performing channel coding on the received semantic symbols by the formula (3):

formula (3) of x=ce(s)

In the above formula (3), CE is a channel coding function, S is a semantic symbol sent by a source, x is a channel code corresponding to the semantic symbol sent by the source, n sources are provided, different messages are sent by the n sources, each message includes semantic information, n destinations simultaneously exist, and the n destinations receive the different semantic information, then the semantic symbol sent by the source is represented by a set S: s= [ S ] ₁ ,s ₂ ,…,s _n ]The channel coding corresponding to the semantic symbol sent by the source is represented by set X: x= [ X ] ₁ ,x ₂ ,…,x _n ]；

In step 312, the channel code X is transmitted on the mimo physical channel, and the signal received by the receiving end is denoted by Y:

y=hx+n formula (4)

In the above formula (4), X represents a channel coding set corresponding to semantic symbols transmitted by a source, H represents a channel matrix, and N represents channel noise;

step 313, performing channel decoding, obtaining reconstructed semantic symbols based on the formula (5) according to the signal Y received by the receiving end, and implementing channel decoding:

S′＝CE ^-1 (Y) formula (5)

In the above formula (5), CE ^-1 Is a channel decoding function, Y is a signal received by the receiving end, S 'is a reconstructed semantic symbol, and the reconstructed semantic symbol is represented by a set S': s '= [ S ]' ₁ ,s′ ₂ ,…,s′ _n ]；

Step 314, obtaining corrected semantic symbols by using a correction algorithm of the knowledge-graph embedding technique according to the formula (6):

s "=correct (S') equation (6)

In the above formula (6), correct is a correction algorithm function using a knowledge-graph embedding technique, S' is a reconstructed semantic symbol, S "is a corrected semantic symbol, and the corrected semantic symbol is represented by a set S": s "= [ S ] ₁ ,s″ ₂ ,…,s″ _n ]；

And step 315, the reconstructed semantic symbol is sent to a semantic symbol user distinguishing module.

Further, the step 32 specifically includes the following steps: according to the received reconstructed semantic symbols, distinguishing the users to which the reconstructed semantic symbols belong based on a message recovery algorithm of a private knowledge graph, and particularly distinguishing semantic symbols corresponding to different users based on an NLP technology.

Further, the method comprises the steps of,

in step 4, the pre-training model T5 is adopted to convert the reconstructed semantic symbol into a reconstructed message, namely, restore the reconstructed message into a text, and the method specifically comprises the following steps:

Step 41, based on the pre-training model T5, fine tuning the pre-training model T5 using a training set sample, the training set sample being a public data sample set;

step 42, saving the parameters of the trimmed model T5;

step 43, performing semantic decoding, modifying the reconstructed semantic symbol sent by the semantic information processing module into an input format of the model T5, and inputting the reconstructed semantic symbol in the format into the trimmed model T5 to obtain a reconstructed message corresponding to the reconstructed semantic symbol, where the reconstructed message is represented by a set M': m '= [ M ]' ₁ ,m′ ₂ ,…,m′ _n ]；

And step 44, the reconstructed message is sent to a message output module, and the message output module outputs messages corresponding to different users. The beneficial effects are that: compared with the prior art, the invention has the advantages that:

1. the knowledge graph is utilized to realize the functions of reasoning and correcting errors in the semantic communication process, and compared with the traditional method under the guidance of shannon information theory, the method greatly improves the compression rate of the information source and the reliability of communication;

2. aiming at the problems that the existing semantic communication technology lacks uniform semantic information theory guidance and does not have uniform semantic measurement standards, so that an application scene cannot be popularized, a large-scale deep neural network lacks interpretability, limitation is caused under a multi-user communication scene, and the like, the invention provides a knowledge graph-driven multi-user cognitive semantic communication system, wherein a triplet in the knowledge graph is used as a uniform semantic symbol, so that the problem of lacking uniform semantic measurement standards is avoided, and simultaneously the triplet is used as a semantic organization form and has readability, so that the system has the advantage of interpretability;

3. The invention uses the knowledge graph to represent the learning method to deeply mine the reasoning rules contained in the knowledge graph. On this basis, a semantic error correction algorithm is studied. In particular to a multi-user cognitive semantic system, a message recovery algorithm is provided, and messages of different users are distinguished by matching knowledge levels between a source and a target;

4. compared with the traditional semantic communication system, the multi-user cognitive semantic communication system based on knowledge graph driving provided by the invention adopts an improved, passed and interpretable semantic information detection algorithm, so that the extraction of semantic information is realized, and the transmission efficiency of the communication system is improved;

5. according to the invention, a correction algorithm based on a knowledge graph embedding technology is adopted to obtain the reconstructed semantic symbols, and an effective semantic error correction algorithm obtained by mining reasoning rules from the knowledge graph is adopted, so that the robustness of the multi-user cognitive semantic communication system is improved;

6. the invention utilizes a message recovery algorithm based on a private knowledge graph to distinguish semantic symbols of different users through a symbol recognition module, matches knowledge levels between a source and a target to distinguish messages of different users, and provides a message recovery algorithm for a multi-user cognitive semantic system, thereby realizing multi-user cognitive semantic communication and greatly improving the compression rate of the information source;

7. According to the invention, the problem of lack of unified semantic measurement standard is avoided by taking the triples in the knowledge graph as unified semantic symbols, and meanwhile, the triples are taken as semantic organization forms and have readability, so that the system has the advantage of interpretability;

8. the invention utilizes the knowledge graph to represent the inference rules contained in the learning method to deeply mine the knowledge graph, on the basis, a semantic error correction algorithm is researched, the functions of inference and error correction are realized by utilizing the knowledge graph, and the reliability of semantic communication is improved;

9. the invention provides a message recovery algorithm particularly aiming at a multi-user cognitive semantic system, and the messages of different users are distinguished by matching knowledge levels between a source and a target.

Drawings

FIG. 1 is a block diagram of a multi-user cognitive semantic communication system according to the present invention.

Fig. 2 is a structural diagram of a semantic information processing module according to the present invention.

Fig. 3 is a block diagram of an implementation of the multi-user cognitive semantic communication system of the present invention.

Fig. 4 is an example of an implementation of the present invention for text messaging.

Fig. 5 is a graph comparing semantic similarity scores of text at a transceiver end under different channel conditions using the present invention and the prior art.

Detailed Description

The invention will be further described with reference to the drawings and examples.

Example 1

The knowledge graph driving-based multi-user cognitive semantic communication system of the embodiment comprises a message input module, a semantic information acquisition module, a semantic information processing module, a semantic information conversion module and a message output module which are sequentially connected, with reference to fig. 1;

the message input module is used for inputting a message comprising semantic information into the multi-user cognitive semantic communication system;

the semantic information acquisition module is used for acquiring semantic information in the message according to the received message comprising the semantic information and sending the semantic information to the semantic information processing module;

the semantic information processing module is used for processing the received semantic information to obtain processed semantic information, and sending the processed semantic information to the semantic information conversion module;

the semantic information conversion module is used for converting the received processed semantic information into a reconstructed message and sending the reconstructed message to the message output module;

the message output module is used for outputting the received reconstructed message for multi-user cognition, so as to realize the multi-user cognition semantic communication system.

When the knowledge graph driving-based multi-user cognitive semantic communication system works, firstly, a message input module inputs a message comprising semantic information into the multi-user cognitive semantic communication system; then, the semantic information acquisition module acquires semantic information in the message according to the received message comprising the semantic information and sends the semantic information to the semantic information processing module; then, the semantic information processing module processes the received semantic information to obtain processed semantic information, and the processed semantic information is sent to the semantic information conversion module; the semantic information conversion module is used for converting the received processed semantic information into a reconstructed message and sending the reconstructed message to the message output module; and finally, outputting the received reconstructed message by a message output module for multi-user cognition.

Example 2

Based on embodiment 1, referring to fig. 1, the system for multi-user cognitive semantic communication based on knowledge graph driving of the present embodiment includes a message input module, a semantic information acquisition module, a semantic information processing module, a semantic information conversion module and a message output module which are sequentially connected; the message input module sends a message comprising semantic information to the semantic information acquisition module; the semantic information acquisition module sends semantic information to the semantic information processing module; the semantic information processing module sends the processed semantic information to the semantic information conversion module; the semantic information conversion module sends the reconstructed message to the message output module;

The connection mode between the message input module and the semantic information acquisition module, the connection mode between the semantic information acquisition module and the semantic information processing module, the connection mode between the semantic information processing module and the semantic information conversion module and the connection mode between the semantic information conversion module and the message output module are not limited to one type; the connection mode between the message input module and the semantic information acquisition module, the connection mode between the semantic information acquisition module and the semantic information processing module, the connection mode between the semantic information processing module and the semantic information conversion module and the connection mode between the semantic information conversion module and the message output module can be the same or different.

The information input module and the semantic information acquisition module perform information transmission or/and the semantic information acquisition module and the semantic information processing module perform information transmission or/and the semantic information processing module and the semantic information conversion module perform information transmission or/and the semantic information conversion module and the information output module perform information transmission through the communication module; the above communication module is not limited to one type, and it is preferable that the communication module adopts a conventional communication module that communicates through Zigbee.

Example 3

Based on embodiment 2, referring to fig. 1, the system for multi-user cognitive semantic communication based on knowledge graph driving of the present embodiment includes a message input module, a semantic information acquisition module, a semantic information processing module, a semantic information conversion module and a message output module, which are sequentially connected;

the message input module is used for inputting the message comprising the semantic information into the multi-user cognitive semantic communication system, the message comprising the semantic information is from a signal transmitting source (short for message source), the message comprising the semantic information is sent to the message input module by the message source, and then the message input module sends the message comprising the semantic information to the semantic information acquisition module.

Example 4

Based on embodiment 3, referring to fig. 1, the system for multi-user cognitive semantic communication based on knowledge graph driving of the present embodiment includes a message input module, a semantic information acquisition module, a semantic information processing module, a semantic information conversion module and a message output module, which are sequentially connected; the semantic information acquisition module is used for acquiring semantic information in the message according to the received message comprising the semantic information and sending the semantic information to the semantic information processing module;

In this embodiment, the semantic information acquisition module extracts semantic information in the message according to formula (1):

s=f (m) formula (1)

In the above formula (1), m represents a message including semantic information transmitted from a source, f () represents a semantic information extraction function, and s represents semantic information in the extracted message.

The semantic information acquisition module acquires the semantic information in the message by using the method and sends the semantic information to the semantic information processing module.

Example 5

Based on embodiment 4, referring to fig. 1, the system for multi-user cognitive semantic communication based on knowledge graph driving of the present embodiment includes a message input module, a semantic information acquisition module, a semantic information processing module, a semantic information conversion module and a message output module, which are sequentially connected; the semantic information processing module is used for processing the received semantic information to obtain processed semantic information, and sending the processed semantic information to the semantic information conversion module; referring to fig. 2, the semantic information processing module includes a semantic symbol reconstruction module and a semantic symbol user distinguishing module connected to each other;

the semantic symbol reconstruction module reconstructs the semantic symbol according to the received semantic symbol to obtain a reconstructed semantic symbol, and sends the reconstructed semantic symbol to the semantic symbol user distinguishing module;

The semantic symbol user distinguishing module distinguishes different user semantic symbols according to the received reconstructed semantic symbols and sends the different user semantic symbol information to the semantic information conversion module.

Example 6

Based on embodiment 5, referring to fig. 1, the system for multi-user cognitive semantic communication based on knowledge graph driving of the present embodiment includes a message input module, a semantic information acquisition module, a semantic information processing module, a semantic information conversion module and a message output module, which are sequentially connected; the semantic information processing module is used for processing the received semantic information to obtain processed semantic information, and sending the processed semantic information to the semantic information conversion module; referring to fig. 2, the semantic information processing module in this embodiment includes a semantic anomaly information processing module, a semantic symbol reconstruction module, and a semantic symbol user distinguishing module that are sequentially connected;

the semantic anomaly information processing module is used for carrying out anomaly value processing on the received semantic information, and the semantic information obtained through the anomaly value processing further improves the accuracy of the semantic information, so that the communication quality and the communication efficiency of the semantic information are improved; the semantic symbol reconstruction module reconstructs the semantic symbol according to the received semantic symbol to obtain a reconstructed semantic symbol, and sends the reconstructed semantic symbol to the semantic symbol user distinguishing module; the semantic symbol user distinguishing module distinguishes semantic symbols of different users according to the received reconstructed semantic symbols, and sends semantic symbol information corresponding to the different users to the semantic information conversion module.

Example 7

Based on embodiment 6, referring to fig. 1, the system for multi-user cognitive semantic communication based on knowledge graph driving of the present embodiment includes a message input module, a semantic information acquisition module, a semantic information processing module, a semantic information conversion module and a message output module, which are sequentially connected; the semantic information conversion module is used for converting the received processed semantic information into a reconstructed message and sending the reconstructed message to the message output module, wherein the semantic symbol user distinguishing module in the semantic information processing module sends semantic symbol information corresponding to different users to the semantic information conversion module, so that the semantic information conversion module converts the semantic symbol information corresponding to the different users into the reconstructed message corresponding to the different users and sends the message corresponding to the different users to the message output module; the semantic information conversion module of the embodiment converts semantic symbol information corresponding to different users into a reconstructed message corresponding to different users, specifically converts the semantic symbol information corresponding to different users into a reconstructed message corresponding to different users through a T5 model after fine tuning, sends the reconstructed message corresponding to different users to the message output module, and the message output module outputs the reconstructed message corresponding to different users for multi-user cognitive use, thereby realizing multi-user cognitive semantic communication.

Example 8

The knowledge graph driving-based multi-user cognitive semantic communication method of the embodiment is based on the multi-user cognitive semantic communication system in the embodiment, and the system comprises a message input module, a semantic information acquisition module, a semantic information processing module, a semantic information conversion module and a message output module which are sequentially connected, wherein the method comprises the following steps:

The message including the semantic information in the step 1 is from a signal transmitting source (abbreviated as a source), the source transmits the message including the semantic information to the message input module, and the message input module transmits the message including the semantic information to the semantic information acquisition module. Typically, in a multi-user cognitive semantic communication system, there are multiple sources, and different messages are sent by multiple different sources, where each message includes semantic information.

Example 9

In step 2, according to the received message including semantic information, the semantic information acquisition module acquires the semantic information in the message and sends the semantic information to the semantic information processing module;

in this embodiment, the semantic information acquisition module extracts semantic information in the message by using a Text2KG alignment algorithm, so as to realize extraction of semantic symbols, and specifically includes the following steps:

step 21, aligning the message including the semantic information sent by each source with the triplet in the knowledge graph, specifically, aligning the message including the semantic information sent by each source with the triplet in the knowledge graph by using a Text2KG alignment algorithm, setting n sources, sending different messages by the n sources, where the messages include the semantic information, and then representing the message including the semantic information sent by the source by using a set M: m= (M ₁ ,m ₂ ,…,m _n ) The triples in the knowledge graph are expressed as (h, r, t);

step 23, inquiring all words in the message m including semantic information sent by the source, judging whether the head entity and the tail entity of one triplet (h, r, t) and all synonyms thereof are in the message m including semantic information sent by the source, if the head entity and the tail entity of one triplet (h, r, t) and all synonyms thereof are in the message including semantic information sent by the source, determining that the alignment is successful,

s=text 2KG (m) formula (2)

In the above formula (2), m represents a message including semantic information sent by a source, s represents semantic information in the extracted message, and Text2KG () represents a semantic symbol extraction function, i.e., a Text2KG alignment algorithm function;

in the above steps, semantic symbol extraction is performed on the message sent by each source, and the purpose of the semantic symbol extraction is to detect and extract specific semantic information contained in the message. The implementation process of semantic symbol extraction is to align the input Text (i.e. the message m including semantic information sent by the source) with the triples in the knowledge graph, and the semantic symbol extraction is implemented by the proposed Text2KG alignment algorithm. Semantic symbol extraction is performed on the messages sent by each source, and the purpose of the semantic symbol extraction is to detect and extract specific semantic information contained in the messages.

In addition, a synonym set is utilized to query all synonyms of an entity, in this embodiment, wordNet is utilized as a synonym set, nouns, verbs, adjectives and adverbs are all stored in this database, and the Text2KG alignment algorithm in this embodiment considers all synonyms of a head entity and a tail entity, so that the defect that the alignment on the traditional character level is not effective in all cases is overcome.

Example 10

In step 3, the semantic information processing module processes the received semantic information to obtain processed semantic information, and sends the processed semantic information to the semantic information conversion module; in summary, the semantic information collection module extracts semantic information in the message by using the Text2KG alignment algorithm, so as to realize extraction of semantic symbols, and sends the semantic symbols to the semantic information processing module, and the semantic information processing module processes the received semantic symbols, which specifically comprises the following steps:

step 31, obtaining a reconstructed semantic symbol according to the received semantic symbol by a semantic symbol reconstruction module in the semantic information processing module, and sending the reconstructed semantic symbol to a semantic symbol user distinguishing module; in this embodiment, the semantic symbol reconstruction module obtains a reconstructed semantic symbol by using a correction algorithm based on a knowledge graph embedding technology, and specifically includes the following steps:

formula (3) of x=ce(s)

In the above formula (3), CE is a channel coding function, S is a semantic symbol (semantic symbol received by the semantic information processing module) sent by a source, x is a channel code corresponding to the semantic symbol sent by the source, n sources are provided, different messages are sent by the n sources, each message includes semantic information, n destinations exist at the same time, and the n destinations receive different semantic information, and the semantic symbol sent by the source is represented by a set S: s= [ S ] ₁ ,s ₂ ,…,s _n ]The channel coding corresponding to the semantic symbol sent by the source is represented by set X: x= [ X ] ₁ ,x ₂ ,…,x _n ]；

y=hx+n formula (4)

In the above formula (4), X represents a channel coding set corresponding to semantic symbols sent by a source, and the channel coding X is represented by a convolutional code; h represents a channel matrix, which is constant; n represents channel noise, which is constant;

in the above-mentioned formula (4),

the transpose of X is n rows l _c A real matrix of columns; />

Represents H is l _c Line l _c A real matrix of columns; n is l _c A real matrix of rows and columns; then->

Represents Y is l _c A real matrix of rows and columns; obtaining a signal Y received by a receiving end through the calculation;

S′＝CE ^-1 (Y) formula (5)

s "=correct (S') equation (6)

And step 315, the corrected reconstructed semantic symbol is sent to a semantic symbol user distinguishing module.

After obtaining the triples, i.e. semantic symbols, the symbols are transmitted using conventional communication modules in order to complete the semantic communication, in particular: first, to implement semantic symbol encoding, a dictionary (keys, values) is created; it enables the head entity, tail entity and relationship of the knowledge graph to be uniquely mapped into integers; secondly, by encoding each fixed-length integer, a binary vector x is obtained, which is denoted as semantic symbol encoding (semantic encoding); in order to improve the reliability of communication, a channel coding module is adopted, and after a coded binary vector set is obtained, the set is transmitted through the channel. Through the steps, the semantic symbol sent by the information source is reconstructed to obtain a reconstructed semantic symbol, and then the semantic symbol reconstruction module sends the corrected reconstructed semantic symbol to the semantic symbol user distinguishing module;

Step 32, distinguishing the user to which the semantic symbol belongs according to the received reconstructed semantic symbol by a semantic symbol user distinguishing module in the semantic information processing module, and sending the semantic symbol information of different users to a semantic information conversion module; in general, in a multi-user cognitive semantic communication system, there are multiple sources, and different messages are sent by multiple different sources, where each message includes semantic information; meanwhile, a plurality of signal destinations (also called users) exist, semantic information sent by the signal sources is received by the plurality of signal destinations, the number of the signal destinations (users) is the same as that of the signal sources, n signal sources are arranged, different messages are sent by the n signal sources, the messages all comprise the semantic information, and semantic symbols sent by the signal sources are received by the n signal destinations (users); because semantic symbols reaching the receiving end are required to be distinguished as to which information sink user at the receiving end belongs respectively, and because different users have different background knowledge, the different users all have respective private knowledge patterns, in fact, each user only can understand semantic information matched with own knowledge patterns, and therefore, the reconstructed semantic symbols can be distinguished by using the private knowledge patterns of the kth (k is less than or equal to 1 and less than or equal to n) information sink (users). In this embodiment, the semantic symbol user distinguishing module distinguishes semantic symbols corresponding to different users based on a message recovery algorithm of the private knowledge graph, specifically distinguishes semantic symbols corresponding to different users based on an NLP technology, queries all the reconstructed semantic symbols for the kth user, calculates the similarity between the reconstructed semantic symbols and triples in the private knowledge graph of the kth user, and judges which one belongs to the private knowledge graph of the kth user according to the similarity so as to distinguish the semantic symbols corresponding to different users.

Example 11

In step 4, the semantic information conversion module converts the received processed semantic information into a reconstructed message and sends the reconstructed message to the message output module, that is, the semantic information processing module sends the reconstructed semantic symbol to the semantic information conversion module, based on embodiment 10, the embodiment converts the reconstructed semantic symbol into the reconstructed message by using the pre-training model T5, that is, the text is recovered, and specifically includes the following steps:

step 42, saving the parameters of the trimmed model T5;

step 43, performing semantic decoding, modifying the reconstructed semantic symbol sent by the semantic information processing module into an input format of the model T5, and inputting the reconstructed semantic symbol in the format into the trimmed model T5 to obtain a reconstructed message corresponding to the reconstructed semantic symbol, where the reconstructed message is represented by a set M': m '= [ M ]' ₁ ,m′ ₂ ,…,m′ _n ]。

And step 44, obtaining a reconstructed message through the method, sending the reconstructed message to a message output module, and outputting the reconstructed message to a corresponding user cognition by the message output module to realize multi-user cognition semantic communication.

Example 12

In the knowledge-graph-driven multi-user cognitive semantic communication method of the embodiment, based on embodiment 11, a multi-user cognitive semantic communication system is provided, which comprises a plurality of information sources and a plurality of information sinks;

referring to fig. 3, in fig. 3, the number of sources is set to n, and the n sources are represented by set S as s= [ S ] ₁ ,S ₂ ,…,S _n ]；

Let the number of the signal sinks be n, the n signal sinks are represented by a set D: d= { D ₁ ,D ₂ ,…,D _n }；

Will be the kth source S _k The message transmitted by (1. Ltoreq.k. Ltoreq.n) is defined as m _k (k＝1,2,…,n)；

Setting the kth source S _k Message m transmitted _k Comprises the kth source S _k To the kth sink D _k Semantic information s of (2) _k (k＝1,2,…,n)；

Performing semantic coding: from the kth source S is achieved by equation (7) _k Message m transmitted _k Extracting semantic symbols to obtain the kth information source S _k To the kth sink D _k Semantic information s of (2) _k ：

s _k ＝Text2KG(m _k ) K=1, 2, …, n formula (7)

In the above formula (7), m _k Represents the kth source S _k A transmitted message; s is(s) _k Representing the kth source T _k To the kth sink D _k Semantic information of (2); text2KG () represents a semantic symbol extraction function, i.e., a Text2KG alignment algorithm function; semantic information s for delivering a source to a corresponding sink in this embodiment _k The method is expressed in a unified form of triples, triples in the knowledge graph are used as unified semantic symbols, the problem of lack of unified semantic measurement standards is avoided, and meanwhile, the triples are used as semantic organization forms and have readability, so that the system has the advantage of interpretability.

After semantic coding is performed through the steps, channel coding is performed, and the method specifically comprises the following steps:

performing channel coding on the received semantic symbols by a formula (8) to obtain x _k ：

x _k ＝CE(s _k ) (1. Ltoreq.k. Ltoreq.n), k=1, 2, …, n formula (8)

In the above formula (8), CE is a channel coding function, s _k Representing the kth source T _k To the kth sink D _k Semantic information of (2); x is x _k Representing the kth source T _k Channel coding of (a); thereby obtaining a channel coding set X representation corresponding to the semantic symbols sent by the source: x= [ X ] ₁ ,x ₂ ,…,x _k ,…,x _n ]；

Knowing that the channel code X is transmitted on a multiple-input multiple-output physical channel, the signal received by the receiver is denoted by Y:

y=hx+n formula (4)

in the above-mentioned formula (4),

and +.>

And +.>

And +.>

The transpose of X is n rows l _c A real matrix of columns; />

Represents H is l _c Line l _c A real matrix of columns; />

N is l _c A real matrix of rows and columns; then->

Performing channel decoding, obtaining reconstructed semantic symbols based on a formula (5) according to a signal Y received by a receiving end, and realizing channel decoding:

S′＝CE ^-1 (Y) formula (5)

In the above formula (5), CE ^-1 Is a channel decoding function, Y is a signal received by the receiving end, S 'is a reconstructed semantic symbol, and the reconstructed semantic symbol is represented by a set S': s '= [ S ]' ₁ ,s′ ₂ ,…,s′ _k ,…,s′ _n ]；

Obtaining corrected semantic symbols by using a correction algorithm of a knowledge graph embedding technology according to a formula (6):

s "=correct (S') equation (6)

In the above formula (6), correct is a correction algorithm function using a knowledge-graph embedding technique, S' is a reconstructed semantic symbol, S "is a corrected semantic symbol, and the corrected semantic symbol is represented by a set S": s "= [ S ] ₁ ,s″ ₂ ,…,s″ _k ,…,s″ _n ]；

Secondly, benefitWith the kth (1. Ltoreq.k. Ltoreq.n) sink (user) D _k In the embodiment, the semantic symbol user distinguishing module distinguishes semantic symbols corresponding to different users based on a message recovery algorithm of the private knowledge spectrum, in particular to distinguish semantic symbols corresponding to different users based on an NLP technology.

Then performs semantic decoding, uses the pre-training model T5 to reconstruct the semantic symbol s' _k Conversion to reconstructed message m' _k And recovering the text, sending the reconstructed message to a message output module, and outputting the reconstructed message to a corresponding user cognition by the message output module to realize multi-user cognition semantic communication.

In order to improve the robustness of the system, the knowledge graph is utilized to correct transmission errors, and channel decoding is realized. The error correction algorithm derives inference rules by mining the relationships between triples in the knowledge graph, which can evaluate the semantic rationality of triples consisting of h e H, r e R, T e T, or obtain the most relevant hidden entities and relationships for a given entity identified from the source signal. Note that the set of all triples in the message at time t is Φ _t Then the ith triplet of time slot t is noted as

Note here +.>

The head entity, the relation and the tail entity in the ith triplet of time slot t, respectively. The key of the error correction algorithm is to mine observable and hidden relations between entities in the triples and hidden relations between entities and relations. The method can automatically judge the rationality of the signal sink side triples and correct the triples inconsistent with semantic logic. In the error correction algorithm, the relation between the entity and the relation is mined by using the representation learning of the knowledge graph. In particular, the evaluation of the rationality of a triplet is achieved using a knowledge-based graph embedding method whose main idea is to learn a mapping function that maps entities and relationships in a high-dimensional graph representation space into a low-dimensional embedding space The low-dimensional representation of entities and relationships, also referred to as entity embedding and relationship embedding, aims to preserve structural information about the graph while facilitating reasoning. />

Embedding vectors representing head and tail entities, respectively, < >>

The embedded vector for the entire triplet is represented. An inference function is introduced to measure the rationality of the triples, denoted +.>

Training through a knowledge graph embedding algorithm. The reasoning function we get has the following two features:

(1) If it is

In agreement with the reasoning logic contained in the knowledge graph, then +.>

Should be maximized;

(2) If it is

Inconsistent with the reasoning logic of the knowledge graph, then +.>

Is a ratio of the score to the meeting of (2)

Much smaller and->

And->

The difference between them is proportional to the difference in their meaning. The principle of the error correction algorithm is the most similar, most reasonable semantic symbols observed at the receiver. The details are as follows: first, the ternary received by the receiving end is searched forThe first three similar entities and relations in the head entity, the relation and the tail entity of the group are combined into a candidate triplet set, the score of each candidate triplet is obtained by utilizing an inference function learned through a knowledge graph embedding algorithm, and the triplet with the highest score is found and used as the output of an error correction algorithm.

And at the information destination end, the reconstructed semantic symbol is obtained by using the proposed error correction algorithm. Because all the semantic symbols of the users are mixed together, a method for distinguishing semantic symbols of different users by a message recovery algorithm based on a private knowledge graph is provided. Specifically, for the kth user, semantic recovery is performed by determining which is a triplet belonging to the private knowledge-graph of the kth user. It essentially distinguishes between different users depending on the content of the message. And finally, converting the reconstructed semantic symbol into a reconstructed message by using the trimmed T5 model. The specific process is that a WebNLG 2020 data set is used as a training set and a verification set of a knowledge graph text generation model, fine tuning training of the knowledge graph text generation model is carried out, and the learning rate is set to be 5e ^-5 After training 100 epochs with a batch size of 2, saving the generated model parameter file model. Pt, inputting the reconstructed triplet into the trimmed model, and outputting to obtain the reconstructed text message.

According to the method, the reconstructed information is obtained through semantic coding, channel decoding and semantic decoding of the information sent by different information sources, the information output module sends the reconstructed information to the corresponding user, the corresponding user is aware of the reconstructed information, and multi-user cognitive semantic communication is achieved.

Example 13

The knowledge-graph-driven multi-user cognitive semantic communication method of the present embodiment is based on embodiment 12, and referring to fig. 4, a source S is provided ₁ (User 1) the message including semantic information transmitted is "Sam Hewson was born in Bolton"; set up the information source S ₂ (User 2) the message including semantic information transmitted is "John Wooley is male"; set up the information source S ₃ (User 3) the message including semantic information transmitted is "Sam Hewson is male";

firstly, semantic coding is carried out, text messages sent by a message source are aligned with triples in a knowledge graph by using a Text2KG alignment algorithm, all words in the Text messages sent by the message source are divided into the words shown in figure 4, after the alignment is determined to be successful, the Text messages are subjected to semantic coding and channel coding processes, channel decoding and semantic decoding are carried out through a physical channel to obtain reconstructed messages, the reconstructed messages in the embodiment are shown in figure 4, and the reconstructed messages are sent to corresponding users, as shown in figure 4, the message source S ₁ (User 1) transmitted "Sam Hewson was born in Bolton" to the sink User1, the source S ₂ (User 2) transmitted "John Wooley is male" to the sink User2, the source S ₃ The "Sam Hewson is male" sent by the (User 3) is sent to the information sink User3, and the information sent by the information source is sent to the corresponding User to realize multi-User cognitive semantic communication.

Example 14

The effect of the method according to the embodiment of the invention is further described by the present embodiment in combination with a simulation experiment:

firstly, acquiring a data set and preparing a related knowledge graph:

the knowledge graph serves as a core of cognitive semantic communication, and is essentially a semantic network which uses a graph structure to reveal the relationship between entities. Knowledge maps are typically composed of forms of triples, including both (head entity, relationship, tail entity) and (entity, attribute value). Knowledge maps are generally classified into general knowledge maps and domain knowledge maps. The general knowledge graph is generally oriented to the general field, the content is mainly common sense knowledge, the breadth of the related knowledge is emphasized, and the universal knowledge graph is more suitable for a semantic communication scene, so that the universal knowledge graph is used as the basis of the semantic communication.

The invention adopts a data set which is a published WebNLG 2020 data set and comprises a training set, a verification set and a test set, wherein the data format of the data set is a triplet text pair, such as "< H > Aarhus_Airport < R > city_service < T > Aarlus, denmark| The Aarhus is the Airport of Aarhus, denmark|and the like, the head and tail entities and the relations of the triplet are found to have obvious identifiers, and the triplet and the generated sentence can be well separated through" || ", the invention utilizes the identifiers to extract the triplet information, separate the head and tail entities and the relations, clean error and repeated data, form a format-regular triplet CSV data set and a pure text sentence data set, and the sentence is processed to be used as a source of a sentence in a communication process, and the triplet data is used as background knowledge of a transmitting end and a receiving end in a cognitive semantic communication frame.

Then, simulation conditions and parameter settings are carried out:

we use the knowledge graph yaco 3-10 as a training set and test set for the knowledge graph embedding algorithm, yaco 3-10 being a knowledge graph in which facts are extracted from wikipedia and aligned with WordNet to exploit the large amount of information contained in WordNet. It contains general facts about public personas, geographical entities, movies and further entities and classifies these concepts. The ComplEx model adopted by the invention is used as a model of a knowledge graph embedding algorithm, and because the model is one of interaction models with the best performance on YAGO3-10, the ComplEx model is set to train 1000 steps on a training set, the learning rate is 0.0004, and the embedding dimension is 100.

We use a training set of WebNLG datasets containing knowledge and text from different fields including airports, artists, astronauts, athletes, buildings, celestial bodies, cities, cartoon characters, food, traffic patterns, monuments, politics, sports teams, universities, etc. the test set includes three additional fields, namely movies, scientists and music. We set the T5 model to perform 100-step fine tuning on the training set, learning rate was 0.0001. We compare the performance of the proposed system with the baseline system by transmitting text in the test set.

Since the conventional communication performance evaluation criteria such as symbol error rate are not suitable for semantic communication, the present invention measures performance using semantic similarity score, calculates a semantic similarity score between an original sentence m and a recovered sentence m' from cosine similarity by the following formula, whereinB _Φ Representing the Bert coding layer, its output is the coding vector of sentence m:

simulation results: fig. 5 shows the comparison of semantic similarity scores obtained by our proposed multi-user cognitive semantic communication system with conventional communication systems under different error parameters p of BSC (binary symmetric channel). Note that the simulation of our multi-user cognitive semantic communication system consists of two transmitters and two receivers. It can be seen that when p <0.05, the semantic similarity score of the baseline system is higher than that of the multi-user cognitive semantic communication system we propose, and is close to 1. The reason for this is that when the channel environment is good, the message transmitted from the source through the reference system and the message reconstructed at the destination are identical. However, with our proposed multi-user cognitive semantic communication system, the information transmitted at the source and reconstructed at the destination is semantically equivalent, but with a different structure. Meanwhile, the semantic similarity score of the multi-user cognitive semantic communication system is higher than 0.9, which indicates that semantic information contained in the message is successfully transferred to a signal sink. When p >0.05, it is obvious that the proposed multi-user cognitive semantic communication system is more competitive and robust in a poor channel environment.

The foregoing is only a preferred embodiment of the invention, it being noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the present invention, and such modifications and adaptations are intended to be comprehended within the scope of the invention.

Claims

1. The multi-user cognitive semantic communication system based on knowledge graph driving is characterized by comprising a message input module, a semantic information acquisition module, a semantic information processing module, a semantic information conversion module and a message output module which are connected in sequence;

2. The knowledge-graph-driven multi-user cognitive semantic communication system according to claim 1, wherein the semantic information acquisition module is used for acquiring semantic information in a received message comprising semantic information according to the message, and sending the semantic information to the semantic information processing module; the semantic information acquisition module extracts semantic information in the message according to the formula (1):

s=f (m) formula (1)

3. The knowledge-graph-driven multi-user cognitive semantic communication system according to claim 1, wherein the semantic information processing module comprises a semantic symbol reconstruction module and a semantic symbol user distinguishing module which are connected with each other, wherein the semantic symbol reconstruction module reconstructs semantic symbols according to received semantic symbols to obtain reconstructed semantic symbols, and sends the reconstructed semantic symbols to the semantic symbol user distinguishing module; the semantic symbol user distinguishing module distinguishes different user semantic symbols according to the received reconstructed semantic symbols and sends the different user semantic symbol information to the semantic information conversion module.

4. The knowledge-graph-driven multi-user cognitive semantic communication system according to claim 3, wherein the semantic information conversion module converts semantic symbol information corresponding to different users into reconstructed messages corresponding to different users, specifically converts the semantic symbol information corresponding to different users into the reconstructed messages corresponding to different users through a fine-tuned T5 model, and sends the reconstructed messages corresponding to different users to the message output module.

5. A knowledge-graph-driven multi-user cognitive semantic communication method based on the knowledge-graph-driven multi-user cognitive semantic communication system as claimed in claims 1 to 4, characterized in that the method comprises the following steps:

6. The knowledge-graph-driven multi-user cognitive semantic communication method according to claim 5, wherein the method comprises the following steps of: in step 2, the method specifically comprises the following steps:

step 21, transmitting each sourceAligning the message comprising the semantic information with the triples in the knowledge graph, specifically, aligning the message comprising the semantic information sent by each information source with the triples in the knowledge graph by using a Text2KG alignment algorithm, setting n information sources, and sending different messages by the n information sources, wherein the messages comprise the semantic information, and the messages comprising the semantic information sent by the information sources are represented by a set M: m= (M ₁ ,m ₂ ,…,m _n ) The triples in the knowledge graph are expressed as (h, r, t);

s=text 2KG (m) formula (2)

7. The knowledge-graph-driven multi-user cognitive semantic communication method according to claim 6, wherein in step 3, the method specifically comprises the following steps: step 31, obtaining a reconstructed semantic symbol according to the received semantic symbol by a semantic symbol reconstruction module in the semantic information processing module, and sending the reconstructed semantic symbol to a semantic symbol user distinguishing module; and step 32, distinguishing users to which the semantic symbols belong by a semantic symbol user distinguishing module in the semantic information processing module according to the received reconstructed semantic symbols, and sending the semantic symbol information of different users to a semantic information conversion module.

8. The knowledge-graph-driven multi-user cognitive semantic communication method according to claim 7, wherein the step 31 specifically comprises the following steps:

formula (3) of x=ce(s)

y=hx+n formula (4)

S′＝CE ^-1 (Y) formula (5)

s "=correct (S') equation (6)

9. The knowledge-graph-driven multi-user cognitive semantic communication method according to claim 8, wherein the step 32 specifically comprises the following steps: and distinguishing the user to which the reconstructed semantic symbol belongs according to the received corrected reconstructed semantic symbol and a message recovery algorithm based on a private knowledge graph, specifically distinguishing semantic symbols corresponding to different users based on an NLP technology.

10. The knowledge-graph-driven multi-user cognitive semantic communication method according to claim 9, wherein in step 4, the reconstructed semantic symbols are converted into reconstructed messages, i.e. restored into texts, by adopting a pre-training model T5, specifically comprising the following steps:

step 42, saving the parameters of the trimmed model T5;

And step 44, the reconstructed message is sent to a message output module, and the message output module outputs messages corresponding to different users.