CN114519346A - Decoding processing method, device, equipment and medium based on language model - Google Patents
Decoding processing method, device, equipment and medium based on language model Download PDFInfo
- Publication number
- CN114519346A CN114519346A CN202210129816.9A CN202210129816A CN114519346A CN 114519346 A CN114519346 A CN 114519346A CN 202210129816 A CN202210129816 A CN 202210129816A CN 114519346 A CN114519346 A CN 114519346A
- Authority
- CN
- China
- Prior art keywords
- decoding
- code words
- candidate
- words
- code
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/284—Lexical analysis, e.g. tokenisation or collocates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
- H04L1/0056—Systems characterized by the type of code used
- H04L1/0057—Block codes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
- H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
- H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
- H04L1/1812—Hybrid protocols; Hybrid automatic repeat request [HARQ]
- H04L1/1816—Hybrid protocols; Hybrid automatic repeat request [HARQ] with retransmission of the same, encoded, message
Abstract
The application relates to a decoding processing method, a device, equipment and a medium based on a language model, wherein the method comprises the following steps: receiving an input code word after channel decoding; selecting legal code words of input code words from a code table; the legal code words comprise all code words of which the distance from the code words to the input code words in the code table is less than a set threshold value; performing source decoding on all legal code words to obtain corresponding all source decoding results which are used as decoding result candidates of input code words; calculating to obtain the co-occurrence probability of the candidate permutation and combination of the decoding result candidate items according to the semantic association probability among the contexts of the decoding result candidate items; and determining the candidate permutation and combination with the maximum co-occurrence probability as a semantic decoding output result of the input code word. The method and the device can realize high-efficiency and accurate error correction of the received information, and improve the effectiveness, stability and reliability of a communication system.
Description
Technical Field
The present application relates to the field of intelligent semantic communication technologies, and in particular, to a method, an apparatus, a device, and a medium for decoding processing based on a language model.
Background
The classical communication system aims at accurate transmission of communication symbols, ignores semantics of transmission contents, and is different from a transmission technology concerned by the classical communication system, so that the semantic communication system is closer to essence of communication, and is more concerned about a semantic problem of how to accurately and efficiently transmit. In a classical communication system, when a communication environment is poor, after received information is subjected to channel decoding check and error correction, the situation that the receiving end information source decoding fails and retransmits when an error code word or an error bit still exists. However, in the process of implementing the present invention, the inventor finds that there is a possibility that an error occurs in the existing semantic communication system after the source channel is decoded, so that the receiving end obtains the error information, and at present, there is still a technical problem that the efficient and accurate error correction of the received information cannot be implemented.
Disclosure of Invention
In view of the above, it is necessary to provide a decoding processing method based on a language model, a decoding processing device based on a language model, a communication device and a computer readable storage medium, which can realize efficient and accurate error correction of received information.
In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:
in one aspect, an embodiment of the present invention provides a decoding processing method based on a language model, including the steps of:
receiving an input code word after channel decoding;
selecting legal code words of input code words from a code table; the legal code words comprise all code words of which the distance from the code words to the input code words in the code table is less than a set threshold value;
performing source decoding on all legal code words to obtain corresponding all source decoding results which are used as decoding result candidate items of input code words;
calculating to obtain the co-occurrence probability of the candidate permutation and combination of the decoding result candidate items according to the semantic association probability among the contexts of the decoding result candidate items;
and determining the candidate permutation and combination with the maximum co-occurrence probability as a semantic decoding output result of the input code word.
In another aspect, a decoding processing apparatus based on a language model is also provided, including:
a code receiving module for receiving the input code after channel decoding;
the code word selecting module is used for selecting legal code words of input code words in the code table; the legal code words comprise all code words of which the distance from the code words to the input code words in the code table is less than a set threshold value;
the source decoding module is used for performing source decoding on all legal code words to obtain corresponding all source decoding results and using the source decoding results as decoding result candidate items of the input code words;
the probability calculation module is used for calculating the co-occurrence probability of the candidate permutation and combination of the decoding result candidate items according to the semantic association probability among the contexts of the decoding result candidate items;
and the output determining module is used for determining the candidate permutation and combination with the maximum co-occurrence probability as a semantic decoding output result of the input code word.
In still another aspect, a communication device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of any one of the above-mentioned language model-based decoding processing methods when executing the computer program.
In still another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any one of the above-mentioned language model-based transcoding processing methods.
One of the above technical solutions has the following advantages and beneficial effects:
the decoding processing method, the decoding processing device, the decoding processing equipment and the decoding processing medium based on the language model provide a new idea for a receiving end to restore information process by introducing the language model into a communication system, namely realize information semantic restoration from a semantic level. In a poor communication environment, error code words and bits can still appear in received information after channel decoding error correction check in classical communication, and an error retransmission mechanism can be adopted in the classical communication. In the semantic communication system, the receiving end further recovers the semantic of the transmission information by using the semantic association between the transmission contents on the basis of the classical communication technology. On the basis of accurately recovering information in the technical level of transmission information, the method further considers and solves the problem of accurately recovering information in the semantic level of transmission content, can efficiently and accurately correct the received information, and improves the effectiveness, stability and reliability of a communication system.
Drawings
FIG. 1 is a flowchart illustrating a method for language model based transcoding in one embodiment;
FIG. 2 is a schematic diagram of a communication system based on a language model in one embodiment;
FIG. 3 is a diagram illustrating comparison of simulation results of BLEU scores of a classical communication system and a latest semantic communication system on a Gaussian white noise channel according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating comparison of METEOR score simulation results of the present method with a classical communication system and a latest semantic communication system on a Gaussian white noise channel according to an embodiment;
fig. 5 is a schematic diagram illustrating comparison of BLEU score simulation results of the method of the present application on rayleigh fading channels with a classical communication system and a latest semantic communication system in one embodiment;
FIG. 6 is a diagram illustrating a comparison of METEOR score simulation results on Rayleigh fading channels between the method of the present application and the classical communication system and the latest semantic communication system in one embodiment;
FIG. 7 is a block diagram of a decoding apparatus based on a language model according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
In addition, the technical solutions in the embodiments of the present invention may be combined with each other, but it must be based on the realization of those skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination of technical solutions should be considered to be absent and not within the protection scope of the present invention.
Aiming at the technical problem that the traditional semantic communication system still cannot efficiently and accurately correct the received information at present, the invention provides a novel decoding processing method based on a language model, which is used for defining a candidate range at a receiving end of the semantic communication system according to the distance of code words and recovering the received information by using the context semantics of transmission contents in combination with the language model and a classical decoding method. It can be understood that, at the sending end of the semantic communication system, the encoding objects of the encoding modules correspond to words, not letters, that is, each word corresponds to a code word. The invention is explained in detail below in the context of a receiving end.
Referring to fig. 1, in one aspect, the present invention provides a decoding processing method based on a language model, including the following steps S12 to S20:
s12, receiving the channel decoded input code word.
It can be understood that, at the decoding end of the semantic communication system, after channel decoding, the receiving end obtains a codeword (for convenience of description, it may be denoted as codeword y), and information needs to be restored by source decoding. However, codeword y may have erroneous bits or codeword y is not in the source coded legal table. Processing by subsequent steps is therefore required to ensure accurate recovery of the source decoded information. The input codeword here and in the following can be represented by codeword y.
S14, selecting legal code word of input code word from code table; the legal code words comprise all code words in the code table, wherein the distance between the code words and the input code words is less than a set threshold value.
It can be understood that the set threshold may be specifically set according to the requirements of time limit, accuracy index, and the like of information recovery at the decoding end of the semantic communication system in an actual application scenario, and for convenience of description, the set threshold may be denoted as T hereinafter. After receiving the codeword y, the receiving end may select the codeword x satisfying that the codeword distance from the codeword y is smaller than a certain threshold T in the code table, that is:
l(x,y)＜T，x∈X
wherein, l (X, y) represents the codeword distance between codeword X and codeword y, and X represents the code table, i.e. the legal code table of the receiving end.
And S16, performing source decoding on all legal code words to obtain corresponding all source decoding results as decoding result candidates of the input code words.
It can be understood that after all the legal codewords x meeting the codeword distance condition in the code table are selected, the information source decoding is performed on all the legal codewords x meeting the codeword distance condition, and the information source decoding results corresponding to the legal codewords x are obtained. And taking all the currently obtained source decoding results as decoding result candidates of the code word y.
As shown in fig. 2, it is a simple schematic diagram of a communication system based on a language model of the method of the present invention, specifically, first, based on a classical communication system, a receiving end performs channel decoding on transmission information; due to the influence of the channel, the codeword y after channel decoding may have the situation that the error bits or the codeword y is not in the legal code table. Then, the receiving end of the semantic communication system combines the information source decoding and the code word candidate mechanism to obtain the candidate decoding result range of each code word.
S18, calculating the co-occurrence probability of the candidate permutation and combination of the decoding result candidates according to the semantic association probability among the contexts of the decoding result candidates;
it can be understood that after the decoding result candidate of the codeword y is obtained, the context knowledge k can be usedcAnd (represented by the permutation and combination co-occurrence probability) selecting the candidate permutation and combination with the maximum semantic association degree (represented by the inter-context semantic association probability) from the candidate permutation and combination of the decoding result candidates as a final semantic decoding output result s.
It is to be understood that each bit codeword in the sequence outputted by channel coding corresponds to multiple candidates, i.e. the previous word and the next word of the current core word correspond to multiple candidates, so the coding result of a sequence can be regarded as a permutation combination of candidates, for example, assuming that a sequence has three words, the first word corresponds to 2 candidates, the second word corresponds to 3 candidates, and the third word corresponds to 4 candidates, then a sequence has 2 × 3 × 4 — 24 permutation combinations of candidates, i.e. 24 possible source coding results. Only one of the 24 candidates is an accurate decoding result, that is, the co-occurrence probability of three words, i.e., the co-occurrence probability is the maximum, which is taken as the final source decoding result, that is, the co-occurrence probability of the 24 permutations is calculated and the permutation combination corresponding to the maximum value is taken as the decoding result. The context association of three words in the sequence is used, and the context association refers to the co-occurrence probability.
In some embodiments, the co-occurrence probability of the candidate permutation combination of the decoding result candidates is calculated by the following model:
wherein, Pr (w)1...wn) Co-occurrence probability, w, of candidate permutation combination of each decoding result candidateiRepresenting the decoding result candidate corresponding to the ith code word of the sequence, n representing the total number of code words in the sequence,
as can be appreciated, the first and second,according to the existing language model, a word wiThe probability of semantic association with its N-1 contexts can be denoted Pr (w)i|wi-N+1...wi-1),
Assuming that the information s contains n words and each word has a candidate, the co-occurrence probability of the candidate permutation combination of the n words in the information s can be expressed as
Therefore, the co-occurrence probability Pr (s | k) of the candidate permutation and combination of the decoding result candidatesc) Can be written as
Correspondingly, the decoding result can be denoted as s*＝argmaxs∈SlnPr(w1...wn) Wherein the capital S represents all candidate permutation combinations corresponding to the candidate candidates of each decoding result.
In some embodiments, the obtaining of the semantic association probability between contexts of the candidate permutation and combination of the decoding result candidates includes obtaining by utilizing bag-of-words model calculation or obtaining by utilizing neural network calculation based on long-term memory and short-term memory.
It can be appreciated that the inter-context semantic association probability Pr (w)i|wi-N+1...wi-1),
The method can be obtained by two existing deep learning methods, namely a bag-of-words model and a neural network based on long-term and short-term memory. The difference between the two approaches is that the former does not take into account the order between context words, and the latter takes into account the context word order.
Specifically, as shown in tables 1 and 2, the method of the present invention is provided for learning the inter-context semantic association probability value Pr (w)i|wi-N+1...wi-1) The structural diagram of the deep learning network of (1), wherein,table 1 shows a network structure parameter setting instruction of the bag-of-words model, and table 2 shows a network structure parameter setting instruction based on a long-and-short-term memory network model. The bag-of-words model consists of three layers of networks, and the long-time and short-time memory-based network model consists of five layers of networks. The input of the two models is the unique heat vector of the context word of the central word (the unique heat vector is the vector taking 0 at the rest position of 'one-bit effective'), and the output is the unique heat vector of the central word. The principle of the model is to predict the core word using the context word of the core word. And (3) minimizing a loss function of the network in training by a gradient descent method, wherein the output of the network is the probability of the context-based deduction of the central word.
TABLE 1
| Layer name | Number of nodes | Activating a function |
| Input layer | Number of words in thesaurus | Linearity |
| Intermediate layer | 300 | Linearity |
| Output layer | Number of words in thesaurus | Softmax |
TABLE 2
| Layer name | Number of nodes | Activating a function |
| Input layer | Number of words in thesaurus | Linearity |
| LSTM I | 256 | (Sigmoid,Sigmoid,Tanh,Sigmoid) |
| LSTM II | 256 | (Sigmoid,Sigmoid,Tanh,Sigmoid) |
| Dense | 256 | Relu |
| Output of | Number of words in thesaurus | Softmax |
And S20, determining the candidate permutation and combination with the maximum co-occurrence probability as the semantic decoding output result of the input code word.
It will be appreciated that the formula is based on a language model
And inter-context semantic association probability value Pr (w) obtained by deep learningi|wi-N+1...wi-1) The co-occurrence probability Pr (w) of all candidate decoding results (i.e. the candidate permutation combination) can be calculated1...wn)。
In some embodiments, a state-compressed dynamic programming algorithm is used to find the candidate permutation combination with the highest co-occurrence probability. Specifically, the candidate result (i.e., the candidate permutation and combination) corresponding to the maximum co-occurrence probability is found by using the existing state compression dynamic programming algorithm in the field as the semantic decoding output result s*＝argmaxs∈SlnPr(w1...wn)。
Specifically, when the number of words in the sequence is more and more, the number of the calculated permutation and combination is more and more, and therefore, in order to save calculation, storage and time resources, a state compression dynamic programming algorithm is adopted to recur the problem into sub-problems to obtain a global optimal solution. Meanwhile, each subproblem establishes a context window during calculation, and the central word is assumed to be related to (N-1) words adjacent to the context of the central word, because the words farther away from the central word are less associated with the context of the central word, and the influence of the words beyond the context window on the central word is less negligible.
For ease of understanding, for example and without limitation, the following examples:
assume that a sequence has n codewords. Code word y1With corresponding candidate decoding results W1Let W be1Comprises w1 1、w1 2And w1 3I.e. W1＝{w1 1，w1 2，w1 3}, then | W 13, |; code word y2With corresponding candidate decoding results W2Let W be2Comprises w2 1And w2 2I.e. W2＝{w2 1，w2 2}, then | W 22, |; code word y3With corresponding candidate decoding results W3If W is assumed3Comprises w3 1、w3 2、w3 3And w3 4I.e. W3＝{w3 1，w3 2,w3 3，w3 4}, then | W3And 4. I.e. the code word yiWith corresponding candidate decoding results Wi，i＝1,2,…,n。
According to language model formula
And inter-context semantic association probability value Pr (w) obtained by deep learningi|wi-N+1...wi-1) The co-occurrence probability Pr (w) of all candidate decoding results (i.e. the aforementioned candidate permutation and combination) can be calculated1...wn). Finding the maximum value of co-occurrence probability by using a state compression dynamic programming algorithm, and taking the corresponding candidate permutation combination as a decoding result s*＝argmaxs∈SlnPr(w1...wn)。
In particular, the contextual semantic association probability value Pr (w)i|wi-N+1...wi-1) An example of the calculation process of (c) is: assuming that the size of the context window N-1 is 1, a codeword y within one context window1And y2Together, there are 6 candidate permutation combinations, 3 × 2 ═ 6; sliding of context window, codeword y2And y3Together, there are 8 candidate permutation combinations, and so on. Assume that the size of the context window N-1 is 2: code y within a context window1、y2And y3Together, there are 24 candidate permutation combinations, 3 × 2 × 4, and the context window slides the codeword y2、y3And y4And so on.
The decoding processing method based on the language model provides a new idea for the receiving end to restore the information process by introducing the language model into the communication system, namely, the information semantic restoration is realized on the semantic level. In a poor communication environment, error code words and bits can still appear in received information after channel decoding error correction check in classical communication, and an error retransmission mechanism can be adopted in the classical communication. In the semantic communication system, the receiving end further recovers the semantic of the transmission information by using the semantic association between the transmission contents on the basis of the classical communication technology. On the basis of accurately recovering information in the technical level of transmission information, the method further considers and solves the problem of accurately recovering information in the semantic level of transmission content, can efficiently and accurately correct the received information, and improves the effectiveness, stability and reliability of a communication system.
It should be understood that, although the various steps in the flowchart of fig. 1 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps of fig. 1 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
In one embodiment, in order to more intuitively and fully describe the above method for decoding based on the language model, the following provides a simulation example of the semantic communication system based on the language model using the above method of the present application in the rayleigh fading channel and the white gaussian noise model.
It should be noted that, the simulation example given in this specification is only illustrative and is not only limited to the specific implementation case of the present invention, and those skilled in the art can similarly adopt the above-mentioned decoding processing method based on the language model to realize simulation, actual application or experiment on different application scenarios under the meaning of the implementation case provided by the present invention.
In Rayleigh fading channels, the transmitted and received signals are x and y, respectively, and the channel response h follows a complex Gaussian distribution
I.e. y ═ hx + n, white gaussian noise(Vector)
The transmission and reception signals satisfy y-x + n in a white gaussian noise channel,
two evaluation criteria are introduced, firstly BLEU (Bilingual evaluation robustness) is used for calculating semantic similarity between sentences, and BLEU (4-gram) calculates similarity of words in two texts by comprehensively comparing occupation ratios of common occurrence of 4-tuple words in the two texts. METEOR (Metric for evaluation of translation with explicit ordering of translation evaluation indicators) also takes synonyms, word shapes (single-word number, temporal variation) of words, and the like into consideration on the basis of BLEU. The method of the invention is compared with a classical communication system architecture (Huffman information source coding and decoding and LDPC channel coding and decoding with a code rate of 2/3) and a latest semantic communication architecture DeepsC Network method based on deep learning under two channel models. As shown in fig. 3 and 4, in the gaussian white noise channel, the semantic similarity score of BLEU (4-gram) and METEOR between transmission and recovery information of contextsc (lstm) + LDPC and contextsc (cbow) + LDPC in the method of the present invention is higher than that of the classic communication Huffman source coding plus LDPC channel coding method. Compared with deep learning based DeepsC Network, when the signal-to-noise ratio is larger than-1 dB, the semantic similarity score between the information recovered by the method and the original transmitted information is better than that of the DeepsC Network. Also, fig. 5 and 6 reflect in rayleigh fading channels, respectively. Compared with a classical information source and channel coding method, the method has higher BLEU and METEOR scores; compared with deep learning-based DeepsC Network, the method provided by the invention has higher semantic similarity score when the signal-to-noise ratio is higher than a certain signal-to-noise ratio. The effectiveness of the decoding processing method based on the language model is verified.
Referring to fig. 7, in an embodiment, a decoding processing apparatus 100 based on a language model is further provided, which includes a codeword receiving module 11, a codeword selecting module 13, a source decoding module 15, a probability calculating module 17, and an output determining module 19. The code receiving module 11 is configured to receive an input code after channel decoding. The code word selecting module 13 is used for selecting a legal code word of the input code word from the code table; the legal code words comprise all code words in the code table, wherein the distance between the code words and the input code words is less than a set threshold value. The signal source decoding module 15 is configured to perform signal source decoding on all legal codewords to obtain corresponding decoding results of all signal sources and use the decoding results as decoding result candidates of the input codewords. The probability calculation module 17 is configured to calculate a co-occurrence probability of the candidate permutation and combination of the decoding result candidates according to the inter-context semantic association probability of each decoding result candidate. The output determining module 19 is configured to determine the candidate permutation combination with the largest co-occurrence probability as the semantic decoding output result of the input codeword.
The decoding processing device 100 based on the language model introduces the language model into the communication system through cooperation of the modules, and provides a new idea for the receiving end to recover information, namely, the information semantic recovery is realized on the semantic level. In a poor communication environment, error code words and bits can still appear in received information after channel decoding error correction check in classical communication, and a retransmission mechanism is adopted in the classical communication. In the semantic communication system, the receiving end further recovers the semantic of the transmission information by using the semantic association between the transmission contents on the basis of the classical communication technology. On the basis of accurately recovering information in the technical level of transmission information, the method further considers and solves the problem of accurately recovering information in the semantic level of transmission content, can efficiently and accurately correct the received information, and improves the effectiveness, stability and reliability of a communication system.
In one embodiment, the co-occurrence probability of the candidate permutation and combination of the decoding result candidates is calculated by the following model:
wherein, Pr (w)1...wn) Representing the co-occurrence probability of the candidate permutation combination of the decoding result candidates,wirepresents the candidate item of the decoding result corresponding to the ith code word of the sequence, n represents the total number of the code words in the sequence,
in one embodiment, the obtaining of the semantic association probability between contexts of the candidate permutation and combination of the decoding result candidates includes calculation by a bag-of-words model or calculation by a neural network based on long-time and short-time memory.
In one embodiment, a state-compressed dynamic programming algorithm is used to find the candidate permutation combination with the highest co-occurrence probability.
For specific limitations of the decoding processing apparatus 100 based on the language model, reference may be made to the above corresponding limitations of the decoding processing method based on the language model, and details are not repeated here. The modules in the decoding processing apparatus 100 based on the language model may be implemented in whole or in part by software, hardware, and a combination thereof. The modules may be embedded in a hardware form or a device independent of a specific data processing function, or may be stored in a memory of the device in a software form, so that a processor may invoke and execute operations corresponding to the modules.
In yet another aspect, a communication device is provided, which includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the following steps: receiving an input code word after channel decoding; selecting legal code words of input code words from a code table; the legal code words comprise all code words of which the distance from the code words to the input code words in the code table is less than a set threshold value; performing source decoding on all legal code words to obtain corresponding all source decoding results which are used as decoding result candidates of input code words; calculating to obtain the co-occurrence probability of the candidate permutation and combination of the decoding result candidate items according to the semantic association probability among the contexts of the decoding result candidate items; and determining the candidate permutation and combination with the maximum co-occurrence probability as a semantic decoding output result of the input code word.
It should be noted that the communication device may be a receiving end device in various semantic communication systems in the field, and in addition to the memory and the processor, the communication device may further include other necessary components that are not listed in detail in this specification, depending on the specific model of the communication device.
In one embodiment, the processor, when executing the computer program, may further implement the sub-steps of the embodiments of the method for processing decoding based on a language model.
In yet another aspect, there is also provided a computer readable storage medium having a computer program stored thereon, the computer program when executed by a processor implementing the steps of: receiving an input code word after channel decoding; selecting legal code words of input code words from a code table; the legal code words comprise all code words of which the distance from the code words to the input code words in the code table is less than a set threshold value; performing source decoding on all legal code words to obtain corresponding all source decoding results which are used as decoding result candidates of input code words; calculating to obtain the co-occurrence probability of the candidate permutation and combination of the decoding result candidate items according to the semantic association probability among the contexts of the decoding result candidate items; and determining the candidate permutation and combination with the maximum co-occurrence probability as a semantic decoding output result of the input code word.
In one embodiment, the computer program, when executed by the processor, may further implement the sub-steps of the embodiments of the method for processing decoding based on a language model.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), synchronous link DRAM (Synchlink) DRAM (SLDRAM), Rambus DRAM (RDRAM), and interface DRAM (DRDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for those skilled in the art, various changes and modifications can be made without departing from the spirit of the present application, and all of them fall within the scope of the present application. Therefore, the protection scope of the present patent should be subject to the appended claims.
Claims (10)
1. A decoding processing method based on language model is characterized by comprising the following steps:
receiving an input code word after channel decoding;
selecting a legal code word of the input code word from a code table; the legal code words comprise all code words of which the distance from the code words in the code table to the input code words is less than a set threshold value;
performing source decoding on all the legal code words to obtain corresponding all source decoding results which are used as decoding result candidates of the input code words;
calculating to obtain the co-occurrence probability of the candidate permutation and combination of the decoding result candidate items according to the semantic association probability among the contexts of the decoding result candidate items;
and determining the candidate permutation and combination with the maximum co-occurrence probability as the semantic decoding output result of the input code word.
2. The method of claim 1, wherein the obtaining of the semantic association probability between contexts of the candidate permutation and combination of the decoding result candidates comprises a bag-of-words model calculation or a neural network calculation based on long-term memory and short-term memory.
3. The language model-based decoding processing method of claim 1 or 2, wherein the co-occurrence probability of the candidate permutation combination of each of the decoding result candidates is calculated by the following model:
wherein, Pr (w)1...wn) Indicating co-occurrence probability, w, of candidate permutation combinations of each of the decoding result candidatesiRepresenting the decoding result candidate corresponding to the ith code word of the sequence, n representing the total number of code words in the sequence,
4. the language model-based decoding processing method of claim 1, wherein the candidate permutation combination with the highest co-occurrence probability is found by using a state compression dynamic programming algorithm.
5. A decoding processing apparatus based on a language model, comprising:
a code receiving module for receiving the input code after channel decoding;
the code word selecting module is used for selecting the legal code words of the input code words from a code table; the legal code words comprise all code words of which the distance from the code words in the code table to the input code words is less than a set threshold value;
the source decoding module is used for performing source decoding on all the legal code words to obtain corresponding all source decoding results and using the source decoding results as decoding result candidate items of the input code words;
the probability calculation module is used for calculating the co-occurrence probability of the candidate permutation and combination of the decoding result candidate items according to the semantic association probability among the contexts of the decoding result candidate items;
and the output determining module is used for determining the candidate permutation combination with the maximum co-occurrence probability as a semantic decoding output result of the input code word.
6. The apparatus of claim 5, wherein the means for obtaining the semantic association probability between contexts of the candidate combinations of the decoding result candidates comprises a bag-of-words model calculation or a long-term-memory-based neural network calculation.
7. The language model-based transcoding processing apparatus of claim 5 or 6, wherein the co-occurrence probability of each candidate permutation and combination of the transcoding result candidates is calculated by the following model:
wherein, Pr (w)1...wn) Indicating co-occurrence probability, w, of candidate permutation combinations of each of the decoding result candidatesiRepresents the candidate item of the decoding result corresponding to the ith code word of the sequence, n represents the total number of the code words in the sequence,
8. the language model-based decoding processing apparatus of claim 5, wherein a state compression dynamic programming algorithm is used to find the candidate permutation combination with the highest co-occurrence probability.
9. A communication device comprising a memory and a processor, the memory storing a computer program, wherein the processor when executing the computer program implements the steps of the language model based transcoding method of any of claims 1 to 4.
10. A computer-readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the language model based transcoding processing method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210129816.9A CN114519346A (en) | 2022-02-11 | 2022-02-11 | Decoding processing method, device, equipment and medium based on language model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210129816.9A CN114519346A (en) | 2022-02-11 | 2022-02-11 | Decoding processing method, device, equipment and medium based on language model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114519346A true CN114519346A (en) | 2022-05-20 |
Family
ID=81596137
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210129816.9A Pending CN114519346A (en) | 2022-02-11 | 2022-02-11 | Decoding processing method, device, equipment and medium based on language model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114519346A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115955297A (en) * | 2023-03-14 | 2023-04-11 | 中国人民解放军国防科技大学 | Semantic coding method, semantic coding device, semantic decoding method and device |
-
2022
- 2022-02-11 CN CN202210129816.9A patent/CN114519346A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115955297A (en) * | 2023-03-14 | 2023-04-11 | 中国人民解放军国防科技大学 | Semantic coding method, semantic coding device, semantic decoding method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109492202B (en) | Chinese error correction method based on pinyin coding and decoding model | |
| CN111128137B (en) | Training method and device for acoustic model, computer equipment and storage medium | |
| CN111294058B (en) | Channel coding and error correction decoding method, equipment and storage medium | |
| CN111626062B (en) | Text semantic coding method and system | |
| CN108959388B (en) | Information generation method and device | |
| WO2016124070A1 (en) | Data processing method and device | |
| CN112039531A (en) | Jielin code error correction optimization method and device | |
| CN110569505A (en) | text input method and device | |
| CN114519346A (en) | Decoding processing method, device, equipment and medium based on language model | |
| CN112463924A (en) | Text intention matching method for intelligent question answering based on internal correlation coding | |
| Teng et al. | Convolutional neural network-aided tree-based bit-flipping framework for polar decoder using imitation learning | |
| Li et al. | Recognition of error correcting codes based on CNN with block mechanism and embedding | |
| CN116645971A (en) | Semantic communication text transmission optimization method based on deep learning | |
| WO2022141844A1 (en) | Text error correction method, apparatus and device, and readable storage medium | |
| CN111030708B (en) | Iterative adjustable soft serial cancellation list decoding method and device for polarization codes | |
| KR20210004897A (en) | A method and apparatus for fast decoding a linear code based on bit matching | |
| Teng et al. | Convolutional neural network-aided bit-flipping for belief propagation decoding of polar codes | |
| KR102517661B1 (en) | Method for identify a word corresponding to a target word in text information | |
| Doan et al. | Neural successive cancellation flip decoding of polar codes | |
| JP2021039220A (en) | Speech recognition device, learning device, speech recognition method, learning method, speech recognition program, and learning program | |
| CN112735392B (en) | Voice processing method, device, equipment and storage medium | |
| CN115694515A (en) | Neural network assisted polarization code decoding method and device based on key bits | |
| CN114912441A (en) | Text error correction model generation method, error correction method, system, device and medium | |
| KR102144732B1 (en) | A method and apparatus for fast decoding a linear code based on soft decision | |
| Zhang et al. | A highly reliable encoding and decoding communication framework based on semantic information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |