CN117914459A - Semantic synchronization method, device and storage medium based on fixed image - Google Patents
Semantic synchronization method, device and storage medium based on fixed image Download PDFInfo
- Publication number
- CN117914459A CN117914459A CN202311835425.XA CN202311835425A CN117914459A CN 117914459 A CN117914459 A CN 117914459A CN 202311835425 A CN202311835425 A CN 202311835425A CN 117914459 A CN117914459 A CN 117914459A
- Authority
- CN
- China
- Prior art keywords
- data
- vector
- sequence
- semantic
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 239000013598 vector Substances 0.000 claims abstract description 196
- 239000011159 matrix material Substances 0.000 claims abstract description 52
- 230000001360 synchronised effect Effects 0.000 claims abstract description 34
- 238000004891 communication Methods 0.000 claims description 24
- 238000001514 detection method Methods 0.000 claims description 24
- 230000003190 augmentative effect Effects 0.000 claims description 2
- 238000004590 computer program Methods 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 description 23
- 238000004422 calculation algorithm Methods 0.000 description 15
- 238000012545 processing Methods 0.000 description 8
- 230000002093 peripheral effect Effects 0.000 description 6
- 238000000540 analysis of variance Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000000556 factor analysis Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000007792 addition Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000010219 correlation analysis Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 230000002441 reversible effect Effects 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 238000003646 Spearman's rank correlation coefficient Methods 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- WYTGDNHDOZPMIW-RCBQFDQVSA-N alstonine Natural products C1=CC2=C3C=CC=CC3=NC2=C2N1C[C@H]1[C@H](C)OC=C(C(=O)OC)[C@H]1C2 WYTGDNHDOZPMIW-RCBQFDQVSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Abstract
The invention provides a semantic synchronization method, a semantic synchronization device and a storage medium based on fixed images, comprising the following steps: encoding the preset shared picture through a semantic encoder to obtain a corresponding first feature matrix, and encoding the current data to be transmitted through the preset encoder to obtain a corresponding second feature matrix; expanding the first feature matrix and the second feature matrix into one-dimensional vectors to obtain a first vector sequence corresponding to the first feature matrix and a second vector sequence corresponding to the second feature matrix; sorting each vector in the first vector sequence in a descending order according to the importance degree to obtain a third vector sequence; determining the first n vectors of the sequence in the third vector sequence as synchronous heads; splicing the second vector sequence to the synchronous head to obtain a data sequence; and transmitting the data sequence to a receiving end. The problem that the accuracy of data synchronization is low due to the fact that a system clock may drift due to hardware and software differences can be solved.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a semantic synchronization method, apparatus, and storage medium based on fixed images.
Background
In the last decades, research in the field of communications has focused on how to more efficiently transmit encoded symbols from a transmitting end to a receiving end. With the proliferation of intelligent applications, the inevitable need for large amounts of data transmission presents serious challenges to the low latency and high data transmission rates of existing communication systems, prompting researchers to consider the paradigm of next generation synchronous communication systems.
Conventional synchronous communication is a method based on symbol transmission, which aims to ensure that a transmitting end and a receiving end keep synchronous in time and frequency. In conventional synchronous communications, a transmitting end encodes data to be transmitted into a series of symbols and transmits the symbols to a receiving end through a time stamp; the receiving end demodulates the received symbols using the same time stamp and restores the original data.
However, in the conventional synchronous communication, due to the difference of hardware and software, the system clock may drift, resulting in inaccurate time stamp, thereby affecting the accuracy of data synchronization, and having a problem of low accuracy of data synchronization.
Disclosure of Invention
In view of this, the embodiments of the present invention provide a semantic synchronization method, apparatus and storage medium based on fixed images, so as to eliminate or improve one or more defects existing in the prior art, and solve the problem that the accuracy of data synchronization is low due to possible drift of a system clock caused by differences of hardware and software.
One aspect of the present invention provides a semantic synchronization method based on a fixed image, which is used in a transmitting end, wherein a communication channel is established between the transmitting end and a receiving end, and the method comprises the following steps:
encoding the preset shared picture through a semantic encoder to obtain a corresponding first feature matrix, and encoding the current data to be transmitted through the preset encoder to obtain a corresponding second feature matrix;
expanding the first feature matrix and the second feature matrix into one-dimensional vectors to obtain a first vector sequence corresponding to the first feature matrix and a second vector sequence corresponding to the second feature matrix;
Sorting each vector in the first vector sequence in a descending order according to the importance degree to obtain a third vector sequence;
determining the first n vectors of the sequence in the third vector sequence as synchronous heads; n is an integer greater than 0 and less than the total number of vectors in the third vector sequence;
splicing the second vector sequence to the synchronous head to obtain a data sequence;
And transmitting the data sequence to a receiving end.
Optionally, ordering each vector in the first vector sequence in descending order according to importance, to obtain a third vector sequence, including:
Determining the entropy of each vector in the first sequence of vectors;
And ordering the vectors in the first vector sequence in a descending order according to the entropy of each vector to obtain a third vector sequence.
Optionally, before determining the first n vectors of the sequence in the third vector sequence as the synchronization header, the method further includes:
Acquiring current channel information of a communication channel between the receiving end and the receiving end;
And determining the vector quantity corresponding to the current channel information from the vector quantity based on a preset relation between the channel information and the vector quantity as n.
Optionally, the data type of the data to be transmitted includes image data, voice data, text data, video data, point cloud data, or augmented reality data.
Optionally, the preset encoder comprises a preset semantic encoder or a preset base encoder.
Optionally, in the case that the data type of the data to be transmitted is picture data, the model structure of the semantic encoder is different from the model structure of the preset semantic encoder.
Optionally, the preset expansion strategy corresponding to the first feature matrix is the same as or different from the preset expansion strategy corresponding to the second feature matrix.
Optionally, the receiving end is configured to decode the data sequence through a detection window with a window length of n and a semantic decoder after receiving the data sequence, and determine the synchronization position based on the similarity between the current image obtained by decoding and the preset shared picture.
Another aspect of the present invention provides an apparatus for semantic synchronization based on fixed images, the apparatus comprising: a processor and a memory, wherein the memory has stored therein computer instructions, the processor being adapted to execute the computer instructions stored in the memory, the apparatus implementing the steps of the fixed image based semantic synchronization method described above when the computer instructions are executed by the processor.
Another aspect of the present invention provides a computer readable storage medium having stored thereon a computer program, characterized in that the program when executed by a processor implements the steps of the above-described fixed image based semantic synchronization method.
The invention has the advantages that:
According to the semantic synchronization method, the semantic synchronization device and the storage medium based on the fixed image, the preset shared image is encoded through the semantic encoder to obtain the corresponding first feature matrix, and the preset encoder is used for encoding the current data to be transmitted to obtain the corresponding second feature matrix; expanding the first feature matrix and the second feature matrix into one-dimensional vectors to obtain a first vector sequence corresponding to the first feature matrix and a second vector sequence corresponding to the second feature matrix; sorting each vector in the first vector sequence in a descending order according to the importance degree to obtain a third vector sequence; determining the first n vectors of the sequence in the third vector sequence as synchronous heads; splicing the second vector sequence to the synchronous head to obtain a data sequence; and transmitting the data sequence to a receiving end. The problem that the accuracy of data synchronization is low due to the fact that a system clock may drift due to the difference of hardware and software can be solved; the method comprises the steps that a preset shared picture is encoded through a preset encoder and then unfolded into a first vector sequence, and in the first vector sequence, a synchronization head required by synchronization is determined based on the importance degree of each vector, and a time stamp is not needed, so that inaccuracy of data synchronization caused by clock drift can be avoided, and the accuracy of the data synchronization is improved; meanwhile, vectors in the first vector sequence are ordered according to the descending order of importance, and the first n vectors with higher importance in the third vector sequence are selected as synchronous heads, so that the correct pictures can be obtained by decoding after the receiving end receives the data sequence, and the success rate of data synchronization is improved.
Further, the transmitting end and the receiving end may determine the value of n in advance according to the current channel state of the communication channel. Under the condition that the current channel state is good, the determined value of n is smaller, so that the length of the synchronous head sequence can be reduced, the overhead and the transmission time of synchronous head transmission are reduced, the occupation of bandwidth is reduced, the instantaneity of data transmission is ensured, and the transmission efficiency is improved; under the condition that the current channel state is poor, the determined value of n is larger, so that the anti-interference capability of the synchronous head in the transmission process can be improved, the synchronous precision is ensured, and the accuracy and the success rate of data synchronization are further improved.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.
It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present invention are not limited to the above-described specific ones, and that the above and other objects that can be achieved with the present invention will be more clearly understood from the following detailed description.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate and together with the description serve to explain the application.
FIG. 1 is a flow chart of a semantic synchronization method based on fixed images according to an embodiment of the present invention;
FIG. 2 is a flowchart of a semantic synchronization method based on fixed images according to another embodiment of the present invention;
FIG. 3 is a block diagram of a fixed image based semantic synchronization apparatus according to a further embodiment of the present invention;
fig. 4 is a block diagram of a semantic synchronization apparatus based on a fixed image according to another embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following embodiments and the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent. The exemplary embodiments of the present invention and the descriptions thereof are used herein to explain the present invention, but are not intended to limit the invention.
It should be noted here that, in order to avoid obscuring the present invention due to unnecessary details, only structures and/or processing steps closely related to the solution according to the present invention are shown in the drawings, while other details not greatly related to the present invention are omitted.
It should be emphasized that the term "comprises/comprising" when used herein is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.
It is also noted herein that the term "coupled" may refer to not only a direct connection, but also an indirect connection in which an intermediate is present, unless otherwise specified.
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.
The semantic synchronization method based on the fixed image provided by the application is described in detail below.
As shown in fig. 1, an embodiment of the present application provides a semantic synchronization method based on a fixed image, where the method is used for a transmitting end, and includes but is not limited to electronic devices such as a mobile phone, a computer, or a server, and specifically, the semantic synchronization method based on a fixed image includes at least the following steps S101 to S106:
Step S101, coding a preset shared picture through a semantic coder to obtain a corresponding first feature matrix, and coding current data to be transmitted through the preset coder to obtain a corresponding second feature matrix.
The preset shared picture refers to a preset picture shared by the transmitting end and the receiving end, including but not limited to a picture with a resolution of 1920×1080, 512×512 or 800×600, and meanwhile, the preset shared picture may be a square picture, a rectangular picture or a circular picture, etc., which is not limited in implementation manner in the embodiment.
The data type of the data to be transmitted includes image data, voice data, text data, video data, point cloud data, or Extended Reality (XR). In practical implementation, the data to be transmitted may be other data types, which are not listed here.
In this embodiment, the preset shared picture and the data to be transmitted may be stored in advance in a local storage medium of the transmitting end, or in a remote server having a communication connection with the transmitting end, or in a removable storage medium independent of the transmitting end, such as a usb disk, a hard disk, or the like. The present embodiment does not limit the storage form of the preset shared picture and the data to be transmitted.
In this embodiment, in the case where the data type of the data to be transmitted is image data, the semantic encoder that encodes the preset shared picture is different from the preset encoder that encodes the data to be transmitted.
Optionally, the preset encoder comprises a preset semantic encoder or a preset base encoder.
In the case where the data type of the data to be transmitted is image data, the model structure of the semantic encoder is different from that of the preset semantic encoder.
Step S102, the first feature matrix and the second feature matrix are unfolded into one-dimensional vectors, and a first vector sequence corresponding to the first feature matrix and a second vector sequence corresponding to the second feature matrix are obtained.
In this embodiment, the transmitting end expands the first feature matrix and the second feature matrix into a one-dimensional vector according to a preset expansion strategy. The preset deployment strategy comprises, but is not limited to, a row-first deployment strategy, a leveling deployment strategy, a column-first deployment strategy or a serpentine deployment strategy.
Optionally, the preset expansion strategy corresponding to the first feature matrix is the same as or different from the preset expansion strategy corresponding to the second feature matrix.
Step S103, each vector in the first vector sequence is ordered in descending order according to the importance degree, and a third vector sequence is obtained.
Wherein the importance of each vector refers to the contribution or influence degree of each vector to the whole vector sequence, and is used for measuring the relative importance of each vector in the vector sequence.
In this embodiment, the vectors in the first vector sequence are reordered according to the importance of the vectors in the first vector sequence, so that the vector with higher importance is ranked before in the third vector sequence obtained after reordering.
In one example, the importance of each vector in the first sequence of vectors is determined by the entropy of each vector.
In this embodiment, the entropy of each vector is calculated by a preset entropy formula, which is expressed as:
where H (m) represents entropy and p (m) represents the probability of each vector in the first sequence of vectors.
Wherein the probability of each vector in the first sequence of vectors is calculated using a predetermined neural network including, but not limited to, a recurrent neural network (Recurrent Neural Network, RNN), a convolutional neural network (Convolutional Neural Network, CNN), or a transducer network.
After determining the entropy of each vector in the first vector, sequencing each vector in the first vector sequence according to the entropy of each vector, and sequencing each vector in the first vector sequence according to the importance degree to obtain a third vector sequence.
Specifically, sorting each vector in the first vector sequence according to importance degree to obtain a third vector sequence, including: determining the entropy of each vector in the first sequence of vectors; and ordering the vectors in the first vector sequence according to the entropy of each vector to obtain a third vector sequence.
In another example, the importance of each vector is determined by a preset importance algorithm.
In this embodiment, the preset importance degree algorithm refers to a preset algorithm for calculating the importance degree of each vector in the first vector sequence. The preset importance degree algorithm comprises, but is not limited to, an analysis of variance algorithm or a correlation analysis algorithm, wherein the analysis of variance algorithm comprises single-factor analysis of variance (One-way analysis), multi-factor analysis of variance (Multi-factor analysis), repeated measurement analysis of variance (Repeated Measures ANOVA) and the like; the correlation analysis algorithm includes pearson correlation coefficient (Pearson correlation coefficient) or spearman correlation coefficient (spearman's rank correlation coefficient) and the like.
Step S104, the first n vectors of the third vector sequence are determined as synchronous heads. Wherein n is an integer greater than 0 and less than the total number of vectors in the third vector sequence.
Before the transmitting end determines the first n vectors of the third vector sequence as the synchronization header, the value of n needs to be determined based on the channel state of the communication channel between the transmitting end and the receiving end.
Specifically, before determining the first n vectors of the sequence in the third vector sequence as the synchronization header, the method further includes: acquiring current channel information of a communication channel between the receiving end and the receiving end; and determining the vector quantity corresponding to the current channel information from the vector quantity based on a preset relation between the channel information and the vector quantity as n.
The channel information is used to indicate the current communication state of the communication channel, including but not limited to the information of signal-to-noise ratio, multipath effect or interference of the communication channel.
In actual implementation, the value of n may also be adjusted by the user based on the actual situation of the communication channel, and the implementation manner of the value of n is not limited in this embodiment.
Conventional synchronous communications typically rely on strict clock synchronization. In a large-scale heterogeneous network, different devices may have differences in hardware and software, transmission media and network topology, so that problems of clock drift, transmission delay, bandwidth isomerism and the like are caused, and stability and accuracy of data synchronization are low. Furthermore, complex transmission tasks typically require high bandwidth, low latency, and high throughput. The clock synchronization mechanism of conventional synchronous communications may not meet these requirements, thereby limiting the real-time and efficiency of the transmission tasks.
In the embodiment, the preset shared picture is encoded and then unfolded into the first vector sequence, and in the first vector sequence, the synchronization head required by synchronization is determined based on the importance degree of each vector, and a time stamp is not required, so that inaccuracy of data synchronization caused by clock drift can be avoided, and the accuracy of data synchronization is improved; and the vectors in the first vector sequence are ordered according to the importance degree, and the first n vectors with higher importance degrees in the third vector sequence are selected as the synchronization head, so that the correct picture can be decoded after the receiving end receives the synchronization head, and the success rate of data synchronization can be improved.
Meanwhile, the transmitting end and the receiving end may determine the value of n in advance according to the current channel state of the communication channel. Under the condition that the current channel state is good, the determined value of n is smaller, so that the length of a synchronous head sequence can be reduced, the overhead and the transmission time of synchronous head transmission are reduced, the occupation of bandwidth is reduced, the instantaneity of data transmission is ensured, and the transmission efficiency is improved; under the condition that the current channel state is poor, the determined value of n is larger, so that the anti-interference capability of the synchronous head in the transmission process can be improved, the synchronous precision is ensured, and the accuracy and the success rate of data synchronization are further improved.
Step S105, the second vector sequence is spliced to the synchronization head to obtain a data sequence.
In this embodiment, the number of data to be transmitted is at least one, and correspondingly, the number of second vector sequences is also at least one.
In the case that the number of the second vector sequences is 2 or more, splicing the second vector sequences after synchronization to obtain a data sequence includes: splicing each second vector sequence to an independent synchronous head respectively to obtain a data subsequence; and splicing each data subsequence in sequence to obtain a data sequence.
Such as: taking an example that the synchronization header comprises ase:Sub>A synchronization header A, the second vector sequence comprises ase:Sub>A second vector sequence B and ase:Sub>A second vector sequence C, splicing the second vector sequence B to the synchronization header A to obtain ase:Sub>A datase:Sub>A subsequence A-B, splicing the second vector sequence C to the synchronization header A to obtain ase:Sub>A datase:Sub>A subsequence A-C, and splicing the datase:Sub>A subsequence A-B and the datase:Sub>A subsequence A-C to obtain ase:Sub>A datase:Sub>A sequence A-B-A-C.
Step S106, the data sequence is sent to the receiving end.
In this embodiment, the receiving end is configured to decode the data sequence through a detection window with a window length of n and a semantic decoder after receiving the data sequence, and determine the synchronization position based on the similarity between the current image obtained by decoding and the preset shared picture. And under the condition that the similarity is greater than or equal to a preset similarity threshold value, determining the current position of the detection window as a synchronous position. And under the condition that the similarity is smaller than a preset similarity threshold, moving a detection window, decoding to obtain a new current image, and calculating the similarity with a preset shared image.
Wherein the semantic decoder is a decoder corresponding to a semantic encoder that encodes a preset shared picture. The storage medium corresponding to the receiving end stores model weights corresponding to the preset decoder.
In this embodiment, the receiving end uses a detection window with a window length of n to slide the data sequence, places n data in the detection window at corresponding positions in the decoding vector equal to the first vector sequence, and since n is an integer greater than 0 and less than the total number of vectors in the third vector sequence, blank positions exist in the decoding vector, and the blank positions in the decoding vector are complemented in a zero-filling manner, and then the current image is decoded by a preset decoder.
Specifically, sliding a window on a data sequence through a detection window with a window length of n, decoding the sequence in the detection window through a preset decoder to obtain a current image corresponding to the current position of the detection window, including: using a detection window, starting from a first position of the data sequence, performing sliding window operation, and placing data in the detection window at a corresponding position in the decoding vector according to the index; decoding vectors of equal length to the first vector sequence; zero padding is carried out on blank positions in the decoding vector; and decoding the decoding vector through a preset decoder to obtain the current image.
In practical implementation, other decoders may be used for decoding, such as a Reverse Decoder (Reverse Decoder), and the implementation of the receiver-side preset Decoder is not limited in this embodiment.
In this embodiment, according to the difference of the similarity calculation methods, the values of the preset similarity thresholds are also different.
In one example, the receiving end determines the similarity between the current image and the preset shared picture by calculating a structural similarity index (Structural Similarity Index, SSIM) or a peak signal-to-noise ratio (PSNR) between the current image and the preset shared picture.
In another example, the receiving end determines the similarity between the current image and the preset shared image through a preset similarity algorithm.
The preset similarity algorithm comprises a perceptual hash algorithm (Perceptual Hashing) or a local feature matching algorithm (such as algorithms of scale-invariant feature transformation, acceleration robust features and the like); or the preset similarity algorithm may be a deep learning method, and the implementation manner of the preset similarity algorithm is not limited in this embodiment.
Under the condition that the similarity between the current image and the preset shared image is calculated through the peak signal-to-noise ratio, the preset similarity threshold is the preset peak signal-to-noise ratio, and the value can be 20 dB, 40 dB or 50 dB; under the condition that the similarity between the current image and the preset shared image is calculated through the structural similarity index, the preset similarity threshold is the preset structural similarity index, and the value can be 0.4, 0.5 or 0.7.
Such as: taking the structural similarity index between the current image and a preset shared image as an example, and setting a preset similarity threshold value to be 0.7; under the condition that the structural similarity instruction between the image and the preset shared picture is smaller than 0.7, sliding the detection window according to the preset moving bit number; in the case where the structural similarity instruction between the image and the preset shared picture is greater than or equal to 0.7, the position of the detection window at this time is recorded as a synchronization position.
The semantic synchronization method based on the fixed image provided by the application is described in detail by taking a transmitting end and a receiving end as examples. The sending end and the receiving end refer to terminals which need to perform semantic synchronization based on fixed images, and the equipment type of the sending end and the equipment type of the receiving end are the same or different.
It should be noted that the transmitting end and the receiving end are only for functional distinction, and in actual implementation, the same terminal may be either the transmitting end, i.e., a terminal that generates and transmits a data sequence, or the receiving end, i.e., a terminal that determines a synchronization position in the data sequence. Comprising at least the following steps S201 to S210:
In step S201, the transmitting end encodes the preset shared picture through a semantic encoder to obtain a corresponding first feature matrix, and encodes the current data to be transmitted through the preset encoder to obtain a corresponding second feature matrix.
Step S202, a transmitting end expands the first feature matrix and the second feature matrix into one-dimensional vectors to respectively obtain a first vector sequence corresponding to the first feature matrix and a second vector sequence corresponding to the second feature matrix.
In step S203, the transmitting end sorts each vector in the first vector sequence in descending order according to the importance degree, so as to obtain a third vector sequence.
In step S204, the transmitting end determines the first n vectors in the third vector sequence as the synchronization header.
In step S205, the transmitting end splices the second vector sequence in the synchronization header to obtain a data sequence.
In step S206, the transmitting end transmits the data sequence to the receiving end.
In step S207, the receiving end receives the data sequence.
Step S208, the receiving end slides the data sequence through a detection window with the window length of n, and decodes the sequence in the detection window through a preset decoder to obtain a current image corresponding to the current position of the detection window.
In step S209, the receiving end determines a similarity between the current image and the preset shared image.
Step S210, determining the current position of the detection window as a synchronous position under the condition that the similarity is larger than or equal to a preset similarity threshold value.
Under the condition that the similarity is smaller than a preset similarity threshold, moving the detection window according to the preset number of moving bits; and executing the step of decoding the sequence in the detection window through a preset decoder to obtain a current image corresponding to the current position of the detection window.
According to the semantic synchronization method based on the fixed image, a sending end obtains a preset shared picture and data to be sent; respectively encoding and expanding a preset shared picture and data to be transmitted to obtain a first vector sequence and a second vector sequence; after the vectors in the first vector sequence are sequenced according to the importance degree, determining n vectors in the front of the sequence as synchronous heads and splicing the first vector sequence with the second vector sequence to obtain a data sequence; transmitting the data sequence to a receiving end based on a communication channel; the receiving end carries out sliding window on the data sequence through a detection window with the window length of n, and decodes the data sequence through a preset decoder to obtain a current image; and under the condition that the similarity between the current image and the preset shared image is greater than or equal to a preset similarity threshold value, determining the current position of the detection window as a synchronous position, and if not, continuing sliding the window. The problem that the accuracy of data synchronization is low due to the fact that a system clock may drift due to the difference of hardware and software can be solved; the method comprises the steps that a preset shared picture is encoded through a preset encoder and then unfolded into a first vector sequence, and in the first vector sequence, a synchronization head required by synchronization is determined based on the importance degree of each vector, and a time stamp is not needed, so that inaccuracy of data synchronization caused by clock drift can be avoided, and the accuracy of the data synchronization is improved; meanwhile, vectors in the first vector sequence are ordered according to the descending order of importance, and the first n vectors with higher importance in the third vector sequence are selected as synchronous heads, so that after a receiving end receives a data sequence, the receiving end can decode the data sequence to obtain correct pictures through a detection window with the window length of n and a preset decoder, and the success rate of data synchronization is improved.
Further, the transmitting end and the receiving end may determine the value of n in advance according to the current channel state of the communication channel. Under the condition that the current channel state is good, the determined value of n is smaller, so that the length of the synchronous head sequence can be reduced, the overhead and the transmission time of synchronous head transmission are reduced, the occupation of bandwidth is reduced, the instantaneity of data transmission is ensured, and the transmission efficiency is improved; under the condition that the current channel state is poor, the determined value of n is larger, so that the anti-interference capability of the synchronous head in the transmission process can be improved, the synchronous precision is ensured, and the accuracy and the success rate of data synchronization are further improved.
Fig. 3 is a block diagram of a semantic synchronization apparatus based on a fixed image according to an embodiment of the present application. The embodiment is applied to a transmitting end by the device, and the device at least comprises the following modules: a data encoding module 310, a matrix unfolding module 320, a vector ordering module 330, a synchronization header determination module 340, a sequence splicing module 350, and a sequence transmitting module 270.
The data encoding module 310 is configured to encode the preset shared picture by using a semantic encoder to obtain a corresponding first feature matrix, and encode the current data to be transmitted by using the preset encoder to obtain a corresponding second feature matrix.
The matrix expansion module 320 is configured to expand the first feature matrix and the second feature matrix into one-dimensional vectors, so as to obtain a first vector sequence corresponding to the first feature matrix and a second vector sequence corresponding to the second feature matrix.
The vector sorting module 330 is configured to sort each vector in the first vector sequence in descending order according to the importance degree, so as to obtain a third vector sequence.
The synchronization header determining module 340 is configured to determine the first n vectors of the sequence in the third vector sequence as a synchronization header.
The sequence splicing module 350 is configured to splice the second vector sequence to the synchronization header to obtain a data sequence.
The sequence sending module 360 is configured to send the data sequence to a receiving end.
For relevant details reference is made to the method embodiments described above.
It should be noted that: in the semantic synchronization apparatus based on fixed images provided in the foregoing embodiments, only the division of the functional modules is used for illustration when the semantic synchronization based on fixed images is performed, and in practical applications, the functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the semantic synchronization apparatus based on fixed images is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the semantic synchronization device based on the fixed image provided in the above embodiment belongs to the same concept as the semantic synchronization method embodiment based on the fixed image, and the specific implementation process of the semantic synchronization device based on the fixed image is detailed in the method embodiment, which is not described herein.
This embodiment provides a semantic synchronization apparatus based on a fixed image, as shown in fig. 4, which includes at least a processor 410 and a memory 420.
Processor 410 may include one or more processing cores such as: 4 core processors, 8 core processors, etc. The processor 410 may be implemented in at least one hardware form of DSP (DIGITAL SIGNAL processing), FPGA (field-programmable gate array), PLA (Programmable Logic Array ). Processor 410 may also include a main processor, which is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 410 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 410 may also include an AI (ARTIFICIAL INTELLIGENCE ) processor for processing computing operations related to machine learning.
Memory 420 may include one or more computer-readable storage media, which may be non-transitory. Memory 420 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 420 is used to store at least one instruction for execution by processor 410 to implement the fixed image based semantic synchronization method provided by the method embodiments of the present application.
In some embodiments, the apparatus may further optionally include: a peripheral interface and at least one peripheral. The processor 410, memory 420, and peripheral interfaces may be connected by buses or signal lines. The individual peripheral devices may be connected to the peripheral device interface via buses, signal lines or circuit boards. Illustratively, peripheral devices include, but are not limited to: radio frequency circuitry, touch display screens, audio circuitry, and power supplies, among others.
Of course, the fixed image based semantic synchronization apparatus may also include fewer or more components, which is not limited in this embodiment.
Optionally, the present application further provides a computer readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the fixed image-based semantic synchronization method of the above method embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or a combination of both. The particular implementation is hardware or software dependent on the specific application of the solution and the design constraints. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the vectors of the present invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave.
It should be understood that the invention is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. The method processes of the present invention are not limited to the specific steps described and shown, but various changes, modifications and additions, or the order between steps may be made by those skilled in the art after appreciating the spirit of the present invention.
In this disclosure, features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, and various modifications and variations can be made to the embodiments of the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A semantic synchronization method based on fixed images, comprising:
encoding the preset shared picture through a semantic encoder to obtain a corresponding first feature matrix, and encoding the current data to be transmitted through the preset encoder to obtain a corresponding second feature matrix;
Expanding the first feature matrix and the second feature matrix into one-dimensional vectors to obtain a first vector sequence corresponding to the first feature matrix and a second vector sequence corresponding to the second feature matrix;
sorting each vector in the first vector sequence in a descending order according to the importance degree to obtain a third vector sequence;
Determining the first n vectors of the sequence in the third vector sequence as synchronous heads; the n is an integer greater than 0 and less than the total number of vectors in the third vector sequence;
Splicing the second vector sequence after the synchronization head to obtain a data sequence;
And sending the data sequence to a receiving end.
2. The fixed image based semantic synchronization method of claim 1, wherein the ordering each vector in the first vector sequence in descending order of importance to obtain a third vector sequence comprises:
Determining the entropy of each vector in the first sequence of vectors;
and sorting vectors in the first vector sequence in a descending order according to the entropy of each vector to obtain the third vector sequence.
3. The fixed image based semantic synchronization method of claim 1, wherein before determining the first n vectors of the third vector sequence as synchronization heads, further comprising:
acquiring current channel information of a communication channel between the receiving end and the receiving end;
And determining the vector quantity corresponding to the current channel information from the vector quantity based on a preset relation between the channel information and the vector quantity, wherein the vector quantity is used as n.
4. The fixed image based semantic synchronization method according to claim 1, wherein the preset encoder comprises a preset semantic encoder or a preset base encoder.
5. The fixed image-based semantic synchronization method according to claim 4, wherein a model structure of the semantic encoder is different from a model structure of the preset semantic encoder in a case where a data type of the data to be transmitted is the picture data.
6. The fixed image-based semantic synchronization method according to claim 1, wherein the preset expansion strategy corresponding to the first feature matrix is the same as or different from the preset expansion strategy corresponding to the second feature matrix.
7. The semantic synchronization method based on fixed images according to claim 1, wherein the receiving end is configured to decode the data sequence through a detection window with a window length of n and a semantic decoder after receiving the data sequence, and determine a synchronization position based on a similarity between a current image obtained by decoding and a preset shared picture.
8. The fixed image-based semantic synchronization method according to claim 1, wherein the data type of the data to be transmitted includes image data, voice data, text data, video data, point cloud data, or augmented reality data.
9. A fixed image based semantic synchronization apparatus comprising a processor and a memory, wherein the memory has stored therein computer instructions for executing the computer instructions stored in the memory, which when executed by the processor, implement the steps of the method of any one of claims 1 to 8.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311835425.XA CN117914459A (en) | 2023-12-28 | 2023-12-28 | Semantic synchronization method, device and storage medium based on fixed image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311835425.XA CN117914459A (en) | 2023-12-28 | 2023-12-28 | Semantic synchronization method, device and storage medium based on fixed image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117914459A true CN117914459A (en) | 2024-04-19 |
Family
ID=90689000
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311835425.XA Pending CN117914459A (en) | 2023-12-28 | 2023-12-28 | Semantic synchronization method, device and storage medium based on fixed image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117914459A (en) |
-
2023
- 2023-12-28 CN CN202311835425.XA patent/CN117914459A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102513867B1 (en) | Point cloud geometry compression | |
| US11454710B2 (en) | Point cloud compression using a space filling curve for level of detail generation | |
| US10798405B2 (en) | Image encoding method and apparatus, and image decoding method and apparatus | |
| US8265407B2 (en) | Method for coding and decoding 3D data implemented as a mesh model | |
| RU2565505C2 (en) | Method and apparatus for encoding and decoding transformation coefficients | |
| CN110413812B (en) | Neural network model training method and device, electronic equipment and storage medium | |
| WO2019001024A1 (en) | Method for determining motion vector predicted value and related device | |
| WO2020119487A1 (en) | Point cloud encoding and decoding method and device | |
| CN110659639B (en) | Chinese character recognition method and device, computer readable medium and electronic equipment | |
| CN109151477B (en) | Image data encoding and decoding methods and devices | |
| CN103581684A (en) | Compression encoding and decoding method and apparatus | |
| CN103369315A (en) | Coding and decoding methods, equipment and system of intra-frame chroma prediction modes | |
| US8582876B2 (en) | Hybrid codec for compound image compression | |
| CN113645600A (en) | Data transmission method, device, terminal and storage medium | |
| CN107431809A (en) | The method and apparatus of image procossing | |
| CN117914459A (en) | Semantic synchronization method, device and storage medium based on fixed image | |
| CN116912556A (en) | Picture classification method and device, electronic equipment and storage medium | |
| CN111405293B (en) | Video transmission method and device | |
| CN113327302A (en) | Picture processing method and device, storage medium and electronic device | |
| CN106874979B (en) | Bar code processing, displaying and reading method and device | |
| US8996361B2 (en) | Method and device for determining a decoding mode of in-band signaling | |
| CN112114760A (en) | Image processing method and device | |
| CN117411608A (en) | Semantic synchronization method, device and storage medium based on fixed image | |
| CN112261065A (en) | Method and device for compression transmission and calculation decompression of satellite communication | |
| US9955163B2 (en) | Two pass quantization of video data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |