CN110401836B - Image decoding and encoding method, device and equipment - Google Patents

Image decoding and encoding method, device and equipment Download PDF

Info

Publication number
CN110401836B
CN110401836B CN201810380105.2A CN201810380105A CN110401836B CN 110401836 B CN110401836 B CN 110401836B CN 201810380105 A CN201810380105 A CN 201810380105A CN 110401836 B CN110401836 B CN 110401836B
Authority
CN
China
Prior art keywords
neural network
index value
image
bit stream
sequence data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810380105.2A
Other languages
Chinese (zh)
Other versions
CN110401836A (en
Inventor
陈方栋
王莉
武晓阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN201810380105.2A priority Critical patent/CN110401836B/en
Priority to CN202210623408.9A priority patent/CN115115720A/en
Priority to PCT/CN2019/079807 priority patent/WO2019205871A1/en
Publication of CN110401836A publication Critical patent/CN110401836A/en
Application granted granted Critical
Publication of CN110401836B publication Critical patent/CN110401836B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream

Abstract

The application provides an image decoding method, an image encoding method, an image decoding device, an image encoding device and equipment, wherein the image decoding method comprises the following steps: acquiring a coded bit stream, wherein the coded bit stream carries coded image sequence data; obtaining neural network decision information, and selecting a neural network corresponding to the neural network decision information from a neural network set; decoding the encoded image sequence data using the selected neural network. By the technical scheme, the coding performance and the decoding performance can be higher. Furthermore, the neural network used at the decoding end is the same as the neural network used at the encoding end, so that the encoded image sequence data can be correctly decoded.

Description

Image decoding and encoding method, device and equipment
Technical Field
The present application relates to the field of video encoding and decoding, and in particular, to a method, an apparatus, and a device for image decoding and encoding.
Background
Neural Networks (NN) are a non-programmed, brain-like style information processing approach, and the essence of Neural networks is: the parallel distributed information processing function is obtained through network transformation and dynamic behaviors, and the information processing function of the human cranial nerve system is simulated in different degrees and layers. The neural network is an operation model, and the processing unit of the neural network can comprise an input unit, an output unit and a hidden unit. The input unit is used for receiving external signals and data, the output unit is used for outputting a processing result, and the hidden unit is a unit which is positioned between the input unit and the output unit and cannot be observed from the outside.
The neural network can adaptively construct feature description under the drive of training data, and has higher flexibility and universality, so that the neural network is widely applied to the fields of image classification, target detection, image coding, image decoding and the like. In order to realize image coding and image decoding, the same neural network is arranged at an encoding end and a decoding end, the encoding end uses the neural network to carry out image coding, and the decoding end uses the neural network to carry out image decoding, so that the neural network is utilized to realize image coding and image decoding.
The above method is to use a fixed neural network to realize image encoding and image decoding, and the encoding performance and the decoding performance may be low. For example, the neural network a is provided at the encoding end and the decoding end, and the neural network a may not be suitable for image encoding and image decoding, and thus, the encoding performance and the decoding performance are low.
Disclosure of Invention
The application provides an image decoding method, an image encoding method, an image decoding device, an image encoding device and image decoding equipment, which can improve encoding performance and decoding performance.
The application provides an image decoding method, which is applied to a decoding end and comprises the following steps:
acquiring a coded bit stream, wherein the coded bit stream carries coded image sequence data;
obtaining neural network decision information, and selecting a neural network corresponding to the neural network decision information from a neural network set;
decoding the encoded image sequence data using the selected neural network.
The application provides an image coding method, which is applied to a coding end and comprises the following steps:
selecting a neural network from a set of neural networks;
coding the image sequence data by using the selected neural network to obtain coded image sequence data;
and sending a coded bit stream to a decoding end, wherein the coded bit stream carries the coded image sequence data.
The application provides an image decoding device, is applied to the decoding end, the device includes:
the device comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a coded bit stream, and the coded bit stream carries coded image sequence data;
the selection module is used for acquiring neural network decision information and selecting a neural network corresponding to the neural network decision information from a neural network set;
a decoding module for decoding the encoded image sequence data using the selected neural network.
The application provides an image coding device, is applied to the encoding end, the device includes:
a selection module for selecting a neural network from the set of neural networks;
the encoding module is used for encoding the image sequence data by using the selected neural network to obtain encoded image sequence data;
and the sending module is used for sending a coded bit stream to a decoding end, wherein the coded bit stream carries the coded image sequence data.
The application provides a decoding side device, a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions capable of being executed by the processor; the processor is used for executing machine executable instructions to realize the image decoding method.
The application provides a coding end device, a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions capable of being executed by the processor; the processor is used for executing machine executable instructions to realize the image coding method.
According to the technical scheme, in the embodiment of the application, the encoding end can select one neural network from a plurality of neural networks in the neural network set and encode the image sequence data by using the neural network instead of encoding the image sequence data by using a fixed neural network, so that the encoding performance is higher. The decoding end may select one neural network from a plurality of neural networks of the neural network set and decode the encoded image sequence data using the neural network, instead of decoding the encoded image sequence data using a fixed neural network, thereby making the decoding performance higher. In addition, the neural network used by the decoding side is the same as the neural network used by the encoding side, so that the encoded image sequence data can be correctly decoded.
Drawings
FIG. 1 is a flow chart of an image encoding method in one embodiment of the present application;
FIG. 2 is a flow chart of an image encoding method in another embodiment of the present application;
FIG. 3 is a flow chart of an image decoding method in one embodiment of the present application;
FIG. 4 is a flow chart of an image decoding method in another embodiment of the present application;
fig. 5 is a block diagram of an image decoding apparatus according to an embodiment of the present application;
FIG. 6 is a block diagram of an image encoding apparatus according to an embodiment of the present application;
fig. 7 is a hardware configuration diagram of a decoding-side device according to an embodiment of the present application;
fig. 8 is a hardware configuration diagram of an encoding-side device according to an embodiment of the present application.
Detailed Description
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein is meant to encompass any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Depending on the context, moreover, the word "if" as used may be interpreted as "at … …" or "when … …" or "in response to a determination".
The embodiment of the application provides an image coding method and an image decoding method corresponding to the image coding method. At the encoding end, a neural network set can be set, and the neural network set can comprise at least one neural network; when encoding the image sequence data, instead of encoding the image sequence data using a fixed neural network, one neural network is selected from a set of neural networks and the image sequence data is encoded using the selected neural network. At the decoding end, a neural network set identical to the encoding end is set, when decoding the encoded image sequence data, one neural network is selected from the neural network set (identical to the neural network selected by the encoding end), and the selected neural network is used for decoding the encoded image sequence data, instead of decoding the encoded image sequence data by using a fixed neural network.
In the traditional method, a fixed neural network is used for encoding and decoding, and since the neural network is still in a fast iterative process, a new neural network is layered endlessly, and therefore, which neural network has the best performance cannot be determined. For example, when encoding the image sequence data a, the performance of the neural network 1 is superior to that of the neural network 2, and when encoding the image sequence data B, the performance of the neural network 2 is superior to that of the neural network 1. Obviously, when the encoding end encodes the image sequence data B by using a fixed neural network (e.g., the neural network 1), the encoding performance is low. When the decoding end decodes the encoded image sequence data of the image sequence data B using a fixed neural network (e.g., the neural network 1), the decoding performance is low.
In a manner different from the above, in the embodiment of the present application, the neural network 1 and the neural network 2 may be included in the neural network set at both the encoding side and the decoding side. When the image sequence data A is coded, the coding end can select the neural network 1 from the neural network set and code the image sequence data A by using the neural network 1, so that the coding performance is improved; when encoding the image sequence data B, the encoding end may select the neural network 2 from the neural network set, and encode the image sequence data B using the neural network 2, thereby improving encoding performance. Similarly, the decoding end may select the neural network 1 from the neural network set, and decode the encoded image sequence data of the image sequence data a using the neural network 1, thereby improving the decoding performance. The decoding side may select the neural network 2 from the neural network set, and decode the encoded image sequence data of the image sequence data B using the neural network 2, thereby improving decoding performance.
The following describes an image encoding method and an image decoding method in detail with reference to specific embodiments.
The first embodiment is as follows:
referring to fig. 1, a flowchart of an image encoding method is shown, where the method may include:
step 101, the encoding end selects a neural network from a neural network set, wherein the neural network set comprises at least one neural network. The neural networks in the neural network set may include, but are not limited to: convolutional neural networks (CNN for short), recurrent neural networks (RNN for short), fully-connected networks, and the like.
Before step 101, the encoding end may establish a neural network set, for example, the neural network set may include a neural network 1, a neural network 2, and a neural network 3, and the neural network set may be the same as the neural network set of the decoding end. In step 101, the encoding end may select one neural network from the neural network 1, the neural network 2, and the neural network 3 of the neural network set, for example, to select the neural network 1.
In step 101, the encoding end selects a neural network from the neural network set, which may include but is not limited to: one neural network is selected from the set of neural networks based on the performance parameters of each neural network in the set of neural networks. Alternatively, a first neural network is selected from the set of neural networks. Alternatively, the last neural network is selected from the set of neural networks. Alternatively, a neural network is randomly selected from a set of neural networks.
Of course, the above-mentioned manner is only an example, and the selection manner is not limited, and the example of "selecting one neural network from the neural network set according to the performance parameter of each neural network in the neural network set" is taken as an example.
Further, selecting a neural network from the set of neural networks according to the performance parameter of each neural network in the set of neural networks may further include, but is not limited to, the following: in the first mode, one neural network can be selected from the neural network set according to the computing capability of the encoding end and the computing complexity of each neural network in the neural network set. Or, in the second mode, the coding performance of each neural network in the neural network set can be obtained, and one neural network with the optimal coding performance is selected from the neural network set.
In the first method, the encoding end can know its own computation capability, and taking the computation capability as an addition operation as an example, if the encoding end processes at most 500 ten thousand addition operations per second, the computation capability of the encoding end is 500 ten thousand. Of course, other ways of representing the computing power may be used, and this is not a limitation. In addition, for each neural network in the neural network set, the encoding end can know the operation complexity of each neural network, and taking the operation complexity as how many times of addition operation needs to be executed per second as an example, if the neural network 1 needs to execute 300 ten thousand times of addition operation per second, the neural network 2 needs to execute 400 ten thousand times of addition operation per second, and the neural network 3 needs to execute 600 ten thousand times of addition operation per second, the encoding end knows that the operation complexity of the neural network 1 is 300 ten thousand, the operation complexity of the neural network 2 is 400 ten thousand, and the operation complexity of the neural network 3 is 600 ten thousand. Of course, other ways to represent the computational complexity of the neural network may also be used, without limitation.
Based on the computing power of the encoding end and the computing complexity of each neural network in the neural network set, the encoding end selects one neural network from the neural network set, including but not limited to: and selecting a neural network with the operation complexity smaller than the operation capability and the largest difference between the operation capability and the operation complexity from the neural network set. For example, the operational complexity of 300 ten thousand of the neural network 1 and the operational complexity of 400 ten thousand of the neural network 2 are both smaller than the operational capability of 500 ten thousand of the encoding end, and the difference between the operational capability of 500 ten thousand and the operational complexity of 300 ten thousand is the largest, so the neural network 1 is selected. Or selecting a neural network from the neural network set, wherein the operation complexity is smaller than the operation capability, and the difference between the operation capability and the operation complexity is minimum. For example, the difference between 500 ten thousand of computation power and 400 ten thousand of computation complexity is the smallest, and therefore the neural network 2 is selected. Of course, the above-mentioned method is only an example, and the method is not limited thereto, as long as the operation complexity of the selected neural network is smaller than the operation capability.
In the second mode, the encoding end may encode the image sequence data by using each neural network in the neural network set, determine the encoding performance of each neural network according to the encoding result, and select the neural network with the optimal encoding performance. For example, if the encoding performance of the neural network 1 is better than that of the neural network 2, and the encoding performance of the neural network 2 is better than that of the neural network 3, the neural network 1 may be selected.
Of course, when the encoding end encodes the image sequence data by using each neural network in the neural network set, the encoding end only selects the neural network with the optimal encoding performance, rather than the real encoding process, and therefore, the encoded image sequence data is not sent to the decoding end by the encoding end, but is discarded by the encoding end.
In order to determine the encoding performance of each neural network, RDO (Rate Distortion Optimized) may be used to determine the encoding performance of each neural network, that is, after the image sequence data is encoded by using the neural network, the RDO may be used to determine the encoding performance of the neural network. Of course, the encoding performance of the neural network may also be determined in other ways, which is not limited in this regard.
In step 101, the initial order of each neural network in the set of neural networks may be pre-configured, without limitation. For example, the initial order is the neural network 1, the neural network 2, and the neural network 3, and the initial order of the neural network set at the decoding end is also the neural network 1, the neural network 2, and the neural network 3.
During subsequent processing, the order of each neural network in the set of neural networks may be fixed, i.e., the order does not change. Alternatively, the order of each neural network in the set of neural networks may also be changed. For example, the neural networks may be sorted in the order from the highest to the lowest in actual use frequency (i.e., the first neural network is the neural network with the highest use frequency, and the last neural network is the neural network with the lowest use frequency); or sorting is performed according to the sequence of the actual use frequency from small to large. (i.e., the first neural network is the least frequently used neural network and the last neural network is the most frequently used neural network).
In step 102, the encoding end encodes the image sequence data by using the selected neural network to obtain encoded image sequence data (for convenience of distinction, the encoded image sequence data is referred to as encoded image sequence data).
The encoding end encodes the image sequence data by using the selected neural network to obtain encoded image sequence data, and the encoding method comprises the following steps of: coding the image sequence by using the selected neural network to obtain coded image sequence data; or, coding the image by using the selected neural network to obtain coded image sequence data; or, coding the image block by using the selected neural network to obtain coded image sequence data.
For example, when encoding image sequence a, assume that image sequence a includes image a1 and image a2, image a1 includes tile a11 and tile a12, and image a2 includes tile a21 and tile a 22.
If the selected neural network is suitable for the image sequence, a neural network may be selected for the image sequence a, and taking the neural network 1 as an example, the entire image sequence a is encoded using the neural network 1.
If the selected neural network is applicable to the entire image, one neural network may be selected for image A1, taking neural network 1 as an example, and image A1 is encoded using neural network 1. A neural network may also be selected for image a2, and in the example of neural network 2, image a2 is encoded using neural network 2.
Furthermore, if the selected neural network is only applicable to the image block, one neural network may be selected for image block a11, and for example, neural network 1 is used to encode image block a 11. A neural network may be selected for tile a12, and using neural network 2 for example, tile a12 is encoded. Furthermore, a neural network may also be selected for tile A21, and for example, neural network 3 may be used to encode tile A21. A neural network may be selected for tile a22, and using neural network 2 for example, tile a22 is encoded.
When encoding the image sequence data by using the selected neural network, the prediction module may select a neural network from the set of neural networks and predict the image sequence data by using the selected neural network, considering that the encoding process involves modules such as prediction, transformation, quantization, entropy encoding, filtering, and the like; the transformation module can select a neural network from the neural network set and transform the image sequence data by using the selected neural network; the quantization module can select a neural network from the neural network set and quantize the image sequence data by using the selected neural network; the entropy coding module can select a neural network from the neural network set and carry out entropy coding on the image sequence data by using the selected neural network; the filtering module may select a neural network from the set of neural networks and filter the image sequence data using the selected neural network.
Step 103, the encoding end sends a coded bit stream to the decoding end, and the coded bit stream carries coded image sequence data.
If the explicit policy is adopted to send the coded bit stream to the decoding end, before the coding end sends the coded bit stream to the decoding end, the index value of the selected neural network in the neural network set may also be determined (also referred to as an index position, for example, when the first neural network of the neural network set is selected, the index value is 1). Based on this, when the encoding end sends the encoded bit stream to the decoding end, the index value may also be added to the encoded bit stream, that is, the encoded bit stream may carry the encoded image sequence data and the index value. When the encoding end adds an index value to the encoded bit stream, the encoding end may encode the index value and add the encoded index value to the encoded bit stream.
A neural network decision policy may be configured at the encoding end and the decoding end, where the neural network decision policy may be a default index value, for example, the default index value is 1. Based on this, before the encoding end sends the encoded bit stream to the decoding end, the index value of the selected neural network in the neural network set may also be determined, and if the index value is 1, the encoded bit stream may be sent to the decoding end by using an implicit policy, that is, the encoded bit stream does not carry the index value 1, for example, only the encoded image sequence data may be carried. If the index value is not 1, for example, the index value is 2, an explicit policy may be adopted to send the encoded bitstream to the decoding end, that is, the encoded bitstream carries the index value 2, for example, the encoded image sequence data and the index value 2 may be carried. In addition, the neural network decision policy may also be an index value of the last time, that is, an index value corresponding to a coded bit stream that is sent by the coding end to the decoding end last time (the coded bit stream that is sent last time may be a coded bit stream that is sent by using an explicit policy, or may be a coded bit stream that is sent by using an implicit policy), for example, the last index value is 1. Based on this, before the encoding end sends the encoded bit stream to the decoding end, the index value of the selected neural network in the neural network set may also be determined, and if the index value is 1, the encoded bit stream may be sent to the decoding end by using an implicit policy, that is, the encoded bit stream does not carry the index value 1, for example, only the encoded image sequence data may be carried. If the index value is not 1, for example, the index value is 2, an explicit policy may be adopted to send the encoded bitstream to the decoding end, that is, the encoded bitstream carries the index value 2, for example, the encoded image sequence data and the index value 2 may be carried. Of course, the default index value and the last index value are only examples of the neural network decision policy, and are not limited thereto, as long as the encoding end can determine the index value of the neural network in the neural network set by using the neural network decision policy.
The encoding end encodes the index value, and when the encoded index value is added to the encoded bit stream, the index value can be encoded by adopting a variable length encoding strategy, and the encoded index value is added to the encoded bit stream; or the index value is coded by adopting a fixed-length coding strategy, and the coded index value is added into the coded bit stream.
When the index value is encoded using the variable length coding scheme, the length of the encoded index value is variable. For example, when the index value is 1, the length of the encoded index value is 1 bit, for example, the encoded index value is 1; when the index value is 2, the length of the encoded index value is 2 bits, for example, the encoded index value is 10; when the index value is 4, the length of the encoded index value is 3 bits, for example, the encoded index value is 100; and so on.
When the index value is encoded by using the fixed-length encoding strategy, the length of the encoded index value is fixed, for example, fixed to 4 bits. For example, when the index value is 1, the encoded index value is 0001; when the index value is 2, the coded index value is 0010; when the index value is 4, the coded index value is 0100; and so on.
According to the technical scheme, in the embodiment of the application, the encoding end can select one neural network from a plurality of neural networks in the neural network set and encode the image sequence data by using the neural network instead of encoding the image sequence data by using a fixed neural network, so that the encoding performance is higher.
Example two:
referring to fig. 2, another flow chart of an image encoding method is shown, which may include:
in step 201, an encoding end acquires a neural network set from a plurality of neural network sets.
The encoding end can configure a plurality of neural network sets, and each neural network set can comprise at least one neural network; for example, the encoding end may configure a neural network set 1, a neural network set 2, and a neural network set 3. Based on this, the encoding side can select one neural network set from the neural network set 1, the neural network set 2, and the neural network set 3 when transmitting the image sequence data to the decoding side.
Further, assuming that the encoding end configures a neural network set 1 for the decoding end a, and configures a neural network set 2 and a neural network set 3 for the decoding end B, the encoding end may select the neural network set 1 when sending the image sequence data to the decoding end a; in addition, when sending the image sequence data to the decoding end B, the encoding end may select the neural network set 2 or the neural network set 3, and this selection method is not limited.
In step 202, the encoding end selects a neural network from the acquired neural network set.
The processing procedure of step 202 may refer to the processing of step 101, and is not described herein again.
And step 203, the encoding end encodes the image sequence data by using the selected neural network to obtain encoded image sequence data. The processing procedure of step 203 may refer to the processing of step 102, and is not described herein again.
Step 204, the encoding end sends the encoded bit stream to the decoding end, and the encoded bit stream carries the encoded image sequence data.
In step 103, when the encoding end sends the encoded bit stream to the decoding end, if a display policy is adopted, the encoded bit stream may carry the index value of the selected neural network in the neural network set, and if an implicit policy is adopted, the encoded bit stream does not carry the index value of the selected neural network in the neural network set.
After acquiring the neural network set from the plurality of neural network sets, the encoding end may further determine an identifier of the neural network set, for example, the identifier of the neural network set 1 may be 1, and so on.
Then, if the encoded bitstream is sent to the decoding end by using an explicit strategy, the encoding end may add the identification of the neural network set to the encoded bitstream. When the coding end adds the identification of the neural network set in the coded bit stream, the identification can be coded, and the coded identification is added in the coded bit stream.
In addition, a neural network set decision strategy can be configured at the encoding end and the decoding end, and the neural network set decision strategy can be a default identifier, such as a default identifier of 1. Based on this, if the identification of the neural network set is 1, an implicit strategy is adopted to send the coded bit stream to the decoding end, namely the coded bit stream does not carry the identification 1; and if the identification of the neural network set is 2, sending the coded bit stream to a decoding end by adopting an explicit strategy, namely the coded bit stream carries the identification 2. Or, the neural network set decision policy may be a last identifier, that is, an identifier corresponding to a coded bit stream that is sent by the coding end to the decoding end last time, for example, the last identifier is 1. Based on this, if the identification of the neural network set is 1, an implicit strategy is adopted to send the coded bit stream to the decoding end, namely the coded bit stream does not carry the identification 1; and if the identification of the neural network set is 2, sending the coded bit stream to a decoding end by adopting an explicit strategy, namely the coded bit stream carries the identification 2. Of course, the default identifier and the last identifier are only examples of the neural network set decision policy, and no limitation is made to the default identifier and the last identifier as long as the identifiers of the neural network set can be determined by using the neural network set decision policy.
The encoding end encodes the identification of the neural network set, and when the encoded identification is added to the encoded bit stream, the identification of the neural network set can be encoded by adopting a variable length encoding strategy, and the encoded identification of the neural network set is added to the encoded bit stream; alternatively, the fixed-length coding strategy can be adopted to code the identification of the neural network set, and the coded identification of the neural network set is added in the coded bit stream.
According to the technical scheme, in the embodiment of the application, the encoding end can select one neural network set from the plurality of neural network sets, select one neural network from the plurality of neural networks of the neural network set, and encode the image sequence data by using the neural network instead of encoding the image sequence data by using a fixed neural network, so that the encoding performance is higher, and the use experience of a user is improved.
Example three:
referring to fig. 3, a flowchart of an image decoding method is shown, where the method may include:
step 301, a decoding end obtains a coded bit stream, where the coded bit stream carries coded image sequence data.
The coded bit stream may be sent by the coding end to the decoding end, or obtained by locally coding the image sequence data by the decoding end, which is not limited to this, and then the coding end is taken as an example to send the coded bit stream to the decoding end, and the specific sending process may refer to the first embodiment or the second embodiment.
Step 302, a decoding end acquires neural network decision information and selects a neural network corresponding to the neural network decision information from a neural network set; wherein the set of neural networks includes at least one neural network.
The neural networks in the neural network set may include, but are not limited to: convolutional neural networks (CNN for short), recurrent neural networks (RNN for short), fully-connected networks, and the like.
Before step 301, the decoding end may establish a neural network set, for example, the neural network set may include a neural network 1, a neural network 2 and a neural network 3, and the neural network set may be the same as the neural network set of the encoding end. In step 302, the decoding end may select one neural network from the neural network 1, the neural network 2, and the neural network 3 of the neural network set, for example, the neural network 1 is selected.
The obtaining, by the decoding end, the neural network decision information, and selecting, from the neural network set, the neural network corresponding to the neural network decision information may include: and if the coded bit stream carries the index value of the neural network, the decoding end determines the index value as the neural network decision information, and selects the neural network corresponding to the index value from the neural network set. Or the decoding end determines an index value of the neural network according to the neural network decision strategy, determines the index value as neural network decision information, and selects the neural network corresponding to the index value from the neural network set. The determining, by the decoding end, the index value of the neural network according to the neural network decision policy may include: if the neural network decision policy is a default index value, determining the default index value as an index value of the neural network; or, if the neural network decision policy is the last index value, the last used index value may be determined as the index value of the neural network.
For example, if the encoding end sends the encoded bitstream to the decoding end by using an explicit policy, the encoded bitstream may also carry an index value of the neural network (i.e., an index value of the neural network in the neural network set), and if the index value of the neural network is 1, the decoding end may select a first neural network from the neural network set.
For another example, a neural network decision policy may be configured at the encoding end and the decoding end, the neural network decision policy may be a default index value, and the default index value may be 1. Based on this, if the encoding end sends the encoded bit stream to the decoding end by using the implicit strategy, the encoded bit stream does not carry the index value of the neural network, and the decoding end can determine the default index value 1 as the index value of the neural network and can select the first neural network from the neural network set. Alternatively, a neural network decision strategy can be configured at the encoding end and the decoding end, and the neural network decision strategy can be a last-time index value. Based on this, if the encoding end sends the coded bit stream to the decoding end by using the implicit policy, the coded bit stream does not carry an index value of the neural network, and the decoding end can determine an index value (for example, an index value 1) corresponding to the coded bit stream received last time (i.e., the coded bit stream sent by the encoding end by using the explicit policy or the coded bit stream sent by using the implicit policy) as an index value of the neural network, and can select a first neural network from the neural network set. Of course, the default index value and the last index value are only examples of the neural network decision policy, and the decoding end may determine the index value of the neural network in the neural network set by using the neural network decision policy without limitation.
When the coded bit stream carries the index value of the neural network, the index value may be the index value coded by using the variable length coding strategy, and therefore, after the decoding end acquires the coded bit stream, the decoding end may also decode the coded bit stream by using the variable length decoding strategy (i.e., the decoding strategy corresponding to the variable length coding strategy), so as to obtain the index value of the neural network. Or, the index value may be an index value encoded by using a fixed-length encoding policy, so that after the decoding end acquires the encoded bit stream, the decoding end may also decode the encoded bit stream by using a fixed-length decoding policy (i.e., a decoding policy corresponding to the fixed-length encoding policy) to obtain the index value of the neural network.
In step 302, the initial order of each neural network in the set of neural networks may be pre-configured, without limitation. For example, the initial order is the neural network 1, the neural network 2, and the neural network 3, and the initial order of the neural network set at the encoding end is also the neural network 1, the neural network 2, and the neural network 3.
During subsequent processing, the order of each neural network in the set of neural networks may be fixed, i.e., the order does not change. Alternatively, the order of each neural network in the set of neural networks may also be changed. For example, the neural networks may be sorted in the order from the highest to the lowest in actual use frequency (i.e., the first neural network is the neural network with the highest use frequency, and the last neural network is the neural network with the lowest use frequency); or sorting is performed according to the sequence of the actual use frequency from small to large. (i.e., the first neural network is the least frequently used neural network and the last neural network is the most frequently used neural network).
If the sequence of the neural network at the encoding end adopts a fixed mode, the sequence of the neural network at the decoding end also adopts a fixed mode; or, if the neural network at the encoding end is sequenced from the high to low of the actual use frequency, the neural network at the decoding end is also sequenced from the high to low of the actual use frequency; or, if the neural network at the encoding end is sorted in the order of the actual use frequencies from small to large, the neural network at the decoding end is also sorted in the order of the actual use frequencies from small to large.
And step 303, the decoding end decodes the coded image sequence data by using the selected neural network.
The decoding end decoding the encoded image sequence data by using the selected neural network may include: the image sequence can be decoded using the selected neural network; alternatively, the image may be decoded using the selected neural network; alternatively, the image block may be decoded using the selected neural network.
For example, assume that the image sequence a includes image a1 and image a2, image a1 includes tile a11 and tile a12, and image a2 includes tile a21 and tile a 22.
On this basis, if the selected neural network is applied to the image sequence, one neural network may be selected for the image sequence a, and taking the example of selecting the neural network 1, the encoded image sequence data corresponding to the entire image sequence a is decoded using the neural network 1.
If the selected neural network is applicable to the entire image, one neural network may be selected for the image a1, and taking the neural network 1 as an example, the encoded image sequence data corresponding to the image a1 may be decoded using the neural network 1. Further, a neural network may be selected for the image a2, and taking the neural network 2 as an example, the encoded image sequence data corresponding to the image a2 may be decoded using the neural network 2.
If the selected neural network is only applicable to the image block, one neural network may be selected for image block a11, and taking neural network 1 as an example, encoded image sequence data corresponding to image block a11 may be decoded using neural network 1. A neural network may be selected for image block a12, and for example, neural network 2 may be used to decode the encoded image sequence data corresponding to image block a 12. A neural network may also be selected for image block a21, and for example, neural network 3 may be used to decode the encoded image sequence data corresponding to image block a 21. A neural network may be selected for image block a22, and for example, neural network 2 may be used to decode the encoded image sequence data corresponding to image block a 22.
In summary, the decoding end may select one neural network from a plurality of neural networks of the neural network set, and decode the encoded image sequence data using the neural network, instead of decoding the encoded image sequence data using a fixed neural network, so that the decoding performance is high. The neural network used by the decoding end is the same as the neural network used by the encoding end, so that the encoded image sequence data is correctly decoded.
Example four:
referring to fig. 4, another flowchart of an image decoding method is shown, where the method may include:
step 401, a decoding end obtains a coded bit stream, where the coded bit stream carries coded image sequence data.
The processing procedure in step 401 may refer to the processing in step 301, and is not described herein again.
Step 402, the decoding end obtains the neural network set decision information and determines the neural network set corresponding to the neural network set decision information.
The decoding end can configure at least one neural network set, and each neural network set comprises at least one neural network; for example, the decoding side configures a neural network set 1 and a neural network set 2, and the decoding side may select one neural network set from the neural network set 1 and the neural network set 2.
The decoding end obtains the neural network set decision information and determines the neural network set corresponding to the neural network set decision information, which may include but is not limited to: if the coded bit stream carries an identifier of the neural network set, the decoding end may determine the identifier as the neural network set decision information, and determine the neural network set corresponding to the identifier. Or the decoding end determines the identification of the neural network set according to the neural network set decision strategy, determines the identification as the neural network set decision information, and determines the neural network set corresponding to the identification. Further, the decoding end determines the identity of the neural network set according to the neural network set decision policy, which may include but is not limited to: if the decision strategy of the neural network set is a default identification, the decoding end determines the default identification as the identification of the neural network set; or, if the decision strategy of the neural network set is the last identifier, the decoding end determines the identifier used last time as the identifier of the neural network set.
For example, if the encoding end sends the encoded bit stream to the decoding end by using an explicit policy, the encoded bit stream also carries an identifier of the neural network set, such as identifier 1, and the decoding end may select the neural network set 1 from all the neural network sets. For another example, a neural network aggregation decision policy is configured at the encoding end and the decoding end, the neural network aggregation decision policy may be a default identifier, and the default identifier may be 1. And if the encoding end sends the encoding bit stream to the decoding end by adopting an implicit strategy, the encoding bit stream does not carry the identification of the neural network set, the decoding end determines the default identification 1 as the identification of the neural network set, and selects the neural network set 1 from all the neural network sets. For another example, the decision policy of the neural network set may be a last identifier, if the encoding end sends the coded bit stream to the decoding end by using the implicit policy, the coded bit stream does not carry an identifier of the neural network set, the decoding end determines an identifier (for example, identifier 1) corresponding to the coded bit stream received last time as an identifier of the neural network set, and selects the neural network set 1 from all the neural network sets. Of course, the default identifier and the last identifier are only examples of the neural network set decision policy, and the neural network set decision policy is not limited as long as the decoding end can determine the identifier of the neural network set by using the neural network set decision policy.
When the coded bit stream carries the identifier of the neural network set, the identifier may be an identifier coded by using a variable length coding strategy, and therefore, after the decoding end obtains the coded bit stream, the decoding end may also decode the coded bit stream by using a variable length decoding strategy (i.e., a decoding strategy corresponding to the variable length coding strategy), so as to obtain the identifier of the neural network set. Or, the identifier may be an identifier encoded by using a fixed-length encoding policy, so that after the decoding end acquires the encoded bit stream, the decoding end may also decode the encoded bit stream by using a fixed-length decoding policy (i.e., a decoding policy corresponding to the fixed-length encoding policy), so as to obtain an identifier of the neural network set.
Step 403, the decoding end obtains neural network decision information, and selects a neural network corresponding to the neural network decision information from the neural network set; the set of neural networks includes at least one neural network.
The processing procedure in step 403 may refer to the processing in step 302, which is not described herein again.
And step 404, the decoding end decodes the coding image sequence data by using the selected neural network.
The processing procedure of step 404 may refer to the processing of step 303, which is not described herein again.
In summary, the decoding end may select a neural network set from at least one neural network set, select a neural network from a plurality of neural networks of the neural network set, and use the neural network to decode the encoded image sequence data, instead of using a fixed neural network to decode the encoded image sequence data, so that the decoding performance is higher, and the user experience is improved. The neural network used by the decoding end is the same as the neural network used by the encoding end, so that the encoded image sequence data is correctly decoded.
Based on the same application concept as the method, an image decoding apparatus applied to a decoding end is further provided in the embodiment of the present application, as shown in fig. 5, which is a structural diagram of the apparatus, and the apparatus specifically includes:
an obtaining module 501, configured to obtain a coded bit stream, where the coded bit stream carries coded image sequence data;
a selecting module 502, configured to obtain neural network decision information, and select a neural network corresponding to the neural network decision information from a neural network set;
a decoding module 503, configured to decode the encoded image sequence data by using the selected neural network.
The selecting module 502 is specifically configured to: if the coded bit stream carries an index value of a neural network, determining the index value as the neural network decision information, and selecting the neural network corresponding to the index value from a neural network set; or determining an index value of a neural network according to a neural network decision strategy, determining the index value as the neural network decision information, and selecting the neural network corresponding to the index value from a neural network set; further, the selecting module 502 is specifically configured to, when determining the index value of the neural network according to the neural network decision policy: if the neural network decision strategy is a default index value, determining the default index value as an index value of the neural network; or, if the neural network decision strategy is the last index value, determining the last used index value as the index value of the neural network.
The selecting module 502 is further configured to obtain neural network set decision information, and determine a neural network set corresponding to the neural network set decision information; further, the selecting module 502 obtains decision information of a neural network set, and when determining the neural network set corresponding to the decision information of the neural network set, is specifically configured to: if the coded bit stream carries an identifier of a neural network set, determining the identifier as decision information of the neural network set, and determining the neural network set corresponding to the identifier; or determining the identification of the neural network set according to the neural network set decision strategy, determining the identification as the neural network set decision information, and determining the neural network set corresponding to the identification.
Based on the same application concept as the method, an embodiment of the present application further provides an image encoding apparatus applied to an encoding end, as shown in fig. 6, which is a structural diagram of the apparatus, and the apparatus includes: a selecting module 601, configured to select a neural network from a neural network set; an encoding module 602, configured to encode the image sequence data by using the selected neural network to obtain encoded image sequence data; a sending module 603, configured to send a coded bit stream to a decoding end, where the coded bit stream carries the coded image sequence data.
The selection module is specifically configured to 601: selecting a neural network from the set of neural networks according to the performance parameters of each neural network in the set of neural networks;
the selecting module 601 specifically is configured to, when selecting a neural network from the neural network set according to the performance parameter of each neural network in the neural network set: selecting a neural network from the neural network set according to the operational capability of the encoding end and the operational complexity of each neural network in the neural network set; or acquiring the coding performance of each neural network in the neural network set, and selecting the neural network with the optimal coding performance from the neural network set.
Before the sending module 603 sends the encoded bitstream to the decoding end, it is further configured to: adding an index value of the selected neural network in the set of neural networks in the encoded bitstream.
Before the sending module 603 sends the encoded bitstream to the decoding end, it is further configured to: adding an identification of a set of neural networks in the encoded bitstream.
In terms of hardware, the hardware architecture diagram of the decoding-side device provided in the embodiment of the present application may specifically refer to fig. 7. The method comprises the following steps: a processor and a machine-readable storage medium, wherein: the machine-readable storage medium stores machine-executable instructions executable by the processor; the processor is used for executing machine executable instructions to realize the image decoding method disclosed by the above example of the application.
In terms of hardware, the hardware architecture diagram of the encoding end device provided in the embodiment of the present application may specifically refer to fig. 8. The method comprises the following steps: a processor and a machine-readable storage medium, wherein: the machine-readable storage medium stores machine-executable instructions executable by the processor; the processor is used for executing machine executable instructions to realize the image coding method disclosed by the above example of the application.
The machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: a RAM (random Access Memory), a volatile Memory, a non-volatile Memory, a flash Memory, a storage drive (e.g., a hard drive), a solid state drive, any type of storage disk (e.g., an optical disk, a dvd, etc.), or similar storage medium, or a combination thereof.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Furthermore, these computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (23)

1. An image decoding method, applied to a decoding end, the method comprising:
acquiring a coded bit stream, wherein the coded bit stream carries coded image sequence data;
obtaining neural network decision information, and selecting a neural network corresponding to the neural network decision information from a neural network set;
decoding the encoded image sequence data using the selected neural network; wherein the decoding end comprises a filtering module, and the decoding of the coded image sequence data by using the selected neural network comprises: the filtering module filters the encoded image sequence data using the selected neural network;
the obtaining of the neural network decision information and the selecting of the neural network corresponding to the neural network decision information from the neural network set include:
if the coded bit stream carries an index value of a neural network, determining the index value as the neural network decision information, and selecting the neural network corresponding to the index value from a neural network set; if the index value carried by the coded bit stream is the index value coded by adopting the variable length coding strategy, decoding the coded bit stream by adopting the variable length decoding strategy to obtain the index value of the neural network;
wherein if the image sequence data includes a plurality of frames of images:
the multi-frame images correspond to the same neural network, or the multi-frame images correspond to different neural networks; if the multiple frames of images correspond to the same neural network, the coded bit stream carries an index value of the neural network corresponding to the image sequence data; if the multiple frames of images correspond to different neural networks, the coded bit stream carries an index value of the neural network corresponding to each frame of image in the image sequence data;
alternatively, the first and second electrodes may be,
if the image sequence data comprises a plurality of frames of images, and each frame of image comprises a plurality of image blocks, then:
the plurality of image blocks correspond to the same neural network, or the plurality of image blocks correspond to different neural networks; if the plurality of image blocks correspond to the same neural network, the coded bit stream carries an index value of a neural network corresponding to a frame of image; and if the plurality of image blocks correspond to different neural networks, the coded bit stream carries the index value of the neural network corresponding to each image block in one frame of image.
2. The method of claim 1, wherein obtaining neural network decision information and selecting a neural network from a set of neural networks that corresponds to the neural network decision information further comprises:
and determining an index value of a neural network according to a neural network decision strategy, determining the index value as the neural network decision information, and selecting the neural network corresponding to the index value from a neural network set.
3. The method of claim 2,
the determining the index value of the neural network according to the neural network decision strategy includes: if the neural network decision strategy is a default index value, determining the default index value as the index value of the neural network; or, if the decision strategy of the neural network is the last index value, determining the last used index value as the index value of the neural network.
4. The method of claim 1, wherein prior to the selecting the neural network from the set of neural networks that corresponds to the neural network decision information, the method further comprises:
acquiring neural network set decision information, and determining a neural network set corresponding to the neural network set decision information.
5. The method of claim 4, wherein obtaining neural network set decision information and determining the neural network set corresponding to the neural network set decision information comprises:
and if the coded bit stream carries the identification of the neural network set, determining the identification as the decision information of the neural network set, and determining the neural network set corresponding to the identification.
6. The method of claim 5, wherein the flag is a flag encoded by using a variable length coding strategy, and after the obtaining the coded bit stream, the method further comprises: and decoding the coded bit stream by adopting a variable length decoding strategy to obtain the identification of the neural network set.
7. The method of claim 4, wherein obtaining neural network set decision information and determining the neural network set corresponding to the neural network set decision information comprises:
and determining the identification of the neural network set according to the neural network set decision strategy, determining the identification as the neural network set decision information, and determining the neural network set corresponding to the identification.
8. The method of claim 1,
at least one neural network included in the neural network set is sorted according to the sequence of the actual use frequency from large to small, or sorted according to the sequence of the actual use frequency from small to large.
9. An image encoding method applied to an encoding side, the method comprising:
selecting a neural network from a set of neural networks;
coding the image sequence data by using the selected neural network to obtain coded image sequence data; the encoding end comprises a filtering module, and the encoding of the image sequence data by using the selected neural network to obtain encoded image sequence data comprises the following steps: the filtering module filters the image sequence data by using the selected neural network to obtain coded image sequence data;
sending a coded bit stream to a decoding end, wherein the coded bit stream carries the coded image sequence data; and the coded bit stream also carries index values of the selected neural network in the set of neural networks;
when the index value is added to the coded bit stream, coding the index value by adopting a variable length coding strategy, and adding the coded index value to the coded bit stream;
wherein if the image sequence data includes a plurality of frames of images:
the multi-frame images correspond to the same neural network, or the multi-frame images correspond to different neural networks; if the multiple frames of images correspond to the same neural network, the coded bit stream carries an index value of the neural network corresponding to the image sequence data; if the multiple frames of images correspond to different neural networks, the coded bit stream carries an index value of the neural network corresponding to each frame of image in the image sequence data;
alternatively, the first and second electrodes may be,
if the image sequence data comprises a plurality of frames of images, and each frame of image comprises a plurality of image blocks, then:
the plurality of image blocks correspond to the same neural network, or the plurality of image blocks correspond to different neural networks; if the plurality of image blocks correspond to the same neural network, the coded bit stream carries an index value of a neural network corresponding to a frame of image; and if the plurality of image blocks correspond to different neural networks, the coded bit stream carries the index value of the neural network corresponding to each image block in one frame of image.
10. The method of claim 9,
the selecting a neural network from a set of neural networks, comprising: selecting a neural network from the set of neural networks based on the performance parameters of each neural network in the set of neural networks.
11. The method of claim 10, wherein selecting a neural network from the set of neural networks based on the performance parameters of each neural network in the set of neural networks comprises:
selecting a neural network from the neural network set according to the operational capability of the encoding end and the operational complexity of each neural network in the neural network set; alternatively, the first and second electrodes may be,
and acquiring the coding performance of each neural network in the neural network set, and selecting the neural network with the optimal coding performance from the neural network set.
12. The method of claim 9,
prior to the selecting a neural network from the set of neural networks, the method further comprises:
acquiring a neural network set and determining an identifier of the neural network set;
and when the coded bit stream is sent to a decoding end, the identification is added in the coded bit stream.
13. The method of claim 12,
said adding said identification in said coded bitstream comprises: and coding the identifier by adopting a variable length coding strategy, and adding the coded identifier in the coded bit stream.
14. The method of claim 9,
at least one neural network included in the neural network set is sorted according to the sequence of the actual use frequency from large to small, or sorted according to the sequence of the actual use frequency from small to large.
15. An image decoding apparatus applied to a decoding side, the apparatus comprising:
the device comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a coded bit stream, and the coded bit stream carries coded image sequence data;
the selection module is used for acquiring neural network decision information and selecting a neural network corresponding to the neural network decision information from a neural network set;
a decoding module for decoding the encoded image sequence data using the selected neural network; wherein the decoding module comprises a filtering module that filters the encoded image sequence data with the selected neural network;
wherein the selection module is specifically configured to:
if the coded bit stream carries an index value of a neural network, determining the index value as the neural network decision information, and selecting the neural network corresponding to the index value from a neural network set; if the index value carried by the coded bit stream is the index value coded by adopting the variable length coding strategy, decoding the coded bit stream by adopting the variable length decoding strategy to obtain the index value of the neural network;
wherein if the image sequence data includes a plurality of frames of images:
the multi-frame images correspond to the same neural network, or the multi-frame images correspond to different neural networks; if the multiple frames of images correspond to the same neural network, the coded bit stream carries an index value of the neural network corresponding to the image sequence data; if the multiple frames of images correspond to different neural networks, the coded bit stream carries an index value of the neural network corresponding to each frame of image in the image sequence data;
alternatively, the first and second electrodes may be,
if the image sequence data comprises a plurality of frames of images, and each frame of image comprises a plurality of image blocks, then:
the plurality of image blocks correspond to the same neural network, or the plurality of image blocks correspond to different neural networks; if the plurality of image blocks correspond to the same neural network, the coded bit stream carries an index value of a neural network corresponding to a frame of image; and if the plurality of image blocks correspond to different neural networks, the coded bit stream carries the index value of the neural network corresponding to each image block in one frame of image.
16. The apparatus of claim 15, wherein the selection module is further configured to: and determining an index value of a neural network according to a neural network decision strategy, determining the index value as the neural network decision information, and selecting the neural network corresponding to the index value from a neural network set.
17. The apparatus of claim 16,
the selection module is specifically configured to, when determining an index value of the neural network according to a neural network decision policy: if the neural network decision strategy is a default index value, determining the default index value as an index value of the neural network; or, if the neural network decision strategy is the last index value, determining the last used index value as the index value of the neural network.
18. The apparatus of claim 15,
the selection module is further used for acquiring neural network set decision information and determining a neural network set corresponding to the neural network set decision information;
the selection module acquires neural network set decision information, and is specifically configured to, when determining a neural network set corresponding to the neural network set decision information: if the coded bit stream carries an identifier of a neural network set, determining the identifier as decision information of the neural network set, and determining the neural network set corresponding to the identifier; or determining the identification of the neural network set according to the neural network set decision strategy, determining the identification as the neural network set decision information, and determining the neural network set corresponding to the identification.
19. An image encoding apparatus applied to an encoding side, the apparatus comprising:
a selection module for selecting a neural network from the set of neural networks;
the encoding module is used for encoding the image sequence data by using the selected neural network to obtain encoded image sequence data; the encoding module comprises a filtering module, and the filtering module is used for filtering the image sequence data by using the selected neural network to obtain encoded image sequence data;
a sending module, configured to send a coded bit stream to a decoding end, where the coded bit stream carries the coded image sequence data; and the coded bit stream also carries index values of the selected neural network in the set of neural networks; when the index value is added to the coded bit stream, coding the index value by adopting a variable length coding strategy, and adding the coded index value to the coded bit stream;
wherein if the image sequence data includes a plurality of frames of images:
the multi-frame images correspond to the same neural network, or the multi-frame images correspond to different neural networks; if the multiple frames of images correspond to the same neural network, the coded bit stream carries an index value of the neural network corresponding to the image sequence data; if the multiple frames of images correspond to different neural networks, the coded bit stream carries an index value of the neural network corresponding to each frame of image in the image sequence data;
alternatively, the first and second electrodes may be,
if the image sequence data comprises a plurality of frames of images, and each frame of image comprises a plurality of image blocks, then:
the plurality of image blocks correspond to the same neural network, or the plurality of image blocks correspond to different neural networks; if the plurality of image blocks correspond to the same neural network, the coded bit stream carries an index value of a neural network corresponding to a frame of image; and if the plurality of image blocks correspond to different neural networks, the coded bit stream carries the index value of the neural network corresponding to each image block in one frame of image.
20. The apparatus of claim 19,
the selection module is specifically configured to: selecting a neural network from the set of neural networks according to the performance parameters of each neural network in the set of neural networks;
the selecting module is specifically configured to, when selecting a neural network from the neural network set according to the performance parameter of each neural network in the neural network set: selecting a neural network from the neural network set according to the operational capability of the encoding end and the operational complexity of each neural network in the neural network set; or acquiring the coding performance of each neural network in the neural network set, and selecting the neural network with the optimal coding performance from the neural network set.
21. The apparatus of claim 19, wherein the sending module is further configured to, before sending the encoded bitstream to the decoding end: adding an identification of a set of neural networks in the encoded bitstream.
22. A decoding-side apparatus, comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute machine executable instructions to perform the method steps of any of claims 1-8.
23. An encoding side device, comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute machine executable instructions to perform the method steps of any of claims 9 to 14.
CN201810380105.2A 2018-04-25 2018-04-25 Image decoding and encoding method, device and equipment Active CN110401836B (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201810380105.2A CN110401836B (en) 2018-04-25 2018-04-25 Image decoding and encoding method, device and equipment
CN202210623408.9A CN115115720A (en) 2018-04-25 2018-04-25 Image decoding and encoding method, device and equipment
PCT/CN2019/079807 WO2019205871A1 (en) 2018-04-25 2019-03-27 Image decoding and encoding methods and apparatuses, and device thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810380105.2A CN110401836B (en) 2018-04-25 2018-04-25 Image decoding and encoding method, device and equipment

Related Child Applications (1)

Application Number Title Priority Date Filing Date
CN202210623408.9A Division CN115115720A (en) 2018-04-25 2018-04-25 Image decoding and encoding method, device and equipment

Publications (2)

Publication Number Publication Date
CN110401836A CN110401836A (en) 2019-11-01
CN110401836B true CN110401836B (en) 2022-04-26

Family

ID=68294729

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201810380105.2A Active CN110401836B (en) 2018-04-25 2018-04-25 Image decoding and encoding method, device and equipment
CN202210623408.9A Pending CN115115720A (en) 2018-04-25 2018-04-25 Image decoding and encoding method, device and equipment

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN202210623408.9A Pending CN115115720A (en) 2018-04-25 2018-04-25 Image decoding and encoding method, device and equipment

Country Status (2)

Country Link
CN (2) CN110401836B (en)
WO (1) WO2019205871A1 (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230188713A1 (en) * 2020-06-04 2023-06-15 Interdigital Vc Holdings France, Sas Neural network based filter in video coding
FR3112661B1 (en) * 2020-07-17 2024-02-02 Fond B Com Methods for decoding a data stream, devices and associated data streams
CN114501031B (en) * 2020-11-13 2023-06-02 华为技术有限公司 Compression coding and decompression method and device
US11716469B2 (en) 2020-12-10 2023-08-01 Lemon Inc. Model selection in neural network-based in-loop filter for video coding
CN115883831A (en) * 2021-08-05 2023-03-31 华为技术有限公司 Encoding and decoding method and device
CN117440185A (en) * 2022-07-14 2024-01-23 杭州海康威视数字技术股份有限公司 Image decoding and encoding method, device and equipment based on neural network

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102523452A (en) * 2011-12-29 2012-06-27 西安空间无线电技术研究所 Method for conversion, compression and transmission of images
CN104145281A (en) * 2012-02-03 2014-11-12 安秉益 Neural network computing apparatus and system, and method therefor
CN107172428A (en) * 2017-06-06 2017-09-15 西安万像电子科技有限公司 The transmission method of image, device and system
CN107197260A (en) * 2017-06-12 2017-09-22 清华大学深圳研究生院 Video coding post-filter method based on convolutional neural networks

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2646575A1 (en) * 1989-04-26 1990-11-02 Labo Electronique Physique METHOD AND STRUCTURE FOR DATA COMPRESSION
NO332189B1 (en) * 2010-02-17 2012-07-23 Cisco Systems Int Sarl Video Encoding Procedure
AU2013322008B2 (en) * 2012-09-26 2016-10-27 Sun Patent Trust Image coding method, image decoding method, image coding apparatus, image decoding apparatus, and image coding and decoding apparatus
US10685279B2 (en) * 2016-09-26 2020-06-16 Splunk Inc. Automatically generating field extraction recommendations
CN107018422B (en) * 2017-04-27 2019-11-05 四川大学 Still image compression method based on depth convolutional neural networks
CN107147603B (en) * 2017-05-05 2019-10-08 西安电子科技大学 DBPSK demodulation method based on multiple neural network
CN107396124B (en) * 2017-08-29 2019-09-20 南京大学 Video-frequency compression method based on deep neural network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102523452A (en) * 2011-12-29 2012-06-27 西安空间无线电技术研究所 Method for conversion, compression and transmission of images
CN104145281A (en) * 2012-02-03 2014-11-12 安秉益 Neural network computing apparatus and system, and method therefor
CN107172428A (en) * 2017-06-06 2017-09-15 西安万像电子科技有限公司 The transmission method of image, device and system
CN107197260A (en) * 2017-06-12 2017-09-22 清华大学深圳研究生院 Video coding post-filter method based on convolutional neural networks

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
DeepCoder: A Deep Neural Network Based Video Compression;Tong Chen等;《2017 IEEE Visual Communications and Image Processing (VCIP)》;20171213;第1-4页 *

Also Published As

Publication number Publication date
CN115115720A (en) 2022-09-27
WO2019205871A1 (en) 2019-10-31
CN110401836A (en) 2019-11-01

Similar Documents

Publication Publication Date Title
CN110401836B (en) Image decoding and encoding method, device and equipment
Wiedemann et al. Deepcabac: A universal compression algorithm for deep neural networks
US20230056351A1 (en) Coded-block-flag coding and derivation
CA3066204C (en) Data processing device, data processing method, and non-transitory computer-readable storage medium
KR101461771B1 (en) An image compression method with random access capability
KR102299958B1 (en) Systems and methods for image compression at multiple, different bitrates
EP2916543A1 (en) Method for coding/decoding depth image and coding/decoding device
KR20130114734A (en) Dynamic video switching
RU2750644C2 (en) Encoding and decoding of spectral peak positions
US11791837B2 (en) Method and apparatus for neural network model compression/decompression
EP3744008B1 (en) Electronic apparatus for compression of data and compression method thereof
Cohen et al. Lightweight compression of intermediate neural network features for collaborative intelligence
CN104717507A (en) Video transcoding method and device
Kang et al. Pilc: Practical image lossless compression with an end-to-end gpu oriented neural framework
JP6545570B2 (en) Encoding device, decoding device and image processing device
US10608664B2 (en) Electronic apparatus for compression and decompression of data and compression method thereof
CN111045726B (en) Deep learning processing device and method supporting coding and decoding
EP2731342A1 (en) Electronic device and method for splitting image
EP2731340A1 (en) Electronic device and method for splitting image
CN115442609A (en) Characteristic data encoding and decoding method and device
US9369734B2 (en) Method and apparatus for encoding and decoding by wavelet transform
Chen et al. CNN-optimized image compression with uncertainty based resource allocation
JP7437426B2 (en) Inter prediction method and device, equipment, storage medium
JP2021072540A (en) Image encoding device, decoding device, transmission system, and control method for the same
US10938412B2 (en) Decompression of model parameters using functions based upon cumulative count distributions

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant