CN109741254B - Dictionary training and image super-resolution reconstruction method, system, equipment and storage medium - Google Patents
Dictionary training and image super-resolution reconstruction method, system, equipment and storage medium Download PDFInfo
- Publication number
- CN109741254B CN109741254B CN201811519177.7A CN201811519177A CN109741254B CN 109741254 B CN109741254 B CN 109741254B CN 201811519177 A CN201811519177 A CN 201811519177A CN 109741254 B CN109741254 B CN 109741254B
- Authority
- CN
- China
- Prior art keywords
- resolution
- low
- sample
- sample image
- dictionary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
Abstract
The invention is suitable for the technical field of computers, and provides a dictionary training and image super-resolution reconstruction method, a system, equipment and a storage medium, wherein the method comprises the following steps: after obtaining the low-resolution to-be-processed image, carrying out image super-resolution reconstruction on the low-resolution to-be-processed image by using the obtained high-resolution dictionary and low-resolution dictionary to obtain a high-resolution imaging image, wherein the method comprises the following steps: defining and machine learning the mapping relation between the corresponding high-resolution sample characteristic data and the corresponding low-resolution sample characteristic data; performing machine learning classification screening on the obtained multiple mapping relations to obtain a final mapping relation; and performing machine learning according to the final mapping relation to obtain corresponding high-resolution and low-resolution dictionaries. Therefore, the low-resolution imaging image obtained by imaging with low radiation dose and short CT time can be used for reconstructing to obtain a high-resolution image, and the occurrence of artifact caused by the movement of a patient can be reduced, so that the quality of the CT image is ensured, meanwhile, the rapid scanning is realized, and the radiation dose is reduced.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a dictionary training and image super-resolution reconstruction method, system, equipment and storage medium.
Background
Computed Tomography (CT) uses a precisely collimated X-ray beam, gamma rays, ultrasonic waves, etc. to scan a cross section of a certain part of a human body together with a detector with extremely high sensitivity, can be used for examination of various diseases, and is one of medical image diagnostic devices with the largest machine loading amount in China.
The image obtained by the low-dose CT has low resolution, generally has high noise, is fuzzy and obvious in granulation, is unreliable for clinical medical observation, and is very likely to cause misdiagnosis.
Disclosure of Invention
The invention aims to provide a dictionary training and image super-resolution reconstruction method, a system, equipment and a storage medium, and aims to solve the problem that the radiation dose to a patient cannot be reduced while the quality of a CT image cannot be ensured in the prior art.
In one aspect, the present invention provides a dictionary obtaining method, including the following steps:
obtaining a high resolution sample image and a low resolution sample image corresponding to the high resolution sample image;
selecting a high-resolution sample image block from the high-resolution sample image, and selecting a low-resolution sample image block corresponding to the high-resolution sample image block from the low-resolution sample image to form a sample image block pair;
performing feature extraction on the high-resolution sample image blocks and the low-resolution sample image blocks in the sample image block pairs to obtain high-resolution sample feature data and low-resolution sample feature data which form sample image block pair data;
defining a mapping relationship between the high-resolution sample feature data and the low-resolution sample feature data in the sample image block pair data based on the consistency of the sparse representation of the high-resolution sample image block on a high-resolution dictionary and the sparse representation of the low-resolution sample image block on a corresponding low-resolution dictionary, the mapping relationship depending on the low-resolution sample feature data;
respectively adopting a first machine learning algorithm to learn a data group of each sample image block pair obtained after the data of the sample image block pairs are grouped, and obtaining a temporary optimal solution of a mapping relation corresponding to each first machine learning algorithm;
determining a temporary optimal solution of the mapping relation as a final solution of the mapping relation from the set of temporary optimal solutions of the mapping relation through a second machine learning algorithm;
calculating to obtain high-resolution temporary sample characteristic data according to the final solution of the mapping relation and the low-resolution sample characteristic data, wherein the high-resolution temporary sample characteristic data and the low-resolution sample characteristic data form temporary sample image block pair data;
and learning the temporary sample image block pair data by adopting a corresponding third machine learning algorithm to solve the high-resolution dictionary and the low-resolution dictionary, wherein the third machine learning algorithm defines the constraint relation among the high-resolution temporary sample feature data, the low-resolution sample feature data, the high-resolution dictionary and the low-resolution dictionary.
Further, the first machine learning algorithm specifically includes: a decision tree algorithm having defined therein a node splitting function conditioned by a response function dependent on the low resolution sample characteristic data.
Further, a quality function dependent on the low resolution sample feature data and the high resolution sample feature data is defined in the decision tree algorithm.
Furthermore, the number and depth of the decision trees of the decision tree algorithm are adaptively selected according to requirements.
On the other hand, the invention also provides an image super-resolution reconstruction method, which is based on the dictionary obtaining method and comprises the following steps:
obtaining a low-resolution image to be processed;
and performing image super-resolution reconstruction on the low-resolution image to be processed by utilizing the high-resolution dictionary and the low-resolution dictionary.
Furthermore, the dictionary acquisition method or the image super-resolution reconstruction method is applied to an electronic Computed Tomography (CT) image, a positron emission computed tomography (PET) image or a Single Photon Emission Computed Tomography (SPECT) image.
In another aspect, the invention provides a computing system, the system comprising:
a preprocessing unit for obtaining a high resolution sample image and a low resolution sample image corresponding to the high resolution sample image; selecting a high-resolution sample image block from the high-resolution sample image, and selecting a low-resolution sample image block corresponding to the high-resolution sample image block from the low-resolution sample image to form a sample image block pair; performing feature extraction on the high-resolution sample image blocks and the low-resolution sample image blocks in the sample image block pairs to obtain high-resolution sample feature data and low-resolution sample feature data which form sample image block pair data; and the number of the first and second groups,
a dictionary obtaining unit, configured to define a mapping relationship between the high-resolution sample feature data and the low-resolution sample feature data in the sample image block pair data, based on that a sparse representation of the high-resolution sample image block on a high-resolution dictionary is consistent with a sparse representation of the low-resolution sample image block on a corresponding low-resolution dictionary, where the mapping relationship depends on the low-resolution sample feature data; respectively adopting a first machine learning algorithm to learn a data group of each sample image block pair obtained after the data of the sample image block pairs are grouped, and obtaining a temporary optimal solution of a mapping relation corresponding to each first machine learning algorithm; determining a temporary optimal solution of the mapping relation as a final solution of the mapping relation from the set of temporary optimal solutions of the mapping relation through a second machine learning algorithm; calculating to obtain high-resolution temporary sample characteristic data according to the final solution of the mapping relation and the low-resolution sample characteristic data, wherein the high-resolution temporary sample characteristic data and the low-resolution sample characteristic data form temporary sample image block pair data; and learning the temporary sample image block pair data by adopting a corresponding third machine learning algorithm to solve the high-resolution dictionary and the low-resolution dictionary, wherein the third machine learning algorithm establishes a constraint relation among the high-resolution temporary sample feature data, the low-resolution sample feature data, the high-resolution dictionary and the low-resolution dictionary.
Further, the system further comprises:
the reconstruction unit is used for obtaining a low-resolution image to be processed; and performing image super-resolution reconstruction on the low-resolution image to be processed by utilizing the high-resolution dictionary and the low-resolution dictionary.
In another aspect, the present invention further provides a computing device, which includes a memory and a processor, and the processor implements the steps in the method when executing the computer program stored in the memory.
In another aspect, the present invention also provides a computer-readable storage medium, which stores a computer program, which when executed by a processor implements the steps in the method as described above.
After obtaining a low-resolution to-be-processed image, the invention utilizes the obtained high-resolution dictionary and the low-resolution dictionary to carry out image super-resolution reconstruction to obtain a high-resolution imaging image, wherein the method comprises the following steps: the mapping relation between the corresponding high-resolution sample feature data and the corresponding low-resolution sample feature data is defined and learned through a machine; performing machine learning classification screening on the obtained multiple mapping relations to obtain a final mapping relation; and performing machine learning according to the final mapping relation to obtain corresponding high-resolution and low-resolution dictionaries. Therefore, the low-resolution imaging image obtained by imaging with low radiation dose and short CT time can be used for reconstructing to obtain a high-resolution image, and the occurrence of artifact caused by the movement of a patient can be reduced, so that the quality of the CT image is ensured, meanwhile, the rapid scanning is realized, and the radiation dose is reduced.
Drawings
Fig. 1 is a flowchart illustrating an implementation of a dictionary obtaining method according to an embodiment of the present invention;
FIG. 2 is a high resolution sample image according to an embodiment of the present invention;
FIG. 3 is a low resolution sample image according to an embodiment of the present invention;
fig. 4 is a flowchart of an implementation of a super-resolution image reconstruction method according to a fourth embodiment of the present invention;
FIG. 5 is a high resolution imaged image according to embodiments of the present invention;
FIG. 6 is an enlarged view of detail A in FIG. 2;
FIG. 7 is an enlarged view of detail B of FIG. 3;
FIG. 8 is an enlarged view of detail C of FIG. 5;
FIG. 9 is a schematic structural diagram of a computing system provided in the fifth embodiment of the present invention;
FIG. 10 is a schematic structural diagram of a computing system according to a sixth embodiment of the present invention;
fig. 11 is a schematic structural diagram of a computing device according to a seventh embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The following detailed description of specific implementations of the present invention is provided in conjunction with specific embodiments:
the first embodiment is as follows:
fig. 1 shows an implementation flow of a dictionary obtaining method provided in the first embodiment of the present invention, and for convenience of description, only the parts related to the first embodiment of the present invention are shown, which are detailed as follows:
in step S101, a high-resolution sample image and a low-resolution sample image corresponding to the high-resolution sample image are obtained.
In this embodiment, before performing the dictionary acquisition process, a sample image set is prepared, where a sample image pair exists in the sample image set, and the sample image pair includes a high-resolution sample image and a low-resolution sample image corresponding to the same scene, as shown in fig. 2 and 3. The high-resolution sample image and the low-resolution sample image can be obtained by performing high-dose and low-dose CT on the same scene, or by performing bicubic interpolation on the high-resolution sample image and downsampling to simulate image degradation, so that the low-resolution sample image is obtained, or by performing super-resolution reconstruction on the low-resolution sample image, the high-resolution sample image is obtained, and the like. No matter what way to obtain the high-resolution and low-resolution sample images, the quality of the sample images needs to meet the expected requirements as much as possible, and thus a dictionary which meets the expectation can be trained.
In step S102, a high-resolution sample image block is selected from the high-resolution sample image, and a low-resolution sample image block corresponding to the high-resolution sample image block is selected from the low-resolution sample image to form a sample image block pair.
In this embodiment, the high-resolution sample image blocks and the low-resolution sample image blocks are also corresponding to each other, that is, the high-resolution sample image blocks and the low-resolution sample image blocks correspond to the same area in the scene.
In step S103, feature extraction is performed on the high-resolution sample image blocks and the low-resolution sample image blocks in the sample image block pair to obtain high-resolution sample feature data X constituting the sample image block pair data H And low resolution sample feature data X L . The sample image block pair data may be { X H ，X L }。
In this embodiment, corresponding high-frequency and/or medium-frequency components may be extracted from the sample image block to obtain corresponding sample feature data, or corresponding sample feature data may be obtained from the first-order gradient feature and/or the second-order gradient feature of the sample image block.
Correspondingly, the step may also involve adding corresponding feature operator processing to the extracted data, for example: and multiplying the extracted data by a corresponding characteristic operator to finally obtain sample characteristic data.
In step S104, the high-resolution dictionary D is based on the high-resolution sample image blocks H Sparse representation of (D) and low resolution dictionary D corresponding to low resolution sample image block L The sparse expression is consistent, and high-resolution sample characteristic data X in the data of sample image blocks is defined H With low resolution sample feature data X L A mapping relation W (X) between L ) Mapping relation W (X) L ) Relying on low resolution sample feature data.
In this embodiment, the high resolution sample image block is in the high resolution dictionary D H The sparseness in (c) is expressed as:
X H ＝D H ×α
where α is a sparse expression.
And the low resolution sample image block is in the low resolution dictionary D L The sparse representation above is:
X L ＝D L ×α
in the two sparse expressions, the sparse expression α is the same.
Based on this, high resolution sample feature data X in the data of sample image block pairs can be defined H With low resolution sample feature data X L The mapping relation W (X) between L ) Mapping relation W (X) L ) Relying on low resolution sample feature data, namely:
X H ＝X L ×W(X L )
in step S105, sample image block pair data { X } H ，X L And (4) learning the data set of each sample image block pair obtained after grouping by adopting a first machine learning algorithm respectively to obtain a temporary optimal solution of a mapping relation corresponding to each first machine learning algorithm.
In this embodiment, the first machine learning algorithm may be a decision tree algorithm, a K Nearest Neighbors (KNN) algorithm, or other machine learning algorithms, and may be configured to pair the N sample image blocks with data { X ″ H ，X L Dividing the image into M (N, M is a natural number) sample image block pairs, and learning the sample image block pairs by using a first machine learning algorithm to obtain M corresponding mapping relationships W (X) L ) A provisional optimal solution.
In step S106, the second machine learning algorithm follows the mapping W (X) L ) Determining a mapping relation W (X) in the temporary optimal solution set L ) The temporary optimal solution is used as a mapping relation W (X) L ) And (5) finally solving.
In this embodiment, M corresponding mapping relationships W (X) are obtained L ) When the temporary optimal solutions are selected, one of the temporary optimal solutions is selected as the mapping relation W (X) L ) The final solution can be a temporary optimal solution which is closest to most of the temporary optimal solutions and is used as the final mapping relation solution W through a voting selection algorithm best (X L )。
In step S107, W is finally solved in the mapping relation best (X L ) And low resolution sample feature data X L Calculating to obtain high-resolution temporary sample characteristic data X H ', high resolution temporal sample feature data X H ' and low resolution sample feature data X L Composing temporary sample image Block Pair data { X H ’，X L }。
In this embodiment, the high-resolution temporary sample feature data X can be obtained by the following formula H ’：
X H ’＝W best (X L )×X L
In step S108, the temporary sample image block pair data { X H ’，X L Learning by adopting a corresponding third machine learning algorithm to solve a high-resolution dictionary D H And low resolution dictionary D L The third machine learning algorithm defines high-resolution temporary sample characteristic data X H ', low resolution sample characteristic data X L High resolution dictionary D H And low resolution dictionary D L The constraint relationship of (2).
In this embodiment, a defining function of the constraint relationship may be determined:
wherein E is a high resolution dictionary D H And low resolution dictionary D L The connection matrix therebetween, Γ (E) is the canonical constraint of E,
to get l 2 And (4) norm.
When the above definition function satisfies the minimum value, the corresponding high resolution dictionary D can be determined H And low resolution dictionary D L 。
In this embodiment, the obtaining the high-resolution dictionary and the low-resolution dictionary specifically involves: defining and machine learning the mapping relation between the corresponding high-resolution sample characteristic data and the corresponding low-resolution sample characteristic data; performing machine learning classification screening on the obtained multiple mapping relations to obtain a final mapping relation; and performing machine learning according to the final mapping relation to obtain corresponding high-resolution dictionaries and low-resolution dictionaries, and performing image super-resolution reconstruction on the high-resolution dictionaries and the low-resolution dictionaries by using the obtained high-resolution dictionaries and low-resolution dictionaries to obtain a high-resolution imaging image. Therefore, the low-resolution imaging image obtained by imaging with low radiation dose and short CT time can be used for reconstructing to obtain the high-resolution image, the occurrence of artifact caused by the movement of a patient can be reduced, the quality of the CT image is ensured, meanwhile, the rapid scanning is realized, and the radiation dose is reduced. In addition, since the dictionary acquisition is different from the traditional dictionary training method with large calculation amount and long running time, the dictionary acquisition method of the embodiment includes the step of calculating the mapping relation, the calculation amount is effectively reduced, the running time is relatively short, and the overall calculation efficiency can be improved.
The second embodiment:
the embodiment further provides the following contents on the basis of the first embodiment:
the first machine learning algorithm is specifically: decision tree algorithm defined thereinNode splitting function delta (X) L θ), node splitting function δ (X) L θ) with a data dependent low resolution sample feature X L Response function r of θ (X L ) As the constraint conditions, the following are specified:
r θ (X L )＝X L [θ 1 ]-θ th ，
wherein Θ th is a threshold value [ ·]In order to take the dimension of XL,
the first machine learning algorithm also defines a characteristic data X depending on the low resolution sample L And high resolution sample feature data X H Quality function Q (delta, theta, X) L ,X H ) Therefore, as a basis for feature selection, the following is specifically made:
wherein Le, Ri are left, right nodes of decision tree split down, namely the selection made according to relevant constraint and condition to achieve the effect of classifier classification, namely:
wherein the content of the first and second substances,
for regular expressions, σ is a selection function, which may be a basis function, in generalThe classifier work needs to be completed by defining a basis function in machine learning, the machine learning in the text belongs to a weak classifier, the basis function cannot be too complex, otherwise, the calculation amount is extremely large, and a general linear function is adopted.
Example three:
the embodiment further provides the following contents on the basis of the second embodiment:
in this embodiment, the number n and the depth ξ of the decision tree algorithm max Can be adaptively selected according to requirements.
In a specific application, the number n of decision trees is different, the effect of obtaining the final result is also different, the effect is better if n is larger, but a saturation state is also achieved, namely saturation is achieved approximately when n is about 10, the algorithm of each decision tree is the same, and the node splitting function delta (X) of each decision tree is the same L And θ) are the same.
Example four:
the embodiment provides an image super-resolution reconstruction method, which is based on the dictionary obtaining method, and the image super-resolution reconstruction method mainly includes the processes as shown in fig. 4:
in step S401, a low-resolution image to be processed is obtained.
In step S402, the high-resolution dictionary D acquired in the dictionary acquisition method is used H And low resolution dictionary D L And performing image super-resolution reconstruction on the low-resolution image to be processed.
In this embodiment, the high-resolution dictionary D obtained by the dictionary obtaining method can be directly used for performing image super-resolution reconstruction on the low-resolution image to be processed H And low resolution dictionary D L The processing may specifically include: partitioning and feature extracting are carried out on the low-resolution image to be processed, sparse expression alpha is obtained when the cost function meets the minimum, and then sparse expression alpha and high-resolution dictionary D are obtained H Obtaining the characteristic data of the high-resolution imaging image, and finally reconstructing the high-resolution imaging image, wherein the cost function can be as follows:
wherein | · | purple sweet 1 To get l 1 Norm, λ is a constant parameter.
The dictionary acquisition method or the image super-resolution reconstruction method can be applied to CT images, and can also be applied to processing of Positron Emission Computed Tomography (PET) images or Single Photon Emission Computed Tomography (SPECT) images and the like.
With reference to fig. 2, 3 and 5, it can be found that: by the image super-resolution reconstruction method of the embodiment, the low-resolution image to be processed as shown in fig. 3 can be reconstructed into the high-resolution imaging image as shown in fig. 5, and the quality of the high-resolution imaging image as shown in fig. 5 is equivalent to that of the high-resolution sample image as shown in fig. 2. Corresponding enlarged detail views in fig. 2, 3 and 5 can refer to fig. 6, 7 and 8, respectively.
The image super-resolution reconstruction of the low-resolution image to be processed can also comprise a high-resolution dictionary D acquired by the dictionary acquisition method H And low resolution dictionary D L The process of (2).
Example five:
fig. 9 shows a structure of a computing system provided in a fifth embodiment of the present invention, and for convenience of explanation, only a part related to the embodiment of the present invention is shown, where the structure includes:
a preprocessing unit 901 for obtaining a high resolution sample image and a low resolution sample image corresponding to the high resolution sample image; selecting a high-resolution sample image block from the high-resolution sample image, and selecting a low-resolution sample image block corresponding to the high-resolution sample image block from the low-resolution sample image to form a sample image block pair; performing feature extraction on a high-resolution sample image block and a low-resolution sample image block in the sample image block pair to obtain high-resolution sample feature data and low-resolution sample feature data which form sample image block pair data; and (c) a second step of,
a dictionary obtaining unit 902, configured to define a mapping relationship between high-resolution sample feature data and low-resolution sample feature data in data of a sample image block based on that a sparse representation of the high-resolution sample image block on the high-resolution dictionary is consistent with a sparse representation of the low-resolution sample image block on a corresponding low-resolution dictionary, where the mapping relationship depends on the low-resolution sample feature data; respectively adopting a first machine learning algorithm to learn each sample image block pair data group obtained after grouping the sample image block pair data to obtain a temporary optimal solution of a mapping relation corresponding to each first machine learning algorithm; determining a temporary optimal solution of the mapping relation as a final solution of the mapping relation from the set of temporary optimal solutions of the mapping relation through a second machine learning algorithm; calculating to obtain high-resolution temporary sample characteristic data according to the final solution of the mapping relation and the low-resolution sample characteristic data, wherein the high-resolution temporary sample characteristic data and the low-resolution sample characteristic data form temporary sample image block pair data; and learning the data of the temporary sample image block by adopting a corresponding third machine learning algorithm to solve the high-resolution dictionary and the low-resolution dictionary, wherein the third machine learning algorithm establishes the constraint relation among the high-resolution temporary sample feature data, the low-resolution sample feature data, the high-resolution dictionary and the low-resolution dictionary.
In the embodiment of the present invention, each unit may be implemented by a corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein. Each unit implements the steps in the corresponding method embodiment, such as steps S101 to S108 shown in fig. 1, which are not described herein again.
Example six:
the present embodiment further provides the following contents on the basis of the fifth embodiment:
as shown in fig. 10, in this embodiment, the computing system further includes:
a reconstruction unit 1001 configured to obtain a low-resolution image to be processed; and performing image super-resolution reconstruction on the low-resolution image to be processed by utilizing the high-resolution dictionary and the low-resolution dictionary.
Similarly, in the embodiment of the present invention, each unit may be implemented by a corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into one software or hardware unit, which is not limited herein. Each unit implements the steps in the corresponding method embodiment, such as steps S401 to S402 shown in fig. 4, which are not described herein again.
Example seven:
fig. 11 shows a structure of a computing device according to a seventh embodiment of the present invention, and for convenience of description, only a part related to the embodiment of the present invention is shown.
The computing device according to an embodiment of the present invention includes a processor 1101 and a memory 1102, and when the processor 1101 executes the computer program 1103 stored in the memory 1102, the steps in the above-described method embodiments, such as steps S101 to S108 shown in fig. 1, are implemented. Alternatively, the processor 1101, when executing the computer program 1103, implements the functions of the units in the above-described device embodiments, such as the functions of the units 901 to 902 shown in fig. 9.
The computing device of embodiments of the present invention may be a computer or a computer network system. For the steps implemented when the processor 1101 executes the computer program 1103 to implement the above methods in the computing device, reference may be made to the description of the foregoing method embodiments, and details are not repeated here.
Example eight:
in an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps in the above-described method embodiments, for example, steps S101 to S108 shown in fig. 1. Alternatively, the computer program, when executed by a processor, implements the functions of the units in the above-described apparatus embodiments, such as the functions of the units 901 to 902 shown in fig. 9.
The computer readable storage medium of the embodiments of the present invention may include any entity or device capable of carrying computer program code, a recording medium, such as a ROM/RAM, a magnetic disk, an optical disk, a flash memory, or the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (9)
1. A dictionary acquisition method, characterized by comprising the steps of:
obtaining a high resolution sample image and a low resolution sample image corresponding to the high resolution sample image;
selecting a high-resolution sample image block from the high-resolution sample image, and selecting a low-resolution sample image block corresponding to the high-resolution sample image block from the low-resolution sample image to form a sample image block pair;
performing feature extraction on the high-resolution sample image blocks and the low-resolution sample image blocks in the sample image block pairs to obtain high-resolution sample feature data and low-resolution sample feature data which form sample image block pair data;
defining a mapping relation between the high-resolution sample feature data and the low-resolution sample feature data in the sample image block pair data based on the consistency of the sparse representation of the high-resolution sample image block on a high-resolution dictionary and the sparse representation of the low-resolution sample image block on a corresponding low-resolution dictionary, wherein the mapping relation depends on the low-resolution sample feature data;
respectively adopting a first machine learning algorithm to learn a data group of each sample image block pair obtained after the data of the sample image block pairs are grouped, and obtaining a temporary optimal solution of a mapping relation corresponding to each first machine learning algorithm; the first machine learning algorithm specifically comprises: a decision tree algorithm, wherein a node splitting function is defined in the decision tree algorithm, and the node splitting function takes a response function depending on the low-resolution sample characteristic data as a constraint condition;
determining a temporary optimal solution of the mapping relation as a final solution of the mapping relation from the set of temporary optimal solutions of the mapping relation through a second machine learning algorithm;
calculating to obtain high-resolution temporary sample characteristic data according to the final solution of the mapping relation and the low-resolution sample characteristic data, wherein the high-resolution temporary sample characteristic data and the low-resolution sample characteristic data form temporary sample image block pair data;
learning the temporary sample image block pair data by adopting a corresponding third machine learning algorithm to solve the high-resolution dictionary and the low-resolution dictionary, wherein the third machine learning algorithm defines the constraint relationship among the high-resolution temporary sample feature data, the low-resolution sample feature data, the high-resolution dictionary and the low-resolution dictionary;
the definition function of the constraint relation is as follows:
wherein E is a high resolution dictionary D H And low resolution dictionary D L The connection matrix therebetween, Γ (E) is the specification constraint of E,
to get l 2 Norm, X H ' is high resolution temporal sample feature data, X L Is low resolution sample feature data;
wherein the high resolution temporal sample feature data is:
X H ’＝W best (X L )×X L
wherein, X H ' is high resolution temporal sample feature data, X L For low resolution sample feature data, W best (X L ) Is the most mapping relationAnd (5) final solution.
2. The method of claim 1, wherein a quality function dependent on the low resolution sample feature data and the high resolution sample feature data is further defined in the decision tree algorithm.
3. The method of claim 1, wherein the number and depth of decision trees of the decision tree algorithm are adaptively selected according to requirements.
4. An image super-resolution reconstruction method, characterized in that the image super-resolution reconstruction method is based on the dictionary acquisition method of any one of claims 1 to 3, and the image super-resolution reconstruction method comprises:
obtaining a low-resolution image to be processed;
and performing image super-resolution reconstruction on the low-resolution image to be processed by utilizing the high-resolution dictionary and the low-resolution dictionary.
5. The image super-resolution reconstruction method according to claim 4, wherein the image super-resolution reconstruction method is applied to an electron computed tomography CT image, a positron emission tomography PET image, or a single photon emission computed tomography SPECT image.
6. A computing system, the system comprising: a preprocessing unit for obtaining a high resolution sample image and a low resolution sample image corresponding to the high resolution sample image; selecting a high-resolution sample image block from the high-resolution sample image, and selecting a low-resolution sample image block corresponding to the high-resolution sample image block from the low-resolution sample image to form a sample image block pair; performing feature extraction on the high-resolution sample image blocks and the low-resolution sample image blocks in the sample image block pairs to obtain high-resolution sample feature data and low-resolution sample feature data which form sample image block pair data; and (c) a second step of,
a dictionary obtaining unit, configured to define a mapping relationship between the high-resolution sample feature data and the low-resolution sample feature data in the sample image block pair data, based on that a sparse representation of the high-resolution sample image block on a high-resolution dictionary is consistent with a sparse representation of the low-resolution sample image block on a corresponding low-resolution dictionary, where the mapping relationship depends on the low-resolution sample feature data; respectively adopting a first machine learning algorithm to learn a data group of each sample image block pair obtained after the data of the sample image block pairs are grouped, and obtaining a temporary optimal solution of a mapping relation corresponding to each first machine learning algorithm; determining a temporary optimal solution of the mapping relation as a final solution of the mapping relation from the set of temporary optimal solutions of the mapping relation through a second machine learning algorithm; calculating to obtain high-resolution temporary sample characteristic data according to the final solution of the mapping relation and the low-resolution sample characteristic data, wherein the high-resolution temporary sample characteristic data and the low-resolution sample characteristic data form temporary sample image block pair data; learning the temporary sample image block pair data by adopting a corresponding third machine learning algorithm to solve the high-resolution dictionary and the low-resolution dictionary, wherein the third machine learning algorithm establishes a constraint relation among the high-resolution temporary sample feature data, the low-resolution sample feature data, the high-resolution dictionary and the low-resolution dictionary;
wherein the first machine learning algorithm specifically is: a decision tree algorithm, wherein a node splitting function is defined in the decision tree algorithm, and the node splitting function takes a response function depending on the low-resolution sample characteristic data as a constraint condition;
the definition function of the constraint relation is as follows:
wherein E is a high resolution dictionary D H And low resolution dictionary D L The connection matrix therebetween, Γ (E) is the specification constraint of E,
to get l 2 Norm, X H ' is high resolution temporal sample feature data, X L Is low resolution sample feature data;
wherein the high resolution temporal sample feature data is:
X H ’＝W best (X L )×X L
wherein, X H ' is high resolution temporal sample feature data, X L For low resolution sample feature data, W best (X L ) Is the final solution of the mapping relation.
7. The system of claim 6, wherein the system further comprises: the reconstruction unit is used for obtaining a low-resolution image to be processed; and performing image super-resolution reconstruction on the low-resolution image to be processed by utilizing the high-resolution dictionary and the low-resolution dictionary.
8. A computing device comprising a memory and a processor, wherein the processor implements the steps of the image super-resolution reconstruction method according to any one of claims 4 to 5 when executing the computer program stored in the memory.
9. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method for super-resolution reconstruction of an image according to one of claims 4 to 5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811519177.7A CN109741254B (en) | 2018-12-12 | 2018-12-12 | Dictionary training and image super-resolution reconstruction method, system, equipment and storage medium |
| PCT/CN2019/071173 WO2020118830A1 (en) | 2018-12-12 | 2019-01-10 | Dictionary training and image super-resolution reconstruction method, system and device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811519177.7A CN109741254B (en) | 2018-12-12 | 2018-12-12 | Dictionary training and image super-resolution reconstruction method, system, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109741254A CN109741254A (en) | 2019-05-10 |
| CN109741254B true CN109741254B (en) | 2022-09-27 |
Family
ID=66358881
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811519177.7A Active CN109741254B (en) | 2018-12-12 | 2018-12-12 | Dictionary training and image super-resolution reconstruction method, system, equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109741254B (en) |
| WO (1) | WO2020118830A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111325695B (en) * | 2020-02-29 | 2023-04-07 | 深圳先进技术研究院 | Low-dose image enhancement method and system based on multi-dose grade and storage medium |
| CN113450427B (en) * | 2021-06-29 | 2023-09-01 | 深圳高性能医疗器械国家研究院有限公司 | PET image reconstruction method based on joint dictionary learning and depth network |
| CN113628109B (en) * | 2021-07-16 | 2023-11-28 | 上海交通大学 | Face five sense organs super-resolution method, system and medium based on learning dictionary |
| CN115841866B (en) * | 2023-02-20 | 2023-05-23 | 北京唯迈医疗设备有限公司 | Low radiation dose image generation method, terminal equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013026659A (en) * | 2011-07-15 | 2013-02-04 | Univ Of Tsukuba | Super-resolution image processing device and dictionary creating device for super-resolution image processing |
| WO2017075768A1 (en) * | 2015-11-04 | 2017-05-11 | 北京大学深圳研究生院 | Super-resolution image reconstruction method and device based on dictionary matching |
| CN106780342A (en) * | 2016-12-28 | 2017-05-31 | 深圳市华星光电技术有限公司 | Single-frame image super-resolution reconstruction method and device based on the reconstruct of sparse domain |
| CN107767337A (en) * | 2017-09-29 | 2018-03-06 | 深圳先进技术研究院 | A kind of image super-resolution rebuilding method, device, mobile terminal and storage medium |
| CN108550111A (en) * | 2018-03-30 | 2018-09-18 | 西安工程大学 | A kind of residual error example recurrence super-resolution reconstruction method based on multistage dictionary learning |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9542725B2 (en) * | 2012-11-29 | 2017-01-10 | Nec Corporation | Image processing device, image processing method and medium |
| CN106296583B (en) * | 2016-08-01 | 2019-06-04 | 西北工业大学 | Based on image block group sparse coding and the noisy high spectrum image ultra-resolution ratio reconstructing method that in pairs maps |
| CN107133915A (en) * | 2017-04-21 | 2017-09-05 | 西安科技大学 | A kind of image super-resolution reconstructing method based on study |
| CN107341765B (en) * | 2017-05-05 | 2020-04-28 | 西安邮电大学 | Image super-resolution reconstruction method based on cartoon texture decomposition |
| CN107492070B (en) * | 2017-07-10 | 2019-12-03 | 华北电力大学 | A kind of single image super-resolution calculation method of binary channels convolutional neural networks |
-
2018
- 2018-12-12 CN CN201811519177.7A patent/CN109741254B/en active Active
-
2019
- 2019-01-10 WO PCT/CN2019/071173 patent/WO2020118830A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013026659A (en) * | 2011-07-15 | 2013-02-04 | Univ Of Tsukuba | Super-resolution image processing device and dictionary creating device for super-resolution image processing |
| WO2017075768A1 (en) * | 2015-11-04 | 2017-05-11 | 北京大学深圳研究生院 | Super-resolution image reconstruction method and device based on dictionary matching |
| CN106780342A (en) * | 2016-12-28 | 2017-05-31 | 深圳市华星光电技术有限公司 | Single-frame image super-resolution reconstruction method and device based on the reconstruct of sparse domain |
| CN107767337A (en) * | 2017-09-29 | 2018-03-06 | 深圳先进技术研究院 | A kind of image super-resolution rebuilding method, device, mobile terminal and storage medium |
| CN108550111A (en) * | 2018-03-30 | 2018-09-18 | 西安工程大学 | A kind of residual error example recurrence super-resolution reconstruction method based on multistage dictionary learning |
Non-Patent Citations (3)
| Title |
|---|
| Couple dictionary training for image super-resolution;Yang J et al;《IEEE Trans. Image Process》;20121231;第3467-3478页 * |
| 基于稀疏编码和随机森林的多帧图像超分辨率算法;包莹莹;《电子设计工程》;20170430;第158-162页 * |
| 基于随机森林的单幅图像超分辨重建算法研究;卢欢;《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》;20180215;第I138-2473页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020118830A1 (en) | 2020-06-18 |
| CN109741254A (en) | 2019-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109146988B (en) | Incomplete projection CT image reconstruction method based on VAEGAN | |
| CN109741254B (en) | Dictionary training and image super-resolution reconstruction method, system, equipment and storage medium | |
| CN111325686B (en) | Low-dose PET three-dimensional reconstruction method based on deep learning | |
| Heinrich et al. | Residual U-net convolutional neural network architecture for low-dose CT denoising | |
| CN107481297B (en) | CT image reconstruction method based on convolutional neural network | |
| Whiteley et al. | FastPET: near real-time reconstruction of PET histo-image data using a neural network | |
| KR20190135618A (en) | Method for processing interior computed tomography image using artificial neural network and apparatus therefor | |
| He et al. | Downsampled imaging geometric modeling for accurate CT reconstruction via deep learning | |
| Li et al. | Learning non-local perfusion textures for high-quality computed tomography perfusion imaging | |
| CN114358285A (en) | PET system attenuation correction method based on flow model | |
| US11455755B2 (en) | Methods and apparatus for neural network based image reconstruction | |
| CN111325695B (en) | Low-dose image enhancement method and system based on multi-dose grade and storage medium | |
| US20230386036A1 (en) | Methods and systems for medical imaging | |
| Jiao et al. | Fast PET reconstruction using multi-scale fully convolutional neural networks | |
| Whiteley et al. | FastPET: Near real-time PET reconstruction from histo-images using a neural network | |
| CN116245969A (en) | Low-dose PET image reconstruction method based on deep neural network | |
| Chen et al. | Low-dose CT reconstruction method based on prior information of normal-dose image | |
| Liang et al. | A model-based unsupervised deep learning method for low-dose CT reconstruction | |
| KR102329938B1 (en) | Method for processing conebeam computed tomography image using artificial neural network and apparatus therefor | |
| Xie et al. | 3D few-view CT image reconstruction with deep learning | |
| Pereira et al. | Extreme sparse X-ray computed laminography via convolutional neural networks | |
| Liang et al. | A model-based deep learning reconstruction for X-Ray CT | |
| Liang et al. | High quality low-dose SPECT reconstruction using CGAN-based transformer network with geometric tight framelet | |
| Yang et al. | Multi-material beam hardening correction (mmbhc) in computed tomography | |
| Fedrigo et al. | Investigating tomographic reconstruction with a priori geometrical information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |