CN112348742B - Image nonlinear interpolation acquisition method and system based on deep learning - Google Patents
Image nonlinear interpolation acquisition method and system based on deep learning Download PDFInfo
- Publication number
- CN112348742B CN112348742B CN202011210475.5A CN202011210475A CN112348742B CN 112348742 B CN112348742 B CN 112348742B CN 202011210475 A CN202011210475 A CN 202011210475A CN 112348742 B CN112348742 B CN 112348742B
- Authority
- CN
- China
- Prior art keywords
- image
- unqualified
- neural network
- interpolation
- target detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 238000013135 deep learning Methods 0.000 title claims abstract description 40
- 238000001514 detection method Methods 0.000 claims abstract description 112
- 238000013528 artificial neural network Methods 0.000 claims abstract description 111
- 238000012549 training Methods 0.000 claims abstract description 33
- 238000012545 processing Methods 0.000 claims abstract description 14
- 230000008569 process Effects 0.000 claims description 14
- 238000002372 labelling Methods 0.000 claims description 9
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000010276 construction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a deep learning-based image nonlinear interpolation acquisition method and system, belonging to the technical field of image processing, and comprising the following steps: 1. establishing a data set; 2. detecting an unqualified target in the interpolated image by using a target detection method of deep learning to obtain a first target detection deep neural network; 3. detecting the position of an unqualified target in an original image and pixels nearby the unqualified target by using a deep learning target detection method; obtaining a second target detection depth neural network; 4. based on the data set, the first target detection depth neural network and the second target detection depth neural network, training to obtain the depth neural network, and outputting correct interpolation through the depth neural network. The invention uses the deep learning target detection technology to construct a data set corrected by manual interpolation, combines the image background, identifies unqualified targets and related nearby related pixels in the traditional interpolation algorithm, combines the manual interpolation example, and outputs correct interpolation.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an image nonlinear interpolation acquisition method and system based on deep learning.
Background
Image processing (also known as image processing) is a technique for performing image processing on an image by a computer to achieve a desired result. Originating in the 20 th century, typically digital image processing. The main content of the image processing technology comprises image compression, enhancement restoration, matching description identification of 3 parts, and common processes comprise image digitization, image coding, image enhancement, image restoration, image segmentation, image analysis and the like. The image processing is to process the image information by using a computer so as to meet the behaviors of visual psychology or application requirements of people, and has wide application, and is mostly used for mapping, atmospheric science, astronomy, image beautifying, image improvement identification and the like.
In the image processing process, the image interpolation method is a technical means which is frequently used, the traditional image interpolation is usually carried out by adopting methods such as nearest neighbor interpolation, bilinear interpolation, bicubic interpolation and the like, and because the methods do not consider the background and the application scene, the methods generally cause a plurality of adverse phenomena such as blurring, distortion, twisting and saw tooth of the image, and the interpolation result is not available in a plurality of occasions (such as content examination, character recognition and the like).
Disclosure of Invention
The invention provides a deep learning-based nonlinear interpolation acquisition method and system for solving the technical problems in the prior art, which mainly solve a key problem in the current image preprocessing field: the user inputs images of unequal sizes and unequal definition, possibly high-definition photos and pixel diagrams; in order to obtain a better algorithm detection effect, various pictures input by a user are subjected to uniform geometric transformation and are arranged into images with the same size and similar definition, and then the images are input into an algorithm module for recognition; the interpolation method proposed by the patent is adopted to perform interpolation processing on the user input image, so that the blurred image becomes clear, and block images and mosaic images are avoided.
The first object of the present invention is to provide a deep learning-based nonlinear interpolation acquisition method for an image, comprising:
s1, establishing a data set, which specifically comprises the following steps:
s101, interpolating an original image through nearest neighbor interpolation and/or bilinear interpolation and/or bicubic interpolation to obtain an interpolated image and recording the interpolated image;
s102, manually inspecting the interpolated image, marking out unqualified targets and recording;
s103, combining the original image and the unqualified target, manually interpolating to obtain an image after manual interpolation, and recording an interpolation result;
s104, combining the original image and the image after manual interpolation, and manually marking and recording the original position and background pixels near the original position, which are related to the image after manual interpolation, in the original image;
s105, combining the records of S1-S4 to form one piece of data in the data set;
s106, repeating the steps of S1-S4 on different original images to form a complete data set;
s2, detecting an unqualified target in the interpolated image by using a target detection method of deep learning, and obtaining a first target detection deep neural network;
s3, detecting positions of unqualified targets in the original image and pixels nearby the unqualified targets by using a target detection method of deep learning; obtaining a second target detection depth neural network;
and S4, training to obtain a deep neural network based on the data set, the first target detection deep neural network and the second target detection deep neural network, and outputting correct interpolation through the deep neural network.
Preferably, in said S2: and taking the interpolated image as a first target detection depth neural network to be input, and calculating loss by manually labeling unqualified target data.
Preferably, in said S3: and taking the unqualified targets in the original image and the interpolated image as second target detection depth neural network input, and outputting the positions of the unqualified targets in the original image and the background pixels near the positions according to implicit logic of manually labeling the unqualified targets and the positions of the unqualified targets in the original image and the examples of the nearby background pixels.
Preferably, in said S4: the input of the deep neural network is the original position of the manually marked unqualified target and the background pixels nearby the original position, the manually marked unqualified target and the image interpolated by the traditional method, and the loss is calculated by using manual interpolation in the training process.
A second object of the present invention is to provide a deep learning based image nonlinear interpolation system; at least comprises:
the data set establishment module comprises the following establishment processes:
s101, interpolating an original image through nearest neighbor interpolation and/or bilinear interpolation and/or bicubic interpolation to obtain an interpolated image and recording the interpolated image;
s102, manually inspecting the interpolated image, marking out unqualified targets and recording;
s103, combining the original image and the unqualified target, manually interpolating to obtain an image after manual interpolation, and recording an interpolation result;
s104, combining the original image and the image after manual interpolation, and manually marking and recording the original position and background pixels near the original position, which are related to the image after manual interpolation, in the original image;
s105, combining the records of S1-S4 to form one piece of data in the data set;
s106, repeating the steps of S1-S4 on different original images to form a complete data set;
the first target detection depth neural network acquisition module is used for detecting unqualified targets in the interpolated image by using a target detection method of deep learning to obtain a first target detection depth neural network;
the second target detection depth neural network acquisition module detects positions and pixels nearby of the unqualified target in the original image by using a target detection method of deep learning; obtaining a second target detection depth neural network;
and the correct interpolation acquisition module is used for training to obtain the depth neural network based on the data set, the first target detection depth neural network and the second target detection depth neural network, and outputting correct interpolation through the depth neural network.
Preferably, in the first target detection depth neural network acquisition module: and taking the interpolated image as a first target detection depth neural network to be input, and calculating loss by manually labeling unqualified target data.
Preferably, in the second target detection depth neural network acquisition module: and taking the unqualified targets in the original image and the interpolated image as second target detection depth neural network input, and outputting the positions of the unqualified targets in the original image and the background pixels nearby the positions according to the manual labeling unqualified targets and the implicit logic of the manual labeling unqualified targets in the original image and the nearby background pixel examples.
Preferably, in the correct interpolation acquisition module: the input of the deep neural network is the original position of the manually marked unqualified target and the background pixels nearby the original position, the manually marked unqualified target and the image interpolated by the traditional method, and the loss is calculated by using manual interpolation in the training process.
A third object of the present invention is to provide an information data processing terminal that implements the above-described image nonlinear interpolation acquisition method based on deep learning.
A fourth object of the present invention is to provide a computer-readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the above-described deep learning-based image nonlinear interpolation acquisition method.
The invention has the advantages and positive effects that:
the invention uses a deep learning target detection technology, by constructing a data set corrected by manual interpolation, combining an image background, identifying an unqualified target and related nearby related pixels in the traditional interpolation algorithm, combining an example of manual interpolation, and finally outputting correct interpolation. The interpolation result is finer and more appropriate for the scene.
Drawings
FIG. 1 is a flow chart of data set construction in a preferred embodiment of the invention;
FIG. 2 is a training flow diagram of a rejected object recognition model in an interpolated image in accordance with a preferred embodiment of the invention;
FIG. 3 is a flow chart of the inferred use of the rejected target recognition model in the interpolated image in the preferred embodiment of the invention;
FIG. 4 is a training flow diagram of a failed target home location and its nearby background pixel target recognition model in a preferred embodiment of the invention;
FIG. 5 is a flow chart of the inferred use of the failed target home location and its nearby background pixel target recognition model in the preferred embodiment of the present invention;
FIG. 6 is a training flow diagram for correct interpolation in a preferred embodiment of the present invention;
FIG. 7 is a flow chart of the application of correct interpolation in a preferred embodiment of the present invention;
fig. 8 is a flow chart of the application of the preferred embodiment of the present invention.
Detailed Description
For a further understanding of the invention, its features and advantages, reference is now made to the following examples, which are illustrated in the accompanying drawings in which:
the invention mainly solves a key problem in the current image preprocessing field: the user inputs images of unequal sizes and unequal definition, possibly high-definition photos and pixel diagrams; in order to obtain a better algorithm detection effect, various pictures input by a user are subjected to uniform geometric transformation and are arranged into images with the same size and similar definition, and then the images are input into an algorithm module for recognition; the interpolation method proposed by the patent is adopted to perform interpolation processing on the user input image, so that the blurred image becomes clear, and block images and mosaic images are avoided.
The invention mainly relates to a data set construction method and three deep learning model construction methods.
Please refer to fig. 1 to 8:
a nonlinear interpolation acquisition method of an image based on deep learning comprises the following steps:
s1, establishing a data set, which specifically comprises the following steps:
1) The method comprises the steps of firstly, interpolating an original image through a plurality of traditional interpolation methods to obtain an interpolated image and recording the interpolated image. The conventional interpolation method includes:
2) And (5) manually inspecting the interpolated image, marking out unqualified targets and recording.
3) And combining the original image and the unqualified target in the interpolated image, manually interpolating, and recording an interpolation result.
4) Combining the original image and the image after manual interpolation, and manually marking and recording the relevant original position and the background pixels nearby the original position in the original image.
5) Combining the recordings of 1) 2) 3) 4) forms one piece of data in the dataset.
6) Repeating steps 1) -5) for different original images, forming a complete data set.
S2, on the basis of establishing a data set in S1, the method detects the unqualified target in the interpolated image by using a currently popular target detection method for deep learning. The goal is to form a target detection depth neural network (denoted as target detection depth neural network 1, i.e. first target detection depth neural network), the training structure is shown in fig. 2, the training structure is input as an interpolated image in the dataset, and unqualified target data is artificially marked for calculating the loss. When the method is applied, the network can take any interpolated image as input and output the unqualified target formed according to the logic implicit in the manual marking example learning in 1), so that the unqualified target can be automatically found for the traditional interpolated image in the subsequent application, and the application structure is shown in figure 3.
S3, on the basis that the data set is established in S1 and the target detection deep neural network 1 is established and fully trained in S2, the position of an unqualified target in an original image and pixels nearby the unqualified target are detected by using a currently popular target detection method of deep learning. The target is formed into another target detection depth neural network (denoted as target detection depth neural network 2, namely a second target detection depth neural network), and the training structure is shown in fig. 4. When the method is applied, the network can take any unqualified target (obtained by the target detection depth neural network 1 established in the S2) in the original image and the interpolated image as input, and the position of the unqualified target in the original image and the nearby background pixels are output according to the implicit logic of the example of manually marking the unqualified target and the position and the nearby background pixels in the original image, and the application structure is shown in fig. 5.
S4, on the basis that a data set is established in S1, and the target detection depth neural network 1 and the target detection depth neural network 2 are established and fully trained in S2 and S3, the invention trains a depth neural network to output correct interpolation, the training structure is shown in figure 6, the input is an image obtained by manually marking unqualified target original positions and background pixels nearby the unqualified target original positions, manually marking unqualified targets and interpolating by a traditional method, and the loss is calculated by using manual interpolation in the training process. When the method is applied, the network can take any original image, an unqualified target output by the target detection depth neural network 1 established in the step S2, an unqualified target original position output by the target detection depth neural network 2 established in the step S3 and a background pixel nearby the unqualified target original position as input, output correct interpolation and an application structure is shown in fig. 7.
S5, as shown in FIG. 8, the original image passes through the 3 depth neural networks to obtain the correct interpolation of the example internal logic when the data set is built.
An image nonlinear interpolation system based on deep learning; comprising the following steps:
the data set establishment module comprises the following establishment processes:
1) The method comprises the steps of firstly, interpolating an original image through a plurality of traditional interpolation methods to obtain an interpolated image and recording the interpolated image. The conventional interpolation method includes:
2) And (5) manually inspecting the interpolated image, marking out unqualified targets and recording.
3) And combining the original image and the unqualified target in the interpolated image, manually interpolating, and recording an interpolation result.
4) Combining the original image and the image after manual interpolation, and manually marking and recording the relevant original position and the background pixels nearby the original position in the original image.
5) Combining the recordings of 1) 2) 3) 4) forms one piece of data in the dataset.
6) Repeating steps 1) -5) for different original images, forming a complete data set.
The first target detection depth neural network acquisition module is used for detecting unqualified targets in the interpolated image by using a target detection method of deep learning to obtain a first target detection depth neural network; based on the data set, the invention uses the current popular target detection method of deep learning to detect the disqualified target in the interpolated image. The goal is to form a target detection depth neural network (denoted as target detection depth neural network 1, i.e. first target detection depth neural network), the training structure is shown in fig. 2, the training structure is input as an interpolated image in the dataset, and unqualified target data is artificially marked for calculating the loss. When the method is applied, the network can take any interpolated image as input and output the unqualified target formed according to the logic implicit in the manual marking example learning in 1), so that the unqualified target can be automatically found for the traditional interpolated image in the subsequent application, and the application structure is shown in figure 3.
The second target detection depth neural network acquisition module detects positions and pixels nearby of the unqualified target in the original image by using a target detection method of deep learning; obtaining a second target detection depth neural network; based on the data set and the target detection deep neural network 1, the present invention uses the currently popular target detection method of deep learning to detect the position of a failed target in an original image and its nearby pixels. The target is formed into another target detection depth neural network (denoted as target detection depth neural network 2, namely a second target detection depth neural network), and the training structure is shown in fig. 4. When the method is applied, the network can take any unqualified target (obtained by the target detection depth neural network 1) in the original image and the interpolated image as input, and according to the implicit logic of the example of manually marking the unqualified target and the position and the nearby background pixels of the unqualified target in the original image, the position and the nearby background pixels of the unqualified target in the original image are output, and the application structure is shown in fig. 5;
the correct interpolation acquisition module is used for training to obtain a depth neural network based on the data set, the first target detection depth neural network and the second target detection depth neural network, and outputting correct interpolation through the depth neural network; on the basis of the data set, the target detection depth neural network 1 and the target detection depth neural network 2, the invention trains a depth neural network to output correct interpolation, the training structure is shown in figure 6, the input is manually labeling original positions of unqualified targets and background pixels nearby the original positions, manually labeling unqualified targets, interpolating images by a traditional method, and the loss is calculated by using manual interpolation in the training process. When the method is applied, the network can take any original image, an unqualified target output by the target detection depth neural network 1 established in the step S2, an unqualified target original position output by the target detection depth neural network 2 established in the step S3 and a background pixel nearby the unqualified target original position as input, output correct interpolation and an application structure is shown in fig. 7.
An information data processing terminal for implementing a deep learning-based image nonlinear interpolation acquisition method, the deep learning-based image nonlinear interpolation acquisition method comprising:
s1, establishing a data set, which specifically comprises the following steps:
1) The method comprises the steps of firstly, interpolating an original image through a plurality of traditional interpolation methods to obtain an interpolated image and recording the interpolated image. The conventional interpolation method includes:
2) And (5) manually inspecting the interpolated image, marking out unqualified targets and recording.
3) And combining the original image and the unqualified target in the interpolated image, manually interpolating, and recording an interpolation result.
4) Combining the original image and the image after manual interpolation, and manually marking and recording the relevant original position and the background pixels nearby the original position in the original image.
5) Combining the recordings of 1) 2) 3) 4) forms one piece of data in the dataset.
6) Repeating steps 1) -5) for different original images, forming a complete data set.
S2, on the basis of establishing a data set in S1, the method detects the unqualified target in the interpolated image by using a currently popular target detection method for deep learning. The goal is to form a target detection depth neural network (denoted as target detection depth neural network 1, i.e. first target detection depth neural network), the training structure is shown in fig. 2, the training structure is input as an interpolated image in the dataset, and unqualified target data is artificially marked for calculating the loss. When the method is applied, the network can take any interpolated image as input and output the unqualified target formed according to the logic implicit in the manual marking example learning in 1), so that the unqualified target can be automatically found for the traditional interpolated image in the subsequent application, and the application structure is shown in figure 3.
S3, on the basis that the data set is established in S1 and the target detection deep neural network 1 is established and fully trained in S2, the position of an unqualified target in an original image and pixels nearby the unqualified target are detected by using a currently popular target detection method of deep learning. The target is formed into another target detection depth neural network (denoted as target detection depth neural network 2, namely a second target detection depth neural network), and the training structure is shown in fig. 4. When the method is applied, the network can take any unqualified target (obtained by the target detection depth neural network 1 established in the S2) in the original image and the interpolated image as input, and the position of the unqualified target in the original image and the nearby background pixels are output according to the implicit logic of the example of manually marking the unqualified target and the position and the nearby background pixels in the original image, and the application structure is shown in fig. 5.
S4, on the basis that a data set is established in S1, and the target detection depth neural network 1 and the target detection depth neural network 2 are established and fully trained in S2 and S3, the invention trains a depth neural network to output correct interpolation, the training structure is shown in figure 6, the input is an image obtained by manually marking unqualified target original positions and background pixels nearby the unqualified target original positions, manually marking unqualified targets and interpolating by a traditional method, and the loss is calculated by using manual interpolation in the training process. When the method is applied, the network can take any original image, an unqualified target output by the target detection depth neural network 1 established in the step S2, an unqualified target original position output by the target detection depth neural network 2 established in the step S3 and a background pixel nearby the unqualified target original position as input, output correct interpolation and an application structure is shown in fig. 7.
S5, as shown in FIG. 8, the original image passes through the 3 depth neural networks to obtain the correct interpolation of the example internal logic when the data set is built.
A computer-readable storage medium comprising instructions that when executed on a computer cause the computer to perform a deep learning based image nonlinear interpolation acquisition method comprising:
s1, establishing a data set, which specifically comprises the following steps:
1) The method comprises the steps of firstly, interpolating an original image through a plurality of traditional interpolation methods to obtain an interpolated image and recording the interpolated image. The conventional interpolation method includes:
2) And (5) manually inspecting the interpolated image, marking out unqualified targets and recording.
3) And combining the original image and the unqualified target in the interpolated image, manually interpolating, and recording an interpolation result.
4) Combining the original image and the image after manual interpolation, and manually marking and recording the relevant original position and the background pixels nearby the original position in the original image.
5) Combining the recordings of 1) 2) 3) 4) forms one piece of data in the dataset.
6) Repeating steps 1) -5) for different original images, forming a complete data set.
S2, on the basis of establishing a data set in S1, the method detects the unqualified target in the interpolated image by using a currently popular target detection method for deep learning. The goal is to form a target detection depth neural network (denoted as target detection depth neural network 1, i.e. first target detection depth neural network), the training structure is shown in fig. 2, the training structure is input as an interpolated image in the dataset, and unqualified target data is artificially marked for calculating the loss. When the method is applied, the network can take any interpolated image as input and output the unqualified target formed according to the logic implicit in the manual marking example learning in 1), so that the unqualified target can be automatically found for the traditional interpolated image in the subsequent application, and the application structure is shown in figure 3.
S3, on the basis that the data set is established in S1 and the target detection deep neural network 1 is established and fully trained in S2, the position of an unqualified target in an original image and pixels nearby the unqualified target are detected by using a currently popular target detection method of deep learning. The target is formed into another target detection depth neural network (denoted as target detection depth neural network 2, namely a second target detection depth neural network), and the training structure is shown in fig. 4. When the method is applied, the network can take any unqualified target (obtained by the target detection depth neural network 1 established in the S2) in the original image and the interpolated image as input, and the position of the unqualified target in the original image and the nearby background pixels are output according to the implicit logic of the example of manually marking the unqualified target and the position and the nearby background pixels in the original image, and the application structure is shown in fig. 5.
S4, on the basis that a data set is established in S1, and the target detection depth neural network 1 and the target detection depth neural network 2 are established and fully trained in S2 and S3, the invention trains a depth neural network to output correct interpolation, the training structure is shown in figure 6, the input is an image obtained by manually marking unqualified target original positions and background pixels nearby the unqualified target original positions, manually marking unqualified targets and interpolating by a traditional method, and the loss is calculated by using manual interpolation in the training process. When the method is applied, the network can take any original image, an unqualified target output by the target detection depth neural network 1 established in the step S2, an unqualified target original position output by the target detection depth neural network 2 established in the step S3 and a background pixel nearby the unqualified target original position as input, output correct interpolation and an application structure is shown in fig. 7.
S5, as shown in FIG. 8, the original image passes through the 3 depth neural networks to obtain the correct interpolation of the example internal logic when the data set is built.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When used in whole or in part, is implemented in the form of a computer program product comprising one or more computer instructions. When loaded or executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL), or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
The above-described embodiments are only for illustrating the technical spirit and features of the present invention, and it is intended to enable those skilled in the art to understand the content of the present invention and to implement it accordingly, and the scope of the present invention is not limited to the embodiments, i.e. equivalent changes or modifications to the spirit of the present invention are still within the scope of the present invention.
Claims (4)
1. An image nonlinear interpolation acquisition method based on deep learning; characterized in that it comprises at least:
s1, establishing a data set, which specifically comprises the following steps:
s101, interpolating an original image through nearest neighbor interpolation and/or bilinear interpolation and/or bicubic interpolation to obtain an interpolated image and recording the interpolated image;
s102, manually inspecting the interpolated image, marking out unqualified targets and recording;
s103, combining the original image and the unqualified target, manually interpolating to obtain an image after manual interpolation, and recording an interpolation result;
s104, combining the original image and the image after manual interpolation, and manually marking and recording the original position and background pixels near the original position, which are related to the image after manual interpolation, in the original image;
s105, combining the records of S101-S104 to form one piece of data in the data set;
s106, repeating the steps of S101-S104 on different original images to form a complete data set;
s2, detecting an unqualified target in the interpolated image by using a target detection method of deep learning, and obtaining a first target detection deep neural network; the method comprises the following steps: taking the interpolated image as a first target detection depth neural network to be input, and calculating loss by manually marking unqualified target data;
s3, detecting positions of unqualified targets in the original image and pixels nearby the unqualified targets by using a target detection method of deep learning; obtaining a second target detection depth neural network; the method comprises the following steps: taking the unqualified targets in the original image and the interpolated image as a second target detection depth neural network to be input, and outputting the positions of the unqualified targets in the original image and the background pixels near the positions according to implicit logic of manually labeling the positions of the unqualified targets and the unqualified targets in the original image and the examples of the background pixels near the positions;
s4, training to obtain a depth neural network based on the data set, the first target detection depth neural network and the second target detection depth neural network, and outputting correct interpolation through the depth neural network; the method comprises the following steps: the input of the deep neural network is the original position of the manually marked unqualified target and the background pixels nearby the original position, the manually marked unqualified target and the image interpolated by the traditional method, and the loss is calculated by using manual interpolation in the training process.
2. An image nonlinear interpolation acquisition system based on deep learning; characterized in that it comprises at least:
the data set establishment module comprises the following establishment processes:
s101, interpolating an original image through nearest neighbor interpolation and/or bilinear interpolation and/or bicubic interpolation to obtain an interpolated image and recording the interpolated image;
s102, manually inspecting the interpolated image, marking out unqualified targets and recording;
s103, combining the original image and the unqualified target, manually interpolating to obtain an image after manual interpolation, and recording an interpolation result;
s104, combining the original image and the image after manual interpolation, and manually marking and recording the original position and background pixels near the original position of the original image related to the image after manual interpolation;
s105, combining the records of S101-S104 to form one piece of data in the data set;
s106, repeating the steps of S101-S104 on different original images to form a complete data set;
the first target detection depth neural network acquisition module is used for detecting unqualified targets in the interpolated image by using a target detection method of deep learning to obtain a first target detection depth neural network; the method comprises the following steps: taking the interpolated image as a first target detection depth neural network to be input, and calculating loss by manually marking unqualified target data;
the second target detection depth neural network acquisition module detects positions and pixels nearby of the unqualified target in the original image by using a target detection method of deep learning; obtaining a second target detection depth neural network; the method comprises the following steps: the unqualified targets in the original image and the interpolated image are used as second target detection depth neural network input, and the positions of the unqualified targets in the original image and the background pixels near the positions are output according to implicit logic of manually labeling the unqualified targets and the positions of the unqualified targets in the original image and the examples of the background pixels near the positions;
the correct interpolation acquisition module is used for training to obtain a depth neural network based on the data set, the first target detection depth neural network and the second target detection depth neural network, and outputting correct interpolation through the depth neural network; the method comprises the following steps: the input of the deep neural network is the original position of the manually marked unqualified target and the background pixels nearby the original position, the manually marked unqualified target and the image interpolated by the traditional method, and the loss is calculated by using manual interpolation in the training process.
3. An information data processing terminal implementing the deep learning-based image nonlinear interpolation acquisition method of claim 1.
4. A computer readable storage medium comprising instructions that when run on a computer cause the computer to perform the deep learning based image nonlinear interpolation acquisition method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011210475.5A CN112348742B (en) | 2020-11-03 | 2020-11-03 | Image nonlinear interpolation acquisition method and system based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011210475.5A CN112348742B (en) | 2020-11-03 | 2020-11-03 | Image nonlinear interpolation acquisition method and system based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112348742A CN112348742A (en) | 2021-02-09 |
| CN112348742B true CN112348742B (en) | 2024-03-26 |
Family
ID=74356335
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011210475.5A Active CN112348742B (en) | 2020-11-03 | 2020-11-03 | Image nonlinear interpolation acquisition method and system based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112348742B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007328735A (en) * | 2006-06-09 | 2007-12-20 | Victor Co Of Japan Ltd | Image processor and image processing program |
| CN106600538A (en) * | 2016-12-15 | 2017-04-26 | 武汉工程大学 | Human face super-resolution algorithm based on regional depth convolution neural network |
| CN108154199A (en) * | 2017-12-22 | 2018-06-12 | 任俊芬 | A kind of quick single class object detection method of high-precision based on deep learning |
| CN109754024A (en) * | 2019-01-29 | 2019-05-14 | 广州云测信息技术有限公司 | Image classification method and device |
| CN110942425A (en) * | 2019-11-26 | 2020-03-31 | 贵州师范学院 | Reconstruction method and reconstruction system of super-resolution image and electronic equipment |
| CN111062383A (en) * | 2019-11-04 | 2020-04-24 | 南通大学 | Image-based ship detection depth neural network algorithm |
| US10635943B1 (en) * | 2018-08-07 | 2020-04-28 | General Electric Company | Systems and methods for noise reduction in medical images with deep neural networks |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7187811B2 (en) * | 2003-03-18 | 2007-03-06 | Advanced & Wise Technology Corp. | Method for image resolution enhancement |
-
2020
- 2020-11-03 CN CN202011210475.5A patent/CN112348742B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007328735A (en) * | 2006-06-09 | 2007-12-20 | Victor Co Of Japan Ltd | Image processor and image processing program |
| CN106600538A (en) * | 2016-12-15 | 2017-04-26 | 武汉工程大学 | Human face super-resolution algorithm based on regional depth convolution neural network |
| CN108154199A (en) * | 2017-12-22 | 2018-06-12 | 任俊芬 | A kind of quick single class object detection method of high-precision based on deep learning |
| US10635943B1 (en) * | 2018-08-07 | 2020-04-28 | General Electric Company | Systems and methods for noise reduction in medical images with deep neural networks |
| CN109754024A (en) * | 2019-01-29 | 2019-05-14 | 广州云测信息技术有限公司 | Image classification method and device |
| CN111062383A (en) * | 2019-11-04 | 2020-04-24 | 南通大学 | Image-based ship detection depth neural network algorithm |
| CN110942425A (en) * | 2019-11-26 | 2020-03-31 | 贵州师范学院 | Reconstruction method and reconstruction system of super-resolution image and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度神经网络与自动标记的人脸识别算法;赵卫东;秦锋;;攀枝花学院学报;20200915(05);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112348742A (en) | 2021-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107633218B (en) | Method and apparatus for generating image | |
| CN105261013B (en) | A kind of scan image quality overall evaluation method and evaluation system | |
| CN109523470B (en) | Depth image super-resolution reconstruction method and system | |
| CN109872305B (en) | No-reference stereo image quality evaluation method based on quality map generation network | |
| CN111784570A (en) | Video image super-resolution reconstruction method and device | |
| CN109389096B (en) | Detection method and device | |
| CN106169173B (en) | Image interpolation method | |
| CN107680043B (en) | Single image super-resolution output method based on graph model | |
| CN110751635B (en) | Oral cavity detection method based on interframe difference and HSV color space | |
| CN110738128A (en) | repeated video detection method based on deep learning | |
| CN104200468B (en) | Method for obtaining correction parameter of spherical perspective projection model | |
| WO2023010750A1 (en) | Image color mapping method and apparatus, electronic device, and storage medium | |
| CN112801904A (en) | Hybrid degraded image enhancement method based on convolutional neural network | |
| CN114926336A (en) | Video super-resolution reconstruction method and device, computer equipment and storage medium | |
| CN111429371A (en) | Image processing method and device and terminal equipment | |
| CN110837781B (en) | Face recognition method, face recognition device and electronic equipment | |
| CN116309483A (en) | DDPM-based semi-supervised power transformation equipment characterization defect detection method and system | |
| CN109034070B (en) | Blind separation method and device for replacement aliasing image | |
| CN112348742B (en) | Image nonlinear interpolation acquisition method and system based on deep learning | |
| CN110211164B (en) | Picture processing method of characteristic point operator based on neural network learning basic graph | |
| Li et al. | RGSR: A two-step lossy JPG image super-resolution based on noise reduction | |
| CN116468625A (en) | Single image defogging method and system based on pyramid efficient channel attention mechanism | |
| CN116977190A (en) | Image processing method, apparatus, device, storage medium, and program product | |
| CN116645268A (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| CN114140429A (en) | Real-time parking space detection method and device for vehicle end |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |