CN111445388A - Image super-resolution reconstruction model training method, ship tracking method and ship tracking device - Google Patents
Image super-resolution reconstruction model training method, ship tracking method and ship tracking device Download PDFInfo
- Publication number
- CN111445388A CN111445388A CN201911381391.5A CN201911381391A CN111445388A CN 111445388 A CN111445388 A CN 111445388A CN 201911381391 A CN201911381391 A CN 201911381391A CN 111445388 A CN111445388 A CN 111445388A
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- training
- sparse
- training sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 212
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 49
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 25
- 238000013135 deep learning Methods 0.000 claims abstract description 16
- 238000013507 mapping Methods 0.000 claims abstract description 15
- 238000012360 testing method Methods 0.000 claims description 44
- 238000001514 detection method Methods 0.000 claims description 31
- 230000015654 memory Effects 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 9
- 230000011218 segmentation Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 2
- 230000000875 corresponding effect Effects 0.000 description 12
- 238000004364 calculation method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image super-resolution reconstruction model training method, a ship tracking method and a ship tracking device, wherein the image super-resolution reconstruction model training method comprises the following steps: acquiring a ship image training sample, wherein the ship image training sample is a high-resolution training sample; obtaining a low-resolution training sample according to the high-resolution training sample; carrying out sparse expression decomposition on the low-resolution training sample according to a low-resolution sparse expression algorithm to obtain a low-resolution sparse training sample; mapping the low-resolution sparse expression algorithm to the high resolution to obtain a high-resolution sparse expression algorithm; carrying out sparse expression decomposition on the high-resolution training sample according to a high-resolution sparse expression algorithm to obtain a high-resolution sparse training sample; and training the sparse domain deep learning network model according to the low-resolution sparse training sample and the high-resolution sparse training sample to obtain an image super-resolution reconstruction model. By implementing the invention, the reconstruction precision of the high-resolution image is improved.
Description
Technical Field
The invention relates to the field of computer vision, in particular to an image super-resolution reconstruction model training method, a ship tracking method and a ship tracking device.
Background
As an important transportation vehicle, the ship plays an important role in production and life, and monitoring and tracking of the ship are essential to ensure the safety of the ship. In the related technology, a video satellite is used for tracking a ship, but the inherent low-resolution imaging characteristic of the video satellite and a high-magnification compression method adopted for adapting to the transmission capacity of a channel make the traditional video super-resolution technology based on multi-frame fusion difficult to recover enough detail information, the obtained ship image is fuzzy, and the ship identification accuracy is low.
Disclosure of Invention
Therefore, the technical problem to be solved by the invention is to overcome the defect of low ship identification accuracy in the prior art, and provide an image super-resolution reconstruction model training method, a ship tracking method and a ship tracking device.
According to a first aspect, the embodiment of the invention discloses an image super-resolution reconstruction model training method, which is applied to ship image recognition and comprises the following steps: acquiring a ship image training sample, wherein the ship image training sample is a high-resolution training sample; obtaining a low-resolution training sample according to the high-resolution training sample; carrying out sparse expression decomposition on the low-resolution training sample according to a low-resolution sparse expression algorithm to obtain a low-resolution sparse training sample; mapping the low-resolution sparse expression algorithm to the high resolution to obtain a high-resolution sparse expression algorithm; carrying out sparse expression decomposition on the high-resolution training sample according to the high-resolution sparse expression algorithm to obtain a high-resolution sparse training sample; and training a sparse domain deep learning network model according to the low-resolution sparse training sample and the high-resolution sparse training sample to obtain an image super-resolution reconstruction model.
With reference to the first aspect, in a first implementation manner of the first aspect, after the obtaining of the low-resolution training samples from the high-resolution training samples, the method further includes: segmenting the high resolution training samples and the low resolution samples according to superpixel segmentation.
With reference to the first aspect, in a second embodiment of the first aspect, the method further includes: in the training process, the image super-resolution reconstruction model is updated iteratively according to the following formula:
the PSNR represents the similarity between the high-resolution sparse training sample and the image obtained by training, n represents the bit number of each pixel, and MSE represents the mean square error between the high-resolution sparse training sample and the image obtained by training.
With reference to the second embodiment of the first aspect, in a third embodiment of the first aspect, the method further includes: acquiring a ship image test sample, wherein the ship image test sample is a high-resolution test sample; obtaining a low-resolution test sample according to the high-resolution test sample; obtaining a test result according to the low-resolution test sample and the image super-resolution reconstruction model; and when the test result meets a preset condition, determining the image super-resolution reconstruction model as a usable image super-resolution reconstruction model.
According to a second aspect, an embodiment of the present invention further discloses a ship tracking method, including the following steps: acquiring a video image acquired by a video satellite; inputting the video image into an image super-resolution reconstruction model to obtain a ship detection image; the image super-resolution reconstruction model is generated by training through the image super-resolution reconstruction model training method according to the first aspect or any embodiment of the first aspect; and correlating the ship detection image by using a target algorithm to obtain the running track of the ship.
With reference to the second aspect, in the first embodiment of the second aspect, after the acquiring the video image acquired by the video satellite, before inputting the video image into an image super-resolution reconstruction model, the method further includes: and carrying out image enhancement and denoising processing on the video image.
According to a third aspect, the embodiment of the present invention further discloses an image super-resolution reconstruction model training apparatus, which is applied to ship image recognition, and includes: the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring ship image training samples which are high-resolution training samples; the low-resolution training sample acquisition module is used for acquiring a low-resolution training sample according to the high-resolution training sample; the first decomposition module is used for carrying out sparse expression decomposition on the low-resolution training sample according to a low-resolution sparse expression algorithm to obtain a low-resolution sparse training sample; the mapping module is used for mapping the low-resolution sparse representation algorithm to the high resolution to obtain a high-resolution sparse representation algorithm; the second decomposition module is used for carrying out sparse expression decomposition on the high-resolution training sample according to the high-resolution sparse expression algorithm to obtain a high-resolution sparse training sample; and the training module is used for training the sparse domain deep learning network model according to the low-resolution sparse training sample and the high-resolution sparse training sample to obtain an image super-resolution reconstruction model.
According to a fourth aspect, an embodiment of the present invention further discloses a ship tracking device, including: the second acquisition module is used for acquiring a video image acquired by a video satellite; the detection module is used for inputting the video image into an image super-resolution reconstruction model to obtain a ship detection image; the image super-resolution reconstruction model is generated by training through the image super-resolution reconstruction model training method according to the first aspect or any embodiment of the first aspect; and the association module is used for associating the ship detection image by using a target algorithm to obtain the running track of the ship.
According to a fifth aspect, an embodiment of the present invention further discloses a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the steps of the image super-resolution reconstruction model training method according to the first aspect or any of the embodiments of the first aspect or the ship tracking method according to any of the embodiments of the second aspect or the second aspect when executing the program.
According to a sixth aspect, the present invention further discloses a storage medium, on which computer instructions are stored, and the instructions, when executed by a processor, implement the steps of the image super-resolution reconstruction model training method according to the first aspect or any of the embodiments of the first aspect or the vessel tracking method according to any of the embodiments of the second aspect or the second aspect.
The technical scheme of the invention has the following advantages:
1. the invention provides a training method and a training device for an image super-resolution reconstruction model. By implementing the method, the mapping model of high-low resolution sparse expression based on the sparse domain deep learning network is established, the high-resolution image reconstruction precision is improved, the ship identification accuracy is improved, and the ship can be conveniently tracked.
2. According to the ship tracking method and device, the video images acquired by the video satellites are acquired, the video images are input into the image super-resolution reconstruction model to obtain ship detection images, and the ship detection images are correlated by using a target algorithm to obtain the running track of a ship. According to the invention, the video images acquired by the video satellite are input into the image super-resolution reconstruction model, so that the detail information of the video images is richer, the ship contour is clearer, and the ship can be conveniently tracked.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a specific example of a training method for an image super-resolution reconstruction model in embodiment 1 of the present invention;
FIG. 2 is a flowchart showing a specific example of a ship tracking method according to embodiment 2 of the present invention;
fig. 3 is a schematic block diagram of a specific example of the training apparatus for image super-resolution reconstruction model in embodiment 3 of the present invention;
FIG. 4 is a schematic block diagram showing a specific example of the ship tracking apparatus according to embodiment 4 of the present invention;
fig. 5 is a schematic block diagram of a specific example of a computer device in embodiment 5 of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In addition, the technical features involved in the different embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Example 1
The embodiment of the invention provides an image super-resolution reconstruction model training method, which is applied to ship image recognition and comprises the following steps as shown in figure 1:
s11: and acquiring a ship image training sample, wherein the ship image training sample is a high-resolution training sample.
In order to improve the identification accuracy, in the embodiment of the present invention, the ship image training sample should ensure the diversity of images, including different shooting environments, different sensors, different degradation models, and the like, and the ship image training sample may be collected and processed by a certain camera, stored in a terminal, and directly called from the terminal, or acquired from a network database.
S12: and obtaining a low-resolution training sample according to the high-resolution training sample.
Exemplarily, in the embodiment of the present invention, a high resolution image training set is downsampled by 3 times first, and then upsampled by 3 times by a double cubic interpolation method, so as to obtain a low resolution image training set.
S13: and carrying out sparse expression decomposition on the low-resolution training sample according to a low-resolution sparse expression algorithm to obtain the low-resolution sparse training sample.
Illustratively, the low-resolution training samples are subjected to sparse representation decomposition to obtain samples for sparse domain deep learning network training, and the purpose of sparse representation is to select representative atoms from the low-resolution training samples to represent the input image. A low resolution training sample will usually be an overcomplete dictionary, so that the vector obtained when encoding usually has only a few elements that are non-zero and others that are zero, and thus such an encoded vector is called sparse coding.
S14: and mapping the low-resolution sparse expression algorithm to the high resolution to obtain the high-resolution sparse expression algorithm.
Illustratively, a least square method is used for calculating a projection matrix between the low-resolution blocks and the high-resolution atoms, traversing the whole dictionary atoms, and finally obtaining a high-resolution sparse expression algorithm according to the projection matrix, wherein each atom corresponds to a cluster of neighbor atoms and the projection matrix.
S15: and carrying out sparse expression decomposition on the high-resolution training sample according to a high-resolution sparse expression algorithm to obtain the high-resolution sparse training sample. In the high resolution training sample dictionary, the atoms corresponding to the low resolution training samples are also taken out, and the specific embodiment is described in the above step S13, which is not described herein again.
S16: and training the sparse domain deep learning network model according to the low-resolution sparse training sample and the high-resolution sparse training sample to obtain an image super-resolution reconstruction model.
Illustratively, low-resolution sparse training samples are input into a sparse domain deep learning network model for training, corresponding high-resolution sparse training samples are used for monitoring the training process, the weight of the sparse domain deep learning network model is continuously adjusted according to the low-resolution sparse training samples, the sparse domain deep learning network model is continuously optimized, and an image super-resolution reconstruction model is obtained.
The invention provides a training method of an image super-resolution reconstruction model, which comprises the steps of obtaining a ship image training sample, wherein the ship image training sample is a high-resolution training sample, obtaining a low-resolution training sample according to the high-resolution training sample, carrying out sparse expression decomposition on the low-resolution training sample according to a low-resolution sparse expression algorithm to obtain a low-resolution sparse training sample, mapping the low-resolution sparse expression algorithm to the high resolution to obtain a high-resolution sparse expression algorithm, carrying out sparse expression decomposition on the high-resolution training sample according to the high-resolution sparse expression algorithm to obtain a high-resolution sparse training sample, and training a sparse domain deep learning network model according to the low-resolution sparse training sample and the high-resolution sparse training sample to obtain the image super-resolution reconstruction model. By implementing the method, the mapping model of high-low resolution sparse expression based on the sparse domain deep learning network is established, the high-resolution image reconstruction precision is improved, the ship identification accuracy is improved, and the ship can be conveniently tracked.
As an optional embodiment of the present application, after obtaining the low-resolution training sample according to the high-resolution training sample, the training method for the image super-resolution reconstruction model further includes:
the high resolution training samples and the low resolution samples are segmented according to superpixel segmentation.
Illustratively, the image superpixel segmentation refers to a process of segmenting an image into a plurality of image sub-regions, and in the embodiment of the present invention, the image superpixel segmentation may be simple linear iterative clustering (S L IC), the image segmentation process includes sampling pixels at a regular grid step S to initialize a cluster center, moving the cluster center to a lowest gradient position in a 3x3 domain, setting a label for each pixel, setting a distance for each pixel, traversing each pixel point in a 2Sx2S domain for each cluster center, calculating a distance to determine whether to update the label and the distance of the pixel, updating the cluster center, and repeating the steps of traversing each pixel point in the 2Sx2S domain for each cluster center until convergence.
As an optional embodiment of the present application, the training method for the image super-resolution reconstruction model further includes: in the training process, the image super-resolution reconstruction model is updated iteratively according to the following formula:
the PSNR represents the similarity between the high-resolution sparse training sample and the image obtained by training, n represents the bit number of each pixel, and MSE represents the mean square error between the high-resolution sparse training sample and the image obtained by training.
Illustratively, in the training process of the image super-resolution reconstruction model, a peak signal-to-noise ratio (PSNR) is adopted to evaluate the quality of an image obtained in the training process, the PSNR refers to the similarity between a high-resolution sparse training sample and the image obtained by training, the difference of the image pixel level is compared, the larger the numerical value is, the smaller the distortion is, when the similarity PSNR reaches a certain preset value, the training is stopped, and when the value of the similarity PSNR does not meet the preset value all the time, the training process of the updated image super-resolution reconstruction model is updated all the time. The preset value may be 50dB, and the preset value is not limited in the embodiment of the present application, and can be set by a person skilled in the art according to actual situations. Using the mean square error as a loss function is beneficial for obtaining a higher PSNR similarity.
As an optional embodiment of the present application, the training method for the image super-resolution reconstruction model further includes:
firstly, a ship image test sample is obtained, and the ship image test sample is a high-resolution test sample.
For example, the ship image test sample may be obtained by randomly dividing a part of the ship image sample obtained in S11, for example, the remaining 30% of the ship image test sample is different from the ship image training sample. The acquisition mode of the ship image test sample is not limited in the embodiment, and can be determined according to the requirement.
And secondly, obtaining a low-resolution test sample according to the high-resolution test sample. The detailed description of the step S12 is omitted here for the detailed description.
And thirdly, obtaining a test result according to the low-resolution test sample and the image super-resolution reconstruction model.
For example, the manner of obtaining the test result according to the low-resolution test sample and the image super-resolution reconstruction model may be to input all the low-resolution test samples to the image super-resolution reconstruction model, perform cross-over ratio calculation according to a plurality of prediction boundary frames output by the image super-resolution reconstruction model and the detection result and the mark in the prediction boundary frames, and take the average value of the calculation results of all the cross-over ratios as the accuracy rate, thereby obtaining the accuracy rate of the output result of the neural network; or performing cross-over ratio calculation according to a plurality of prediction boundary frames output by the image super-resolution reconstruction model and detection results and marks in the prediction boundary frames, judging whether each cross-over ratio calculation result meets a preset threshold value, indicating that the detection result of the low-resolution test sample is accurate when the preset threshold value is met, and taking the ratio of the low-resolution test sample with the accurate detection result to all corresponding high-resolution test samples as the accuracy of the output result of the neural network. The embodiment does not limit what kind of method is specifically adopted to obtain the test result, and can determine according to the needs.
And finally, when the test result meets the preset condition, determining the image super-resolution reconstruction model as an available image super-resolution reconstruction model. The preset threshold may be 98%, and the size of the preset threshold is not limited in this embodiment and may be set as needed.
Example 2
An embodiment of the present invention provides a ship tracking method, as shown in fig. 2, including the following steps:
s21: and acquiring a video image acquired by a video satellite.
Exemplarily, the video images acquired by the video satellite have low spatial resolution and blurred images, and the acquisition mode of the video images acquired by the video satellite can be that the videos acquired by the video satellite are subjected to frame decoding processing to acquire each frame of video images; or frame skipping processing, in which a video image separated by a certain number of frames is used as the acquired video image, for example, a video image is acquired every 3 frames. The mode of obtaining the video image collected by the unmanned aerial vehicle is not limited by the embodiment, and can be determined through requirements.
S22: inputting the video image into an image super-resolution reconstruction model to obtain a ship detection image; the image super-resolution reconstruction model is generated by training through the training method of the image super-resolution reconstruction model as in the above embodiment 1.
For example, the image super-resolution reconstruction model is generated by training according to the image super-resolution reconstruction model training method in embodiment 1, and the ship detection image represents a position coordinate frame detected by the ship in the current video image, and the rest of the image super-resolution reconstruction model is not described herein again.
S23: and correlating the ship detection images by using a target algorithm to obtain the running track of the ship.
Exemplarily, in the embodiment of the present invention, the ship detection images are associated by using a target algorithm to obtain the ship movement track, and the ship detection images in the video images acquired by any two adjacent video satellites are divided into two sets, the maximum matching between the two sets is found, and the matching results are subjected to data association to obtain the ship movement track; or adopting an IOU algorithm to obtain ship detection images in video images acquired by any two adjacent video satellites and sequentially carrying out intersection ratio calculation, and carrying out data association on two ship detection results with the largest intersection ratio calculation result so as to obtain the running track of the ship. The target algorithm is not limited in this embodiment, and may be determined by requirements.
According to the ship tracking method, the video images acquired by the video satellites are acquired, the video images are input into the image super-resolution reconstruction model to obtain ship detection images, and the ship detection images are associated by using a target algorithm to obtain the running track of a ship. According to the invention, the video images acquired by the video satellite are input into the image super-resolution reconstruction model, so that the detail information of the video images is richer, the ship contour is clearer, and the ship can be conveniently tracked.
As an optional embodiment of the present application, after acquiring the video image acquired by the video satellite and before inputting the video image to the image super-resolution reconstruction model, the ship tracking method further includes: and carrying out image enhancement and denoising processing on the video image. The method is favorable for further reducing the influence of noise or weather conditions in the video satellite image data on the detection result.
Example 3
The embodiment of the present invention further provides an image super-resolution reconstruction model training apparatus, which is applied to ship image recognition, and as shown in fig. 3, the image super-resolution reconstruction model training apparatus includes:
the first acquisition module 31 is configured to acquire a ship image training sample, where the ship image training sample is a high-resolution training sample; the specific implementation manner is shown in step S11 in embodiment 1, and details are not described here.
A low resolution training sample obtaining module 32, configured to obtain a low resolution training sample according to the high resolution training sample; the specific implementation manner is shown in step S12 in embodiment 1, and details are not described here.
The first decomposition module 33 is configured to perform sparse representation decomposition on the low-resolution training sample according to a low-resolution sparse representation algorithm to obtain a low-resolution sparse training sample; the specific implementation manner is shown in step S13 in embodiment 1, and details are not described here.
The mapping module 34 is configured to map the low-resolution sparse representation algorithm to the high resolution to obtain a high-resolution sparse representation algorithm; the specific implementation manner is shown in step S14 in embodiment 1, and details are not described here.
The second decomposition module 35 is configured to perform sparse expression decomposition on the high-resolution training sample according to a high-resolution sparse expression algorithm to obtain a high-resolution sparse training sample; the specific implementation manner is shown in step S15 in embodiment 1, and details are not described here.
And the training module 36 is configured to train the sparse domain deep learning network model according to the low-resolution sparse training sample and the high-resolution sparse training sample to obtain an image super-resolution reconstruction model. The specific implementation manner is shown in step S16 in embodiment 1, and details are not described here.
The invention provides an image super-resolution reconstruction model training device which obtains an image super-resolution reconstruction model by obtaining a ship image training sample, wherein the ship image training sample is a high-resolution training sample, obtaining a low-resolution training sample according to the high-resolution training sample, carrying out sparse expression decomposition on the low-resolution training sample according to a low-resolution sparse expression algorithm to obtain a low-resolution sparse training sample, mapping the low-resolution sparse expression algorithm to the high resolution to obtain a high-resolution sparse expression algorithm, carrying out sparse expression decomposition on the high-resolution training sample according to the high-resolution sparse expression algorithm to obtain a high-resolution sparse training sample, and training a sparse domain deep learning network model according to the low-resolution sparse training sample and the high-resolution sparse training sample. By implementing the method, the mapping model of high-low resolution sparse expression based on the sparse domain deep learning network is established, the high-resolution image reconstruction precision is improved, the ship identification accuracy is improved, and the ship can be conveniently tracked.
As an optional embodiment of the present application, the training apparatus for image super-resolution reconstruction model further includes:
and the segmentation module is used for segmenting the high-resolution training sample and the low-resolution sample according to the super-pixel segmentation. The specific implementation manner is shown in the corresponding steps in embodiment 1, and is not described herein again.
As an optional embodiment of the present application, the training apparatus for image super-resolution reconstruction model further includes: in the training process, the image super-resolution reconstruction model is updated iteratively according to the following formula:
the PSNR represents the similarity between the high-resolution sparse training sample and the image obtained by training, n represents the bit number of each pixel, and MSE represents the mean square error between the high-resolution sparse training sample and the image obtained by training. The specific implementation manner is shown in the corresponding steps in embodiment 1, and is not described herein again.
As an optional embodiment of the present application, the training apparatus for image super-resolution reconstruction model further includes:
and the third acquisition module is used for acquiring a ship image test sample, and the ship image test sample is a high-resolution test sample. The specific implementation manner is shown in the corresponding steps in embodiment 1, and is not described herein again.
And the low-resolution test sample acquisition module is used for acquiring a low-resolution test sample according to the high-resolution test sample. The specific implementation manner is shown in the corresponding steps in embodiment 1, and is not described herein again.
And the test module is used for obtaining a test result according to the low-resolution test sample and the image super-resolution reconstruction model. The specific implementation manner is shown in the corresponding steps in embodiment 1, and is not described herein again.
And the determining module is used for determining the image super-resolution reconstruction model as an available image super-resolution reconstruction model when the test result meets the preset condition. The specific implementation manner is shown in the corresponding steps in embodiment 1, and is not described herein again.
Example 4
An embodiment of the present invention further provides a ship tracking device, including:
a second obtaining module 41, configured to obtain a video image acquired by a video satellite; the specific implementation manner is shown in step S21 in embodiment 2, and details are not described here.
The detection module 42 is used for inputting the video image into the image super-resolution reconstruction model to obtain a ship detection image; the image super-resolution reconstruction model is generated by training through the image super-resolution reconstruction model training method in the embodiment 1; the specific implementation manner is shown in step S22 in embodiment 2, and details are not described here.
And the associating module 43 is configured to associate the ship detection image with a target algorithm to obtain a running track of the ship. The specific implementation manner is shown in step S23 in embodiment 2, and details are not described here.
According to the ship tracking device provided by the invention, the video images acquired by the video satellites are acquired, the video images are input into the image super-resolution reconstruction model to obtain the ship detection images, and the ship detection images are associated by using a target algorithm to obtain the running track of the ship. According to the invention, the video images acquired by the video satellite are input into the image super-resolution reconstruction model, so that the detail information of the video images is richer, the ship contour is clearer, and the ship can be conveniently tracked.
As an optional embodiment of the present application, the vessel tracking device further comprises:
and the image processing module is used for carrying out image enhancement and denoising processing on the video image. The specific implementation manner is shown in the corresponding steps in embodiment 2, and is not described herein again.
Example 5
An embodiment of the present invention further provides a computer device, as shown in fig. 5, the computer device may include a processor 51 and a memory 52, where the processor 51 and the memory 52 may be connected by a bus or in another manner, and fig. 5 takes the example of connection by a bus as an example.
The processor 51 may be a Central Processing Unit (CPU). The Processor 51 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or combinations thereof.
The memory 52 is a non-transitory computer readable storage medium, and can be used for storing non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the image super-resolution reconstruction model training method or the ship tracking method in the embodiment of the present invention (for example, the first obtaining module 31, the low resolution training sample obtaining module 32, the first decomposing module 33, the mapping module 34, the second decomposing module 35, and the training module 36 shown in fig. 3 or the second obtaining module 41, the detecting module 42, and the associating module 43 shown in fig. 4). The processor 51 executes various functional applications and data processing of the processor by running non-transitory software programs, instructions and modules stored in the memory 52, namely, implementing the image super-resolution reconstruction model training method or the ship tracking method in the above method embodiments.
The memory 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor 51, and the like. Further, the memory 52 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 52 may optionally include memory located remotely from the processor 51, and these remote memories may be connected to the processor 51 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 52 and when executed by the processor 51, perform an image super-resolution reconstruction model training method as in the embodiment shown in fig. 1 or a vessel tracking method as in the embodiment shown in fig. 2.
The details of the electronic device may be understood with reference to the corresponding descriptions and effects in the embodiments shown in fig. 1 or fig. 2, and are not described herein again.
Example 6
The embodiment of the invention also provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the image super-resolution reconstruction model training method or the ship tracking method in any method embodiment. The storage medium may be a magnetic Disk, an optical Disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a flash Memory (FlashMemory), a Hard Disk (Hard Disk Drive, abbreviated as HDD) or a Solid State Drive (SSD), etc.; the storage medium may also comprise a combination of memories of the kind described above.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.
Claims (10)
1. An image super-resolution reconstruction model training method is applied to ship image recognition and is characterized by comprising the following steps:
acquiring a ship image training sample, wherein the ship image training sample is a high-resolution training sample;
obtaining a low-resolution training sample according to the high-resolution training sample;
carrying out sparse expression decomposition on the low-resolution training sample according to a low-resolution sparse expression algorithm to obtain a low-resolution sparse training sample;
mapping the low-resolution sparse expression algorithm to the high resolution to obtain a high-resolution sparse expression algorithm;
carrying out sparse expression decomposition on the high-resolution training sample according to the high-resolution sparse expression algorithm to obtain a high-resolution sparse training sample;
and training a sparse domain deep learning network model according to the low-resolution sparse training sample and the high-resolution sparse training sample to obtain an image super-resolution reconstruction model.
2. The method of claim 1, wherein after the obtaining low resolution training samples from the high resolution training samples, the method further comprises:
segmenting the high resolution training samples and the low resolution samples according to superpixel segmentation.
3. The method of claim 1, further comprising: in the training process, the image super-resolution reconstruction model is updated iteratively according to the following formula:
the PSNR represents the similarity between the high-resolution sparse training sample and the image obtained by training, n represents the bit number of each pixel, and MSE represents the mean square error between the high-resolution sparse training sample and the image obtained by training.
4. The method of claim 3, further comprising:
acquiring a ship image test sample, wherein the ship image test sample is a high-resolution test sample;
obtaining a low-resolution test sample according to the high-resolution test sample;
obtaining a test result according to the low-resolution test sample and the image super-resolution reconstruction model;
and when the test result meets a preset condition, determining the image super-resolution reconstruction model as a usable image super-resolution reconstruction model.
5. A method of vessel tracking, comprising the steps of:
acquiring a video image acquired by a video satellite;
inputting the video image into an image super-resolution reconstruction model to obtain a ship detection image; the image super-resolution reconstruction model is generated by training through the image super-resolution reconstruction model training method according to any one of claims 1 to 4;
and correlating the ship detection image by using a target algorithm to obtain the running track of the ship.
6. The method of claim 5, wherein after said obtaining the video images acquired by the video satellite and before inputting the video images into an image super-resolution reconstruction model, further comprising:
and carrying out image enhancement and denoising processing on the video image.
7. The utility model provides an image super-resolution reconstruction model training device, is applied to ship image recognition, its characterized in that includes:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring ship image training samples which are high-resolution training samples;
the low-resolution training sample acquisition module is used for acquiring a low-resolution training sample according to the high-resolution training sample;
the first decomposition module is used for carrying out sparse expression decomposition on the low-resolution training sample according to a low-resolution sparse expression algorithm to obtain a low-resolution sparse training sample;
the mapping module is used for mapping the low-resolution sparse representation algorithm to the high resolution to obtain a high-resolution sparse representation algorithm;
the second decomposition module is used for carrying out sparse expression decomposition on the high-resolution training sample according to the high-resolution sparse expression algorithm to obtain a high-resolution sparse training sample;
and the training module is used for training the sparse domain deep learning network model according to the low-resolution sparse training sample and the high-resolution sparse training sample to obtain an image super-resolution reconstruction model.
8. A vessel tracking device, comprising:
the second acquisition module is used for acquiring a video image acquired by a video satellite;
the detection module is used for inputting the video image into an image super-resolution reconstruction model to obtain a ship detection image; the image super-resolution reconstruction model is generated by training through the image super-resolution reconstruction model training method according to any one of claims 1 to 4;
and the association module is used for associating the ship detection image by using a target algorithm to obtain the running track of the ship.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor when executing the program implements the steps of the image super resolution reconstruction model training method according to any of the claims 1-4 or the vessel tracking method according to any of the claims 5-6.
10. A storage medium having stored thereon computer instructions, which when executed by a processor, carry out the steps of the image super-resolution reconstruction model training method according to any one of claims 1 to 4 or the vessel tracking method according to any one of claims 5 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911381391.5A CN111445388A (en) | 2019-12-27 | 2019-12-27 | Image super-resolution reconstruction model training method, ship tracking method and ship tracking device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911381391.5A CN111445388A (en) | 2019-12-27 | 2019-12-27 | Image super-resolution reconstruction model training method, ship tracking method and ship tracking device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111445388A true CN111445388A (en) | 2020-07-24 |
Family
ID=71626921
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911381391.5A Pending CN111445388A (en) | 2019-12-27 | 2019-12-27 | Image super-resolution reconstruction model training method, ship tracking method and ship tracking device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111445388A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111932462A (en) * | 2020-08-18 | 2020-11-13 | Oppo（重庆）智能科技有限公司 | Training method and device for image degradation model, electronic equipment and storage medium |
| CN112258394A (en) * | 2020-11-11 | 2021-01-22 | 珠海大横琴科技发展有限公司 | Data processing method, ship tracking method, device, equipment and storage medium |
| CN112465042A (en) * | 2020-12-02 | 2021-03-09 | 中国联合网络通信集团有限公司 | Generation method and device of classification network model |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104778659A (en) * | 2015-04-15 | 2015-07-15 | 杭州电子科技大学 | Single-frame image super-resolution reconstruction method on basis of deep learning |
| AU2015101167A4 (en) * | 2015-07-26 | 2015-10-01 | Macau University Of Science And Technology | A Single Image Super-Resolution Method Using Transform-Invariant Directional Total Variation with S1/2+L1/2-norm |
| CN105513014A (en) * | 2016-01-21 | 2016-04-20 | 集美大学 | Method and system for reconstruction of multiframe image super resolution |
| CN105931179A (en) * | 2016-04-08 | 2016-09-07 | 武汉大学 | Joint sparse representation and deep learning-based image super resolution method and system |
| CN107392855A (en) * | 2017-07-19 | 2017-11-24 | 苏州闻捷传感技术有限公司 | Image Super-resolution Reconstruction method based on sparse autoencoder network Yu very fast study |
| CN108830792A (en) * | 2018-05-09 | 2018-11-16 | 浙江师范大学 | A kind of image super-resolution method using multiclass dictionary |
| US20190073594A1 (en) * | 2017-09-01 | 2019-03-07 | Thomson Licensing | Apparatus and method to process and cluster data |
| CN109741256A (en) * | 2018-12-13 | 2019-05-10 | 西安电子科技大学 | Image super-resolution rebuilding method based on rarefaction representation and deep learning |
| CN109816606A (en) * | 2019-01-18 | 2019-05-28 | 中国科学院电子学研究所 | A method of target following is carried out using Optical remote satellite |
| CN110570351A (en) * | 2019-08-01 | 2019-12-13 | 武汉大学 | Image super-resolution reconstruction method based on convolution sparse coding |
-
2019
- 2019-12-27 CN CN201911381391.5A patent/CN111445388A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104778659A (en) * | 2015-04-15 | 2015-07-15 | 杭州电子科技大学 | Single-frame image super-resolution reconstruction method on basis of deep learning |
| AU2015101167A4 (en) * | 2015-07-26 | 2015-10-01 | Macau University Of Science And Technology | A Single Image Super-Resolution Method Using Transform-Invariant Directional Total Variation with S1/2+L1/2-norm |
| CN105513014A (en) * | 2016-01-21 | 2016-04-20 | 集美大学 | Method and system for reconstruction of multiframe image super resolution |
| CN105931179A (en) * | 2016-04-08 | 2016-09-07 | 武汉大学 | Joint sparse representation and deep learning-based image super resolution method and system |
| CN107392855A (en) * | 2017-07-19 | 2017-11-24 | 苏州闻捷传感技术有限公司 | Image Super-resolution Reconstruction method based on sparse autoencoder network Yu very fast study |
| US20190073594A1 (en) * | 2017-09-01 | 2019-03-07 | Thomson Licensing | Apparatus and method to process and cluster data |
| CN108830792A (en) * | 2018-05-09 | 2018-11-16 | 浙江师范大学 | A kind of image super-resolution method using multiclass dictionary |
| CN109741256A (en) * | 2018-12-13 | 2019-05-10 | 西安电子科技大学 | Image super-resolution rebuilding method based on rarefaction representation and deep learning |
| CN109816606A (en) * | 2019-01-18 | 2019-05-28 | 中国科学院电子学研究所 | A method of target following is carried out using Optical remote satellite |
| CN110570351A (en) * | 2019-08-01 | 2019-12-13 | 武汉大学 | Image super-resolution reconstruction method based on convolution sparse coding |
Non-Patent Citations (4)
| Title |
|---|
| 吴炜: "《基于学习的图像增强技术》", 28 February 2013 * |
| 曹铁勇等译: "《稀疏与冗余表示----理论及其在信号与图像处理中的应用》", 31 May 2015 * |
| 欧阳合，韩军: "《H.264和MPEG-4视频压缩---新一代多媒体的视频编码技术》", 30 November 2004 * |
| 黄祚继，王春林，孙金彦，董丹丹: "《多源遥感数据目标地物的分类与优化》", 31 May 2017 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111932462A (en) * | 2020-08-18 | 2020-11-13 | Oppo（重庆）智能科技有限公司 | Training method and device for image degradation model, electronic equipment and storage medium |
| CN111932462B (en) * | 2020-08-18 | 2023-01-03 | Oppo（重庆）智能科技有限公司 | Training method and device for image degradation model, electronic equipment and storage medium |
| CN112258394A (en) * | 2020-11-11 | 2021-01-22 | 珠海大横琴科技发展有限公司 | Data processing method, ship tracking method, device, equipment and storage medium |
| CN112258394B (en) * | 2020-11-11 | 2022-04-29 | 珠海大横琴科技发展有限公司 | Data processing method, ship tracking method, device, equipment and storage medium |
| CN112465042A (en) * | 2020-12-02 | 2021-03-09 | 中国联合网络通信集团有限公司 | Generation method and device of classification network model |
| CN112465042B (en) * | 2020-12-02 | 2023-10-24 | 中国联合网络通信集团有限公司 | Method and device for generating classified network model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11222211B2 (en) | Method and apparatus for segmenting video object, electronic device, and storage medium | |
| CN111681273B (en) | Image segmentation method and device, electronic equipment and readable storage medium | |
| CN112446383B (en) | License plate recognition method and device, storage medium and terminal | |
| CN110781756A (en) | Urban road extraction method and device based on remote sensing image | |
| CN111445388A (en) | Image super-resolution reconstruction model training method, ship tracking method and ship tracking device | |
| CN109583345B (en) | Road recognition method, device, computer device and computer readable storage medium | |
| WO2023082453A1 (en) | Image processing method and device | |
| CN114742799B (en) | Industrial scene unknown type defect segmentation method based on self-supervision heterogeneous network | |
| CN113313047B (en) | Lane line detection method and system based on lane structure prior | |
| CN115147418B (en) | Compression training method and device for defect detection model | |
| WO2020043296A1 (en) | Device and method for separating a picture into foreground and background using deep learning | |
| CN115035172A (en) | Depth estimation method and system based on confidence degree grading and inter-stage fusion enhancement | |
| CN113393385B (en) | Multi-scale fusion-based unsupervised rain removing method, system, device and medium | |
| CN112330618B (en) | Image offset detection method, device and storage medium | |
| EP3156943A1 (en) | Method and device for clustering patches of a degraded version of an image | |
| CN110490170B (en) | Face candidate frame extraction method | |
| CN112418229A (en) | Unmanned ship marine scene image real-time segmentation method based on deep learning | |
| CN111582013A (en) | Ship retrieval method and device based on gray level co-occurrence matrix characteristics | |
| CN114429602A (en) | Semantic segmentation method and device, electronic equipment and storage medium | |
| CN110648291B (en) | Unmanned aerial vehicle motion blurred image restoration method based on deep learning | |
| CN110738225B (en) | Image recognition method and device | |
| CN114596580B (en) | Multi-human-body target identification method, system, equipment and medium | |
| Bian et al. | Swin transformer UNet for very high resolution image dehazing | |
| CN112668563B (en) | Method and related device for realizing movement track calculation based on video image | |
| CN116681625B (en) | Multi-scale contrast learning-based unsupervised image rain removing method, device and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200724 |