CN117789041B - Ship defogging method and system based on atmospheric scattering priori diffusion model - Google Patents
Ship defogging method and system based on atmospheric scattering priori diffusion model Download PDFInfo
- Publication number
- CN117789041B CN117789041B CN202410217435.5A CN202410217435A CN117789041B CN 117789041 B CN117789041 B CN 117789041B CN 202410217435 A CN202410217435 A CN 202410217435A CN 117789041 B CN117789041 B CN 117789041B
- Authority
- CN
- China
- Prior art keywords
- picture
- noise
- current
- ship
- clear
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000009792 diffusion process Methods 0.000 title claims abstract description 92
- 238000000034 method Methods 0.000 title claims abstract description 43
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims abstract description 128
- 239000003595 mist Substances 0.000 claims abstract description 119
- 230000007246 mechanism Effects 0.000 claims description 24
- 230000009466 transformation Effects 0.000 claims description 13
- 230000004048 modification Effects 0.000 claims description 4
- 238000012986 modification Methods 0.000 claims description 4
- 239000007921 spray Substances 0.000 claims description 4
- 238000002834 transmittance Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 2
- 238000006243 chemical reaction Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000007423 decrease Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 2
- 230000000452 restraining effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a ship defogging method and system based on an atmospheric scattering priori diffusion model. Wherein the method comprises the following steps: predicting all water mist ship pictures in the training set through a trained target Unet model to obtain all target clear pictures, all target transmissivity pictures and all target pure noise pictures; performing multi-step diffusion and noise adding on each clear ship picture in the training set to obtain a plurality of corresponding diffusion noise pictures; randomly extracting one of all diffusion noise pictures corresponding to each clear ship picture, inputting each target clear picture and each target transmissivity picture corresponding to the clear ship picture into a constructed denoising Unet model, and training to obtain a target denoising Unet model; inputting the current water mist ship picture, the corresponding current target clear picture and the corresponding current target transmissivity picture into the target denoising Unet model for reverse reasoning defogging to obtain the current recovered clear ship picture; the ship picture obtained by the method is more real.
Description
Technical Field
The invention relates to the technical field of ships, in particular to a ship defogging method and a ship defogging system based on an atmospheric scattering priori diffusion model.
Background
With the continuous development of water traffic, the water traffic order is also continuously updated, and the workload of workers maintaining the water traffic is continuously increased, wherein the ship detection is one of the water traffic workload. However, a large amount of water mist can appear in the evening or early morning even in the daytime in autumn and winter, so that the ship on the water surface is shielded by the water mist, and the ship on the river surface cannot be detected.
Aiming at the problem that in the prior art, when a ship is shielded by water mist, clear detection cannot be carried out on the ship, no effective solution is proposed at present.
Disclosure of Invention
The embodiment of the invention provides a ship defogging method and a ship defogging system based on an atmospheric scattering priori diffusion model, which are used for solving the problem that in the prior art, when a ship is shielded by water mist, the ship cannot be clearly detected.
To achieve the above object, in one aspect, the present invention provides a ship defogging method based on an atmospheric scattering prior diffusion model, the method comprising: s1, acquiring a training set, wherein the training set is divided into a plurality of groups, and each group comprises a clear ship picture and a corresponding water mist ship picture; predicting all water mist ship pictures in the training set through a trained target Unet model to obtain all target clear pictures, all target transmissivity pictures and all target pure noise pictures; s2, carrying out multi-step diffusion noise adding on each clear ship picture in the training set to obtain a plurality of diffusion noise pictures corresponding to each clear ship picture, and storing a plurality of noise values loaded by each clear ship picture; s3, randomly extracting one of all diffusion noise pictures in the current clear ship picture, inputting the current target clear picture and the current target transmissivity picture generated by the corresponding current water mist ship picture into a constructed denoising Unet model for training, and obtaining a current prediction noise value; calculating a current iteration loss value according to a noise value loaded by a diffusion noise picture which is extracted randomly and a current predicted noise value; updating the denoising Unet model according to the current iteration loss value; repeating the steps S2-S3 until all the clear ship pictures in the training set are trained and a plurality of rounds of training are carried out, and obtaining a target denoising Unet model; s4, inputting the current water mist ship picture, the corresponding current target clear picture and the corresponding current target transmissivity picture into the target denoising Unet model for reverse reasoning defogging to obtain the current recovered clear ship picture; and S4, repeating until all the water mist ship pictures are trained, and obtaining all the recovered clear ship pictures.
Optionally, the target Unet model is trained by: s11, changing one output of an original Unet model into three outputs to obtain a modified Unet model; s12, inputting a current water mist ship picture in the training set into the modified Unet model to obtain a current original clear picture, a current original transmissivity picture and a current original pure noise picture; s13, inputting the current original clear picture, the current original transmissivity picture and the current original pure noise picture into an atmospheric scattering model to generate a current noise-containing ship picture; s14, calculating a first noise loss value according to the noise value of the current noise-containing ship picture and the noise value of the current water mist ship picture; s15, calculating a second noise loss value according to the noise value of the current original clear picture and the noise value of the current clear ship picture corresponding to the current water spray ship picture; s16, calculating to obtain an iterative noise loss value according to the first noise loss value and the second noise loss value; updating the modified Unet model according to the iterative noise loss value; and S17, repeating the steps S12-S16 until all the water mist ship pictures in the training set are trained and a plurality of training rounds are performed, so that the training is stopped after the iterative noise loss value fluctuates within a preset range, and the finally updated and obtained modified Unet model is used as the target Unet model.
Optionally, a plurality of diffusion noise pictures obtained by performing multi-step diffusion noise adding on the current clear ship picture are calculated according to the following formula:
；
Wherein, Diffusion noise picture obtained by performing last diffusion noise adding on current clear ship picture,/>Performing diffusion noise image obtained by adding noise to current-step diffusion for current clear ship image, wherein T represents the T-th step, T represents the preset total step number, and when T is 1,/>For the current clear ship picture,/>The current target pure noise picture generated for the current water mist ship picture corresponding to the current clear ship picture is represented by YJ, which represents Yeo-Johnson transformation, YJ (/ >) Representing the conversion of the current target pure noise picture into a noise map conforming to a gaussian normal distribution.
Optionally, the denoising Unet model is constructed according to the following method: encoding the original Unet model by adding a multi-head attention mechanism so as to encode each target transmissivity picture to obtain a high-latitude characteristic sequence; adding a cross attention mechanism of each stage to the original Unet model so as to enable the high-latitude feature sequence to cross attention with each stage of the original Unet model, and obtaining a feature map; wherein the original Unet model adds multi-head attention mechanism coding and each stage cross attention mechanism generates a de-noised Unet model.
Optionally, the current restored clear ship picture is calculated according to the following formula:
；
；
Wherein, as t decreases, The value also becomes smaller, T represents the preset total number of steps,/>For the previous denoising picture of the current water mist ship picture,/>For the current water mist ship picture, T is reduced from T until the T becomes 1, and when T is T, the process is carried out by adopting the method of carrying out the denoising of the current water mist ship pictureFor the current water mist ship picture, when t is 1,/>For restoring the current clear ship picture,/>Inputting a previous denoising picture and a corresponding previous target clear picture and a corresponding previous target transmissivity picture of a previous water mist ship picture into a target denoising Unet model to obtain a noise value,/>The current target pure noise picture generated for the current water mist ship picture, YJ represents Yeo-Johnson transformation, YJ (/ >) Representing the conversion of the current target pure noise picture into a noise map conforming to a gaussian normal distribution.
In another aspect, the present invention provides a marine defogging system based on an atmospheric scattering prior diffusion model, the system comprising: the generation unit is used for acquiring training sets, wherein the training sets are divided into a plurality of groups, and each group comprises a clear ship picture and a corresponding water mist ship picture; predicting all water mist ship pictures in the training set through a trained target Unet model to obtain all target clear pictures, all target transmissivity pictures and all target pure noise pictures; the diffusion unit is used for carrying out multi-step diffusion and noise adding on each clear ship picture in the training set to obtain a plurality of diffusion noise pictures corresponding to each clear ship picture, and storing a plurality of noise values loaded by each clear ship picture; the model training unit is used for randomly extracting one from all diffusion noise pictures in the current clear ship picture, inputting the current target clear picture and the current target transmissivity picture generated by the corresponding current water mist ship picture into the constructed denoising Unet model for training, and obtaining the current predicted noise value; calculating a current iteration loss value according to a noise value loaded by a diffusion noise picture which is extracted randomly and a current predicted noise value; updating the denoising Unet model according to the current iteration loss value; repeating the diffusion unit and the model training unit until all clear ship pictures in the training set are trained and a plurality of rounds of training are performed, and obtaining a target denoising Unet model; the denoising unit is used for inputting the current water mist ship picture, the corresponding current target clear picture and the corresponding current target transmissivity picture into the target denoising Unet model for reverse reasoning defogging to obtain the current restored clear ship picture; and repeating the denoising unit until all the water mist ship pictures are trained, and obtaining all the restored clear ship pictures.
Optionally, the target Unet model is trained by: the modification subunit is used for changing one output of the original Unet models into three outputs to obtain a modified Unet model first generation subunit, and is used for inputting a current water mist ship picture in the training set into the modified Unet model to obtain a current original clear picture, a current original transmissivity picture and a current original pure noise picture; the second generation subunit is used for inputting the current original clear picture, the current original transmissivity picture and the current original pure noise picture into an atmospheric scattering model to generate a current noise-containing ship picture; the first calculating subunit is used for calculating a first noise loss value according to the noise value of the current noise-containing ship picture and the noise value of the current water mist ship picture; the second calculating subunit is used for calculating a second noise loss value according to the noise value of the current original clear picture and the noise value of the current clear ship picture corresponding to the current water mist ship picture; an updating subunit, configured to calculate an iteration noise loss value according to the first noise loss value and the second noise loss value; updating the modified Unet model according to the iterative noise loss value; and repeating the training subunit, wherein the training subunit is used for repeating the first generating subunit, the second generating subunit, the first calculating subunit, the second calculating subunit and the updating subunit until all the water mist ship pictures in the training set are trained and a plurality of rounds of training are performed, so that the iterative noise loss value is fluctuated within a preset range, then the training is stopped, and the finally updated and obtained modified Unet model is used as the target Unet model.
Optionally, a plurality of diffusion noise pictures obtained by performing multi-step diffusion noise adding on the current clear ship picture are calculated according to the following formula:
；
Wherein, Diffusion noise picture obtained by performing last diffusion noise adding on current clear ship picture,/>Performing diffusion noise image obtained by adding noise to current-step diffusion for current clear ship image, wherein T represents the T-th step, T represents the preset total step number, and when T is 1,/>For the current clear ship picture,/>The current target pure noise picture generated for the current water mist ship picture corresponding to the current clear ship picture is represented by YJ, which represents Yeo-Johnson transformation, YJ (/ >) Representing the conversion of the current target pure noise picture into a noise map conforming to a gaussian normal distribution.
Optionally, the denoising Unet model is constructed according to the following method: encoding the original Unet model by adding a multi-head attention mechanism so as to encode each target transmissivity picture to obtain a high-latitude characteristic sequence; adding a cross attention mechanism of each stage to the original Unet model so as to enable the high-latitude feature sequence to cross attention with each stage of the original Unet model, and obtaining a feature map; wherein the original Unet model adds multi-head attention mechanism coding and each stage cross attention mechanism generates a de-noised Unet model.
Optionally, the current restored clear ship picture is calculated according to the following formula:
；
；
Wherein, as t decreases, The value also becomes smaller, T represents the preset total number of steps,/>For the previous denoising picture of the current water mist ship picture,/>For the current water mist ship picture, T is reduced from T until the T becomes 1, and when T is T, the process is carried out by adopting the method of carrying out the denoising of the current water mist ship pictureFor the current water mist ship picture, when t is 1,/>For restoring the current clear ship picture,/>Inputting a previous denoising picture and a corresponding previous target clear picture and a corresponding previous target transmissivity picture of a previous water mist ship picture into a target denoising Unet model to obtain a noise value,/>The current target pure noise picture generated for the current water mist ship picture, YJ represents Yeo-Johnson transformation, YJ (/ >) Representing the conversion of the current target pure noise picture into a noise map conforming to a gaussian normal distribution.
The invention has the beneficial effects that:
the invention provides a ship defogging method and a ship defogging system based on an atmospheric scattering priori diffusion model, wherein the method introduces a modified Unet model into the atmospheric scattering model and a clear ship picture to conduct supervision to obtain a target Unet model; the water mist ship picture is subjected to physical analysis through a target Unet model, and a physical analysis result is input into a target denoising Unet model to perform reverse reasoning defogging, so that the obtained restored clear ship picture detail texture is more specific; carrying out Gaussian normal distribution transformation on a target pure noise picture generated by an atmospheric scattering model to enable the target pure noise picture to conform to a denoising Unet model, so that original random sampling Gaussian noise is replaced, and the obtained restored clear ship picture is more real; when the ship is restored, the original water mist ship picture is replaced by the water mist ship picture, the target clear picture and the target transmissivity picture, so that the ship is more real during restoration.
Drawings
FIG. 1 is a flow chart of a ship defogging method based on an atmospheric scattering prior diffusion model provided by an embodiment of the invention;
FIG. 2 is a flowchart of a method for training a model of a target Unet provided by an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a ship defogging system based on an atmospheric scattering model according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a generating unit according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a flowchart of a ship defogging method based on an atmospheric scattering prior diffusion model according to an embodiment of the present invention, as shown in fig. 1, the method includes:
S1, acquiring a training set, wherein the training set is divided into a plurality of groups, and each group comprises a clear ship picture and a corresponding water mist ship picture; predicting all water mist ship pictures in the training set through a trained target Unet model to obtain all target clear pictures, all target transmissivity pictures and all target pure noise pictures;
Specifically, it is assumed that the training set is divided into 100 groups, and each group includes a clear ship picture and a corresponding water mist ship picture.
Wherein, the target Unet model is trained by the following method, fig. 2 is a flowchart of a training method of the target Unet model provided by the embodiment of the invention, as shown in fig. 2:
S11, changing one output of an original Unet model into three outputs to obtain a modified Unet model;
specifically, inputting one picture into the modified Unet model to obtain three pictures; the three outputs correspond to: clear picture, transmittance picture, and pure noise picture.
S12, inputting a current water mist ship picture in the training set into the modified Unet model to obtain a current original clear picture, a current original transmissivity picture and a current original pure noise picture;
specifically, taking one water mist ship picture and one clear ship picture in one group as examples, inputting the current water mist ship picture in the group into the modified Unet model to obtain the current original clear picture, the current original transmissivity picture and the current original pure noise picture.
S13, inputting the current original clear picture, the current original transmissivity picture and the current original pure noise picture into an atmospheric scattering model to generate a current noise-containing ship picture;
Specifically, the formula is as follows:
；
Wherein, For the current noisy ship picture,/>For the current original clear picture,/>For the current original transmittance picture,/>Is the current original pure noise picture.
S14, calculating a first noise loss value according to the noise value of the current noise-containing ship picture and the noise value of the current water mist ship picture;
Specifically, the current noise-containing ship picture and the current water mist ship picture are scaled to the same size, and then a first noise loss value is calculated, wherein the formula is as follows:
；
Wherein, For the first noise loss value,/>Is the noise value of the current water mist ship picture,/>For the noise value of the current noise-containing ship picture,/>The number of pixels for the current water mist ship picture or the current noise ship picture is equal to the width x the height.
S15, calculating a second noise loss value according to the noise value of the current original clear picture and the noise value of the current clear ship picture corresponding to the current water spray ship picture;
specifically, the current clear ship picture in the selected group and the current original clear picture are scaled to the same size, and then a second noise loss value is calculated, wherein the formula is as follows:
；
Wherein, Is the second noise loss value,/>As the noise value of the current clear ship picture,For the noise value of the current original clear picture,/>The number of pixels, which is the current clear ship picture or the current original clear picture, is equal to the width x height.
S16, calculating to obtain an iterative noise loss value according to the first noise loss value and the second noise loss value; updating the modified Unet model according to the iterative noise loss value;
Specifically, the iterative noise loss value, namely the loss value of the clear ship picture and the water mist ship picture in the selected group is calculated according to the following formula:
；
Wherein, For iterative noise loss value,/>For the first noise loss value,/>Is the second noise loss value,/>、/>Is a super-ginseng, and is set manually.
And reversely updating the modified Unet model according to the iterative noise loss value.
And S17, repeating the steps S12-S16 until all the water mist ship pictures in the training set are trained and a plurality of training rounds are performed, so that the training is stopped after the iterative noise loss value fluctuates within a preset range, and the finally updated and obtained modified Unet model is used as the target Unet model.
Specifically, repeating S12-S16 until 100 groups of pictures in the training set are trained, and performing one round of training; updating the revised Unet model once for each training group; repeating S12-S16 to perform multiple training until the iterative noise loss value is within a preset range0.1%) After fluctuation, training is stopped, and the last updated modified Unet model is taken as the target Unet model.
And predicting all water mist ship pictures in the training set through the target Unet model to respectively and correspondingly obtain all target clear pictures, all target transmissivity pictures and all target pure noise pictures.
S2, carrying out multi-step diffusion noise adding on each clear ship picture in the training set to obtain a plurality of diffusion noise pictures corresponding to each clear ship picture, and storing a plurality of noise values loaded by each clear ship picture;
specifically, taking one group of training sets as an example: converting a current target pure noise picture generated by a current water mist ship picture corresponding to a current clear ship picture through a target Unet model, and converting the current target pure noise picture into a noise picture conforming to Gaussian normal distribution;
decomposing the converted noise map into T steps so as to enable the current clear ship picture to carry out T steps of Gaussian noise and obtain T diffusion noise maps; the specific formula is as follows:
；
Wherein, Diffusion noise picture obtained by performing last diffusion noise adding on current clear ship picture,/>Performing diffusion noise image obtained by adding noise to current-step diffusion for current clear ship image, wherein T represents the T-th step, T represents the preset total step number, and when T is 1,/>For the current clear ship picture,/>The current target pure noise picture generated for the current water mist ship picture corresponding to the current clear ship picture is represented by YJ, which represents Yeo-Johnson transformation, YJ (/ >) Representing the conversion of the current target pure noise picture into a noise map conforming to a gaussian normal distribution.
Firstly, diffusing Gaussian noise in a first step to obtain a first diffused noise picture, and secondly, diffusing Gaussian noise in a second step to obtain a second diffused noise picture; carrying out third-step diffusion and Gaussian noise adding on the second diffusion noise picture to obtain a third diffusion noise picture; after the step T is performed (in the present invention, T is set to 2000, and it should be noted that the setting of T is not limited in the present invention), T diffuse noise pictures are obtained. The noise value loaded by each diffusion noise picture needs to be recorded and stored.
S3, randomly extracting one of all diffusion noise pictures in the current clear ship picture, inputting the current target clear picture and the current target transmissivity picture generated by the corresponding current water mist ship picture into a constructed denoising Unet model for training, and obtaining a current prediction noise value; calculating a current iteration loss value according to a noise value loaded by a diffusion noise picture which is extracted randomly and a current predicted noise value; updating the denoising Unet model according to the current iteration loss value; repeating the steps S2-S3 until all the clear ship pictures in the training set are trained and a plurality of rounds of training are carried out, and obtaining a target denoising Unet model;
in the prior art, one water mist ship picture is randomly extracted from T diffusion noise pictures, the current water mist ship picture in the extracted group in the S2 is input into an original Unet model for training, the target denoising Unet model trained by the method is inaccurate in denoising the water mist ship picture, and the obtained restored clear ship picture is not clear enough.
In the invention, one of T diffusion noise pictures is randomly extracted, a current target clear picture and a current target transmissivity picture which are generated by the current water mist ship picture in the extracted group in the S2 through a target Unet model are input into a constructed denoising Unet model for training, and a current predicted noise value is obtained;
the denoising Unet model is constructed according to the following method:
Encoding the original Unet model by adding a multi-head attention mechanism so as to encode each target transmissivity picture to obtain a high-latitude characteristic sequence;
Adding a cross attention mechanism of each stage to the original Unet model so as to enable the high-latitude feature sequence to cross attention with each stage of the original Unet model, and obtaining a feature map;
Wherein the original Unet model adds multi-head attention mechanism coding and each stage cross attention mechanism generates a de-noised Unet model.
During specific training, randomly extracting one diffusion noise picture from T diffusion noise pictures obtained by carrying out T-step diffusion and Gaussian noise on the current stage of clear ship pictures in the extracted group in the S2, scaling the current target clear pictures generated by the current water mist ship pictures in the extracted group in the S2 through a target Unet model to the same size, and superposing the current target clear pictures according to a channel; the superimposed pictures are input into a denoising Unet model, and output of a plurality of stages is obtained;
At this time, the current target transmissivity picture generated by the current water mist ship picture in the extracted group in the S2 through the target Unet model is input into the denoising Unet model for coding, and a high latitude characteristic sequence is obtained; the obtained high latitude characteristic sequence and the outputs of the stages are subjected to cross attention to obtain a characteristic diagram; the characteristic diagram is more in line with the interference law of the physical information of the water mist, so that the obtained restored clear ship picture has higher detail restoration.
The feature map is processed through an output layer of a denoising Unet model to obtain a current predicted noise picture, and then a current predicted noise value is obtained;
Scaling the current predicted noise picture and the randomly extracted diffusion noise picture to the same size, and calculating to obtain a current iteration loss value according to the current predicted noise value and the noise value correspondingly loaded by the randomly extracted diffusion noise picture stored in the step S2; the formula is as follows:
；
Wherein, For the current iteration loss value,/>Corresponding to the noise value of the random extracted diffusion noise picture,/>For the current predicted noise value,/>The number of pixels for the current predicted noise picture or a randomly decimated diffuse noise picture is equal to the width x height.
And updating the denoising Unet model according to the current iteration loss value.
Repeating the steps S2-S3 until 100 groups of pictures in the training set are trained, and performing one round of training; updating the denoising Unet model once every training of a group; repeating S2-S3 to perform multiple training until the current iteration loss value is within a preset rangeAnd stopping training after 0.1 percent of fluctuation, and taking the de-noising Unet model obtained by final updating as the target de-noising Unet model.
S4, inputting the current water mist ship picture, the corresponding current target clear picture and the corresponding current target transmissivity picture into the target denoising Unet model for reverse reasoning defogging to obtain the current recovered clear ship picture; and S4, repeating until all the water mist ship pictures are trained, and obtaining all the recovered clear ship pictures.
Specifically, the current restored clear ship picture is calculated according to the following formula:
；
；
Wherein, as t decreases, The value also becomes smaller, T represents a preset total number of steps (which is the same as T in S2),For the previous denoising picture of the current water mist ship picture,/>The current denoising picture is carried out for the current water mist ship picture, T is reduced from T until the current denoising picture becomes 1, when T is T,For the current water mist ship picture, when t is 1,/>For restoring the current clear ship picture,/>The previous denoising picture of the current water spray ship picture, the corresponding current target clear picture and the corresponding current target transmissivity picture are input into the noise value obtained by the target denoising Unet model,The current target pure noise picture generated for the current water mist ship picture, YJ represents Yeo-Johnson transformation, YJ (/ >) Representing the conversion of the current target pure noise picture into a noise map conforming to a gaussian normal distribution.
During specific training, scaling the current water mist ship picture and the corresponding current target clear picture to the same size, inputting the current water mist ship picture and the corresponding current target clear picture into the target denoising Unet model, and restraining the current target transmissivity picture to obtain a first noise value, namelyThen subtracting the first noise value output by the target denoising Unet model from the noise value of the current water mist ship picture to obtain a first denoising picture;
in the second denoising process, scaling the first denoising picture and the corresponding current target clear picture to the same size, inputting the first denoising picture and the corresponding current target clear picture into a target denoising Unet model, restraining the first denoising picture through the current target transmissivity picture to obtain a second noise value, and subtracting the second noise value output in the target denoising Unet model from the noise value of the first denoising picture to obtain a second denoising picture;
The method comprises the steps of carrying out first denoising on a current water mist ship picture to obtain a first denoising picture, and carrying out second denoising on the first denoising picture to obtain a second denoising picture; carrying out third denoising on the second denoising picture to obtain a third denoising picture; obtaining a current restored clear ship picture after denoising for T times; and denoising all the water mist ship pictures in the training set by a similar method to obtain all the recovered clear ship pictures.
Fig. 3 is a schematic structural diagram of a ship defogging system based on an atmospheric scattering model according to an embodiment of the present invention, and as shown in fig. 3, the system includes:
The generation unit is used for acquiring training sets, wherein the training sets are divided into a plurality of groups, and each group comprises a clear ship picture and a corresponding water mist ship picture; predicting all water mist ship pictures in the training set through a trained target Unet model to obtain all target clear pictures, all target transmissivity pictures and all target pure noise pictures;
the object Unet model is trained by the following method, and fig. 4 is a schematic structural diagram of a generating unit provided by an embodiment of the present invention, as shown in fig. 4, where the generating unit includes:
a modification subunit, configured to change one output of the original Unet models to three outputs, to obtain a modified Unet model;
The first generation subunit is used for inputting the current water mist ship picture in the training set into the modified Unet model to obtain a current original clear picture, a current original transmissivity picture and a current original pure noise picture;
The second generation subunit is used for inputting the current original clear picture, the current original transmissivity picture and the current original pure noise picture into an atmospheric scattering model to generate a current noise-containing ship picture;
the first calculating subunit is used for calculating a first noise loss value according to the noise value of the current noise-containing ship picture and the noise value of the current water mist ship picture;
The second calculating subunit is used for calculating a second noise loss value according to the noise value of the current original clear picture and the noise value of the current clear ship picture corresponding to the current water mist ship picture;
an updating subunit, configured to calculate an iteration noise loss value according to the first noise loss value and the second noise loss value; updating the modified Unet model according to the iterative noise loss value;
And repeating the training subunit, wherein the training subunit is used for repeating the first generating subunit, the second generating subunit, the first calculating subunit, the second calculating subunit and the updating subunit until all the water mist ship pictures in the training set are trained and a plurality of rounds of training are performed, so that the iterative noise loss value is fluctuated within a preset range, then the training is stopped, and the finally updated and obtained modified Unet model is used as the target Unet model.
The diffusion unit is used for carrying out multi-step diffusion and noise adding on each clear ship picture in the training set to obtain a plurality of diffusion noise pictures corresponding to each clear ship picture, and storing a plurality of noise values loaded by each clear ship picture;
specifically, a plurality of diffusion noise pictures obtained by performing multi-step diffusion noise adding on a current clear ship picture are calculated according to the following formula:
；
Wherein, Diffusion noise picture obtained by performing last diffusion noise adding on current clear ship picture,/>Performing diffusion noise image obtained by adding noise to current-step diffusion for current clear ship image, wherein T represents the T-th step, T represents the preset total step number, and when T is 1,/>For the current clear ship picture,/>The current target pure noise picture generated for the current water mist ship picture corresponding to the current clear ship picture is represented by YJ, which represents Yeo-Johnson transformation, YJ (/ >) Representing the conversion of the current target pure noise picture into a noise map conforming to a gaussian normal distribution.
The model training unit is used for randomly extracting one from all diffusion noise pictures in the current clear ship picture, inputting the current target clear picture and the current target transmissivity picture generated by the corresponding current water mist ship picture into the constructed denoising Unet model for training, and obtaining the current predicted noise value; calculating a current iteration loss value according to a noise value loaded by a diffusion noise picture which is extracted randomly and a current predicted noise value; updating the denoising Unet model according to the current iteration loss value; repeating the diffusion unit and the model training unit until all clear ship pictures in the training set are trained and a plurality of rounds of training are performed, and obtaining a target denoising Unet model;
specifically, the denoising Unet model is constructed according to the following method:
Encoding the original Unet model by adding a multi-head attention mechanism so as to encode each target transmissivity picture to obtain a high-latitude characteristic sequence;
Adding a cross attention mechanism of each stage to the original Unet model so as to enable the high-latitude feature sequence to cross attention with each stage of the original Unet model, and obtaining a feature map;
Wherein the original Unet model adds multi-head attention mechanism coding and each stage cross attention mechanism generates a de-noised Unet model.
The denoising unit is used for inputting the current water mist ship picture, the corresponding current target clear picture and the corresponding current target transmissivity picture into the target denoising Unet model for reverse reasoning defogging to obtain the current restored clear ship picture; and repeating the denoising unit until all the water mist ship pictures are trained, and obtaining all the restored clear ship pictures.
Specifically, the current restored clear ship picture is calculated according to the following formula:
；/>
；
Wherein, as t decreases, The value also becomes smaller, T represents the preset total number of steps,/>For the previous denoising picture of the current water mist ship picture,/>For the current water mist ship picture, T is reduced from T until the T becomes 1, and when T is T, the process is carried out by adopting the method of carrying out the denoising of the current water mist ship pictureFor the current water mist ship picture, when t is 1,/>For restoring the current clear ship picture,/>Inputting a previous denoising picture and a corresponding previous target clear picture and a corresponding previous target transmissivity picture of a previous water mist ship picture into a target denoising Unet model to obtain a noise value,/>The current target pure noise picture generated for the current water mist ship picture, YJ represents Yeo-Johnson transformation, YJ (/ >) Representing the conversion of the current target pure noise picture into a noise map conforming to a gaussian normal distribution.
All the restored clear ship pictures obtained by the method are more specific and more real.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. The ship defogging method based on the atmospheric scattering prior diffusion model is characterized by comprising the following steps of:
S1, acquiring a training set, wherein the training set is divided into a plurality of groups, and each group comprises a clear ship picture and a corresponding water mist ship picture; predicting all water mist ship pictures in the training set through a trained target Unet model to obtain all target clear pictures, all target transmissivity pictures and all target pure noise pictures;
s2, carrying out multi-step diffusion noise adding on each clear ship picture in the training set to obtain a plurality of diffusion noise pictures corresponding to each clear ship picture, and storing a plurality of noise values loaded by each clear ship picture;
S3, randomly extracting one of all diffusion noise pictures in the current clear ship picture, inputting the current target clear picture and the current target transmissivity picture generated by the corresponding current water mist ship picture into a constructed denoising Unet model for training, and obtaining a current prediction noise value; calculating a current iteration loss value according to a noise value loaded by a diffusion noise picture which is extracted randomly and a current predicted noise value; updating the denoising Unet model according to the current iteration loss value; repeating the steps S2-S3 until all the clear ship pictures in the training set are trained and a plurality of rounds of training are carried out, and obtaining a target denoising Unet model;
S4, inputting the current water mist ship picture, the corresponding current target clear picture and the corresponding current target transmissivity picture into the target denoising Unet model for reverse reasoning defogging to obtain the current recovered clear ship picture; repeating the step S4 until all the water mist ship pictures are trained, and obtaining all the recovered clear ship pictures;
the target Unet model is trained by the following method:
S11, changing one output of an original Unet model into three outputs to obtain a modified Unet model;
s12, inputting a current water mist ship picture in the training set into the modified Unet model to obtain a current original clear picture, a current original transmissivity picture and a current original pure noise picture;
S13, inputting the current original clear picture, the current original transmissivity picture and the current original pure noise picture into an atmospheric scattering model to generate a current noise-containing ship picture;
s14, calculating a first noise loss value according to the noise value of the current noise-containing ship picture and the noise value of the current water mist ship picture;
S15, calculating a second noise loss value according to the noise value of the current original clear picture and the noise value of the current clear ship picture corresponding to the current water spray ship picture;
s16, calculating to obtain an iterative noise loss value according to the first noise loss value and the second noise loss value; updating the modified Unet model according to the iterative noise loss value;
And S17, repeating the steps S12-S16 until all the water mist ship pictures in the training set are trained and a plurality of training rounds are performed, so that the training is stopped after the iterative noise loss value fluctuates within a preset range, and the finally updated and obtained modified Unet model is used as the target Unet model.
2. The method according to claim 1, characterized in that:
The multiple diffusion noise pictures obtained by carrying out multi-step diffusion noise adding on the current clear ship picture are calculated according to the following formula:
the img_voice t-1 is a diffusion Noise picture obtained after the previous diffusion Noise adding is carried out on the current clear ship picture, the img_voice t is a diffusion Noise picture obtained after the previous diffusion Noise adding is carried out on the current clear ship picture, T represents the T step, T represents the preset total step number, when T is 1, the img_voice t-1 is the current clear ship picture, the noise_img is the current target pure Noise picture generated by the current water mist ship picture corresponding to the current clear ship picture, YJ represents Yeo-Johnson transformation, and YJ (noise_img) represents the Noise picture which is converted from the current target pure Noise picture to be in accordance with Gaussian normal distribution.
3. The method of claim 1, wherein the denoising Unet model is constructed according to the following method:
Encoding the original Unet model by adding a multi-head attention mechanism so as to encode each target transmissivity picture to obtain a high-latitude characteristic sequence;
Adding a cross attention mechanism of each stage to the original Unet model so as to enable the high-latitude feature sequence to cross attention with each stage of the original Unet model, and obtaining a feature map;
Wherein the original Unet model adds multi-head attention mechanism coding and each stage cross attention mechanism generates a de-noised Unet model.
4. The method of claim 1, wherein the current restored clear ship picture is calculated according to the following formula:
The method comprises the steps that as T is reduced, the alpha t value also becomes smaller, T represents the preset total step number, img_noise t is a picture after the previous denoising of a current water mist ship picture, img_noise t-1 is a picture after the previous denoising of the current water mist ship picture, T is reduced from T until T is changed to 1, img_noise t is the current water mist ship picture when T is T, img_noise t-1 is the current restoration clear ship picture when T is 1, noise_pre t is a Noise value obtained by inputting a picture after the previous denoising of the current water mist ship picture and a corresponding current target clear picture and a current target transmittance picture into a target denoising Unet model, noise_img is a current target pure Noise picture generated by the current water mist ship picture, YJ represents Yeo-Johnson transformation, and YJ (noise_img) represents that the current target Noise is converted into a Noise according with a Gaussian distribution.
5. A marine defogging system based on an atmospheric scattering prior diffusion model, comprising:
The generation unit is used for acquiring training sets, wherein the training sets are divided into a plurality of groups, and each group comprises a clear ship picture and a corresponding water mist ship picture; predicting all water mist ship pictures in the training set through a trained target Unet model to obtain all target clear pictures, all target transmissivity pictures and all target pure noise pictures;
the diffusion unit is used for carrying out multi-step diffusion and noise adding on each clear ship picture in the training set to obtain a plurality of diffusion noise pictures corresponding to each clear ship picture, and storing a plurality of noise values loaded by each clear ship picture;
The model training unit is used for randomly extracting one from all diffusion noise pictures in the current clear ship picture, inputting the current target clear picture and the current target transmissivity picture generated by the corresponding current water mist ship picture into the constructed denoising Unet model for training, and obtaining the current predicted noise value; calculating a current iteration loss value according to a noise value loaded by a diffusion noise picture which is extracted randomly and a current predicted noise value; updating the denoising Unet model according to the current iteration loss value; repeating the diffusion unit and the model training unit until all clear ship pictures in the training set are trained and a plurality of rounds of training are performed, and obtaining a target denoising Unet model;
The denoising unit is used for inputting the current water mist ship picture, the corresponding current target clear picture and the corresponding current target transmissivity picture into the target denoising Unet model for reverse reasoning defogging to obtain the current restored clear ship picture; repeating the denoising unit until all the water mist ship pictures are trained, and obtaining all the restored clear ship pictures;
the target Unet model is trained by the following method:
a modification subunit, configured to change one output of the original Unet models to three outputs, to obtain a modified Unet model;
The first generation subunit is used for inputting the current water mist ship picture in the training set into the modified Unet model to obtain a current original clear picture, a current original transmissivity picture and a current original pure noise picture;
The second generation subunit is used for inputting the current original clear picture, the current original transmissivity picture and the current original pure noise picture into an atmospheric scattering model to generate a current noise-containing ship picture;
the first calculating subunit is used for calculating a first noise loss value according to the noise value of the current noise-containing ship picture and the noise value of the current water mist ship picture;
The second calculating subunit is used for calculating a second noise loss value according to the noise value of the current original clear picture and the noise value of the current clear ship picture corresponding to the current water mist ship picture;
an updating subunit, configured to calculate an iteration noise loss value according to the first noise loss value and the second noise loss value; updating the modified Unet model according to the iterative noise loss value;
And repeating the training subunit, wherein the training subunit is used for repeating the first generating subunit, the second generating subunit, the first calculating subunit, the second calculating subunit and the updating subunit until all the water mist ship pictures in the training set are trained and a plurality of rounds of training are performed, so that the iterative noise loss value is fluctuated within a preset range, then the training is stopped, and the finally updated and obtained modified Unet model is used as the target Unet model.
6. The system according to claim 5, wherein:
The multiple diffusion noise pictures obtained by carrying out multi-step diffusion noise adding on the current clear ship picture are calculated according to the following formula:
the img_voice t-1 is a diffusion Noise picture obtained after the previous diffusion Noise adding is carried out on the current clear ship picture, the img_voice t is a diffusion Noise picture obtained after the previous diffusion Noise adding is carried out on the current clear ship picture, T represents the T step, T represents the preset total step number, when T is 1, the img_voice t-1 is the current clear ship picture, the noise_img is the current target pure Noise picture generated by the current water mist ship picture corresponding to the current clear ship picture, YJ represents Yeo-Johnson transformation, and YJ (noise_img) represents the Noise picture which is converted from the current target pure Noise picture to be in accordance with Gaussian normal distribution.
7. The system of claim 5, wherein the denoising Unet model is constructed according to the following method:
Encoding the original Unet model by adding a multi-head attention mechanism so as to encode each target transmissivity picture to obtain a high-latitude characteristic sequence;
Adding a cross attention mechanism of each stage to the original Unet model so as to enable the high-latitude feature sequence to cross attention with each stage of the original Unet model, and obtaining a feature map;
Wherein the original Unet model adds multi-head attention mechanism coding and each stage cross attention mechanism generates a de-noised Unet model.
8. The system of claim 5, wherein the current restored clear ship picture is calculated according to the following formula:
The method comprises the steps that as T is reduced, the alpha t value also becomes smaller, T represents the preset total step number, img_noise t is a picture after the previous denoising of a current water mist ship picture, img_noise t-1 is a picture after the previous denoising of the current water mist ship picture, T is reduced from T until T is changed to 1, img_noise t is the current water mist ship picture when T is T, img_noise t-1 is the current restoration clear ship picture when T is 1, noise_pre t is a Noise value obtained by inputting a picture after the previous denoising of the current water mist ship picture and a corresponding current target clear picture and a current target transmittance picture into a target denoising Unet model, noise_img is a current target pure Noise picture generated by the current water mist ship picture, YJ represents Yeo-Johnson transformation, and YJ (noise_img) represents that the current target Noise is converted into a Noise according with a Gaussian distribution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410217435.5A CN117789041B (en) | 2024-02-28 | 2024-02-28 | Ship defogging method and system based on atmospheric scattering priori diffusion model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410217435.5A CN117789041B (en) | 2024-02-28 | 2024-02-28 | Ship defogging method and system based on atmospheric scattering priori diffusion model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117789041A CN117789041A (en) | 2024-03-29 |
| CN117789041B true CN117789041B (en) | 2024-05-10 |
Family
ID=90385286
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410217435.5A Active CN117789041B (en) | 2024-02-28 | 2024-02-28 | Ship defogging method and system based on atmospheric scattering priori diffusion model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117789041B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107203977A (en) * | 2017-05-17 | 2017-09-26 | 河海大学 | A kind of underwater image restoration method based on dark primary priori and rarefaction representation |
| CN113052776A (en) * | 2021-04-09 | 2021-06-29 | 北京邮电大学 | Unsupervised image defogging method based on multi-scale depth image prior |
| CN115330777A (en) * | 2022-10-13 | 2022-11-11 | 浙江华是科技股份有限公司 | Ship detection method and system for training picture scaling size |
| CN115601657A (en) * | 2022-10-17 | 2023-01-13 | 北京理工大学(Cn) | Method for detecting and identifying ship target in severe weather |
| CN115900582A (en) * | 2022-11-11 | 2023-04-04 | 浙江大学 | Meter-level planar element surface shape detection device and method for eliminating intermediate-frequency coherent noise |
| CN116052094A (en) * | 2023-03-07 | 2023-05-02 | 浙江华是科技股份有限公司 | Ship detection method, system and computer storage medium |
| WO2023116631A1 (en) * | 2021-12-24 | 2023-06-29 | 中国科学院深圳先进技术研究院 | Training method and training apparatus for rotating-ship target detection model, and storage medium |
| CN116883883A (en) * | 2023-03-06 | 2023-10-13 | 华南理工大学 | Marine ship target detection method based on generation of anti-shake of countermeasure network |
| WO2023230412A1 (en) * | 2022-05-24 | 2023-11-30 | The Regents Of The University Of California | Class-specific diffractive camera with all-optical erasure of undesired objects |
| CN117152587A (en) * | 2023-10-27 | 2023-12-01 | 浙江华是科技股份有限公司 | Anti-learning-based semi-supervised ship detection method and system |
| CN117522903A (en) * | 2023-09-21 | 2024-02-06 | 上海电力大学 | SF-Unet model-based high-resolution cultivated land remote sensing image segmentation method |
| CN117557477A (en) * | 2024-01-09 | 2024-02-13 | 浙江华是科技股份有限公司 | Defogging recovery method and system for ship |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4594114B2 (en) * | 2005-01-19 | 2010-12-08 | キヤノン株式会社 | Image processing apparatus and refractive index distribution measuring apparatus |
| US20230103638A1 (en) * | 2021-10-06 | 2023-04-06 | Google Llc | Image-to-Image Mapping by Iterative De-Noising |
-
2024
- 2024-02-28 CN CN202410217435.5A patent/CN117789041B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107203977A (en) * | 2017-05-17 | 2017-09-26 | 河海大学 | A kind of underwater image restoration method based on dark primary priori and rarefaction representation |
| CN113052776A (en) * | 2021-04-09 | 2021-06-29 | 北京邮电大学 | Unsupervised image defogging method based on multi-scale depth image prior |
| WO2023116631A1 (en) * | 2021-12-24 | 2023-06-29 | 中国科学院深圳先进技术研究院 | Training method and training apparatus for rotating-ship target detection model, and storage medium |
| WO2023230412A1 (en) * | 2022-05-24 | 2023-11-30 | The Regents Of The University Of California | Class-specific diffractive camera with all-optical erasure of undesired objects |
| CN115330777A (en) * | 2022-10-13 | 2022-11-11 | 浙江华是科技股份有限公司 | Ship detection method and system for training picture scaling size |
| CN115601657A (en) * | 2022-10-17 | 2023-01-13 | 北京理工大学(Cn) | Method for detecting and identifying ship target in severe weather |
| CN115900582A (en) * | 2022-11-11 | 2023-04-04 | 浙江大学 | Meter-level planar element surface shape detection device and method for eliminating intermediate-frequency coherent noise |
| CN116883883A (en) * | 2023-03-06 | 2023-10-13 | 华南理工大学 | Marine ship target detection method based on generation of anti-shake of countermeasure network |
| CN116052094A (en) * | 2023-03-07 | 2023-05-02 | 浙江华是科技股份有限公司 | Ship detection method, system and computer storage medium |
| CN117522903A (en) * | 2023-09-21 | 2024-02-06 | 上海电力大学 | SF-Unet model-based high-resolution cultivated land remote sensing image segmentation method |
| CN117152587A (en) * | 2023-10-27 | 2023-12-01 | 浙江华是科技股份有限公司 | Anti-learning-based semi-supervised ship detection method and system |
| CN117557477A (en) * | 2024-01-09 | 2024-02-13 | 浙江华是科技股份有限公司 | Defogging recovery method and system for ship |
Non-Patent Citations (3)
| Title |
|---|
| 基于YOLOv3深度学习的海雾气象条件下海上船只实时检测;王飞;刘梦婷;刘雪芹;秦志亮;马本俊;郑毅;;海洋科学;20200815(08);199-206 * |
| 基于暗原色先验算法的雨雾天图片清晰化;徐春雨;司宇航;;科技广场;20170530(05);24-28 * |
| 基于空洞U-Net神经网络的PET图像重建算法;杜倩倩;强彦;李硕;杨晓棠;;太原理工大学学报;20200315(02);37-44 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117789041A (en) | 2024-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113658051B (en) | Image defogging method and system based on cyclic generation countermeasure network | |
| CN109410239B (en) | Text image super-resolution reconstruction method based on condition generation countermeasure network | |
| CN111833277B (en) | Marine image defogging method with unpaired multi-scale mixed coding and decoding structure | |
| CN112365422B (en) | Irregular missing image restoration method and system based on deep aggregation network | |
| CN112508125A (en) | Efficient full-integer quantization method of image detection model | |
| CN110189260B (en) | Image noise reduction method based on multi-scale parallel gated neural network | |
| CN111861886B (en) | Image super-resolution reconstruction method based on multi-scale feedback network | |
| CN116682120A (en) | Multilingual mosaic image text recognition method based on deep learning | |
| CN116051396B (en) | Image denoising method based on feature enhancement network and GRU network | |
| CN112767279A (en) | Underwater image enhancement method for generating countermeasure network based on discrete wavelet integration | |
| CN111768326A (en) | High-capacity data protection method based on GAN amplification image foreground object | |
| CN111539896B (en) | Domain-adaptive-based image defogging method and system | |
| CN111898482A (en) | Face prediction method based on progressive generation confrontation network | |
| CN114004766A (en) | Underwater image enhancement method, system and equipment | |
| CN113283577A (en) | Industrial parallel data generation method based on meta-learning and generation countermeasure network | |
| CN116777764A (en) | Diffusion model-based cloud and mist removing method and system for optical remote sensing image | |
| CN115018725A (en) | Diffraction image restoration method based on block minimum pixel prior | |
| CN117789041B (en) | Ship defogging method and system based on atmospheric scattering priori diffusion model | |
| CN114202473A (en) | Image restoration method and device based on multi-scale features and attention mechanism | |
| CN117333398A (en) | Multi-scale image denoising method and device based on self-supervision | |
| CN111612715B (en) | Image restoration method and device and electronic equipment | |
| CN116645287B (en) | Diffusion model-based image deblurring method | |
| CN117333881A (en) | Oracle auxiliary decoding method based on conditional diffusion model | |
| CN115272131B (en) | Image mole pattern removing system and method based on self-adaptive multispectral coding | |
| CN114663307B (en) | Integrated image denoising system based on uncertainty network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |