GB2255699A - Image restoration using a neural network - Google Patents
Image restoration using a neural network Download PDFInfo
- Publication number
- GB2255699A GB2255699A GB9110119A GB9110119A GB2255699A GB 2255699 A GB2255699 A GB 2255699A GB 9110119 A GB9110119 A GB 9110119A GB 9110119 A GB9110119 A GB 9110119A GB 2255699 A GB2255699 A GB 2255699A
- Authority
- GB
- United Kingdom
- Prior art keywords
- image
- estimate
- original image
- processing
- storage means
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 6
- 238000000034 method Methods 0.000 claims abstract description 41
- 238000006731 degradation reaction Methods 0.000 claims description 15
- 230000015556 catabolic process Effects 0.000 claims description 5
- 230000000593 degrading effect Effects 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 230000003278 mimic effect Effects 0.000 claims description 3
- 239000002356 single layer Substances 0.000 claims description 3
- 230000008447 perception Effects 0.000 claims 1
- 239000010410 layer Substances 0.000 description 13
- 230000000694 effects Effects 0.000 description 4
- 238000012937 correction Methods 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Facsimile Image Signal Circuits (AREA)
Abstract
The method of image restoration comprises processing 5 an estimate of an original image so as to obtain an estimate of a received degraded version of the original image; comparing the estimate of the received degraded image with an actual received degraded version of the original image to obtain an indication of the difference between the actual received degraded image and the estimate of the received degraded image and utilising said difference to improve said estimate of the original image. Normally the steps of the method are executed a number of times using for the first step of each further execution the improved estimate of the original image obtained in the preceding execution. Processing means 5 may comprise a neural network. In such a method in the final step of at least one of the executions, e.g. in alternate executions, instead of utilising the difference to improve the estimate of the original image, the difference is utilised to modify the processing of the first step in such a manner as to improve the estimate of the received degraded image obtained in the first step of the next execution. <IMAGE>
Description
Methods and Apparatus for Image Restoration
This invention relates to methods and apparatus for image restoration.
More particularly the invention relates to methods and apparatus for image restoration of the kind wherein a received degraded version of an original image is processed so as to obtain an image which resembles the original image more closely than the received degraded image. In such methods and apparatus the received degraded image is typically represented by an electrical digital signal containing information about each of the pixels of a received degraded image and the processing is performed electrically, e.g. in a digital computer.
Known such methods and apparatus require a knowledge or estimation of the degradation process to which the received degraded image has been subjected and attempt to obtain the original image from the received degraded image by applying to the received image the reverse of the degradation process. In practice, since the degradation process is accompanied by further degradation due to noise, the restoration process is not entirely satisfactory.
Moreover a knowledge orestimation of the degradation process is not always easily obtainable.
It is an object of the present invention to provide methods and apparatus for image restoration whereby these problems may be alleviated.
According to the invention there is provided a method for image restoration comprising the steps of: processing an estimate of an original image so as to obtain an estimate of a received degraded version of the original image; comparing the estimate of the received degraded image with an actual received degraded version of the original image to obtain an indication of the difference between the actual received degraded image and the estimate of the received degraded image and utilising said difference to improve said estimate of the original image.
Preferably the method according to the invention further comprises further executing said steps a number of times using for the first step of each said further execution the improved estimate of the original image obtained in the preceding execution.
In such a method in the final step of at least one of the executions, e.g. in alternate executions, instead of utilising said difference to improve said estimate of the original image, said difference is utilised to modify said processing of the first step in such a manner as to improve the estimate of the received degraded image obtained in the first step of the next execution.
The invention also provides an apparatus for image restoration comprising: first stoage means for storing an estimate of an original image; second storage means for storing a received degraded version of the original image; processing means for processing the image stored in the first storage means so as to mimic the degradation of the received image; comparator means for comparing the image produced by the processing means from the image stored in said first storage means with the image stored in said second storage means; and means for utilising the output of said comparator to improve the estimate of the original image in said first storage means.
Preferably said apparatus also includes means for utilising the output of said comparator to improve the mimicking process effected by said processing means.
Said means for utilising the output of said comparator to improve said estimate of the original image suitably comprises means for causing said processing means to process the output signals of the comparator in reverse mode and for utilising the resultant signals to modify the signals in said first storage means.
Said processing means suitably comprises a neural network processing means. The neural network processing means is suitably a single layer perceptron.
One method and apparatus in accordance with the invention will now be described by way of example with reference to the accompanying drawing which illustrates the apparatus diagrammatically.
The apparatus includes a first storage means 1 which is arranged to store an image as digital electrical signals respectively indicative of the intensities of respective pixels of the image.
The storage means 1 is connected to a first input port 3 of a digital processing means 5 which is arranged to process digital image signals fed to it from storage means 1 in such a manner as to effect a degradation of the image represented by the signals. The processing means 5 is a neural network processing means of a kind known as a single layer perceptron. Thus the processing means 5 has a first layer 7 having a separate location for each pixel of an input image applied to input port 3 and a second layer 9 having a separate location for each pixel of an output image, each first layer location being separately connected to each second layer location via a separate weighted connection 11.By appropriately setting the weights of connections 11 by means of a control means 13 there can be produced in the second layer 9, on operation of the processing means 5, signals representing a degraded version of the image represented by the signals in the first layer, as might result, for example, from defocussing of an image or viewing an image with a moving camera.
Digital signals of an output image in the second layer 9 of the processing means 5 are fed via a first output port 15 of the processing means 5 to a second electrical signal image storage means 17.
The second storage means 17 is arranged to feed the signals stored therein to one input of a comparator 19. Input signals are fed to the other input of the comparator 19 from a third electrical signal image storage means 21.
The output signal of the comparator 19 is selectively connectable via a two-way switch 23 either to a second input port 25 of the processing means 5, which is connected to the second layer 9, or to the weighting control means 13. A second output port 27 of the processing means 5, connected to the first layer 7, is connected to the first storage means 1.
A visual display of the image represented by the signals in storage means 1 is provided by a display means 29. Operation of the various parts of the apparatus is controlled by a control means 31.
Operation of the apparatus to restore a received degraded image where the degradation process to which the received image has been subjected is known will now be described.
The weightings of the connection 11 of the processing means 5 are first set by the control means 31 via the weighting control means 13 so that the processing means 5 mimics the known degradation process between the first input port 3 and the first output port 15.
Signals representing the received degraded image are stored in storage means 21 and signals representing an estimate of the original, i.e. undegraded, image are stored in storage means 1.
Under control of the control means 31 the processing means 5 is operated to subject the estimate original image in storage means 1 to the known degradation process, the resulting image representing signals being stored in storage means 17.
The signals stored in storage means 17 are then compared in comparator 19 with the signals stored in storage means 21, thereby to obtain signals representing the difference between the actual received degraded image and the estimate degraded image represented by the signals in storage means 17. This difference is the result, of course, of inaccuracy of the estimate of the original image in storage means 1.
The difference signals at the output of the comparator 19 are fed via the switch 23 and the second input port 25 to the second layer 9 of the processing means 5. By operation of the processing means 5 in reverse mode to that described above, signals are produced in the first layer 7 of the processing means 5 representing the difference between the estimate original image in storage means 1 and the actual original image.
That these signals in fact represent this difference can be understood by appreciating that the comparator output signals represent a degraded version of the difference between the estimate and actual original image. Hence the signals produced in layer 7 represent an undergraded version of that difference.
The signals produced in layer 7 are passed via output port 27 to storage means 1 to correct the estimate image stored therein.
The corrected estimate image is displayed by display means 29.
It will be appreciated that due to various effects such as noise in the received degraded image and processing errors, the corrected estimate image will still differ from the actual original image. This difference can be progressively reduced by repetition of the above described process using the corrected estimate image as the initial estimate of the original image for the repetition.
It will be understood that if the original image, but not the degrading process, is known, the apparatus of Figure 1 may be used to effect correction of the weightings in the connections 11 of the processor means 5 so that the processor means 5 correctly mimics the degradation process. This is achieved by storing signals representing the actual original image in storage means 1 and applying the output signals of the comparator 19 to the weighting control means 13 via switch 23 to corrct the weightings of connections 11, these output signals clearly indicating the difference between the degrading process to which the received image has actually been subjected and the degrading process effected by the processor means 5.
The apparatus may hence be utilised to effect restoration of a received degraded image when neither the degrading process to which the received image has been subjected nor the original image is accurately known, by executing alternately (by appropriate operation of switch 23) the process described above for correcting an estimated original image in storage means 1 and the process for correcting the weightings in connections 11.
It will be appreciated that whilst it will normally be preferable to operate switch 23 after each process so that the same process is never carried out twice in succession, it may sometimes be found desirable to carry out one or both processes more than once before operating the switch 23.
It will be understood that there is a possibility that instead of obtaining the desired corrections of the estimate image in storage means 1 and the weightings in connections 11, the above described process may result in storage of an estimate image in storage means 1 which differs from the actual original image in a complementary manner to the difference between the weightings actually obtained in connections 11 and the weightings required to mimic accurately the degradation process. This possibility however can be virtually eliminated by applying various constraints to the processes.
Such a constraint may, for example, be applied by using a good first estimate of the original image for storage in storage means 1. However, the prior knowledge to make such a good estimate may not be available. An alternative more practicable way of imposing such a constraint is to impose constraints on the permissable values of the weightings in connections 11. Thus the weighting values may be constrained so as to be always consistent with very basic and physically meaningful assumptions about the degradation process to which the actual received image has been subjected such that the degradation is non-negative everywhere, such that it monotonically decreases in a radial direction from an origin and such that it has a unit grey volume so as to preserve the mean intensity of the image.
With such constraints applied it has been found possible to restore an image with a method and apparatus according to the invention using even the received degraded original image for the first estimate of the original image.
It is also desirable to provide some form of filtering to reduce noise artifacts in a method and apparatus according to the invention. Such filtering may be effected by filtering of image signals or by filtering of the weighting in connections 11, e.g. by smoothing out the differences between the signals for adjacent pixels of an image or the differences between weightings for different connections.
It will be understood that the apparatus of Figure 1 and other apparatuses according to the invention suitably comprise an appropriately programmed digital computer.
Claims (13)
1. A method for image restoration comprisilng the steps of: processing an estimate of an original image so as to obtain an estimate of a received degraded version of the original image; comparing the estimate of the received degraded image with an actual received degraded version of the original image to obtain an indication of the difference between the actual received degraded image and the estimate of the received degraded image, and utilising said difference to improve said estimate of the original image.
2. A method according to Claim 1 further comprising further executing said steps a number of times using for the first step of each said further execution the improved estimate of the original image obtained in the preceding execution.
3. A method according to Claim 2 wherein in at least one of -the executions, instead of utilising said difference to improve said estimate of the original image, said difference is utilised to modify said processing of the first step in such a manner as to improve the estimate of the received degraded image obtained in the first step of the next execution.
4. A method according to Claim 3 wherein said difference is utilised to modify said processing in alternate executions.
5. A method according to Claim 3 or Claim 4 wherein constraints are imposed on said processing consistent with assumptions regarding the degrading process to which said actual received degraded image has been subjected.
6. A method according to any one of the preceding claims further including a step of filtering to reduce noise artifacts.
7. A method for image restoration substantially as hereinbefore described with reference to the accompanying drawing.
8. An apparatus for image restoration comprising: first storage means for storing an estimate of an original image; second storage means for storing a received degraded version of the original image; processing means for processing the image stored in the first storage means so as to mimic the degradation of the received image; comparator means for comparing the image produced by the processing means from the image stored said first storage means with the image stored in said second storage means; and means for utilising the output of said comparator to improve the estimate of the original image in said first storage means.
9. An apparatus according to Claim 8 including means for utilising the output of said comparator to improve the mimicking process effected by said processing means.
10. An apparatus according to Claim 8 or Claim 9 wherein said means for utilising the output of said comparator to improve said estimate of the original image comprises means for causing said processing means to process the output signals of the comparator in reverse mode and for utilising the resultant signals to modify the signals in said first storage means.
11. An apparatus according to any one of Claims 8 to 10 wherein said processing means comprises a neural network.
12. An apparatus according to Claim 11 wherein said processing means comprises a single layer perception.
13. An apparatus for image restoration substantially as hereinbefore described with reference to the accompanying drawing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB9110119A GB2255699B (en) | 1991-05-10 | 1991-05-10 | Methods and apparatus for image restoration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB9110119A GB2255699B (en) | 1991-05-10 | 1991-05-10 | Methods and apparatus for image restoration |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| GB9110119D0 GB9110119D0 (en) | 1991-07-03 |
| GB2255699A true GB2255699A (en) | 1992-11-11 |
| GB2255699B GB2255699B (en) | 1994-09-28 |
Family
ID=10694759
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| GB9110119A Expired - Fee Related GB2255699B (en) | 1991-05-10 | 1991-05-10 | Methods and apparatus for image restoration |
Country Status (1)
| Country | Link |
|---|---|
| GB (1) | GB2255699B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2567723A (en) * | 2017-10-16 | 2019-04-24 | Adobe Inc | Digital image completion using deep learning |
| US10614557B2 (en) | 2017-10-16 | 2020-04-07 | Adobe Inc. | Digital image completion using deep learning |
| US10672164B2 (en) | 2017-10-16 | 2020-06-02 | Adobe Inc. | Predicting patch displacement maps using a neural network |
| US10699453B2 (en) | 2017-08-17 | 2020-06-30 | Adobe Inc. | Digital media environment for style-aware patching in a digital image |
| US10755391B2 (en) | 2018-05-15 | 2020-08-25 | Adobe Inc. | Digital image completion by learning generation and patch matching jointly |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2187058A (en) * | 1986-02-25 | 1987-08-26 | Licentia Gmbh | Regeneration of a reference image |
-
1991
- 1991-05-10 GB GB9110119A patent/GB2255699B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2187058A (en) * | 1986-02-25 | 1987-08-26 | Licentia Gmbh | Regeneration of a reference image |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10699453B2 (en) | 2017-08-17 | 2020-06-30 | Adobe Inc. | Digital media environment for style-aware patching in a digital image |
| GB2567723A (en) * | 2017-10-16 | 2019-04-24 | Adobe Inc | Digital image completion using deep learning |
| GB2567723B (en) * | 2017-10-16 | 2020-03-18 | Adobe Inc | Digital image completion using deep learning |
| US10614557B2 (en) | 2017-10-16 | 2020-04-07 | Adobe Inc. | Digital image completion using deep learning |
| US10672164B2 (en) | 2017-10-16 | 2020-06-02 | Adobe Inc. | Predicting patch displacement maps using a neural network |
| US11250548B2 (en) | 2017-10-16 | 2022-02-15 | Adobe Inc. | Digital image completion using deep learning |
| US11436775B2 (en) | 2017-10-16 | 2022-09-06 | Adobe Inc. | Predicting patch displacement maps using a neural network |
| US10755391B2 (en) | 2018-05-15 | 2020-08-25 | Adobe Inc. | Digital image completion by learning generation and patch matching jointly |
| US11334971B2 (en) | 2018-05-15 | 2022-05-17 | Adobe Inc. | Digital image completion by learning generation and patch matching jointly |
Also Published As
| Publication number | Publication date |
|---|---|
| GB9110119D0 (en) | 1991-07-03 |
| GB2255699B (en) | 1994-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5394305B2 (en) | Image processing device | |
| CN111669514B (en) | High dynamic range imaging method and apparatus | |
| CN102651122A (en) | Image enhancement apparatus and method | |
| US10089910B2 (en) | Image display device, correction data generation method, and image correction device and method, as well as image correction system | |
| CN110796979A (en) | Driving method and driving device of display panel | |
| JP2001054013A5 (en) | ||
| EP0269993B1 (en) | Method and image processing system for reconstruction of an image | |
| CN111292282B (en) | Method and device for generating low-bit-width HDR image, storage medium and terminal | |
| US20190147572A1 (en) | Low cost color expansion module for expanding colors of an image | |
| CN104301622A (en) | Self-adaption space-based space target imaging exposure time control method and device | |
| US9837013B2 (en) | Methods and systems for display correction | |
| EP0345977B1 (en) | Improvements relating to image processing | |
| GB2255699A (en) | Image restoration using a neural network | |
| EP1832123A1 (en) | Digital image photographing apparatus and method | |
| US5631979A (en) | Pixel value estimation technique using non-linear prediction | |
| CN114549383A (en) | Image enhancement method, device, equipment and medium based on deep learning | |
| Weir | Applications of maximum entropy techniques to HST data | |
| CN111612721B (en) | Image restoration model training method and device and satellite image restoration method and device | |
| KR102276863B1 (en) | Image processing apparatus and image processing method | |
| US7184596B2 (en) | Image processing apparatus for reducing coding load | |
| JPH06118920A (en) | Method and device for image information processing | |
| CN100505830C (en) | Method and device for processing a video signal aimed at compensating for the defects of display devices | |
| US20220366539A1 (en) | Image processing method and apparatus based on machine learning | |
| JP2846780B2 (en) | Image information processing method and image information processing apparatus | |
| US7843607B2 (en) | Method for processing digital image data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PCNP | Patent ceased through non-payment of renewal fee |
Effective date: 19960510 |