WO2022075994A1 - Correction de documents numérisés à base de fonctions correctives déterminées - Google Patents

Correction de documents numérisés à base de fonctions correctives déterminées Download PDF

Info

Publication number
WO2022075994A1
WO2022075994A1 PCT/US2020/054921 US2020054921W WO2022075994A1 WO 2022075994 A1 WO2022075994 A1 WO 2022075994A1 US 2020054921 W US2020054921 W US 2020054921W WO 2022075994 A1 WO2022075994 A1 WO 2022075994A1
Authority
WO
WIPO (PCT)
Prior art keywords
document
corrective function
error
processor
reference sample
Prior art date
Application number
PCT/US2020/054921
Other languages
English (en)
Inventor
Carlos Eduardo LEAO
Lucas Nedel KIRSTEN
Sebastien TANDEL
Original Assignee
Hewlett-Packard Development Company, L.P.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett-Packard Development Company, L.P. filed Critical Hewlett-Packard Development Company, L.P.
Priority to PCT/US2020/054921 priority Critical patent/WO2022075994A1/fr
Publication of WO2022075994A1 publication Critical patent/WO2022075994A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46—Colour picture communication systems
    • H04N1/56—Processing of colour picture signals
    • H04N1/60—Colour correction or control
    • H04N1/603—Colour correction or control controlled by characteristics of the picture signal generator or the picture reproducer
    • H04N1/6033—Colour correction or control controlled by characteristics of the picture signal generator or the picture reproducer using test pattern analysis
    • H04N1/6036—Colour correction or control controlled by characteristics of the picture signal generator or the picture reproducer using test pattern analysis involving periodic tests or tests during use of the machine
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46—Colour picture communication systems
    • H04N1/56—Processing of colour picture signals
    • H04N1/60—Colour correction or control
    • H04N1/6083—Colour correction or control controlled by factors external to the apparatus
    • H04N1/6086—Colour correction or control controlled by factors external to the apparatus by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Definitions

  • Physical scanners allow scanning of documents, such as text documents, photos, and the like.
  • Mobile devices including cameras may provide similar functionality by capturing an image and processing the image to scan the document.
  • FIG. 1 is a block diagram of an example system for correcting scanned documents based on determined corrective functions.
  • FIG. 2 is a block diagram of an example non-transitory machine- readable storage medium storing instructions for correcting scanned documents based on determined corrective functions.
  • FIG. 3 is a flowchart of an example method for correcting scanned documents based on determined corrective functions.
  • FIG. 4 is a flowchart of an example method of determining a corrective function during execution of the method of FIG. 3.
  • An example method of correcting scanned documents allows dynamic determination of a unique corrective function for each scanned document.
  • an image is captured including the document to be scanned and a reference sample.
  • a processor selects a reference portion containing the reference sample and uses the reference portion to determine a corrective function (e.g., to correct color).
  • the processor may apply and continually update or tune parameters of the corrective function until the corrective function recovers a target reference sample from the reference portion.
  • the corrective function may be a convolutional neural network accepting an image as an input and outputting an image adjusted for color. Errors between the target reference sample and the corrected reference portion are back- propagated through the neural network to train the neural network on-the-fly for the image being processed.
  • the corrective function can correct the reference portion to approximate the target reference sample
  • the corrective function is applied to a document portion of the image containing a document to be scanned.
  • the document portion is therefore corrected in the same manner that the reference portion is corrected to obtain a more accurate scanned document, and apply a correction that is dynamically, uniquely, and individually applied per image scan.
  • FIG. 1 shows a top view of an example system 100 for correcting scanned documents based on determined corrective functions.
  • the system 100 is to obtain an image 102 of a document 104 and a reference sample 106 and process the image 102, including applying a determined corrective function, to obtain a scan of the document 104.
  • the document 104 may be a document that a user of the system 100 wishes to scan, such as a text document, a drawing, a photo, other paper files including a combination of text and images, or the like.
  • the reference sample 106 is a predefined sample of colors to facilitate the color correction of the scan of the document 104.
  • the reference sample 106 is a strip having sections of different colors, including different hues, saturations and values, in other examples, other suitable forms of the reference sample 106 are contemplated, in addition, the reference sample 106 may have more or fewer sections of sample colors.
  • the system 100 includes a computing device 108, such as a smart phone, a tablet, or other suitable computing device.
  • the computing device 108 includes an image capture device 110, a processor 1 12 and a memory 1 14.
  • the image capture device 1 10 1s to capture an image, such as the Image 102.
  • the image capture device 110 is to capture the image 102 including the document 104 and the reference sample 106 and generate image data representing the image 102, including the document 104 and the reference sample 106.
  • the image capture device 1 10 may be connected to the processor 112 and/or the memory 114 to send the captured image data to the processor 112 for further processing, or to the memory 114 for storage.
  • the image capture device 110 may be, for example, an optical or color image camera or another suitable imaging device.
  • the processor 112 may include a central processing unit (CPU), a microcontroller, a microprocessor, a processing core, or similar device capable of executing instructions.
  • the processor 1 12 may also include or be interconnected with a non-transitory machine-readable storage medium, such as the memory 114, that may be electronic, magnetic, optical, or other physical storage device that stores executable instructions.
  • the memory 114 may store an application for determining a corrective function and applying the corrective function to a scanned document to correct the scanned document.
  • the memory 114 further stores a target reference sample representing the reference sample 106 for facilitating color correction of the scan of the document 104.
  • the target reference sample represents the true or original colors, including hue, saturation and value, of the sections of the reference sample 106.
  • a corrective function which corrects the image of the reference sample 106 to the target reference sample may also therefore correct the image of the document 104 to its true or original colors.
  • the reference sample 106 may be provided, for example, by a publisher of the application for determining a corrective function and applying the corrective function to a scanned document to correct the scanned document.
  • the reference sample 106 may be embedded into a hardware component, such as a printer cover, or provided as a sheet with the reference sample 106 printed on the sheet, provided by the publisher or developer of said application.
  • the computing device 108 captures image data representing the image 102 of the document 104 to be scanned and the reference sample 106.
  • the processor 112 selects a reference portion of the image data representing the reference sample 106.
  • the processor 1 12 further determines a corrective function to correct the reference portion to match the target reference sample stored in the memory 114. For example, the processor 112 may iteratively apply a prospective corrective function to the reference portion, determine an error rate between the corrected reference portion and the target reference sample, and update the corrective function until the error is below a threshold error.
  • the processor 112 may then apply the determined corrective function to a document portion of the image data representing the document 104 to generate a scanned document, which has been corrected by the corrective function.
  • the processor 112 may then output the scanned document, for example, by storing the scanned document in the memory 1 14, or displaying the scanned document at a display (not shown) of the computing device 108 for approval by a user.
  • the computing device 108 may repeat the process of determining a corrective function for each document to be scanned (i.e., each image captured including a document to be scanned).
  • the corrective function applied to each image is uniquely determined to allow for dynamic color correction based on the specific color distortions or other effects uniquely sustained in each respective image.
  • the memory 114 including an example application 200 for correcting scanned documents based on determined corrective functions is depicted.
  • the application 200 includes image identification instructions 202, corrective function application instructions 204, error determination instructions 206, corrective function updating instructions 208, and scan output instructions 210. It will be understood that execution of the instructions 202, 204, 206, 208, and 210 by the processor 112 configures the processor 112 and the computing device 108 to perform the functionality described herein.
  • the image identification instructions 202 when executed, cause the processor 112 to identify a reference portion of the Image data representing the reference sample 106 and a document portion of the image data representing the document 104.
  • the processor 112 may identify the reference portion and the document portion, for example based on predefined expected shapes and qualities of the reference sample 106 and the document 104, respectively.
  • the processor 112 may also apply image processing filters (e.g., cropping, tilting/skewing, lighting and/or shadow correction, and the like) to the Image 102 as a whole or to each of the document portion and the reference portion.
  • the corrective function application instructions 204 when executed, cause the processor 112 to apply a prospective corrective function to the reference portion of the image data to obtain a corrected reference portion.
  • the processor 112 may input the reference portion to the prospective corrective function.
  • the corrective function may be a convolutional neural network, a transformation matrix, or other suitable functions including parameters which may be tuned to adjust the effect of the corrective function on its input.
  • the error determination instructions 206 when executed, cause the processor 112 to compare the corrected reference portion to the target reference sample and determine an error between the corrected reference portion and the target reference sample.
  • the processor 1 12 may compute the mean-squared error for each pixel of the corrected reference portion relative to a corresponding pixel of the target reference sample.
  • the total error of the corrected reference portion may be obtained by summing or averaging the mean-squared errors for each pixel of the corrected reference portion.
  • the resulting error may be referred to herein as the mean-squared error between the corrected reference portion and the target reference sample.
  • the processor 112 may also compare the determined error to a threshold error.
  • the corrective function updating instructions 208 when executed, cause the processor 1 12 to update the prospective corrective function when the error computed by the error determination instructions 206 is above the threshold error.
  • the processor 112 tunes the parameters of the corrective function to adjust the effect of the corrective function on its input. For example, the processor 112 may back-propagate the error through the convolutional neural network to adjust the weight of each node in the convolutional neural network.
  • the processor 112 may first determine whether a current number of iterations is below a threshold number of iterations (e.g., about 1000 iterations) to avoid infinite looping or overfitting the corrective function.
  • the scan output instructions 210 when executed, cause the processor 112 to apply the selected corrective function to the document portion when the error computed by the error determination instructions 206 is below the threshold error.
  • the processor 1 12 may Input the document portion to the selected corrective function to correct the document portion in the same manner that the reference portion was corrected to recover the target reference sample.
  • the processor 112 may also output the corrected scanned document.
  • FIG. 3 a flowchart of an example method 300 is depicted.
  • the method 300 will be described in conjunction with its performance in the system 100, and in particular, by the processor 112. In other examples, the method 300 may be performed by other suitable systems or devices.
  • the processor 112 receives an image, such as the image 102.
  • the processor 112 may receive the image 102 from the image capture device 110, after the image capture device 110 captures the image 102.
  • the processor 1 12 may receive an image from another source, such as another computing device in communication with the computing device 108, or an image stored in the memory 1 14 (e.g., in association with a gallery of images captured by the image capture device 110, or received from other devices).
  • the image received at block 302 includes a document, such as the document 104, and a reference sample, such as the reference sample 106.
  • the processor 112 selects a reference portion of the image 102 containing the reference sample 106. More particularly, fhe processor 112 may select the reference portion from the image data representing the portion of the image 102 containing the reference sample 106. For example, the processor 112 may detect the reference sample 106 within the image 102 based on a predefined shape and expected color of the reference sample 106. In other examples, the reference sample 106 may include fiducial marks or other identifiers to facilitate the location of the reference sample 106 by the processor 112. Thus, the processor 112 may detect the fiducial marks or other identifiers to identify the reference sample 106 within the image 102.
  • the processor 112 may additionally apply image processing filters to the image 102 to obtain the reference portion.
  • the processor 1 12 may adjust the lighting, remove shadow effects, sharpen the image 102, and the like.
  • the processor 112 may additionally crop the image 102 to include only the reference sample 106 In the reference portion and tilt the reference portion to account for a capture angle of the image 102.
  • the processor 112 may apply the image processing filters to the image 102 as a whole, or to the selected reference portion specifically.
  • the processor 112 may further identify a scale for the image 102.
  • a predefined size of the reference sample 106 may be stored in the memory 114 in association with the target reference sample. Accordingly, the processor 1 12 may determine, based on the predefined size of the reference sample 106 and the proportions of the reference sample 106 relative to the image 102, a size of the image 102, and hence a scale for the image 102 relative to the resolution of the image data.
  • the processor 112 determines a corrective function for the image based on the reference portion selected at block 304 and the predefined target reference sample stored in the memory 114.
  • the corrective function recovers the target reference sample from the reference portion. That is, when the corrective function is applied to the reference portion, the corrected reference portion approximates the target reference sample.
  • the initial prospective corrective function may be a default function defined in the memory 1 14.
  • the prospective corrective function is a function with parameters which may be tuned to adjust the corrective effect of the function on its input.
  • the initial or default function may have randomized values as parameters.
  • the prospective corrective function may be a convolutional neural network receiving image data as its input.
  • other suitable corrective functions such as a transformation matrix, may be utilized.
  • the corrective function may be a color correcting function.
  • the processor 112 applies the prospective corrective function to the reference portion to obtain a corrected reference portion.
  • the processor 1 12 may input the reference portion of the image data to the convolutional neural network.
  • the processor 112 compares the corrected reference portion generated at block 402 to the target reference sample stored in the memory 114.
  • the processor 1 12 may determine an error between the corrected reference portion and the target reference sample.
  • the processor 112 may compute a mean-squared error between the corrected reference portion and the target reference sample.
  • the processor 112 may compute the mean-squared error for each pixel in the corrected reference portion, relative to a corresponding pixel in the target reference sample, and sum the error over all the pixels in the corrected reference portion to obtain the total error for the corrected reference portion.
  • other error functions may be applied to compute the error between the corrected reference portion and the target reference sample.
  • the processor 112 compares the error determined at block 404 to a predefined threshold error, stored, for example, in the memory 114 to determine whether the error is less than the threshold error.
  • the method 400 proceeds to block 407.
  • the processor determines whether a current number of iterations is below the threshold number of iterations.
  • the processor 112 may increment the current number of iterations and the method 400 proceeds to block 408.
  • the processor 112 updates the corrective function to generate a new prospective corrective function. For example, when the corrective function is a convolutional neural network, the processor 112 may back-propagate the error through the convolutional neural network to update the update the weights of each node in the convolutional neural network. In other examples, the processor 1 12 may otherwise tune the parameters of the prospective corrective function to change the corrective effect of the corrective function on the reference portion.
  • the method 400 then returns to block 402 to repeat the application of the prospective corrective function and the comparison of the corrected reference portion to the target reference sample.
  • the method 400 proceeds to block 410.
  • the processor 1 12 selects the prospective corrective function applied at block 402 as the corrective function.
  • the processor 112 determines that the selected corrective function, when applied to the reference portion, produces a corrected reference portion which sufficiently closely approximates the target reference sample. That is, the selected or determined corrective function restores the reference portion to its true colors.
  • the processor 112 After having selected the corrective function at block 410, the processor 112 returns to block 308 of the method 300. Accordingly, returning to FIG. 3, at block 308, the processor 1 12 selects a document portion of the image data. For example, the processor 112 may detect the document 104 within the image 102 based on a predefined expected shape of documents for scanning. For example, the processor 112 may run an edge detection algorithm and select edges which define a quadrilateral shape. The processor 112 may additionally apply image processing filters to the image 102 to obtain the document portion. For example, the processor 112 may adjust the lighting, remove shadow effects, sharpen the image 102, and the like.
  • the processor 1 12 may additionally crop the image 102 to include only the document 104 (i.e., the detected quadrilateral) in the document portion and tilt the reference portion to account for a capture angle of the image 102 (i.e., tilt or skew the quadrilateral to define a rectangular document).
  • the processor 1 12 may additionally apply a determined scale for the image 102 to the document portion selected at block 308.
  • the scale applicable to the reference sample 106 is similarly applicable to the image 102 and hence the document 104.
  • the document portion may therefore be scaled accordingly to allow the subsequently generate scanned document to be appropriately sized, regardless of the resolution of the image capture device 110.
  • the processor 112 applies the corrective function determined at block 306 to the document portion selected at block 308.
  • the processor 112 may input the image data corresponding to the document portion into the convolutional neural network which has been tuned according to the reference portion and the target reference sample.
  • the corrective function which restores the reference sample 106 to the true colors defined in the target reference sample will similarly restore the document 104 to its true colors.
  • the corrected document portion may be designated as the scanned document (i.e., the scanned version of the document 104).
  • the processor 112 may additionally output the scanned document, for example by saving the scanned document in the memory 114, displaying the scanned document at a display output of the computing device 108, sending the scanned document to a server or other computing device, or the like.
  • an example device and method for correcting scanned documents by dynamically determining corrective functions is provided.
  • a reference sample embedded for example in a hardware component associated with the application for correcting scanned documents, is captured in an image including the document to be scanned.
  • a corrective function is determined by applying the corrective function to the reference portion containing the reference sample and continually tuning the corrective function until an error is below a threshold error or a threshold number of iterations of updating the corrective function is reached.
  • the corrective function is then applied to a document portion of the image containing the document to be scanned. Since the document and the reference sample are subject to similar distortions and lighting conditions, the corrective function accurately restores the document to its true colors.
  • the corrective function is dynamically determined for each image, and uniquely applied to each document according to the conditions within said image.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

Selon l'invention, un procédé illustratif de correction de documents numérisés à base de fonctions correctives déterminées comprend les étapes suivantes : recevoir une image comprenant un document et un échantillon de référence ; sélectionner une partie de référence de l'image contenant l'échantillon de référence ; déterminer, en fonction de la partie de référence et d'un échantillon de référence cible prédéfini, une fonction corrective pour récupérer l'échantillon de référence cible prédéfini à partir de la partie de référence ; sélectionner une partie de document de l'image contenant le document ; et appliquer la fonction corrective à la partie de document pour produire un document corrigé.
PCT/US2020/054921 2020-10-09 2020-10-09 Correction de documents numérisés à base de fonctions correctives déterminées WO2022075994A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2020/054921 WO2022075994A1 (fr) 2020-10-09 2020-10-09 Correction de documents numérisés à base de fonctions correctives déterminées

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2020/054921 WO2022075994A1 (fr) 2020-10-09 2020-10-09 Correction de documents numérisés à base de fonctions correctives déterminées

Publications (1)

Publication Number Publication Date
WO2022075994A1 true WO2022075994A1 (fr) 2022-04-14

Family

ID=81127048

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2020/054921 WO2022075994A1 (fr) 2020-10-09 2020-10-09 Correction de documents numérisés à base de fonctions correctives déterminées

Country Status (1)

Country Link
WO (1) WO2022075994A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080316549A1 (en) * 2007-06-19 2008-12-25 Bush Iii James Lesesne Method For Scanning And Processing A Document With An Imaging Apparatus
US20110032570A1 (en) * 2009-08-07 2011-02-10 Daisaku Imaizumi Captured image processing system and recording medium
US20110063456A1 (en) * 2009-09-17 2011-03-17 Hideki Ohnishi Portable terminal apparatus, image output apparatus, method of controlling portable terminal apparatus, and recording medium
US20180256041A1 (en) * 2015-11-29 2018-09-13 Arterys Inc. Medical imaging and efficient sharing of medical imaging information
US20190108396A1 (en) * 2017-10-11 2019-04-11 Aquifi, Inc. Systems and methods for object identification

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080316549A1 (en) * 2007-06-19 2008-12-25 Bush Iii James Lesesne Method For Scanning And Processing A Document With An Imaging Apparatus
US20110032570A1 (en) * 2009-08-07 2011-02-10 Daisaku Imaizumi Captured image processing system and recording medium
US20110063456A1 (en) * 2009-09-17 2011-03-17 Hideki Ohnishi Portable terminal apparatus, image output apparatus, method of controlling portable terminal apparatus, and recording medium
US20180256041A1 (en) * 2015-11-29 2018-09-13 Arterys Inc. Medical imaging and efficient sharing of medical imaging information
US20190108396A1 (en) * 2017-10-11 2019-04-11 Aquifi, Inc. Systems and methods for object identification

Similar Documents

Publication Publication Date Title
US8995012B2 (en) System for mobile image capture and processing of financial documents
US7978900B2 (en) Systems for mobile image capture and processing of checks
US8767232B2 (en) Image processing apparatus, image processing method, and computer-readable storage medium
JP5547243B2 (ja) 画像処理装置、プログラムおよび記録媒体
CN110557515B (zh) 图像处理装置、图像处理方法和存储介质
US9041982B1 (en) Portable device scanner support with contour correction and automatic copying
JP2017046342A (ja) 画像処理装置及び画像処理方法
JP2018006981A (ja) 画像処理装置、画像処理方法、コンピュータプログラム
KR101513431B1 (ko) 매체 이미지의 테두리 자동 검출 방법, 이를 이용한 매체 이미지 처리 시스템 및 처리 방법
JP2012095002A (ja) 画像補正装置および画像補正方法
WO2022075994A1 (fr) Correction de documents numérisés à base de fonctions correctives déterminées
US20210281742A1 (en) Document detections from video images
US10356276B2 (en) Image processing apparatus, image forming apparatus, and computer readable medium
JP6350474B2 (ja) キャリブレーションシステム、キャリブレーション方法、画像形成装置およびキャリブレーションプログラム
JP2000261653A (ja) 画像処理装置
US9521290B2 (en) Image processing apparatus, image forming apparatus, image processing method, and non-transitory computer readable medium for detecting difference between images
JP2023124205A (ja) 画像処理システム、画像処理方法、及びプログラム
JP2016171475A (ja) 画像処理システム、方法およびプログラム
JP6550009B2 (ja) 画像読取方法、画像読取プログラム及び画像処理装置
JP2023076914A (ja) 画像処理システム、情報コード読取方法
JP5960097B2 (ja) 画像形成装置及び画像形成方法
JP2012212229A (ja) 画像処理装置および画像処理方法
JP2019213190A (ja) 画像処理装置、画像処理方法、および、プログラム
JP2011217195A (ja) 画像処理装置、同装置における画像処理方法及び画像処理プログラム

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20956893

Country of ref document: EP

Kind code of ref document: A1