EP3882032B1 - Sheet inspection with global brightness adjustment - Google Patents

Description

The present invention relates to a method for image inspection of printed products with automated brightness compensation.

The invention lies in the technical field of print quality control.

In today's printing industry, quality control is carried out automatically using so-called inline inspection systems, hereinafter referred to as image acquisition systems, particularly in the case of larger printing presses. In this case, inline means that the image acquisition system, more precisely the camera of the image acquisition system, is installed in the printing press. It is usually installed after the last printing unit or, if present, after a further post-processing station, such as a varnishing unit, and records the printed products produced by the printing press. It can be a camera or a camera system with multiple cameras. The use of other image sensors is also possible. For the sake of simplicity, however, the term "camera" is used below. The digital print images generated in this way by means of the camera are then compared in an image processing computer with corresponding good images of the print subject. These good images can either be created from the preliminary stage data or they are taught. In this case, teach-in means that a series of print products is printed with the print subject to be produced and captured by the camera of the image acquisition system. These sample prints should be as error-free as possible and are therefore stored as a good image as a good image after being recorded by the image acquisition system as a digital reference in the image processing computer. In the production run, the camera of the image acquisition system then captures the printed image or parts thereof and compares it with the digitally taught-in good image reference or the good image reference created from the prepress data. If deviations are found between the printed products produced in the production run and the digital reference, these deviations are displayed to the printer, who can then decide whether these deviations are acceptable or whether the deviations generated in this way are acceptable Printed products are to be removed as waste. The printed sheets recognized as waste can be ejected via a waste diverter. It is of enormous importance that both the good image reference is error-free and that the print image that is actually printed and recorded by the image acquisition system really corresponds to the real printed print image. Errors caused by the image recording, e.g. due to insufficient lighting, a dirty camera lens or other sources of influence, must not have a negative impact on the inspection process.

A very specific problem, which negatively influences the inspection in this respect, is irregularities in the transport of the print substrate within the printing machine. For a good image recording, the image acquisition system depends on the transported print substrate being transported past the camera of the image acquisition system as smoothly and evenly as possible. This is particularly challenging with sheetfed presses. Because air turbulence and other effects during sheet transport can lead to small deformations on the printing cylinder of the printing press during image recording. These deformations become noticeable through the changing distance from the surface of the sheet to the camera, both through geometric distortions and through fluctuations in brightness. Since these deviations are of course not present in the digital reference, they are classified as printing errors when the captured, recorded print image is compared with the digital reference. If a printer manually monitors the image inspection process, it will of course recognize that these are not real printing errors and will be able to classify these error displays accordingly. For a fully automated image inspection, however, it would be necessary to rule out such false positive errors or pseudo errors from the outset. However, the previous methods known in the prior art are very sensitive to the occurrence of pseudo-errors caused by changes in brightness caused by arc deformation.

From the German patent DE 10 2018 201 794 B3 a method for image inspection of printed products in a machine processing printing materials using a computer is known, with the image inspection being carried out by an image acquisition system the printed products produced are recorded and digitized by means of at least one image sensor, the digital print images recorded in this way are compared by the computer with a digital reference image, the computer subjects the digital reference image to smoothing beforehand, and in the event of deviations in the recorded digital print images from digital reference image the incorrectly recognized print products are ejected, and which is characterized in that before the digital reference image is smoothed, the computer carries out a comparison of the recorded, digital print images with the digital reference image, the result of the comparison to image areas with blurred regions in the recorded, digital Print image is examined, suitable smoothing factors are calculated for these blurred image areas and the digital reference image is then smoothed using the calculated, suitable smoothing factors for these blurred image areas, so that in In these image areas, the digital reference image has a level of blur that is comparable to that of the captured digital print image. However, this procedure only reveals the rectification of deviations found and suitable for rectification in the recorded printed image before the comparison with the reference image. Deviations that are not found or that are not suitable for equalization cannot be dealt with in this way. Even differences in brightness caused by the deformations cannot be corrected by this method. Such pseudo errors can therefore still occur.

On the subject of differences in brightness, the not yet published German patent application 10 2019 204551.0 discloses a method for image inspection of printed products of a machine processing printing materials using a computer, with the printed products produced being recorded and digitized by means of at least one image sensor as part of the image inspection by an image acquisition system , the recorded, digital print images created in this way are compared by the computer with a digital reference image and, in the event of deviations in the recorded, digital print images from the digital reference image, the printed products that have been recognized as incorrect are ejected, which is characterized in that the computer reads the digital print and Divides reference images into sub-images and compensates for differences in brightness between the sub-images. However, this method has the disadvantage that the division into sub-images for print and reference image must be identical. If the partial images that are compared differ only slightly from one another, this results in further procedural errors during the subsequent inspection. In addition, it is relatively computationally expensive to carry out.

It is therefore the object of the present invention to disclose an alternative method for image inspection of printed products, which increases the efficiency of image inspection by recognizing occurring pseudo-defects and avoiding them as best as possible.

This problem is solved by a method for image inspection of printed products in a machine processing printing materials using a computer, with the printed products produced being recorded and digitized by means of at least one image sensor as part of the image inspection by an image recording system, and the digital print images that are recorded in this way are recorded by the computer are compared with a digital reference image and, in the event of discrepancies between the recorded digital print images and the digital reference image, the incorrectly recognized printed products are ejected, which is characterized in that the computer separates areas with paper white for print and reference images and determines their color values and based on the differences between these two color values, compensates for differences in brightness between print and reference images. Color values here mean the color value of the respective paper white. The paper white is already known from the pre-press stage, since the print substrate to be used is part of the print job data. By determining a corresponding value for the real paper white of the printed image and comparing it with the target value, a corresponding difference in brightness can be determined and compensated for. The target value for the paper white logically corresponds to the paper white of the reference image with which the recorded print image is compared as part of the image inspection. This target value is therefore identical to the paper white known from prepress, since the reference image was created either as a good image from the prepress data. Alternatively, the good image can also be taught from actually printed print images, with only print images generated without printing errors and with paper white that is as realistic as possible being used during the teach-in process. In In this case, the paper white of the reference image may deviate slightly from the paper white of the prepress data.

Advantageous and therefore preferred developments of the method result from the associated subclaims and from the description with the associated drawings.

A preferred further development of the method according to the invention is that the brightness of the print or the reference image is adjusted to compensate for the differences in brightness between the print and reference images. It is of secondary importance whether the brightness of the printed image or the reference image is adjusted, as long as the difference in brightness is compensated for before the two images are compared. A decision criterion could e.g. B. that one always brightens the respective darker image or vice versa darkens the brighter image. Or you can simplify the algorithm by specifying a corresponding image, print or reference image, as the image to be adjusted from the outset.

A further preferred development of the method according to the invention is that the computer checks the standard deviation in the paper white in the recorded print image in order to rule out color spatters in the paper white inadmissibly changing the color value of the paper white. Paint splashes would falsify the determined color value of the respective area with paper white and thus lead to an incorrect brightness value, which in turn negatively influences the method according to the invention with regard to the adjustment of the brightness difference or even makes it completely unusable. It must therefore be ruled out that there are ink spatters in the paper white, and this is done by checking the standard deviation in the paper white in the printed image accordingly.

A further preferred development of the method according to the invention is that the computer determines the color value of the paper white by calculating the mean value or the median of all separated paper white areas. The easiest way to proceed is to simply calculate the mean value of the color values of all separated paper white areas and for each captured digitized print image and the Compare reference or good image with each other. However, one can also take the median value if e.g. B. too many splashes of color in the paper white area in the recorded print image would change the mean value. Other calculation approaches for the paper white are also conceivable, such as B. to always accept the darkest or the lightest value in the separated paper white area.

A further preferred development of the method according to the invention is that the comparison of the recorded, digital print images with a digital reference image takes place by the computer for image inspection by creating and evaluating a difference image between the respective recorded, digital print image with the digital reference image. Several approaches are possible for comparing the recorded print images with the digital reference/good image. However, since procedures such as For example, if the adjustment from pixel to pixel is relatively laborious and expensive, the method using the formation of a difference image is preferred. Ideally, if both images are identical, this difference image is simply black, i.e. the color values are zero. The difference image also has the advantage that it can be formed relatively easily from the recorded print image and reference image using a matrix operation.

A further preferred development of the method according to the invention is that the separation of the paper white is carried out by the computer using a threshold value method, with all areas in the image that are darker than the paper white being masked out. After this operation, the corresponding image only has color values in areas that correspond to the white of the paper or are lighter. All other areas are black or have the color value zero.

A further preferred development of the method according to the invention is that a predetermined paper white value from the prepress stage is used as the threshold value for the paper white. Of course, there must be a specified value for the paper white, which usually comes from the prepress stage or the reference/good image. If the reference image is created from the preliminary stage data, this specified value should logically be identical to the mean or median value from the reference image. Just in case you want to be sure, too to detect all paper white areas and a lack of sharpness in the area assignment is acceptable, the predefined threshold value can also be lower than the paper white value to be expected. In the case of creation by teach-in, the default value should be at least similar.

A further preferred development of the method according to the invention is that the threshold value method is applied to the digital reference image, as a result of which a digital mask is created with which the paper white areas of the respective recorded digital print image are separated. i.e. one applies the thresholding method to the digital reference image and then obtains an image in which only the white areas of the paper have color values above zero, while the rest is correspondingly zero. If you place this image, which corresponds to a bit mask or a digital mask, over the captured digital print image or multiplies it with it, all areas of the print image that are not in the paper white area are hidden in the captured digital print image or set to zero set.

A further preferred development of the method according to the invention is that the paper white is separated and its color values are determined by the computer by declaring a predetermined area of the printed product to be inspected as the paper white area, in which the color values of the paper white are then determined. If, for any reason, the process of separating the white areas of the paper using a threshold value and a mask created in this way from the reference image is not possible or is too complex, the reference or good image can also be analyzed in the preliminary stage and the area can thus be determined directly from the print image data and used for the method according to the invention. It is even easier if the paper white area is known without analysis. For example, if the existing print image data of the print job contains an area that always remains unprinted, e.g. B. at the edge of the print substrate. This fixed and clearly defined area can then be used to determine the respective paper white value.

A further preferred development of the method according to the invention is that before the separation of the paper white for the recorded, digital print images, these are digitally corrected by the computer. A number of algorithms are known from the prior art, with which distortions or deformations of the recorded digital printed image can be compensated. The additional use of these algorithms naturally facilitates the method according to the invention, since not all deformations of the sheet can be covered, recognized and treated by means of the brightness compensation. Furthermore, these algorithms or methods for rectifying the print image usually cannot completely correct the rectification either, so that the method according to the invention becomes even more efficient in combination with these known methods.

The invention as such and constructively and/or functionally advantageous developments of the invention are described in more detail below with reference to the associated drawings using at least one preferred exemplary embodiment. In the drawings, corresponding elements are given the same reference numbers.

Figur 1:

ein Beispiel eines Bildinspektionssystems in einer Bogenoffset-Druckmaschine

Figur 2:

ein Kamerabild mit Bogenverformung im Mittelteil

Figur 3:

ein Beispiel eines Referenzbildes

Figur 4:

ein Differenzbild mit markiertem Bereich mit Helligkeitsunterschied

Figur 5:

eine grafische Darstellung der relevanten Bildwerte

Figur 6:

ein generiertes Maskenbild aus dem Referenzbild

Figur 7:

das Resultat der Anwendung der Maske auf das Kamerabild

Figur 8:

das helligkeitsangepasste Kamerabild

Figur 9:

das Differenzbild zwischen Referenzbild und angepasstem Kamerabild

Figur 10:

die grafisch dargestellten angepassten relevanten Bildwerte

Figur 11:

schematisch den Ablauf des erfindungsgemäßen Verfahrens

The figures show:

Figure 1:

an example of an image inspection system in a sheetfed offset printing press

Figure 2:

a camera image with arc deformation in the middle part

Figure 3:

an example of a reference image

Figure 4:

a difference image with marked area with difference in brightness

Figure 5:

a graphical representation of the relevant image values

Figure 6:

a generated mask image from the reference image

Figure 7:

the result of applying the mask to the camera image

Figure 8:

the brightness-adjusted camera image

Figure 9:

the difference image between the reference image and the adjusted camera image

Figure 10:

the graphed adjusted relevant image values

Figure 11:

schematically shows the course of the method according to the invention

figure 1 shows an example of an image acquisition system 2 which uses the method according to the invention. It consists of at least one image sensor 5, usually a camera 5, which is integrated into the sheet-fed printing press 4. The at least one camera 5 records the print images generated by the printing machine 4 and sends the data to a computer 3, 6 for evaluation. This computer 3, 6 can be its own separate computer 6, for example one or more specialized image processing computers 6, or it can be identical to the control computer 3 of the printing press 4. At least the control computer 3 of the printing machine 4 has a display 7 on which the results of the image inspection are shown.

The process according to the invention itself is shown schematically in terms of its sequence in figure 1 shown in a preferred embodiment. During sheet inspection, the computer compares a reference image from prepress and a print image recorded by a camera. The method presented here provides for the brightness of the entire sheet to be adjusted in such a way that it optimally matches the reference image and camera image. For this purpose, a mean value for brightness is formed over all paper white areas of the recorded sheet and the reference image. If both mean values deviate from each other, the brightness of the reference image should be changed in such a way that the mean value of the white areas of the paper is the same. An important point is the reliable identification of paper white areas. For this purpose, all printed areas in the reference image are identified using a threshold value method. All areas in the image that are darker than paper white do not belong to paper white. A mask is created in this way, which is also placed over the camera image after the rectification of the camera image. The areas that remain are then logically paper white.

The exact course of the process is shown in more detail below using several figures:
Air turbulence and other effects can lead to small sheet deformations on the impression cylinder of the printing machine during image recording. These deformations go through both geometric distortions and Brightness fluctuations noticeable, as exemplified in figure 2 shown in the oval obscuration in the center of the image.

In the difference image of figure 4 the brightness change of the arc deformation is marked accordingly. If the reference image off figure 3 and the camera image figure 2 are well coordinated, then no pseudo-error occurs here. Since in the case shown here, which presumably represents many practical cases, the reference image and camera image are not well matched, the arc deformation results in a large pseudo error. Since pseudo error messages should be avoided under all circumstances, this is undesirable behavior.

The resulting values and tolerance ranges of the current example Figure 2 to 4 be in figure 5 shown graphically.

The aim of the new method is to align the global brightness values of the reference image and the camera image in order to avoid pseudo errors caused by small arc deformations. To do this, the paper white values must be determined in the reference image and in the camera image and then the brightness values must be adjusted. The paper white of the reference image is always set to a defined value in prepress. Here in this example to the value 243. Ie the paper white mean value in the reference image is 243. All values < 243 form the subject and can be removed from the reference image using a threshold value method. The resulting image shows in the form of a digital mask figure 6 .

This mask is then placed over the camera image by the computer. This results in a resulting camera image that only shows the white areas of the paper. figure 7 shows this resulting camera image for the previous example.

An average of the gray values of all pixels in the masked camera image whose gray value is not equal to 0 is now formed. The value for this sample image is 226.8. The mean values in the paper white between the reference image and the camera image therefore deviate by around 13 gray values. The camera image is now brightened by these 13 gray values in order to adapt it to the Adjust reference image. This results in an adjusted camera image, which is figure 8 will be shown.

A difference image is now created again, which in figure 9 you can see. One can clearly see how the pseudo-error disappears in the middle of the arc and the problem is solved. The adjusted values and tolerance ranges are displayed in figure 10 shown.

1. 1. Anstatt die Helligkeit des Referenzbildes zu ändern könnte auch die Helligkeit des Scans geändert werden. Es ist jedoch technisch sinnvoller den Wert des Referenzbildes anzupassen.
2. 2. Zur Erhöhung der Funktionsweise des Verfahrens kann die Standardabweichung im Papierweiß im Druckbild geprüft werden, um auszuschließen, dass Farbspritzer im Papierweiß den Mittelwert unzulässig verändern. Tritt eine zu hohe Abweichung im Papierweiß-Mittelwert auf, so müssen weitere Schritte unternommen werden. Dieser Schritt ist allerdings sehr rechenzeitaufwendig und deshalb bei einer Inlineinspektion möglichst zu vermeiden.
3. 3. Das Papierweiß, bzw. Weiß im Referenzbild wird nicht durch Auffinden der Weiß-Bereiche detektiert, sondern in einem als Metadatum von der Druckmaschine, bzw. deren Digitalsteuerung, der Vorstufe oder durch Benutzerauswahl an einer Bediener-GUI oder anderem geeigneten System, mitgelieferten Bereich des Bildes. D.h. es wird explizit ein Bereich auf dem Bogen ausgezeichnet, in dem Papierweiß vorliegt. Es wird dann nur der explizit ausgezeichnete Bereich zur Bestimmung des Papierweißes herangezogen, und es ist keine Detektion der Papierweiß-Bereich mehrnötig. Evtl. werden die ausgezeichneten Bereiche noch zusätzlich mit Ausreißer-Filtern behandelt, um z.B. Farbspritzer aus der Weißschätzung herauszurechnen.
4. 4. Das mittlere Papierweiß im Kamerabild wird in der Beschreibung bisher durch eine einfache Mittelwertbildung berechnet. Hier kann als Variante auch ein besser geeigneter Schätzer des Mittelwerts verwendet werden, der z.B. robust auf Ausreißer, wie Farbspritzer o.ä., ist. Dazu kann z.B. vor der Mittelung ein Anteil der hellsten und/oder dunkelsten Pixel ausgeschlossen werden, wobei diese typischerweise den Ausreißern entsprechen, oder es wird statt dem arithmetischen Mittel ein Median berechnet. Weitere Möglichkeiten umfassen das Berechnen statt des arithmetischen Mittels eines anderen ausreißer-resistenten mittleren Papierweiß-, bzw. Farbwertes, z.B. eines Iteratively Reweighted Least Squares-Mittelwert.
5. 5. Insbesondere für den vollvariablen Digitaldruck gibt es noch eine weitere Alternative. Wenn auf jedem aufeinanderfolgenden Bogen einer Sequenz das Substrat bekannt ist und gleichbleibt, genügt es das Papierweiß auf dem ersten Bogen eines Druckjobs zu bestimmen und den errechneten Skalierungsfaktor auf alle folgenden Referenz-Bilder nur noch anzuwenden. Dies ermöglicht es, eine evtl. länger dauernde Analyse, z.B. das Auffinden der Weißflächen, durchzuführen und trotzdem bei langlaufenden Jobs schritthaltende Verarbeitung zu bieten. Der erste Bogen braucht dann etwas länger, dies wird aber bei den Folgebogen wieder aufgeholt. Man sollte dann aber das Substrat überwachen, ob es sich ändert. Entweder indem die Inspektionssoftware einen neuen Substratbezeichner von der Drucksteuerung mitgeteilt bekommt oder indem in einem kleinen Bereich, wie z.B. einem speziell ausgezeichneten Bereich oder zwischen den Greifern, usw., das Papierweiß ständig überwacht wird. Stellt die Software hierbei eine signifikante Änderung fest, so wird das Papierweiß automatisch neu eingelernt.

Further alternative embodiments include the following possibilities:

1. 1. Instead of changing the brightness of the reference image, the brightness of the scan could also be changed. However, it makes more technical sense to adjust the value of the reference image.
2. 2. To increase the functionality of the method, the standard deviation in the paper white can be checked in the printed image in order to rule out that color splashes in the paper white change the mean value in an unacceptable manner. If the deviation in the average paper white value is too high, further steps must be taken. However, this step requires a lot of computing time and should therefore be avoided if possible in an inline inspection.
3. 3. The paper white or white in the reference image is not detected by finding the white areas, but in a metadata supplied by the printing press or its digital control, the prepress or by user selection on an operator GUI or other suitable system area of the image. This means that an area on the sheet that contains paper white is explicitly marked. Only the explicitly marked area is then used to determine the paper white, and detection of the paper white area is no longer necessary. Possibly, the marked areas are additionally treated with outlier filters, for example to eliminate color splashes from the white estimation.
4. 4. The average paper white in the camera image has so far been calculated in the description by simple averaging. A more suitable estimator of the mean value can also be used here as a variant, which is, for example, robust to outliers such as paint splashes or the like. For this purpose, for example, a portion of the brightest and/or darkest pixels can be excluded before the averaging, these typically corresponding to the outliers, or a median is calculated instead of the arithmetic mean. Other possibilities include calculating instead of the arithmetic mean another outlier-resistant mean paper white or color value, eg an iteratively reweighted least squares mean.
5. 5. There is another alternative, especially for fully variable digital printing. If the substrate on each successive sheet of a sequence is known and remains the same, it is sufficient to determine the paper white on the first sheet of a print job and then simply apply the calculated scaling factor to all subsequent reference images. This makes it possible to carry out an analysis that may take a long time, for example finding the white areas, and still offer continuous processing for long-running jobs. The first sheet then takes a little longer, but this is made up for in the following sheets. But then you should monitor the substrate to see if it changes. Either by the inspection software receiving a new substrate identifier from the print controller or by constantly monitoring the white of the paper in a small area, such as a specially marked area or between the grippers, etc. If the software detects a significant change, the paper white is automatically re-taught.

In the method according to the invention presented here, it is thus avoided to generate pseudo-defects through small arc deformations, without reducing the inspection quality as is the case in the prior art.

Reference List

Claims (10)

1. Method of inspecting images on printed products in a machine for processing printing materials by means of a computer, wherein an image recording system records and digitizes the produced printed products by means of at least one image sensor in the course of the image inspection process, the computer compares the recorded digital printed images which have been created in this way to a digital reference image and wherein, if deviations are found between the recorded digital printed images and the digital reference image, the printed products which have been found defective are removed,
   characterized
   in that the computer separates regions of paper white for printed and reference images, determines the colour values thereof, and, on the basis of the differences between these two colour values, compensates brightness differences between printed and reference images.
2. Method according to claim 1,
   characterized
   in that to compensate for the brightness differences between printed and reference images, the brightness of the printed image or of the reference image is adapted.
3. Method according to any one of the preceding claims,
   characterized
   in that the computer checks the standard deviation of paper white in the recorded printed image to ensure that color splashes in paper white do not modify the colour value of paper white in an unacceptable way.
4. Method according to any one of the preceding claims,
   characterized
   in that the computer determines the value of paper white by calculating the average or median of all the separated paper white regions.
5. Method according to any one of the preceding claims,
   characterized
   in that the comparison of the recorded digitized printed images and a digital reference image for image inspection purposes by means of the computer is implemented by creating and analyzing a differential image between the respective recorded digital printed image and the digital reference image.
6. Method according to any one of the preceding claims,
   characterized
   in that the separation of paper white by the computer is implemented by a threshold value process wherein all regions in the image which are darker than paper white are excluded.
7. Method according to claim 6,
   characterized
   in that a predefined paper white value from the preprint stage is utilized as the threshold value for paper white.
8. Method according to claim 6 or claim 7,
   characterized
   in that the threshold value process is applied to the digital reference image, creating a digital mask with which the paper white regions of the respective recorded digital printed image are separated.
9. Method according to any one of claims 1 to 5,
   characterized
   in that the separation of paper white and the determination of the colour values thereof by the computer are implemented in that a predefined region of the printed product to be inspected is declared a paper white region, in which subsequently the colour values of paper white are determined.
10. Method according to any one of the preceding claims,
    characterized
    in that before the separation of paper white for the recorded digital printed images distortions in the latter are digitally removed by the computer.

Images