JP7457425B1 - Printed matter inspection system, processing device used in the inspection system, and inspection method - Google Patents

Abstract

[Problem] To improve the accuracy of inspection of printed matter. [Solution] An inspection system is used in combination with a printing machine equipped with a print controller that converts input original print data into head data, and a print head that performs printing based on the head data. The inspection system includes a camera that takes an image of a printed matter printed by the printing machine and generates a camera image, and a processing device that judges the quality of the printed matter based on the camera image and outputs the judgment result. The processing device obtains identification information that specifies at least one of the model of the printing machine and the mode of the printing machine, performs spatial filtering processing on the original print data or the head data to generate a filter image, and compares the filter image with the camera image. The processing device determines the parameters of a filter to be used in the spatial filtering processing based on the identification information. [Selected Figure] Figure 4

Description

The present invention relates to a printed matter inspection system, a processing device used in the inspection system, and an inspection method.

2. Description of the Related Art Various inspection devices have been developed and used that inspect the quality or quality of printing by comparing image data obtained by photographing printed matter with a camera and data from the printing source.

For example, Patent Document 1 discloses an inspection device that performs inspection based on the difference between a master image generated from print data and a read image generated by reading a printed matter with a reading device. It is stated that by generating a master image from print data, it is possible to efficiently inspect printed matter using variable data printing (VDP), for example.

Patent Document 2 reads out and compares an image to be inspected obtained by photographing a printed matter and a reference image generated in advance for each unit of printed matter, and if a discrepancy is found, it is determined that an error has occurred in printing. Discloses the testing method.

Patent Document 3 discloses an inspection device that inspects the printed content and printing condition of a color printed matter in which different data is printed on a page-by-page basis. This inspection device inspects the print content and printing condition by comparing and checking a background pattern common to printed matter and variable data that differs for each printed matter with a reference image.

Patent Document 4 discloses a method for determining the quality of printed matter by comparing reference data for comparison generated by converting reference data according to the conditions of the printed matter and imaged data for comparison based on imaged data obtained by imaging the printed matter. Discloses an inspection method for making the determination. The reference data for comparison is converted from the reference data using a learning model generated by applying deep learning. It is described that this makes it possible to determine the quality of printed matter using reference data according to the conditions of the printed matter.

JP 2015-174307 A Japanese Patent Application Publication No. 2019-132661 JP 2003-54096 A JP 2020-186938 A

In conventional print inspection methods, inspection accuracy may be affected due to differences in the format of the image data obtained by photographing the print and the original data for printing, ink bleeding during printing, deformation of the paper, etc. There were cases where the value decreased. In addition, in environments where multiple printing presses of different models coexist, or when printing presses operate in multiple modes depending on the printing target, it is difficult to properly inspect each of the multiple printing presses or multiple modes. was difficult.

The present disclosure provides a new inspection that can improve the accuracy of image inspection of printed matter in an environment where multiple printing machines of different models coexist or when printing machines operate in multiple modes depending on the printing target. Provide technology.

An inspection system according to one aspect of the present invention is used in combination with a printing machine that includes a print controller that converts input printing source data into head data, and a print head that prints based on the head data. . The inspection system includes a camera that takes a picture of a printed matter printed by the printing machine to generate a camera image, and a processing device. The processing device obtains identification information that specifies at least one of a model of the printing press and a mode of the printing press, and, based on the identification information, (a) processes the printing source data using a first filter. (b) a first process of determining the quality of the printed matter by comparing a first filter image generated by performing one-space filtering process with the camera image; and (b) a second process of using a second filter for the head data. A second process is performed in which the quality of the printed matter is determined by comparing a second filter image generated by performing two-space filtering process with the camera image. The processing device determines parameters of the first filter and parameters of the second filter based on the identification information.

A processing device according to another aspect of the present invention is used in combination with a printing machine including a print controller that converts input printing source data into head data, and a print head that prints based on the head data. be done. The processing device acquires identification information that specifies at least one of a model of the printing press and a mode of the printing press, and includes parameters of a first filter used in a first spatial filtering process applied to the printing source data; parameters of a second filter used in a second spatial filtering process applied to the head data are determined based on the identification information. The processing device compares a first filter image generated by performing the first spatial filtering process on the print source data and a second filter image generated by performing the second spatial filtering process on the head data. , output the matching results.

General or specific aspects of the invention may be implemented by an apparatus, a system, an integrated circuit, a computer program, a recording medium, or any combination thereof.

According to the present invention, it is possible to improve the accuracy of image inspection of printed matter.

1 is a diagram schematically showing an example of a conventional inspection system. 1A and 1B are diagrams illustrating the deformation of paper and printed portions caused by printing; 1 is a diagram schematically showing the configuration of an inspection system according to an exemplary embodiment of the present invention. FIG. 2 is a schematic diagram for explaining an overview of determination processing by a processing device. FIG. 1 is a schematic diagram illustrating an example of a system including multiple printing machines. FIG. 13 is a diagram for explaining another example of the inspection system. It is a figure for explaining still another example of an inspection system. FIG. 2 is a diagram schematically showing the flow of an inspection based on an image that has been printed and filtered. FIG. 3 is a diagram schematically showing an outline of color conversion processing. 3 is a flowchart illustrating an example of processing executed by a processing device. FIG. 2 is a diagram showing an example of a table showing the relationship between printing machines, output data formats, and the like. FIG. 3 is a diagram showing an example of a table showing the relationship between printing press modes and filters used. It is a flowchart which shows another example of the process which a processing apparatus performs. FIG. 3 is a diagram showing the flow of processing for converting head data into a first filter image. FIG. 3 is a diagram showing an example of a Gaussian filter and an averaging filter. FIG. 3 is a diagram showing an example of a color conversion table. 1A and 1B are diagrams illustrating an example of a spatial filter and a method of adjusting parameters of the spatial filter. FIG. 3 is a diagram illustrating an example of a color conversion table and an example of a method for adjusting its parameters. It is a figure which shows an example of the matching process of a 1st filter image and a camera image. 1 is a flowchart showing the flow of a method for generating a trained model. FIG. 1 is a diagram showing an example of a test chart. FIG. 13 is a diagram showing another example of the test chart. FIG. 13 is a diagram showing yet another example of the test chart.

Before describing the embodiments of the present invention, we will explain the findings that form the basis of the present invention.

Digital printing machines that perform variable printing print different content on each page. Such printing machines are widely used for various applications such as address printing, bill printing, and statement printing, for example.

In variable printing, different content is printed on each page, so inspection cannot be performed simply by comparing it with a single image prepared in advance. For this reason, a method can be considered in which the inspection is performed by comparing and collating the image data of the printing source and the image data obtained by imaging the printed matter.

However, the method of comparing two images has the following problems, for example.
-Generally, printing processes data using subtractive color system (YMCK), and cameras process data using additive color system (RGB), so it is not possible to directly compare the data of the two.
- The image developed in the printing machine and the image developed in the inspection device do not necessarily match, and there are differences.
-Due to the characteristics of the paper or ink, ink bleeding or paper deformation may occur, resulting in differences between the image to be printed and the image actually printed.

Due to these issues, it has been difficult for conventional inspection devices to inspect color printed materials with high accuracy. For this reason, inspection systems that focus on matching the colors of important parts, such as text, have traditionally been used.

FIG. 1 is a diagram showing a schematic diagram of an example of a conventional inspection system. The inspection system shown in FIG. 1 includes a printing machine 100 (printer) having a print controller 110 and a print head 120, a processing device 210, and a camera 220 (imaging device). The print head 120 includes four types of heads corresponding to four colors, cyan (C), magenta (M), yellow (Y), and black (K). The print controller 110 converts the original print data (e.g., a PDF file, etc.) input from an external computer 300 into head data and outputs it to each print head 120. The head data is, for example, data of halftone images of each color of CMYK. Each print head 120 performs printing on the paper 10 being conveyed according to the head data. The printed paper 10 is photographed by the camera 220. The camera 220 photographs the printed paper 10 to generate camera image data (hereinafter also referred to as "camera image"). The processing device 210 extracts data on specific colors (e.g., the colors of important parts such as text) from both the head data and the camera image, and compares them to determine whether the print is good or bad.

In such systems, the inspection target is limited to specific color components of the printed matter, and other color components cannot be inspected. In addition, it is difficult to detect defects in the printed matter caused by ink bleeding or paper deformation during printing with a high degree of accuracy.

Generally, paper used for printing expands and contracts by absorbing moisture and oil. When ink containing water or oil is jetted or applied onto paper, the paper may expand or contract, and the printed matter may be deformed relative to the original digital data. For example, an A4 size sheet may be deformed by about 1 mm.

FIG. 2 is a diagram schematically showing that the paper and the printed portion are deformed by printing. As shown in FIG. 2, areas with a large amount of ink have a large amount of shrinkage, and areas with a small amount of ink have a small amount of shrinkage. Since the degree of deformation and the form of deformation change depending on the printed content, it is difficult to predict deformation in advance. In particular, in variable printing in which different content is printed on each page, the amount of ink differs on each page, so the way the ink shrinks can vary irregularly on each page. As a result, it may become difficult to match the camera image with the original data. In addition, the captured image may be deformed due to the optical characteristics (mainly distortion) of the lens inside the camera. Although the deformation in this case is regular, matching with the original data can be difficult as well.

Furthermore, in the system shown in FIG. 1, inspection is performed by comparing the image indicated by the head data with the camera image, but some printers output print source data instead of head data. In an environment where printing machines that output printing source data and printing machines that output head data coexist, it is required to be able to perform inspections appropriately regardless of the type of printing machine. Furthermore, the characteristics (ejection amount, density, etc.) of ink ejected from the print head 120 may vary depending on the model and mode of the printing machine. For example, in a printing machine that prints by selecting one mode from a plurality of modes depending on the type of object to be printed (e.g., paper, film, etc.), when the mode changes, the print controller 11 The head data generated will change. As a result, the color tone and degree of blurring of the printed image may also change depending on the mode. Therefore, it is required to be able to perform accurate inspections regardless of the type or mode of the printing press.

In order to solve the above problems, the present inventors came up with the configuration of the embodiment of the present invention described below. Exemplary embodiments of the present invention will be described below. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of well-known matters or redundant explanations of substantially the same configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. The inventors have provided the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims. do not have. Identical or similar components are provided with the same reference numerals throughout this specification.

(Embodiment 1)
FIG. 3 is a diagram schematically showing the configuration of an inspection system according to an exemplary embodiment of the present invention. This inspection system includes a printing press 100 and an inspection device 200. FIG. 3 also shows a computer 300 connected to the printing press 100 and the inspection device 200. Computer 300 is any information processing device such as a server computer, a personal computer (PC), or a mobile terminal. Computer 300 may be included in the inspection system or may be an element external to the inspection system. The printing press 100 may also be an external element of the inspection system.

The printing press 100 shown in FIG. 3 includes a print controller 110, a print head 120, and a conveyor 130. The conveyor 130 conveys the roll-shaped paper 10. The print controller 110 is a device that controls the overall operation of the printing press 100. Print controller 110 includes an integrated circuit including a processor and memory, such as a microcontroller unit. Print controller 110 receives a print instruction including print source data input from computer 300 and drives conveyor 130 and print head 120 . Thereby, the printing machine 100 executes printing while conveying the paper 10. Print controller 110, print head 120, and conveyor 130 do not need to be integrated as one device, and may be configured as independent devices.

The printing machine 100 is a digital printing machine that performs printing using a known method such as an inkjet method or an electrophotographic method. The printing machine 100 prints on paper 10 based on printing source data input from the computer 300. The printing may be color printing or monochrome printing. In the following description, as an example, it is assumed that color printing is performed using four colors of ink: cyan (C), magenta (M), yellow (Y), and black (K). The printing machine 100 is not limited to CMYK, and may print using inks of other colors.

The printing press 100 in this embodiment can perform variable printing. In variable printing, different characters, figures, images, etc. are continuously printed on each page of paper 10. Although the paper 10 in this embodiment is roll paper, printing may be performed on other print media such as film.

The print controller 110 converts the input printing source data into head data and outputs it to the print head 120. The print source data may be vector format data including image and character information, such as PDF (Portable Document Format). The head data may be, for example, bitmap (raster) data indicating a halftone image expressed by a small number of gradations, about 2 to 4 gradations per color for each pixel. The print controller 110 has a RIP (Raster Image Processor) function that converts, for example, vector format printing source data into head data representing a raster image expressed as a set of points of a smaller number of gradations. RIP processing involves converting print source data such as PDF into raster data, converting RGB (red, green, blue) to CMYK (cyan, magenta, yellow, black), and converting the shading of each color into halftone dots ( This may include various types of processing such as halftone processing (halftone). Note that at least a portion of the RIP functions may be executed by a device different from the print controller 110 (for example, the computer 300). The print source data is not limited to PDF, and may be image data such as JPEG (Joint Photographic Experts Group) or PNG (Portable Network Graphics).

The print head 120 is a device that includes components for performing printing based on input head data and circuits for controlling these components. Although one print head 120 is illustrated in FIG. 3, when color printing is performed, multiple types of print heads corresponding to multiple colors of ink may be provided. For example, four print heads corresponding to four colors of ink, cyan (C), magenta (M), yellow (Y), and black (K), may be provided. When the printing machine 100 is an inkjet printer, the print heads of each color may include multiple inkjet heads arranged in a direction perpendicular to the conveying direction (the direction of the arrow shown in FIG. 1) along the surface of the paper 10. Each inkjet head includes, for example, a nozzle that pressurizes or heats the ink and ejects it. Note that when the printing machine 100 is an electrophotographic printer, the print head 120 may include a light source such as a laser or LED, an optical system such as a polygon mirror, components such as a photosensitive drum, and circuits for controlling the light source and the optical system.

The inspection device 200 operates in conjunction with the printing machine 100, inspects the printed paper 10, and outputs the inspection results. Inspection device 200 is connected to printing press 100 and computer 300. Inspection device 200 includes a camera 220 and a processing device 210.

Camera 220 is placed on the conveyance path of printed paper 10. The camera 220 photographs the paper 10 after printing and generates image data (referred to as a "camera image"). Camera 220 may be configured to output a camera image for each page, for example.

The camera 220 includes an image sensor such as a CCD or CMOS and an optical system such as a lens. The camera 220 may include a two-dimensional image sensor, or a linear image sensor (line sensor) such as a contact image sensor (CIS). When the camera 220 includes a linear image sensor, the camera 220 may capture the conveyed paper 10 line by line, and output the data of the multiple lines as a camera image by page. Alternatively, the camera 220 may output data line by line, and the processing device 210 may generate two-dimensional camera image data by combining the data of the multiple lines for each page. The camera 220 may be provided not only on the front side of the paper 10 but also on the back side. By providing two cameras 220 on both sides of the paper 10, the printing state of both sides of the paper 10 can be inspected.

The processing device 210 includes a processor 212, a storage medium (memory) such as a RAM 214 and a ROM 216, and an input/output interface (I/F) 218. The processor 212 includes an arithmetic processing circuit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). ROM 216 stores computer programs executed by processor 212. The program includes a group of instructions for causing the processor 212 to execute processing to be described later. The RAM 214 is a work memory for expanding the program when the processor 212 executes the program. Processor 212 is connected to computer 300, print controller 110, and camera 220 via interface 218.

The processor 212 of the processing device 210 performs a verification process based on printing source data or head data and camera image data to determine whether the printed matter is good or bad. Further, the processor 212 of the processing device 210 can also determine the normality of the print controller 110 by performing a verification process based on print source data and head data. The processing device 210 can arbitrarily select the matching data to be matched from a pair of printing source data and a camera image, a pair of head data and a camera image, and a pair of printing source data and head data. Details of this determination process will be described below with reference to FIGS. 4 to 6B.

FIG. 4 is a schematic diagram for explaining an example of determination processing by the processing device 210. The processing device 210 in this example acquires at least one of printing source data and head data, performs necessary conversion processing such as spatial filtering processing and color conversion processing on the acquired printing source data or head data, and performs necessary conversion processing such as spatial filtering processing and color conversion processing. Image data (hereinafter sometimes referred to as "printing press filter image") is generated. The processing device 210 also acquires a camera image from the camera 220, performs necessary conversion processing such as spatial filtering processing and color conversion processing on the camera image, and creates a camera image for comparison (hereinafter referred to as a "camera filter image"). ). The processing device 210 compares the printing press filter image with the camera filter image and outputs the comparison result. For example, the processing device 210 executes a process of comparing the printing press filter image and the camera filter image for each relatively small area over the entire image, and determines whether the printing is good or bad based on the degree of agreement between the two. The processing device 210 outputs the determination result to an output device or a storage device, such as a display or a speaker, connected to the processing device 210, for example.

In the example shown in FIG. 4, the processing device 210 can select whether to perform the comparison based on print source data or head data. The processing device 210 may be configured to select which of print source data and head data to use for verification, for example, based on a mode setting specified by a user. In that case, the user can use an input device connected to the processing device 210 to switch between a first mode using print source data and a second mode using head data. Alternatively, the processing device 210 obtains model information of the printing press 100 from a device such as the computer 300 or the print controller 100, and determines whether print source data or head data is to be used for verification, depending on the model information. It may be configured to select. Such a function enables appropriate inspection depending on the model of the printing press 100.

When the print controller 110 outputs head data, the contents of the head data may vary depending on the model of the printing press 100 and the print mode. For example, if the printing machine 100 is a model that can print in multiple print modes depending on the type of paper 10, the density or color setting of the ink used, the printing speed, the image quality, etc., the print mode The head data may differ depending on the This is because the optimal halftone processing varies depending on various factors such as the type of paper 10 and the type of ink used by the printing machine 100. In such a printing machine 100, printing is performed by selecting an optimal mode from a plurality of print modes depending on the type of paper 10, the density or color setting of the ink used, printing speed, image quality, etc. .

The processing device 210 in this embodiment is configured to obtain identification information that specifies at least one of the model and mode of the printing machine 100, and switch filter parameters (e.g., values that determine the filter size and coefficients, etc.) to be applied to the original print data or head data according to the identification information. This allows the filter parameters to be adjusted to optimal values according to the identification information. The processing device 210 obtains identification information of the printing machine 100 from an external device, such as the computer 300, the print controller 110, or an input device used by the user. The identification information may include at least one of model information that specifies the model of the printing machine 100 and mode information that specifies the mode of the printing machine 100.

The processing device 210 may further be configured to change parameters of a filter used in spatial filtering processing applied to the camera image according to the identification information. Further, the processing device 210 may be configured to change parameters of color conversion processing applied to print source data or head data in accordance with identification information. Furthermore, the processing device 210 may be configured to change the parameters of the color conversion process applied to the camera image according to the identification information.

The identification information may include both model information that specifies the model of printing press 100 and mode information that specifies the mode of printing press 100. The mode information has a different data format depending on the model of printing press 100. The processing device 210 can read mode information having a different data format for each model of the printing press 100 based on the model information. The processing device 210 may be configured to select matching data based on model information and to determine parameters of a filter used in each spatial filtering process based on mode information.

In the example shown in FIG. 4, the print head 120 includes four types of print heads corresponding to four colors: cyan (C), magenta (M), yellow (Y), and black (K). This allows color printing. Print head 120 may be configured to perform monochrome printing. When monochrome printing is performed, color adjustment processing for print source data or head data and camera image data may be omitted.

According to the present embodiment, the processing device 210 generates a printing press filter image generated by performing spatial filtering on printing source data or head data, and a spatial filtering process on a camera image obtained by photographing printed matter. The quality of the printed matter is determined based on the captured camera filter image. As spatial filtering processing, processing is performed that reflects the effects of ink bleeding and paper deformation during printing. As a result, defects in printed matter can be detected with high accuracy even if ink smearing, paper deformation, etc. occur during printing. Further, in this embodiment, color adjustment processing is also performed according to the print source data or head data and the characteristics of the camera image. Thereby, inspection can be performed with high accuracy regardless of the difference in color expression between the two.

Furthermore, the processing device 210 in this embodiment performs a first spatial filtering process using a first filter on the printing source data according to identification information that specifies at least one of the model of the printing press 100 and the mode of the printing press 100. and a second process in which a second spatial filtering process using a second filter is performed on the head data to generate a second filter image. That is, the processing device 210 selects matching data based on the identification information of the printing press 100. The processing device 210 also sets each parameter of the first filter and the second filter and various parameters in the color conversion process to optimal values based on the identification information. As a result, even in an environment where a plurality of printing machines 100 of different models coexist or in a case where the printing machine 100 has a plurality of printing modes depending on the printing target (for example, paper, film, etc.), the model of the printing press 100 Alternatively, appropriate inspection depending on the mode becomes possible.

Figure 5 is a diagram showing a schematic example of a system including multiple printing machines of different models. Two printing machines 100A and 100B are shown in Figure 5. The number of printing machines is not limited to two, and may be three or more. In this example, the printing machine 100A is configured to output original printing data and identification information of the printing machine 100A. On the other hand, the printing machine 100B is configured to output head data and identification information of the printing machine 100B. The user can select whether to print using the printing machine 100A or the printing machine 100B. When the printing machine 100A is used, the processing device 210 acquires the original printing data and identification information from the printing machine 100A, and performs spatial filtering processing and color conversion processing according to the model or mode of the printing machine 100A indicated by the identification information to generate a first filter image. On the other hand, when the printing machine 100B is used, the processing device 210 acquires head data and identification information from the printing machine 100B, performs spatial filtering processing and color conversion processing according to the model or mode of the printing machine 100B indicated by the identification information, and generates a second filter image. The processing device 210 judges the quality of the print by comparing the first filter image or the second filter image with a camera filter image converted from a camera image.

In this way, the processing device 210 shown in FIG. 5 acquires the identification information of the printing press 100A or 100B, and performs a first process of performing a spatial filtering process and a color conversion process on the printing source data according to the identification information. It is possible to switch between second processing in which spatial filter processing and color conversion processing are performed on the head data. Furthermore, the processing device 210 can adjust parameters such as the filter size and coefficients used in the spatial filtering process, as well as parameters of the color conversion process, according to the identification information. This makes it possible to perform appropriate inspections depending on the type and mode of the printing press.

In the example of FIG. 5, the printing press 100A is configured to transmit printing source data and identification information to the processing device 210. The computer 300 may be configured to send the printing source data and identification information to the processing device 210 instead of the printing press 100A. Further, the computer 300 may be configured to send the identification information of the printing press 100B to the processing device 210 instead of the printing press 100B.

Note that the processing device 210 may omit the spatial filtering process and color adjustment process for the camera image, and directly compare the filter image based on the print source data or head data with the camera image. FIG. 6A shows an example of the system configuration in that case. By performing appropriate processing such as spatial filtering on print source data or head data (hereinafter sometimes referred to collectively as "print data"), high-precision inspection comparable to that achieved by human visual inspection can be achieved. is possible.

As described above, after acquiring the identification information of the printing press 100, the processing device 210 in this embodiment executes either the first process (a) or the second process (b) described below.
(a) The quality of the print is determined by comparing a first filter image generated by performing a first spatial filtering process on the original print data using a first filter with a camera image generated by a camera that photographs the printout printed by the printing press.
(b) A second spatial filtering process is performed on the head data using a second filter, and a second filtered image is generated. The second filtered image is compared with the camera image to determine the quality of the print.

The processing device 210 selects which of the first process and the second process to perform based on the identification information. Furthermore, the parameters of the first filter and the parameters of the second filter are adjusted based on the identification information. Through such operations, optimal image inspection can be achieved depending on the type or mode of the printing press.

Instead of performing the first process (a) or the second process (b) above, the processing device 210 may perform the following third process (c).
(c) A first filter image generated by performing a first spatial filtering process using a first filter on the print source data, and a second filter image generated by performing a second spatial filtering process using a second filter on the head data. The normality of the print controller is determined by comparing it with the filter image.

In this case, the processing device 210 compares the original print data with the head data without using the camera image. The processing device 210 may select whether to execute the first process, the second process, or the third process based on the identification information.

FIG. 6B is a diagram illustrating an example of the processing device 210 that performs the third process (c), that is, collates print source data and head data. In this example, the camera 220 is not used, and the processing device 210 performs conversion processing such as spatial filtering on the first filter image generated by performing conversion processing such as spatial filtering processing on the print source data and conversion processing such as spatial filtering processing on the head data. and the generated second filter image, and outputs the comparison result (for example, information indicating whether or not the print controller 110 is normal). According to such a configuration, printing defects due to a failure or malfunction of the print controller 110 can be detected without using the camera 220. This function may be used alone or in combination with matching using camera images.

FIG. 7A is a diagram schematically showing the flow of an inspection based on an image subjected to printing and filtering processing in this embodiment. Here, an example will be described in which a filter image generated by filtering an image indicated by head data using a spatial filter is compared with a camera image. As shown in FIG. 7A, the print controller 110 performs halftone processing such as dithering on the print source data to generate head data. The head data is data of halftone dots in which the shading in the printing source data is expressed as the density of dots. Such head data can be generated for each color, for example, cyan, magenta, yellow, and black. The print head 120 prints on paper by driving heads corresponding to each color based on such head data. On paper, changes occur in the image due to the effects of ink bleeding, mixing, etc. Furthermore, when a printed matter is photographed with a camera, the photographed image changes depending on the characteristics of the camera, the photographing environment, and the like. As a result, the camera image does not match the image indicated by the head data, and the two cannot be directly compared. Therefore, in the example of FIG. 7A, the processing device 210 converts the head data into a filter image using a spatial filter that simulates the effects of ink smearing on the paper during printing, deformation of the paper, etc. Match the image with the camera image. This makes it possible to compare head data and camera images, and greatly improves inspection accuracy. The size and density of halftone dots in head data and the degree of ink bleeding on paper change depending on the model and mode of the printing press. In this embodiment, by switching the spatial filter to be used depending on the model and mode of the printing press, appropriate inspection can be performed regardless of the model or mode.

Note that in the example of FIG. 7A, spatial filtering is performed on the head data to generate a filter image for verification, but as described above, processing such as spatial filtering is performed on the printing source data to generate a filter image for verification. In some cases. When the print source data is data in a vector format such as PDF, the processing device 210 may perform spatial filtering after performing necessary processing such as rasterization processing and color conversion processing on the print source data. Since the printing source data is also different from the camera image, it is not possible to compare the two as is. Inspection accuracy can be improved by performing appropriate spatial filtering processing on the printing source data according to the model and mode of the printing press.

In the example shown in FIG. 7A, the camera image is used as is as the data for matching, but as in the example of FIG. 4, conversion processes such as spatial filtering and color conversion may also be performed on the camera image to make it easier to match the two. In this case, too, the spatial filtering process and color conversion process may be appropriately adjusted according to the identification information (model or mode) of the printing press.

In this manner, in the present embodiment, print data (and camera images as necessary) are processed before verification, assuming factors that may cause changes in images due to printing. The processing may be performed using a spatial filter (ie, a low-pass filter) that performs smoothing, such as a Gaussian filter. The spatial filter used is not limited to a linear filter such as a Gaussian filter, but may also be a nonlinear filter. Furthermore, a suitable filter image for comparison may be generated by combining a plurality of spatial filters. The spatial filter is created in advance according to the model and mode of the printing press 100 and the characteristics of the paper 10. The spatial filter may be generated as a learned model trained using machine learning, for example. The processing device 210 itself may generate the spatial filter before inspection. The processing device 210 may generate an optimal spatial filter (learned model) by learning the deformation of known print data during printing using, for example, a machine learning algorithm using a neural network. An example of the neural network used is a convolutional neural network (CNN) that uses error backpropagation. The processing device 210 may generate an optimal spatial filter for each printing press model or mode by using a learning algorithm (such as pix2pix) based on a GAN (Generative Adversarial Network), for example.

In the embodiment shown in Figures 4 to 6, the processing device 210 performs color conversion processing in addition to spatial filtering processing on the print data. This makes it easier to compare colors between the print data and the camera image. Below, an overview of the color conversion processing in this embodiment is explained with reference to Figure 7B.

FIG. 7B is a diagram schematically showing an overview of color conversion processing. Here, similarly to the example shown in FIG. 7A, an example in which a filter image is generated from head data will be described. In this example, the print source data is expressed in an additive RGB color space, and the head data is expressed in a subtractive CMYK color space. Since the image taken by the camera is expressed in the RGB color space, it is not possible to directly compare the head data and the camera image. Therefore, the processing device 210 generates a filter image by performing a conversion process from the CMYK color space to the RGB color space on the head data in addition to the above-described spatial filtering process. By performing such conversion, it is possible to simulate the image taken by the camera from the head data, and it is possible to improve the accuracy of the inspection.

The color conversion is not limited to conversion from CMYK to RGB, and may be other conversions. For example, in order to bring the sensitivity to color changes closer to that of the naked eye, print data (print source data or head data) and camera images are converted to a color system such as CIE-L ^* a ^* b ^* in which each color is weighted equally. May be converted. By performing such conversion, inspection that is closer to the naked eye becomes possible. Furthermore, the color conversion of the print data may be performed using known print data and camera images, using a learned model trained by a machine learning algorithm such as CNN using error backpropagation, for example. The processing device 210 may use an algorithm such as pix2pix to generate a model that simultaneously performs the above-described spatial filtering and color conversion. Such models may be generated for each printing press model or mode.

Next, the processing executed by the processing device 210 will be described in more detail with reference to FIG.

FIG. 8 is a flowchart illustrating an example of processing executed by the processor 212 of the processing device 210. In this example, after starting printing, the processing device 210 repeats the processes from steps S101 to S108 for each page of printed matter until receiving an instruction to end printing (step S109) from the print controller 110. Note that the instruction to end printing may be input from a device other than the print controller 110, such as the computer 300.

In step S100, the processing device 210 acquires identification information of the printing press 100. The identification information includes model information that identifies the model of the printing press 100. The identification information may further include mode information that specifies the mode of printing press 100. The mode information includes, for example, information regarding at least one of the type of object to be printed (paper or film, etc.), the density of ink ejected from the print head, printing speed, color settings, and print image quality. You can stay there. Processing device 200 may be configured to obtain identification information from an external device such as printing press 100, computer 300, or an input device used by a user.

The data format of the mode information differs depending on the model of printing press 100. Based on the acquired model information, the processing device 200 can read mode information having a data format that differs depending on the model. For example, data indicating the relationship between the model information of the printing press 100 and the data format of the corresponding mode information may be stored in advance in the memory of the processing device 200 (for example, the ROM 216 shown in FIG. 3). The processing device 200 can refer to the data, identify the data format corresponding to the model of the printing press 100, and read the mode information according to the reading rule according to the data format.

FIG. 9A shows an example of data indicating the relationship between the model information of the printing press 100 and the data format of the corresponding mode information. The data shown in FIG. 9A includes identifiers (IDs) of multiple printing machines 100 used at the site where printing and inspection are performed, machine type information, output data type (head data or printing source data), and mode. This is a table showing the correspondence between information and data formats. The processing device 200 can identify the data format of the mode information corresponding to the model information by referring to a table as shown in FIG. 9A.

In step S101, the processing device 210 determines whether the printing machine 100 is a model that outputs head data based on the identification information. The processing device 210 determines whether the printing machine 100 corresponding to the acquired identification information is a model that outputs head data, for example, by referring to a table such as that shown in FIG. 9A. If the printing machine 100 is a model that outputs head data (the determination result is Yes), the process proceeds to step S102. If the printing machine 100 is a model that outputs original print data (the determination result is No), the process proceeds to step S103.

Note that the table shown in FIG. 9A is an example, and data in a format different from the table shown may be used as long as the data includes similar information. For example, the information shown in FIG. 9A may be distributed between a table that stores model information for each printing press ID and a table that stores information about the data format of the output data type and mode information for each model.

In step S102, the processing device 210 acquires head data from the print controller 110. After that, the process advances to step S104.

In step S103, the processing device 210 acquires the original print data from the print controller 110. Then, the process proceeds to step S104. The original print data may be sent from the computer 300 to the processing device 210.

Note that the acquisition of head data or printing source data may be performed simultaneously with the acquisition of identification information in step S100. For example, in step S100, the processing device 210 may be configured to acquire data in which head data or print source data and identification information are integrated from the print controller 110 or the computer 300. In such a case, the processes of steps S101, S102, and S103 are omitted, and the process proceeds to step S104 after step S100.

In step S104, the processing device 210 performs spatial filtering processing and color conversion processing on the acquired printing source data or head data to generate a printing press filter image.

Spatial filtering processing may include, for example, processing using a smoothing filter such as a weighted averaging filter (eg, a Gaussian filter). The spatial filtering process may include size compression processing using an averaging filter. Spatial filtering processing can simulate the effects of ink bleeding and paper deformation, making it easier to compare with camera images. When processing print source data in vector format, the processing device 210 may perform rasterization processing to convert the data into a raster (bitmap) format image before performing spatial filtering processing.

The processing device 210 in this embodiment determines one or more filters to use in the spatial filtering process based on the identification information. For example, the processing device 210 refers to the table illustrated in FIG. 9B and selects and uses a filter or a group of filter parameters that correspond to the model and mode of the printing machine indicated by the identification information. This allows the spatial filtering process to be appropriately performed using a filter that corresponds to the type and mode of the printing machine.

In the example shown in FIG. 8, in addition to the spatial filtering process, the processing device 210 also performs a color conversion process to adjust the color of each pixel of the printing press filter image. When the printing press filter image is expressed in a four-color color space of CMYK, the processing device 210 is configured to perform color conversion on the printing press filter image from CMYK to RGB, which is the color space of the camera image. can be done. Color conversion may be performed based on a color conversion table created in advance. Alternatively, the processing device 210 may convert the printing source data or head data into a filter image for comparison using a learned model that is trained in advance using a learning algorithm that uses backpropagation or the like. . Processing device 210 may determine the color conversion table or model based on the identification information of printing press 100. Thereby, it is possible to appropriately execute color conversion processing depending on the type and mode of the printing press.

In step S105, the processing device 210 acquires a camera image from the camera 220. The camera image is, for example, RGB image data.

In step S106, the processing device 210 performs spatial filtering processing on the camera image acquired in step S105 to generate a camera filter image. The spatial filtering process may include processing using a smoothing filter, such as a weighted averaging filter (eg, a Gaussian filter). The spatial filtering process may include size compression processing using an averaging filter. By performing spatial filtering processing, it is possible to generate, for example, a camera filter image that has the same number of pixels and degree of blur as a printing press filter image. This facilitates comparison with the printing press filter image.

The processing device 210 may change one or more filters used in the spatial filtering process for the camera image according to the identification information. For example, the processing device 210 may refer to a pre-prepared table and select and use a filter or a group of filter parameters corresponding to the model and mode of the printing machine indicated by the identification information. This allows the spatial filtering process to be appropriately performed using a filter according to the type and mode of the printing machine.

In addition to the spatial filtering process, the processing device 210 may perform a color conversion process to adjust the color of each pixel of the camera image. In addition to spatial filtering processing and color conversion processing, processing such as rotation correction may be performed to facilitate comparison with printing press filter images. Alternatively, the processing device 210 may convert the camera image into a camera filter image for comparison using a learned model that has been trained in advance using a learning algorithm that uses error backpropagation or the like. Processing device 210 may determine the color conversion table or model based on the identification information of printing press 100. Thereby, color conversion processing can be appropriately executed depending on the type and mode of the printing press.

The processing of steps S105 and S106 may be performed before step S102 or S103, or may be performed in parallel with the processing of step S102 or S103.

In step S107, the processing device 210 compares the printer filter image with the camera filter image to determine whether the printing is good or bad. The processing device 210 determines whether the printing is good or bad based on the difference between the printer filter image and the camera filter image. For example, the processing device 210 can compare the printer filter image and the camera filter image for each corresponding small area, and determine that a printing error has occurred if a location where pixel values differ significantly is detected.

In step S108, the processing device 210 outputs the determination result. For example, the processing device 210 outputs, as a determination result, information indicating whether or not there is a defect in the printing and, if there is a defect, where in the image the defect is located. The determination result may be output to an output device such as a display or a storage device connected to the processing device 210, for example. By looking at this output, the user can know that a defect has occurred in printing. The processing device 210 may create a list of test results in which the determination results are recorded for each page, and record the list in the storage device. This allows a history of printing and inspection to be kept.

In step S109, the processing device 210 determines whether there is an instruction to end printing. If there is no instruction to end printing, the process returns to step S101 and the next page is inspected. The instruction to end printing may be input from print controller 110 or computer 300, for example.

By performing the above operations, the processing device 210 can compare the original print image with the camera image for each page of the printout and determine whether there are any abnormalities in the print. In this embodiment, in steps S104 and S105, spatial filtering and color conversion processes are performed on the original print data or head data and the camera image. These processes make it easier to match the original print data or head data with the camera image, improving the accuracy of the inspection.

In the example of FIG. 8, the camera image is converted into a camera filter image in step S106, but this process may be omitted.

FIG. 10 is a flowchart showing another example of the process executed by the processor 212 of the processing device 210. In the flowchart shown in FIG. 10, steps S106 and S107 in the flowchart shown in FIG. 8 are replaced with step S117. Processing other than step S117 is similar to the example of FIG. 8. In the example of FIG. 10, the processing device 210 uses the camera image as it is as data for matching without performing spatial filtering processing or color conversion processing on the camera image. In step S117, the processing device 210 compares the printing press filter image and the camera image to determine the quality of printing. Even in this case, high accuracy can be achieved by appropriately creating models (spatial filters and color conversion tables) used to generate printing press filter images from print data for each printing press model or mode. It is possible to judge the quality of printing.

Here, a specific example of the data conversion process in step S104 will be described with reference to FIG. 11. FIG. 11 shows the flow of processing for converting head data into a printing press filter image. Here, as an example, the printing machine uses head data (a) containing image information of four colors, cyan (C), magenta (M), yellow (Y), and black (K) with a resolution of 1200 dpi (dots per inch). An example of processing when generating a filter image will be described. In the example of FIG. 11, the pixel value in the image indicated by the head data is expressed by 2 bits for each color (8 bits in total). Pixel value 0 (00 in binary) represents "no ink", pixel value 1 (01 in binary) represents "small" amount of ink, and pixel value 2 (10 in binary) represents "medium" amount of ink. , and a pixel value of 3 (11 in binary) represents a "large" amount of ink. For example, if the pixel value is 3 (11) for cyan, 1 (01) for magenta, 0 (00) for yellow, and 2 (00) for black, the amount of ink for cyan is "large" and the amount of ink for magenta is "large". The amount of ink is "small", the amount of ink for yellow is "no ink", and the amount of ink for black is "medium". In this way, the amount of ink for each color of CMYK is expressed by 2 bits, and the larger the value of the 2 bits, the larger the amount of ink. In this embodiment, the pixel value in the head data and the print density have a nonlinear relationship, and the print density is not proportional to the pixel value.

In the example of FIG. 11, the processing device 210 refers to a grayscale conversion table recorded in advance in memory for each color of CMYK, and converts head data having a pixel value of 2 bits for each color into 8 bits for each color (32 bits in total). The image data is converted into image data (b) having a pixel value of . There is a linear relationship between the pixel value and the density in this image data, and the density is proportional to the pixel value. The processing device 210 performs filtering processing (convolution operation) on this image data using a spatial filter such as a Gaussian filter and an averaging filter (for example, a filter that averages 16 pixels). As a result, the processing device 210 smoothes the CMYK color images and generates image data (c) in which the number of pixels is reduced to 1/16 (300 dpi).

FIG. 12A shows an example of a Gaussian filter and an averaging filter. The numerical values of each element (coefficient) of these filters (kernels) are determined so as to appropriately reflect the effects of ink bleeding and paper deformation due to printing. The size (kernel size) of these filters and each coefficient are appropriately adjusted depending on the model or mode of printing press 100. For example, if a mode is selected that corresponds to a combination of ink and printing object (eg, paper or film) that is likely to cause bleeding, the kernel size may be increased and the gradient of the coefficient may be made gentler. On the other hand, if a mode corresponding to a combination of ink and print object that does not easily cause bleeding is selected, the kernel size of the Gaussian filter may be made small and the gradient of the coefficient may be made large.

The processing device 210 performs gain adjustment on the image data (c). Gain adjustment is a process to bring the color closer to the color of the image indicated by the printing source data, and is performed for each color of C, M, Y, and K based on a gain adjustment table recorded in advance in a storage medium such as memory. obtain. Since the optimal gain value differs depending on the printing press model and printing mode, a gain adjustment table can be created for each printing press model and mode. Processing device 210 may be configured to determine the gain adjustment table to use based on identification information indicating the model or mode of the printing press.

The processing device 210 further performs color conversion from CMYK to RGB based on a color conversion table previously recorded in a storage medium such as a memory, and generates a printer filter image (d).

FIG. 12B shows an example of a color conversion table. This color conversion table defines the correspondence between a combination of four CMYK values and a combination of three RGB values. Each numerical value in the color conversion table is set to an appropriate value depending on the characteristics of the ink, paper, camera, etc. For example, a plurality of color conversion tables may be created in advance for each printing press model or mode. Processing device 210 may be configured to determine the color conversion table to use based on identification information indicating the model or mode of the printing press.

The printing press filter image in the example of FIG. 11 is a 300 dpi image in which the pixel values of each color of RGB are expressed with 8 bits (24 bits in total). The processing device 210 checks the quality of the print by comparing this printing press filter image with a camera image or a camera filter image converted from the camera image. Note that in this specification, not only matching with the camera image itself but also matching with a camera filter image converted from the camera image is sometimes expressed as "matching with the camera image."

The resolution of each image and the number of bits of each pixel in the example of FIG. 11, as well as the values in each filter and each table, are merely examples, and these values are set to appropriate values depending on the system (e.g., the model or mode of the printing machine).

In the example of FIG. 11, the printing press filter image is generated from the head data, but the printing press filter image may be generated based on printing source data such as PDF. In that case, the processing device 210 converts the print source data into bitmap image data expressed in a predetermined color space such as RGB, and then converts the data into a bitmap image data expressed in a predetermined color space such as RGB, Filtering (and optionally color conversion) may be performed to generate a press filter image.

Next, an example of the spatial filter used in this embodiment will be explained in more detail.

ここで、（ｘ，ｙ）は、画像の中心を原点とする座標であり、ｘ軸は画像の横方向に対応し、ｙ軸は画像の縦方向に対応する。σは標準偏差である。 FIG. 13 is a diagram schematically showing an example of a spatial filter applied to print data and a method for adjusting parameters of the spatial filter. The spatial filter in this example is a Gaussian filter based on the following two-dimensional Gaussian distribution function.

Here, (x, y) are coordinates with the center of the image as the origin, the x axis corresponds to the horizontal direction of the image, and the y axis corresponds to the vertical direction of the image. σ is the standard deviation.

The kernel size of the Gaussian filter illustrated in FIG. 13 is 5×5. The kernel size may be other sizes (eg, 3x3, 7x7, etc.). Parameters such as the kernel size and standard deviation σ are set to appropriate values depending on the type of printing press used in the inspection, the mode of the printing press, and the characteristics of the camera and the like. Although a Gaussian filter is used in this example, when other spatial filters are used, the filter parameters (values of each element) can be adjusted depending on the characteristics of the printing press, camera, etc.

Here, the pixel values at coordinates (i, j) in the print data, filter image, and camera image are expressed as A[i,j], B[i,j], and C[i,j], respectively. The pixel value B[i,j] of the filter image is approximated by the following formula using the above two-dimensional Gaussian distribution function.

The deviation d between the camera image and the filter image is expressed by the following formula.

Each parameter of the spatial filter is determined in advance to an appropriate value so as to minimize this deviation d. Each parameter of the spatial filter may be determined using machine learning. For example, by performing supervised learning using a predetermined machine learning algorithm and a large number of training datasets corresponding to pairs of print data A[i,j] and camera images C[i,j], , each parameter of the spatial filter may be determined. As a result of this learning, the spatial filter actually used in the inspection may be a filter different from the Gaussian filter.

Next, we will explain in more detail the color conversion process in this embodiment and an example of how to adjust the color conversion table.

FIG. 14 is a diagram showing an example of a color conversion table and an example of a method for adjusting its parameters. In this example, the processing device 210 converts CMYK print data into an RGB filter image based on a color conversion table recorded in a storage medium such as a memory. This color conversion table is data that defines the relationship between the combination of four color values of CMYK and the combination of three color values of RGB. Instead of a color conversion table, a function or model that implements a similar conversion may be used. Each parameter in the color conversion table is determined to minimize the deviation between the camera image and the filter image. The processing device 210 executes supervised learning using a predetermined machine learning algorithm and a large number of learning data sets corresponding to pairs of print data A[i,j] and camera images C[i,j]. By doing so, the parameters of the color conversion table may be adjusted.

In this embodiment, the spatial filtering process and the color conversion process are performed separately, but a conversion process that includes these processes may also be performed. Such conversion processing may be, for example, conversion using a learned model trained using a machine learning algorithm such as a neural network. The processing device 210 may generate a trained model for generating a pseudo camera image from print data using an algorithm such as GAN, for example.

Next, an example of matching processing between a printing press filter image converted from print data and a camera image will be described.

FIG. 15 is a diagram illustrating an example of a process of matching a printing press filter image and a camera image. In this example, the processing device 210 divides the printing press filter image and the camera image into a plurality of regions, and compares the two regions for each region. In the example of FIG. 15, each image is divided into 12 regions. Overlapping regions are provided between the regions. The processing device 210 compares the area to be cut out from the printing press filter image multiple times while moving it one pixel at a time in each direction, for example, vertically and horizontally, determines the cutting area with the highest degree of matching with the camera image, and combines that area with the camera image. You may also compare the corresponding areas in . An offset may be provided in advance to the coordinates in order to compensate for the effects of predictable deformation, such as deformation of the camera image due to lens aberrations in the camera. The processing device 210 can calculate, pixel by pixel, the difference between the pixel value of the printing press filter image and the corresponding pixel value of the camera image, and detect printing defects based on the pixel value difference. For example, the processing device 210 calculates an index value indicating the image difference, such as SSD (Sum of Squared Difference) or SAD (Sum of Absolute Difference), for each of the above regions, and when the value exceeds a threshold value, , it is possible to detect that there is a defect in the printing in that area. Note that when comparing a camera filter image obtained by performing spatial filtering or color conversion processing on a camera image with a printer filter image, the quality of printing can be determined in the same manner.

As described above, the inspection system in this embodiment is combined with a printing press 100 that includes a print controller 110 that converts input printing source data into head data, and a print head 120 that prints based on the head data. used. The inspection system includes a camera 220 that photographs a printed matter printed by the printing press 100 to generate a camera image, and a processing device 210. The processing device 210 acquires identification information that specifies at least one of a printing press model and a printing press mode, and (a) performs a first spatial filtering process using a first filter on the printing source data to generate the first spatial filtering process. (b) applying a second filter to the head data; and (b) applying a second filter to the head data. A second process is performed in which the quality of the printed matter is determined by comparing the second filter image generated by performing the second spatial filtering process used with the camera image. The processing device 210 determines the parameters of the first filter and the parameters of the second filter based on the identification information.

With such a configuration, it is possible to compensate for the effects of ink smearing and paper deformation during printing, or aberrations of a camera lens, etc., depending on the model and mode of the printing press. Therefore, even in an environment where multiple printing machines of different models coexist, or when using a printing machine with multiple printing modes depending on the type of paper or film, the image quality of the printed matter can be maintained regardless of the model or mode. The accuracy of inspection can be improved.

Note that in this embodiment, spatial filtering processing and color conversion processing are performed on the print source data or head data, but the color conversion processing may be omitted depending on the printed matter. For example, when monochrome printing is inspected, the color conversion process may be omitted. This point also applies to color conversion processing for camera images. Further, although the printing apparatus 100 in this embodiment is an inkjet printer, the technology of this embodiment may be applied to an electrophotographic printer such as a laser printer, or a printing machine that performs plate printing.

Furthermore, the processing device 210 may determine the parameters of the first filter used in the first spatial filtering process applied to the original print data and the parameters of the second filter used in the second spatial filtering process applied to the head data based on the identification information of the printing machine, compare the first filter image generated by performing the first spatial filtering process on the original print data with the second filter image generated by performing the second spatial filtering process on the head data, and output the comparison result. By such comparison, it is possible to detect defects in the print head of the printing machine without using a camera.

(Embodiment 2)
Next, an example of a method for generating a trained model by machine learning to be used for performing conversion processes such as the spatial filtering and color conversion described above will be described.

FIG. 16 is a flowchart showing the flow of the trained model generation method in this embodiment. This method includes the steps of generating a training dataset (step S200) and generating a trained model based on the training dataset (step S210). Step S200 includes steps S201 to S205. Step S210 includes steps S211 to S213. Step S200 and step S210 may be executed by the same device or by different devices. In the following description, it is assumed, as an example, that the computer 300 shown in FIG. 3 executes step S200, and the processing device 210 executes step S210. Note that the respective processes of steps S200 and S210 may be executed not only by the computer 300 and the processing device 210 but also by other devices.

In step S201, the computer 300 acquires learning print data from the storage device. The learning print data may be, for example, data showing an image similar to a printed matter actually inspected, or data showing an image of a test chart including a plurality of color patterns specially created for learning. The learning print data is created in advance and recorded in a storage device within the computer 300 or an external storage device. The learning print data corresponds to the print source data shown in FIG. Note that the learning print data may be data in the same format as the head data.

In step S202, the computer 300 instructs the printing machine 100 to execute printing based on the learning print data. In response, the printing machine 100 executes printing on the paper 10.

In step S203, the computer 300 instructs the camera 220 to take a photograph of the printed matter based on the learning print data. In response to this, the camera 220 generates image data of the printed matter (hereinafter referred to as "learning camera image").

In step S204, computer 300 associates the learning camera image with the learning print data and records them in the storage device.

In step S205, computer 300 determines whether an instruction to end the generation of learning data has been issued. The instruction to end may be input to computer 300 based on, for example, a user's operation. Alternatively, the instruction to end may be issued when a preset number of printing and photographing operations have been completed. Until the instruction to end is issued, computer 300 repeats the operations of steps S201 to S205 for different learning print data.

Through the above operations, a learning data set including one or more pairs of learning print data and learning camera images is generated and recorded in the storage device. This learning data set is used to train a machine learning model for converting print data into a pseudo camera image (hereinafter sometimes referred to as a "verification print image"). In order to enhance the effectiveness of learning, it is preferable to prepare as many pairs of learning print data and learning camera images as possible. A set of print data used in an actual inspection and a camera image corresponding thereto may be included in the learning data set.

Next, we will explain the details of step S210, which generates a trained model using the training dataset.

In step S211, the processing device 210 acquires a learning data set from the storage device. If the learning data set includes a plurality of sets of learning print data and learning camera images, data of all sets are acquired.

In step S212, the processing device 210 uses a predetermined machine learning algorithm to perform machine learning based on the learning data set, thereby creating a model (a learned model) for converting the print data into a print image for verification. ) is generated. As the machine learning algorithm, various algorithms used for converting an input image into another image using deep learning, for example, can be used. For example, an algorithm using a convolutional neural network (CNN), such as GAN or NST (Neural Style Transfer), can be used. The generated trained model may be a model that executes the spatial filtering process and color conversion process described in the first embodiment, for example. Generation of the learned model includes adjusting a plurality of parameters in the model so as to reduce the deviation between the learning print data and the corresponding learning camera image.

In step S213, the processing device 210 records the generated trained model in a storage device (eg, ROM 216).

The above operations can generate a trained model for converting print data into a print image for comparison. As described above, the optimal spatial filtering process and color conversion process differ depending on the model or mode of the printing machine. For this reason, a trained model can be generated for each model or mode of the printing machine. When performing an inspection based on the generated trained model, the processing device 210 executes an image conversion program that implements the trained model to convert the print data to be inspected into a print image for comparison. The image conversion program can be configured to apply different trained models depending on the model or mode of the printing machine. The processing device 210 compares and collates the print image for comparison with a camera image or an image that has been processed by performing rotation correction, color correction, etc. on the camera image (collectively referred to as a "camera image for comparison") to determine whether the print is good or bad.

Next, we will explain an example of a test chart used as learning print data for model generation. The test chart may include, for example, a pattern in which multiple areas of different colors are arranged one-dimensionally or two-dimensionally. In this specification, "color" refers to a combination of hue, lightness, and saturation.

17A to 17C show examples of images of test charts. FIG. 17A shows an example of a test chart in which a plurality of color blocks of different colors are two-dimensionally arranged. This test chart consists of areas of many square color blocks arranged in a direction corresponding to the conveyance direction of the printed material (vertical direction in the figure) and a direction corresponding to the direction perpendicular to the conveyance direction (horizontal direction in the figure). including. Each color block is small, for example smaller than 10 mm x 10 mm. The number of color blocks is, for example, 1000 or more. The colors of these color blocks may be selected to cover most of the colors printable by printing press 100, for example. All colors of the color blocks do not need to be different, and blocks of the same color may be included.

FIG. 17B shows an example of a test chart in which a plurality of color bars of different colors are arranged one-dimensionally. This test chart includes a plurality of color bars of different colors extending in a direction (horizontal direction in the figure) corresponding to the direction perpendicular to the conveyance direction of the printed matter. The plurality of color bars are arranged along a direction corresponding to the transport direction (vertical direction in the figure). The width (dimension in the vertical direction) of each color bar is small, for example smaller than 10 mm. The number of color bars is, for example, 30 or more. The colors of these color bars may be selected to cover most of the colors printable by printing press 100, for example. All colors of the color bar do not need to be different, and color bars of the same color may be included.

If printing machine 100 is an inkjet printer, print head 120 may include multiple heads for each color of CMYK, for example. Multiple heads for each color may be arranged along a direction perpendicular to the transport direction. Since the ink ejection condition may differ depending on the head, the density and spread of the ink may differ depending on the position of the head even for the same color. Therefore, in the example of FIG. 17B, a test chart including a plurality of color bars extending in a direction corresponding to the direction in which the heads are arranged is used. By using such a test chart, it is possible to effectively learn how each color changes depending on the position in the arrangement direction of the heads.

The test chart illustrated in FIGS. 17A and 17B includes a plurality of two-dimensional codes 70 including information indicating the type of the test chart. The two-dimensional code 70 is a code, such as a QR (Quick Response) code, in which information is expressed as two-dimensional shading information according to a predetermined rule. The two-dimensional code 70 can be used to identify the type of test chart and the position of each color area in a model learning device. In the example of FIGS. 17A and 17B, a plurality of two-dimensional codes 70 are arranged at predetermined positions around a color block or color bar. By arranging the two-dimensional code 70 in this manner, positioning during verification can be facilitated. At least a portion of the two-dimensional code 70 may be arranged along a direction (lateral direction) corresponding to a direction perpendicular to the conveyance direction of the printed material. Such a two-dimensional code 70 functions as a mark during verification and facilitates identification of each color area. The two-dimensional code 70 may include information for specifying the position of each color region included in the color chart. For example, the two-dimensional code 70 may include information indicating the correspondence between position coordinates in the coordinate system set in the color chart and colors at the position coordinates. As in the example of FIGS. 17A and 17B, when a plurality of two-dimensional codes 70 are included in the color chart, each two-dimensional code 70 is associated with the two-dimensional code 70 in addition to its own position coordinate information. It may also include information on the position coordinates of regions of multiple colors. By referring to such information, the processing device 210 can specify the position of each color area using the two-dimensional code 70 as a landmark. This facilitates positioning and improves learning accuracy.

In addition, instead of the two-dimensional code, a one-dimensional code (barcode) may be provided on the color chart. The same function can be achieved even when a barcode is used. Also, instead of the two-dimensional code and one-dimensional code, for example, a distinctive mark, character, or symbol may be used to achieve the same function.

FIG. 17C is a diagram showing still another example of the test chart. This test chart includes a plurality of color bars of different colors. The brightness of each color bar changes continuously along a direction (horizontal direction) corresponding to a direction perpendicular to the transport direction. By using such a test chart, it is possible to efficiently learn a plurality of colors having different lightness.

The number of test charts illustrated in FIGS. 17A to 17C is not limited to one type, and multiple types of test charts may be used. In that case, for example, the operations from steps S201 to S205 shown in FIG. 16 are repeated for each test chart. A model that can perform image conversion with higher accuracy may be generated by combining learning using a test chart and learning using test data close to a normal inspection target.

As described above, the present disclosure includes the systems, devices, methods, and programs described in the following items.

[Item 1]
An inspection system used in combination with a printing machine including a print controller that converts input print source data into head data, and a print head that prints based on the head data,
a camera for capturing an image of a printed matter printed by the printing machine and generating a camera image;
A processing device;
Equipped with
The processing device includes:
acquiring identification information that identifies at least one of the model of the printing press and the mode of the printing press;
(a) a first process for judging the quality of the printed matter by comparing a first filtered image generated by performing a first spatial filtering process on the original printing data using a first filter with a camera image generated by a camera that photographs a printed matter printed by the printing press, and (b) a second process for judging the quality of the printed matter by comparing a second filtered image generated by performing a second spatial filtering process on the head data using a second filter with the camera image,
determining a parameter of the first filter and a parameter of the second filter based on the identification information;
Inspection system.

[Item 2]
The inspection system according to item 1, wherein the processing device selects which of the first process and the second process to execute based on the identification information.

[Item 3]
The processing device (c) generates a first filter image generated by performing a first spatial filtering process using a first filter on the print source data, and a second spatial filtering process using a second filter on the head data. It is possible to execute a third process of determining the normality of the print controller by comparing it with a second filter image generated by performing the above process,
selecting which of the first process, the second process, and the third process to perform based on the identification information;
Inspection system described in item 1.

[Item 4]
2. The inspection system of claim 1, wherein the parameters include at least one of a filter size and a coefficient.

[Item 5]
The inspection system according to item 1, wherein the identification information includes model information that specifies the model of the printing press and mode information that specifies the mode of the printing press.

[Item 6]
The processing device includes:
reading the mode information having a different data format for each model of the printing press based on the model information;
Inspection system described in item 5.

[Item 7]
The processing device includes:
selecting either the first process or the second process based on the model information;
determining parameters of the first filter and parameters of the second filter based on the mode information;
Inspection system according to item 5 or 6.

[Item 8]
Item 5 or 5, wherein the mode information of the printing press includes information regarding at least one of the type of object to be printed, the density of ink ejected from the print head, printing speed, color settings, and print image quality. 6. The inspection system described in 6.

[Item 9]
7. The inspection system according to any one of items 1 to 6, wherein the processing device learns parameters of the first filter and the second filter that are suitable for each model and each mode of the printing press using machine learning.

[Item 10]
7. The inspection system according to any one of items 1 to 6, wherein the processing device matches the first filtered image or the second filtered image with the camera image by comparing the first filtered image or the second filtered image with an image generated by performing a spatial filtering process on the camera image.

[Item 11]
11. The inspection system according to item 10, wherein the processing device determines parameters of a filter used in the spatial filtering process performed on the camera image based on the identification information.

[Item 12]
The processing device includes:
In the first process , the first spatial filtering process and the first color conversion process are performed on the original print data to generate the first filtered image;
In the second process , the second spatial filtering process and the second color conversion process are performed on the head data to generate the second filter image;
determining parameters of the first color conversion process and the second color conversion process based on the identification information;
7. An inspection system according to any one of items 1 to 6.

[Item 13]
The processing device includes:
By comparing the first filter image or the second filter image with an image generated by performing spatial filtering processing and color conversion processing on the camera image, the first filter image or the second filter image and the Check with the camera image,
The inspection system according to any one of items 1 to 6, wherein the filter used in the spatial filtering process performed on the camera image and the parameters of the color conversion process performed on the camera image are determined based on the identification information. .

[Item 14]
A printing machine including a print controller that converts input print source data into head data, and a print head that performs printing based on the head data;
a camera for capturing an image of a printed matter printed by the printing machine and generating a camera image;
A processing device;
A processing device for use in an inspection system comprising:
acquiring identification information that identifies at least one of the model of the printing press and the mode of the printing press;
Based on the identification information, the method executes either (a) a first process for judging the quality of the printed matter by comparing a first filtered image generated by performing a first spatial filtering process on the original print data using a first filter with the camera image, or (b) a second process for judging the quality of the printed matter by comparing a second filtered image generated by performing a second spatial filtering process on the head data using a second filter with the camera image,
determining a parameter of the first filter and a parameter of the second filter based on the identification information;
Processing unit.

[Item 15]
A processing device used in combination with a printing machine including a print controller that converts input print source data into head data, and a print head that prints based on the head data, the processing device comprising:
obtaining identification information that specifies at least one of a model of the printing press and a mode of the printing press;
Parameters of a first filter used in a first spatial filtering process applied to the print source data and parameters of a second filter used in a second spatial filtering process applied to the head data are determined based on the identification information. decided,
A first filter image generated by performing the first spatial filtering process on the print source data and a second filter image generated by performing the second spatial filtering process on the head data are compared, and a matching result is output. do,
Processing equipment.

[Item 16]
a printing machine including a print controller that converts input printing source data into head data; and a print head that prints based on the head data;
a camera that generates a camera image by photographing the printed matter printed by the printing machine;
a processing device;
An inspection method executed by the processing device in an inspection system comprising:
acquiring identification information that specifies at least one of a model of the printing press and a mode of the printing press;
Based on the identification information, (a) a first filter image generated by performing a first spatial filtering process using a first filter on the printing source data is compared with the camera image to determine whether the printed matter is good or not; and (b) a second filter image generated by performing a second spatial filtering process using a second filter on the head data and comparing the camera image to determine the quality of the printed matter. performing either of the following: a second process for determining;
determining parameters of the first filter and parameters of the second filter based on the identification information;
Inspection methods including.

[Item 17]
a printing machine including a print controller that converts input printing source data into head data; and a print head that prints based on the head data;
a camera that generates a camera image by photographing the printed matter printed by the printing machine;
a processing device;
A program used in an inspection system comprising:
acquiring identification information that specifies at least one of a model of the printing press and a mode of the printing press;
Based on the identification information, (a) the quality of the printed matter is determined by comparing the first filter image generated by performing a first spatial filtering process on the printing source data using a first filter and the camera image; and (b) a second filter image generated by performing a second spatial filtering process using a second filter on the head data and a comparison between the camera image and the quality of the printed matter. selecting one mode from a plurality of modes including a second mode for determining the
determining parameters of the first filter and parameters of the second filter based on the identification information;
A program to run.

[Item 18]
A method executed by a processing device used in combination with a printing press including a print controller that converts input print source data into head data, and a print head that prints based on the head data. ,
acquiring identification information that specifies at least one of a model of the printing press and a mode of the printing press;
Parameters of a first filter used in a first spatial filtering process applied to the print source data and parameters of a second filter used in a second spatial filtering process applied to the head data are determined based on the identification information. deciding and
A first filter image generated by performing the first spatial filtering process on the print source data and a second filter image generated by performing the second spatial filtering process on the head data are compared, and a matching result is output. to do and
method including.

[Item 19]
A program used in a system including a printing machine including a print controller that converts input print source data into head data and a print head that performs printing based on the head data, and a processing device, the program comprising: The processing device,
acquiring identification information that specifies at least one of a model of the printing press and a mode of the printing press;
Parameters of a first filter used in a first spatial filtering process applied to the print source data and parameters of a second filter used in a second spatial filtering process applied to the head data are determined based on the identification information. deciding and
A first filter image generated by performing the first spatial filtering process on the print source data and a second filter image generated by performing the second spatial filtering process on the head data are compared, and a matching result is output. to do and
A program to run.

The technology of the present disclosure can be used to inspect whether the print result of a printed matter is appropriate. For example, it can be suitably used for inspecting printed matter printed by a digital printing machine that performs color variable printing.

10 Paper 100 Printing machine 110 Print controller 120 Print head 130 Conveyor
200 Inspection device 210 Processing device 212 Processor 214 RAM
216 ROM
220 camera 300 computer

Claims (18)

An inspection system used in combination with a printing machine including a print controller that converts input print source data into head data, and a print head that prints based on the head data,
a camera for capturing an image of a printed matter printed by the printing machine and generating a camera image;
A processing device;
Equipped with
The processing device includes:
acquiring identification information that identifies at least one of the model of the printing press and the mode of the printing press;
(a) a first process for judging the quality of the printed matter by comparing a first filtered image generated by performing a first spatial filtering process on the original print data using a first filter with a camera image generated by a camera that photographs a printed matter printed by the printing machine; ( b) a second process for judging the quality of the printed matter by comparing a second filtered image generated by performing a second spatial filtering process on the head data using a second filter with the camera image; and (c) a third process for judging the normality of the print controller by comparing the first filtered image generated by performing the first spatial filtering process on the original print data using the first filter with the second filtered image generated by performing the second spatial filtering process on the head data using the second filter,
selecting which of the first process, the second process, and the third process is to be executed based on the identification information;
An inspection system that determines a parameter of the first filter and a parameter of the second filter based on the identification information. The inspection system according to claim 1, wherein the parameter is a value that determines at least one of a filter size and a coefficient. The inspection system according to claim 1, wherein the identification information includes model information that specifies the model of the printing press, and mode information that specifies the mode of the printing press. The processing device includes:
reading the mode information having a different data format for each model of the printing press based on the model information;
The inspection system according to claim 3 . The processing device includes:
selecting either the first process or the second process based on the model information;
determining parameters of the first filter and parameters of the second filter based on the mode information;
The inspection system according to claim 3 or 4 . 5. The inspection system of claim 3 , wherein the mode information of the printing press includes information regarding at least one of the type of object to be printed on, the thickness of the ink ejected from the print head, the printing speed, color settings, and print image quality . Any one of claims 1 to 4 , wherein the processing device uses machine learning to learn parameters of each of the first filter and the second filter that are compatible with each model and each mode of the printing press. Inspection system described in. 5. The inspection system according to claim 1, wherein the processing device matches the first filtered image or the second filtered image with the camera image by comparing the first filtered image or the second filtered image with an image generated by performing spatial filtering processing on the camera image. The inspection system according to claim 8 , wherein the processing device determines parameters of a filter used in the spatial filtering process performed on the camera image based on the identification information. The processing device includes:
In the first process , the first filter image is generated by performing the first spatial filtering process and the first color conversion process on the printing source data,
in the second process , generating the second filter image by performing the second spatial filtering process and the second color conversion process on the head data;
determining parameters for the first color conversion process and the second color conversion process based on the identification information;
The inspection system according to any one of claims 1 to 4 . The processing device includes:
By comparing the first filter image or the second filter image with an image generated by performing spatial filtering processing and color conversion processing on the camera image, the first filter image or the second filter image and the Check with the camera image,
determining a filter used in the spatial filtering process performed on the camera image and parameters for the color conversion process performed on the camera image based on the identification information;
The inspection system according to any one of claims 1 to 4 . a printing machine including a print controller that converts input printing source data into head data; and a print head that prints based on the head data;
a camera that generates a camera image by photographing the printed matter printed by the printing machine;
a processing device;
A processing device used in an inspection system comprising:
obtaining identification information that specifies at least one of a model of the printing press and a mode of the printing press;
Based on the identification information, (a) the quality of the printed matter is determined by comparing the first filter image generated by performing a first spatial filtering process on the printing source data using a first filter and the camera image; ( b) performing a second spatial filtering process using a second filter on the head data and comparing a second filter image generated with the camera image to determine the quality of the printed matter; (c) using the first filter image generated by performing the first spatial filtering process using the first filter on the print source data and using the second filter on the head data; It is possible to execute a third process of determining the normality of the print controller by performing comparison with the second filter image generated by performing the second spatial filtering process,
selecting which of the first process, the second process, and the third process to perform based on the identification information;
determining parameters of the first filter and parameters of the second filter based on the identification information;
Processing equipment. a printing machine including a print controller that converts input printing source data into head data; and a print head that prints based on the head data;
a camera that generates a camera image by photographing the printed matter printed by the printing machine;
a processing device;
An inspection method executed by the processing device in an inspection system comprising:
acquiring identification information that specifies at least one of a model of the printing press and a mode of the printing press;
( a) a first process of determining the quality of the printed matter by comparing the camera image with a first filter image generated by performing a first spatial filtering process on the print source data using a first filter; ( b) a second process of determining the quality of the printed matter by comparing the camera image with a second filter image generated by performing a second spatial filtering process on the head data using a second filter; (c) the first filter image generated by performing the first spatial filtering process using the first filter on the print source data; and the second spatial filtering process using the second filter on the head data; Selecting and executing any one of a third process of determining the normality of the print controller by comparing it with the second filter image generated by performing the process, based on the identification information ;
determining parameters of the first filter and parameters of the second filter based on the identification information;
Inspection methods including. a printing machine including a print controller that converts input printing source data into head data; and a print head that prints based on the head data;
a camera that generates a camera image by photographing the printed matter printed by the printing machine;
a processing device;
A program used in an inspection system comprising:
acquiring identification information that specifies at least one of a model of the printing press and a mode of the printing press;
( a) a first process of determining the quality of the printed matter by comparing the camera image with a first filter image generated by performing a first spatial filtering process on the print source data using a first filter; ( b) a second process of determining the quality of the printed matter by comparing a second filter image generated by performing a second spatial filtering process on the head data using a second filter with the camera image; (c) the first filter image generated by performing the first spatial filtering process using the first filter on the print source data; and the second spatial filtering process using the second filter on the head data; Selecting and executing any one of a third process of determining the normality of the print controller by comparing it with the second filter image generated by performing the process, based on the identification information;
determining parameters of the first filter and parameters of the second filter based on the identification information;
A program to run. An inspection system used in combination with a printing machine including a print controller that converts input printing source data into head data, and a print head that prints based on the head data, the inspection system comprising:
a camera for capturing an image of a printed matter printed by the printing machine and generating a camera image;
a processing device;
Equipped with
The processing device includes:
obtaining identification information including model information specifying the model of the printing press and mode information specifying the mode of the printing press;
(a) a first process for judging the quality of the printed matter by comparing a first filtered image generated by performing a first spatial filtering process on the original printing data using a first filter with a camera image generated by a camera that photographs a printed matter printed by the printing press, and (b) a second process for judging the quality of the printed matter by comparing a second filtered image generated by performing a second spatial filtering process on the head data using a second filter with the camera image,
determining parameters of the first filter and parameters of the second filter based on the identification information;
reading the mode information having a different data format for each model of the printing press based on the model information in the identification information;
Inspection system. a printing machine including a print controller that converts input printing source data into head data; and a print head that prints based on the head data;
a camera that generates a camera image by photographing the printed matter printed by the printing machine;
A processing device;
A processing device for use in an inspection system comprising:
obtaining identification information including model information specifying the model of the printing press and mode information specifying the mode of the printing press;
(a) a first process for judging the quality of the printed matter by comparing a first filtered image generated by performing a first spatial filtering process on the original printing data using a first filter with a camera image generated by a camera that photographs a printed matter printed by the printing press, and (b) a second process for judging the quality of the printed matter by comparing a second filtered image generated by performing a second spatial filtering process on the head data using a second filter with the camera image,
determining parameters of the first filter and parameters of the second filter based on the identification information;
reading the mode information having a different data format for each model of the printing press based on the model information in the identification information;
Processing equipment. a printing machine including a print controller that converts input printing source data into head data; and a print head that prints based on the head data;
a camera for capturing an image of a printed matter printed by the printing machine and generating a camera image;
a processing device;
An inspection method executed by the processing device in an inspection system comprising:
obtaining identification information including model information that specifies the model of the printing press and mode information that specifies the mode of the printing press;
Executing either (a) a first process for judging the quality of the print by comparing a first filtered image generated by performing a first spatial filtering process on the original print data using a first filter with the camera image, or (b) a second process for judging the quality of the print by comparing a second filtered image generated by performing a second spatial filtering process on the head data using a second filter with the camera image;
determining parameters of the first filter and parameters of the second filter based on the identification information;
reading the mode information having a different data format for each model of the printing press based on the model information in the identification information;
The inspection method includes: a printing machine including a print controller that converts input printing source data into head data; and a print head that prints based on the head data;
a camera that generates a camera image by photographing the printed matter printed by the printing machine;
a processing device;
A program used in an inspection system comprising:
obtaining identification information including model information that specifies the model of the printing press and mode information that specifies the mode of the printing press;
Executing either (a) a first process for judging the quality of the print by comparing a first filtered image generated by performing a first spatial filtering process on the original print data using a first filter with the camera image, or (b) a second process for judging the quality of the print by comparing a second filtered image generated by performing a second spatial filtering process on the head data using a second filter with the camera image;
determining parameters of the first filter and parameters of the second filter based on the identification information;
reading the mode information having a different data format for each model of the printing press based on the model information in the identification information;
A program to run.