JP2021154683A - Specifying method of attribute of ink set, specifying device of attribute of ink set and computer program - Google Patents

Abstract

To appropriately specify an attribute of an ink set used in a printer.SOLUTION: A specifying method comprises the steps of: printing a first image which includes a first portion that has dots of ink in the first color and does not have dots of ink in the second color, and a second portion that has dots of ink in the second color and does not have dots of ink in the first color; acquiring first image data indicating the printed first image by using an image sensor; detecting a malfunction of a nozzle by using the first image data; acquiring second image data indicating a second image printed by using the printer by using a detection result of the malfunction; and specifying the attribute of the ink set including ink in the first color and ink in the second color by using the acquired second image data. The second image is the image including dots of ink in the first color and dots of ink in the second color. The acquired second image data indicates the second image printed by using the nozzle in which the malfunction does not occur in the first nozzle group and the nozzle in which the malfunction does not occur in the second nozzle group.SELECTED DRAWING: Figure 5

Description

The present specification relates to a technique for specifying a type of printing material used for printing by a printing apparatus.

In a printing apparatus that prints an image using a printing material such as ink or toner, it may be desired to specify the type of printing material used. For example, the image forming apparatus described in Patent Document 1 reads data stored in a memory mounted on a toner cartridge, and uses the read data to determine whether the toner is a genuine product or a non-genuine product. Is specified.

Japanese Unexamined Patent Publication No. 2009-300694

However, with the above technique, there is a possibility that the type of toner cannot be properly specified when there is no memory or when the toner in the cartridge is refilled. Such a problem is not limited to specifying the type of toner, for example, in a printing apparatus using an ink set containing a first color ink and a second color ink, the attributes of the ink set (for example, The same was true when specifying (whether or not non-genuine ink was included).

The present specification discloses a new technique capable of appropriately identifying the attributes of an ink set used in a printing apparatus.

The techniques disclosed herein can be realized as the following application examples.

[Application Example 1] A printing apparatus including a printing head having a first nozzle group for ejecting ink of the first color and a second nozzle group for ejecting ink of a second color different from the first color is used. In the printing process of printing one image, the first image includes a first portion containing dots of the first color ink and not including dots of the second color ink, and the second color ink. The printing process and the printing process including the second portion including the dots of the first color ink but not the dots of the first color ink.
A first image acquisition step of acquiring a first image data showing the printed first image using an image sensor, and a defect of the nozzle of the first nozzle group and the first using the first image data. A detection step for detecting a defect in a nozzle of a group of two nozzles, and a second image acquisition step for acquiring second image data indicating a second image printed by the printing apparatus using the detection result of the defect. The second image is an image including dots of the ink of the first color and dots of the ink of the second color, and the second image data is acquired and acquired by using an image sensor. The second image data to be printed shows the second image printed by using the non-defect-free nozzle of the first nozzle group and the non-defect-free nozzle of the second nozzle group. The second image acquisition step includes a specific step of specifying the attributes of an ink set including the first color ink and the second color ink by using the acquired second image data. How to identify the attributes of an ink set.

According to the above configuration, since the defects of the nozzles of the first nozzle group and the second nozzle group are detected by using the first image data, the defects of these nozzles can be detected with high accuracy. Since the second image data showing the printed second image is acquired by using the detection result of the defect of these nozzles, the second image data showing the second image printed using the nozzle without the defect has occurred. Can be obtained properly. Then, since the attributes of the ink set are specified using the second image data, for example, it is possible to accurately specify the attributes of the ink set such that differences appear in the colors represented by the dots of the inks of a plurality of colors. can. Therefore, according to the above configuration, the attributes of the ink set used in the printing apparatus can be appropriately specified.

The techniques disclosed in the present specification can be realized in various forms, for example, a specific device for specifying the attributes of an ink set, these methods, and a computer program for realizing the function of the device. It can be realized in the form of a recording medium or the like on which the computer program is recorded.

The block diagram which shows the structure of the ink identification system 1000 of this Example. The figure which shows the schematic structure of the printing mechanism 100. The block diagram which shows the structure of the machine learning model DN. The flowchart which shows the process of specifying the attribute of an ink set. Explanatory drawing of the test image TI of 1st Example. Flow chart of nozzle clogging detection processing. Explanatory drawing of nozzle clogging detection processing. Flowchart of input image data acquisition process. A flowchart showing a training process of a machine learning model DN. The figure which shows an example of the test image TIb of a modification.

A. Example A-1. Configuration of Ink Identification System 1000 Next, an embodiment will be described based on an embodiment. FIG. 1 is a block diagram showing a configuration of the ink specifying system 1000 of this embodiment. The ink identification system 1000 includes a data processing device 200 and a scanner 500.

The data processing device 200 is a computer such as a personal computer or a smartphone. The data processing device 200 includes a CPU 210 as a controller of the data processing device 200, a volatile storage device 220 such as RAM, a non-volatile storage device 230 such as a hard disk drive and a flash memory, an operation unit 240, and a display unit 250. , Communication interface (IF) 270. The operation unit 240 is a device that receives a user's operation, and is, for example, a keyboard or a mouse. The display unit 250 is a device for displaying an image, for example, a liquid crystal display. The communication interface 270 is an interface for connecting to an external device, for example, a scanner 500.

The volatile storage device 220 provides a buffer area for temporarily storing various intermediate data generated when the CPU 210 performs processing. The non-volatile storage device 230 stores the computer program PG and the test image data TD.

The computer program PG and the test image data TD are provided, for example, by the manufacturer of the printer 400 and installed in the data processing apparatus 200. By executing the computer program PG, the CPU 210 cooperates with the scanner 500 to specify the attributes of the ink set, which will be described later.

The computer program PG includes a computer program as a module that realizes the function of the machine learning model DN described later in the CPU 210. The test image data TD is image data indicating a test image TI described later.

The scanner 500 uses an image sensor to optically read the document to generate scan data indicating a scanned image including the document. The image sensor is, for example, a one-dimensional image sensor having a structure in which a plurality of photoelectric conversion elements such as a CCD and CMOS are arranged side by side in a row. The scan data is, for example, RGB image data representing a color for each pixel by an RGB value. The RGB value is a gradation value of three color components (hereinafter, also referred to as a component value), that is, a color value of an RGB color system including an R value, a G value, and a B value. The R value, G value, and B value are, for example, gradation values having a predetermined number of gradations (for example, 256). The scanner 500 is communicably connected to the data processing device 200.

The printer 400 includes a control device 410 and a printing mechanism 100 as a printing execution unit. The control device 410 includes a CPU and a memory (not shown) and controls the printing mechanism 100. The printing mechanism 100 is an inkjet printing mechanism capable of printing a color image using four color inks of cyan (C), magenta (M), yellow (Y), and black (K) as a printing material. A group of one or more inks used for printing is called an ink set. The ink set of the printing mechanism 100 of this embodiment is composed of four colors of CMYK ink. The ink specifying system 1000 of this embodiment is a system that specifies the attributes of the ink set, specifically, whether or not the ink set contains non-genuine ink, as will be described later.

The printing mechanism 100 includes a printing head 110, a head driving unit 120, a main scanning unit 130, a transport unit 140, and a cartridge mounting unit 150.

FIG. 2 is a diagram showing a schematic configuration of the printing mechanism 100. As shown in FIG. 2A, the main scanning unit 130 holds the carriage 133 on which the print head 110 is mounted and the carriage 133 so as to reciprocate along the main scanning direction (X-axis direction in FIG. 2). It is provided with a moving shaft 134. The main scanning unit 130 reciprocates the carriage 133 along the sliding shaft 134 by using the power of a main scanning motor (not shown). As a result, the main scanning in which the print head 110 is reciprocated along the main scanning direction with respect to the paper S is realized.

While holding the paper S, the transport unit 140 transports the paper S in the transport direction AR (+ Y direction in FIG. 2) that intersects the main scanning direction. As shown in FIG. 2A, the paper base 145, the upstream roller pair 142, and the downstream roller pair 141 are provided. Hereinafter, the upstream side (−Y side) of the transport direction AR is also simply referred to as an upstream side, and the downstream side (+ Y side) of the transport direction AR is also simply referred to as a downstream side.

The upstream roller pair 142 holds the paper S on the upstream side (−Y side) of the print head 110, and the downstream roller pair 141 holds the paper S on the downstream side (+ Y side) of the print head 110. The paper base 145 is arranged at a position between the upstream roller pair 142 and the downstream roller pair 141 and at a position facing the nozzle forming surface 111 of the print head 110. Paper S is conveyed by driving the downstream roller pair 141 and the upstream roller pair 142 by a transfer motor (not shown).

The head drive unit 120 (FIG. 1) supplies a drive signal to the print head 110 to drive the print head 110 while the main scanning unit 130 is performing the main scan of the print head 110. The print head 110 ejects ink onto the paper conveyed by the conveying unit 140 to form dots according to the drive signal.

FIG. 2B shows the configuration of the print head 110 as viewed from the −Z side (lower side in FIG. 2). As shown in FIG. 2 (B), on the nozzle forming surface 111 of the print head 110, a plurality of nozzle rows composed of a plurality of nozzles, that is, nozzle rows for ejecting the above-mentioned C, M, Y, and K inks. NC, NM, NY, NK are formed. Each nozzle row includes a plurality of nozzles NZ arranged along the transport direction AR. The positions of the plurality of nozzles NZ in the transport direction AR (+ Y direction) are different from each other, and the plurality of nozzles NZ are lined up at a predetermined nozzle interval NT along the transport direction AR. The nozzle spacing NT is the length of the transport direction AR between two nozzles NZ adjacent to the transport direction AR among the plurality of nozzles NZ. Among the nozzles constituting these nozzle rows, the nozzle NZ located on the most upstream side (-Y side) is also referred to as the most upstream nozzle NZu. Further, among these nozzles, the nozzle NZ located on the most downstream side (+ Y side) is referred to as the most downstream nozzle NZd. The length obtained by adding the nozzle interval NT to the length of the transport direction AR from the most upstream nozzle NZu to the most downstream nozzle NZd is also referred to as a nozzle length D.

The positions of the nozzle trains NC, NM, NY, and NK in the main scanning direction (X direction in FIG. 2B) are different from each other, and the positions in the transport direction AR (Y direction in FIG. 2B) overlap each other. ing. For example, in the example of FIG. 2B, the nozzle row NM is arranged in the + X direction of the nozzle row NC that ejects C ink.

The control device 410 of the printer 400 controls the head drive unit 120, the main scanning unit 130, and the transport unit 140, and prints by alternately repeating partial printing and transporting the paper S a plurality of times. I do. In one partial printing, ink is ejected from the nozzle NZ of the print head 110 onto the paper S while the paper S is stopped on the paper base 145 and one main scan is performed. As a result, a partial image that is a part of the image to be printed is printed on the paper S. In one transport of the paper S, the paper S is moved in the transport direction AR by a predetermined transport amount.

The cartridge mounting unit 150 is configured to mount CMYK ink cartridges 420C, 420M, 420Y, and 420K. The ink cartridges 420C, 420M, 420Y, and 420K are cartridges containing inks of cyan C, magenta M, yellow Y, and black K, respectively. Ink is supplied to the print head 110 from these ink cartridges 420C, 420M, 420Y, and 420K.

In this embodiment, two types of ink, genuine ink and non-genuine ink, are assumed to be used. The genuine ink is, for example, an ink manufactured by the manufacturer of the printer 400 and optimized for use in the printer 400. The non-genuine ink is, for example, an ink manufactured by a business operator different from the manufacturer of the printer 400. Genuine ink and non-genuine ink have different characteristics and components such as viscosity. Non-genuine ink tends to cause a failure of the printer 400 (for example, clogging of the nozzle of the print head) as compared with genuine ink. For this reason, for example, when investigating the cause of failure of the printer 400, it is preferable to specify whether the ink used in the printer 400 is genuine ink or non-genuine ink. The ink identification system 1000 of this embodiment is used in such an application, for example. For example, after the ink cartridge of genuine ink is emptied, non-genuine ink may be stored in the ink cartridge (so-called refilling of ink). For this reason, the type of ink may not be properly specified by the method of referring to the appearance of the ink cartridge or the product number of the ink cartridge. Even in such a case, the method using the ink identification system 1000 is effective.

A-2. Configuration of Machine Learning Model FIG. 3 is a block diagram showing the configuration of the machine learning model DN. The machine learning model DN is realized by the CPU 210 executing the computer program PG. The machine learning model DN is a trained model trained by a training process described later.

Input image data Din is input to the machine learning model DN as input data. The input image data Din is image data indicating the input patch IP. The input patch IP is an image printed on paper S using the printer 400, as will be described later. The input image data Din is bitmap data indicating an image including a plurality of pixels, and specifically, RGB image data. Since the input image data Din contains three component values (R value, G value, B value) of each of the (200 × 200) pixels, the data includes (200 × 200 × 3) values. be.

The machine learning model DN is an identification network that identifies whether or not the ink set used to print the input patch IP contains non-genuine ink. Here, when the input image data Din is input, the machine learning model DN executes arithmetic processing using a plurality of arithmetic parameters (described later) to print the input patch IP indicated by the input image data Din. Output data Dout indicating the result of identifying whether or not the used ink set contains non-genuine ink is output.

Specifically, the output data Dout contains a plurality of values corresponding to the number of classes to be identified. In this embodiment, there are two types of classes to be identified: an ink set in which all are genuine ink (also called a genuine ink set) and an ink set containing non-genuine ink (also called a non-genuine ink set). .. Therefore, the output data Dout is two-dimensional data including a value Vo1 corresponding to the genuine ink set and a value Vo2 corresponding to the non-genuine ink set. For example, the two values Vo1 and Vo2 are values in the range of 0 or more and 1 or less, respectively, and the sum of the two values Vo1 and Vo2 is 1. In the training described later, of the two values Vo1 and Vo2, one value corresponding to the class to which the ink set used for printing the input patch IP belongs approaches 1, and the other values that do not correspond approach 0. As such, the machine learning model DN is being trained. Therefore, the machine learning model DN can output output data Dout that appropriately indicates whether or not the ink set used for printing the input patch IP contains non-genuine ink.

The machine learning model DN is a network called a CNN (Convolutional Neural Network), and has an input layer 305, a first convolutional layer 310, a first convolutional layer 320, a second convolutional layer 330, and a second convolutional layer. It has 340, a first fully-bonded layer 350, a second fully-bonded layer 360, and a third fully-bonded layer 370. These layers 305-370 are connected in this order.

The above-mentioned input image data Din is input to the input layer 305. The input image data Din input to the input layer 305 is used as input information by the first convolution layer 310.

The convolution layers 310 and 330 perform a convolution process and a bias addition process. In the convolution process, s filters of (p × q × r) dimension are sequentially applied to the data input from the immediately preceding layer, and a correlation value indicating the correlation between the input data and the filter is obtained. It is a process to calculate. In the process of applying each filter, a plurality of correlation values are sequentially calculated while sliding the filter. One filter contains (p × q × r) weights. The bias addition process is a process of adding one bias prepared for each filter to the calculated correlation value.

The (p × q × r × s) weights included in the s filters and the s biases corresponding to the s filters are arithmetic parameters adjusted by the training process. Each value of the data generated by the convolution layers 310 and 330 is a value obtained by adding a bias to the above-mentioned correlation value. The number of values (number of dimensions) included in the data generated by each convolution layer is determined by the stride (amount of sliding the filter) in the convolution process and the number s of filters.

The data generated by the convolutional layers 310 and 330 is input to the activation function as post-processing, converted, and then input to the next layer (pooling layer 320, 340). In this embodiment, ReLU is used as the activation function.

The pooling layers 320 and 340 perform MaxPooling on the data input from the immediately preceding convolutional layer to reduce the number of dimensions of the data. Max pooling is a process of reducing the number of dimensions by so-called downsampling, and selects the maximum value in a window while sliding a window of a predetermined size (for example, 2 × 2) with a predetermined stride (for example, 2). This reduces the number of dimensions. The data generated by the pooling layers 320 and 340 is directly input to the next layer (second convolution layer 330 and first fully connected layer 350).

The fully connected layers 350, 360, and 370 are layers similar to the fully connected layers used in a general neural network. When m-dimensional data (m values (m is an integer of 2 or more)) is input, the fully connected layer outputs n-dimensional data (n values (n is an integer of 2 or more)). .. Each of the output N values is a value (inner product + bias) obtained by adding a bias to the inner product of the input vector consisting of m values and the vector consisting of n weights. These (m × n) weights and n biases are arithmetic parameters adjusted by the training process.

Each value generated by the fully connected layers 350 and 360 is input to the activation function and converted as a post-processing, and then input to the next layer (fully connected layers 360 and 370). For the activation function, for example, ReLU is used.

The data (two values) generated by the third fully connected layer 370 is input to the activation function and converted as post-processing. SoftMax is used as the activation function. The two values converted by the activation function are the output data Dout described above.

In FIG. 3, the numerical value on the left side of each layer (for example, [200 × 200 × 3]) is the number of dimensions of the data input to that layer, and the numerical value on the right side of each layer (for example, [196 × 196 × 3]). 8]) is the number of dimensions of the data output from that layer.

A-3. Specifying Ink Set Attributes FIG. 4 is a flowchart showing a process of specifying ink set attributes. This step is performed on a CMYK four-color ink set used in the printer 400.

In S100, the CPU 210 of the data processing device 200 acquires the test image data TD (FIG. 1) from the non-volatile storage device 230. The test image data TD is, for example, RGB image data.

In S105, the CPU 210 prints the test image TI on the paper S using the test image data TD. Specifically, the CPU 210 executes a color conversion process on the test image data TD to convert the values of a plurality of pixels included in each image data from RGB values to CMYK values. The CMYK value is a color value including a plurality of component values (CMYK component values) corresponding to the CMYK ink used for printing. The CPU 210 executes halftone processing such as an error diffusion method on the test image data TD that has undergone color conversion processing. As a result, dot data indicating the test image TI is generated. The dot data is data indicating a dot formation state for each color material used for printing and for each pixel. The dot formation state can be, for example, two types of states with or without dots, and four types of states of large dots, medium dots, small dots, and no dots. The CPU 210 transmits dot data indicating the test image TI to the printer 400. The control device 410 of the printer 400 controls the printing mechanism 100 according to the dot data, and causes the printing mechanism 100 to print the test image TI. In the modified example, the CMYK image data and the dot data may be stored in the non-volatile storage device 230 in advance as the test image data TD instead of the RGB image data.

FIG. 5 is an explanatory diagram of the test image TI of the first embodiment. FIG. 5A shows a state in which the test image TI is printed on the paper S. The test image TI is printed in one partial print. FIG. 5A shows a head position P1 when printing a test image TI. The head position P1 is a position relative to the paper S in the transport direction of the print head 110.

The test image TI includes a plurality of detection patch TPs and a plurality of input patch IPs. The detection patch TP is used in the nozzle clogging detection process described later. The input patch IP is used as an image input to the machine learning model DN described above.

FIG. 5B shows one detection patch TP. The detection patch TP is a rectangular image and includes monochromatic portions Pc, Pm, Py, and Pk corresponding to each ink of CMYK. Each monochromatic portion is a strip-shaped portion extending in the transport direction AR. Each monochromatic portion is a portion printed using only the corresponding one color ink. That is, each monochromatic portion contains dots of the corresponding one color ink and does not include dots of the other uncorresponding ink.

For example, the monochromatic portion Pc is a portion corresponding to the ink of C and is a uniform region of cyan. The monochromatic portion Pc includes C dots and does not include MYK dots. The monochromatic portion Pm is a portion corresponding to the ink of M and is a uniform region of magenta. The monochromatic portion Pm includes M dots and does not include CYK dots. The monochromatic portion Py is a portion corresponding to the ink of Y, and is a uniform region of yellow. The monochromatic portion Py includes Y dots and does not include CMK dots. The monochromatic portion Pk is a portion corresponding to the ink of K, and is a uniform region of black. The monochromatic portion Pk includes K dots and does not include CMY dots.

FIG. 5C shows one input patch IP. The input patch IP is a rectangular image and includes mixed color portions Pr, Pg, Pb, and Pbl corresponding to inks of two or more colors out of the four colors of CMYK. Each color mixing portion is a strip-shaped portion extending in the transport direction AR. Each color mixture portion is a portion printed using two or more corresponding inks. That is, each color mixture portion contains dots of two or more corresponding inks and does not include dots of other inks that do not correspond to each other.

For example, the color mixing portion Pr is a portion corresponding to the inks of M and Y, and is a uniform region of red. The mixed color portion Pr contains approximately the same number of M and Y dots, and does not include C and K dots. The mixed color portion Pg is a portion corresponding to the Y and C inks, and is a uniform green region. The mixed color portion Pg contains approximately the same number of Y and C dots, and does not include K and M dots. The mixed color portion Pb is a portion corresponding to the inks of C and M, and is a uniform region of blue. The mixed color portion Pb contains approximately the same number of dots C and M, and does not include dots Y and K.

The mixed color portion Pbl is a portion corresponding to the CMYK ink and is a uniform region of black. The mixed color portion Pbl contains the same number of CMY dots, and further includes K dots. In this embodiment, the ratio of each CMY dot to all the dots included in the mixed color portion Pbl is (1/6), and the ratio of K dots is (1/2).

The length Ht (length in the Y direction) of the detection patch TP in the transport direction AR and the length Hi of the transport direction AR of the input patch IP are equal to each other.

As shown in FIG. 5A, the plurality of detection patch TPs and the plurality of input patch IPs have a one-to-one correspondence. For example, the four detection patches TP1, TP2, TP3, and TP4 in FIG. 5A correspond to the input patches IP1, IP2, IP3, and IP4, respectively. In the two corresponding patches, for example, the detection patch TP1 and the input patch IP1, the positions of the transport directions AR are equal to each other, and the positions in the main scanning direction are deviated from each other. For example, the input patch IP1 is located at a position separated from the detection patch TP1 by a distance DS in the + X direction.

The corresponding two patches are printed using the same nozzle NZ. For example, the detection patch TP1 and the input patch IP1 are printed using each CMYK nozzle group located in the hatched range RG1 of the head position P1 in FIG. 5 (A).

The position of the transport direction AR (Y direction) is shifted by ΔH between the two input patch IPs adjacent to each other in the main scanning direction (X direction). For example, the position of the input patch IP4 in FIG. 5A in the Y direction is deviated from the position of the input patch IP3 in the Y direction by ΔH in the −Y direction. ΔH is shorter than the length Hi of the input patch IP in the Y direction.

In S110, the CPU 210 acquires scan data indicating the test image TI printed using the printer 400. The reading resolution of the transport direction AR of the test image TI is higher than the print resolution of the transport direction AR of the test image TI (resolution based on the nozzle interval NT) in order to accurately detect defects in the nozzle NZ described later. Is preferable. For example, the operator inputs a reading instruction to the scanner 500 in a state where the paper S on which the test image TI is printed is placed on the platen of the scanner 500. The scanner 500 generates scan data indicating the test image TI by reading the paper S, and transmits the scan data to the data processing device 200. The CPU 210 of the data processing device 200 receives the scan data via the communication interface 270. Since the scanned image SI indicated by the scan data indicates the printed test image TI, it can be said that FIG. 5A is an image indicating the scanned image SI.

In S115, the CPU 210 uses the scan data to execute the nozzle clogging detection process. This detection process is a process of detecting whether or not a nozzle clogging problem has occurred in each nozzle used for printing the detection patch TP, and determining the pass / fail of the detection patch TP based on the detection result.

FIG. 6 is a flowchart of the nozzle clogging detection process. In S200, the CPU 210 analyzes the scan data to identify a plurality of detection patch TPs in the scan image SI. Since the content of the test image TI is known in advance, the approximate positions of the plurality of detection patch TPs in the scanned image SI are known in advance. However, there are variations in the positions of the plurality of detection patch TPs in the scanned image SI due to variations in the positions where the paper S is placed on the platen during scanning. For this purpose, the CPU 210 identifies a plurality of detection patch TPs by performing pattern matching within a range in which the detection patch TPs in the scanned image SI can be located.

In S210, the CPU 210 selects one attention detection patch from the plurality of specified detection patch TPs. In S220, the CPU 210 selects one ink of interest from the four color inks of CMYK.

In S230, the CPU 210 analyzes the monochromatic portion (also referred to as the monochromatic portion of interest) corresponding to the ink of interest among the monochromatic portions Pc, Pm, Py, and Pk of the four colors included in the attention detection patch, and converts the monochromatic portion of interest into the monochromatic portion of interest. Search for the included white streaks.

FIG. 7 is an explanatory diagram of a nozzle clogging detection process. FIG. 7 shows one detection patch TP and a part of the print head 110 at the head position P1 (FIG. 5) when printing the detection patch TP. In the print head 110 of FIG. 7, among the nozzles NZ indicated by circles, the nozzles NZxm and NZxx indicated by black circles are nozzles in which the nozzles are clogged. Since the defective nozzles NZxm and NZxx cannot eject ink, the dots to be formed by the nozzles NZxm and NZxx during printing are not formed. For example, in the example of FIG. 7, there is a nozzle NZxm having a defect in the nozzle row NM that ejects M ink. For this reason, a white streak WLm is present in the monochromatic portion Pm of M of the detection patch TP. The position of the transport direction AR of the white streak WLm is a position corresponding to the defective nozzle NZxm. The white streak WLm is a white line (color of paper S) extending over the entire length of the monochromatic portion Pm in the main scanning direction (X direction). Further, in the example of FIG. 7, there is a nozzle NZxx having a defect in the nozzle row NK for ejecting K ink. For this reason, a white streak WLk is present in the monochromatic portion Pk of K of the detection patch TP.

The CPU 210 searches for white streaks by determining whether or not they are white streaks at each position in the Y direction of the monochromatic portion of interest. For example, in the monochromatic portion Pm of M, the RGB value of the region different from the white streaks is a value close to the value indicating M (R, G, B) = (255, 0, 255). On the other hand, the RGB values of the white streaks are close to the values indicating white (R, G, B) = (255, 255, 255). Therefore, when the monochromatic portion of interest is the monochromatic portion Pm of M, the value (G value) of the G component is significantly different between the white streaks and the region different from the white streaks. Therefore, when the monochromatic portion of interest is the monochromatic portion Pm of M, the CPU 210 calculates the average of the G values for each position in the Y direction, and the average of the G values is equal to or greater than the threshold value (for example, 200). When there is a position in the Y direction, it is determined that there is a white streak at the position in the Y direction. When the monochromatic portion of interest is the monochromatic portion Pc and Py of C and Y, white streaks are similarly searched based on the R value and the B value, respectively. When the monochromatic portion of interest is the monochromatic portion Pk of K, the white streaks are similarly searched based on any of the R value, the G value, and the B value.

In S240, the CPU 210 determines whether or not the white streaks are present in the attention monochromatic portion as a result of the search. When a white streak is present in the attention monochromatic portion (S240: YES), in S260, the CPU 210 determines that the attention detection patch has failed. When there are no white streaks in the monochromatic portion of interest (S240: NO), in S250, the CPU 210 determines whether or not all the inks have been processed. If there is unprocessed ink (S250: NO), the CPU 210 returns to S220. When all the inks have been processed (S250: YES), in S270, the CPU 210 determines that the attention detection patch has passed. In this way, if there are no white streaks in all the monochromatic parts, the attention detection patch is determined to pass, and if there are white streaks in at least one monochromatic part, the attention detection patch fails. If there is, it will be half-handed.

In S280, the CPU 210 determines whether or not all the detection patches have been processed. If there is an unprocessed detection patch (S280: NO), the CPU 210 returns to S210. When all the detection patches have been processed (S280: YES), the CPU 210 ends the nozzle clogging detection process.

When the nozzle clogging detection process is completed, in S120 of FIG. 4, the CPU 210 executes the input image data acquisition process. The input image data acquisition process is a process of acquiring a plurality of input image data to be input to the machine learning model DN described above by using the scan data.

FIG. 8 is a flowchart of the input image data acquisition process. In S310, the CPU 210 selects one attention detection patch from the detection patches determined to pass by the above-mentioned detection process among the plurality of detection patch TPs of the scanned image SI.

In S320, the CPU 210 identifies one input patch IP corresponding to the attention detection patch. Specifically, the input patch IP located at a position separated by the distance DS in the main scanning direction (+ X direction in FIG. 5) from the attention detection patch is specified.

In S330, the CPU 210 acquires partial image data indicating the specified input patch IP from the scan data. For example, a cropping process for cutting out the input patch IP from the scanned image SI is executed, and partial image data is acquired.

In S340, the CPU 210 executes enlargement processing or reduction processing on the acquired partial image data to set the size of the partial image data to the size defined as the input image data Din (described above) of the machine learning model DN. Adjust to. In this embodiment, the size of the input image data Din of the machine learning model DN is a size of 200 pixels in the vertical direction and 200 pixels in the horizontal direction. The partial image data after the size is adjusted is stored in the non-volatile storage device 230 as input image data Din.

In S350, the CPU 210 determines whether or not all the detection patches determined to pass have been processed. If there is an unprocessed detection patch (S350: NO), the CPU 210 returns to S310. When all the detection patches have been processed (S350: YES), the CPU 210 ends the input image data acquisition process.

When the input image data acquisition process is completed, in S130 of FIG. 4, the CPU 210 inputs the plurality of input image data Dins acquired in S120 into the machine learning model DN, respectively. As a result, a plurality of output data Douts corresponding to the plurality of input image data Dins are generated. As described above, each output data Dout is two-dimensional data including a value Vo1 corresponding to a genuine ink set and a value Vo2 corresponding to a non-genuine ink set.

In S135, the CPU 210 uses a plurality of output data Douts to calculate the ratio Rd of the input image data Din determined to indicate the genuine ink set. Specifically, when the value Vo1 of the output data Dout corresponding to the input image data Din to be determined is equal to or higher than the threshold value TH1, it is determined that the input image data Din to be determined indicates a genuine ink set. When the value Vo1 of the corresponding output data Dout is less than the threshold value TH1, it is determined that the input image data Din to be determined indicates a non-genuine ink set. The value obtained by dividing the number of input image data bins determined to indicate the genuine ink set by the total number of input image data bins input to the machine learning model DN is calculated as the ratio Rd.

In S140, the CPU 210 determines whether or not the calculated ratio Rd is equal to or greater than the threshold value TH2. When the ratio Rd is equal to or higher than the threshold value TH2 (S140: YES), the CPU 210 specifies that the ink set of the printer 400 is a genuine ink set in S150. When the ratio Rd is less than the threshold value TH2 (S140: NO), in S145, the CPU 210 specifies that the ink set of the printer 400 is a non-genuine ink set.

A-4. Training of Machine Learning Model DN The machine learning model DN described above is pre-trained so that when the input image data Din is input, the desired output data Dout can be generated. The training of the machine learning model DN will be described below. Training of these machine learning model DNs is performed, for example, by a business operator who manufactures the printer 400.

FIG. 9 is a flowchart showing a training process of the machine learning model DN. In S410, the CPU 210 uses a genuine ink set to generate a plurality of input image data Din1, for example, thousands of input image data Din1. The input image data Din1 is generated by repeating S100 to S120 of FIG. 4 a plurality of times. At this time, in S105 of FIG. 4, the genuine ink set is mounted on the cartridge mounting portion 150 of the printer 400, and the test image TI is printed.

In S420, the CPU 210 uses a non-genuine ink set to generate a plurality of input image data Din2, for example, thousands of input image data Din2. The input image data Din2 is generated by repeating S100 to S120 of FIG. 4 a plurality of times. At this time, in S105 of FIG. 4, the non-genuine ink set is mounted on the cartridge mounting portion 150 of the printer 400, and the test image TI is printed. As a non-genuine ink set, one of the four color inks is a non-genuine ink set, and two of the four color inks are non-genuine inks. All patterns of the ink set in which the three colors are non-genuine ink and the ink set in which all the four color inks are non-genuine ink are used.

In S430, the CPU 210 generates teacher data corresponding to each of the generated plurality of input image data Din1 and Din2. The teacher data is ideal output data Dout that correctly indicates the attributes of the ink set used for printing the input patch IP indicated by the corresponding input image data Din1 and Din2. The input image data Din1 generated in S410 is associated with teacher data indicating a genuine ink set, and the input image data Din2 generated in S420 is associated with teacher data indicating a non-genuine ink set. .. The generated teacher data is stored in the non-volatile storage device 230 in association with the input image data Din1 and Din2.

In S435, the CPU 210 initializes a plurality of arithmetic parameters of the machine learning model DN. For example, the initial values of these arithmetic parameters are set to random numbers obtained independently from the same distribution (for example, normal distribution).

In S440, the CPU 210 selects input image data Din for a batch size from a plurality of input image data Din1 and Din2. For example, a plurality of input image data Din1 and Din2 are divided into a plurality of groups (batch) each containing V input image data (V is an integer of 2 or more, for example, V = 100). The CPU 210 selects V input image data Dins to be used by sequentially selecting one group from the plurality of groups. Instead of this, V input image data Dins may be randomly selected from a plurality of input image data Din1 and Din2 each time.

In S445, the CPU 210 inputs the selected V input image data Dins into the machine learning model DN to generate V output data Douts. The V output data Douts may include output data Dout1 corresponding to the input image data Din1 of the genuine ink set and output data Dout2 corresponding to the input image data Din2 of the non-genuine ink.

In S450, the CPU 210 calculates an error value EV between the output data Dout and the teacher data corresponding to the output data Dout for each of the V output data Douts. The error value EV is calculated based on a predetermined loss function. For example, a mean squared error (MSE) is used to calculate the error value EV.

In S455, the CPU 210 adjusts a plurality of arithmetic parameters of the machine learning model DN by using V error values EV. Specifically, the CPU 210 adjusts the calculation parameters according to a predetermined algorithm so that the error value EV becomes small, that is, the difference between the output data Dout and the teacher data becomes small. As the predetermined algorithm, for example, an algorithm using the backpropagation method and the gradient descent method (for example, adam) is used.

In S460, the CPU 210 determines whether or not the training has been completed. In this embodiment, it is determined that the training is completed when the completion instruction from the worker is input, and it is determined that the training is not completed when the training continuation instruction is input. For example, the CPU 210 inputs a plurality of test input image data Dins different from the input image data Din used for training into the machine learning model DN to generate a plurality of output data Douts. The operator evaluates the plurality of output data Douts to determine whether or not to end the training. The operator inputs a training completion instruction or a continuation instruction via the operation unit 240 according to the confirmation result. In the modified example, for example, when the processes of S440 to S455 are repeated a predetermined number of times, it may be determined that the training is completed.

If it is determined that the training has not been completed (S460: NO), the CPU 210 returns the process to S440. When it is determined that the training is completed (S460: YES), the CPU 210 ends the training of the machine learning model DN. At the end of training, the machine learning model DN is a trained model with adjusted arithmetic parameters. Therefore, it can be said that this training is a process of generating (manufacturing) a trained machine learning model DN.

In specifying the attributes of the ink set of this embodiment (FIG. 4), the printing process for printing the detection patch TP (S100 and S105 in FIG. 4) and the scan data indicating the detection patch TP using the image sensor of the scanner 500 are used. First image acquisition step (S110 in FIG. 4), a detection step (S115 in FIG. 4) for detecting defects in each nozzle NZ of the CMYK of the print head 110 using scan data, and defect detection results. The second image acquisition step (S100 to S110, S120 in FIG. 4) for acquiring the input image data Din indicating the input patch IP printed by the printer 400 and the acquired input image data Din are used. Then, a specific step (S130 to S150 in FIG. 4) of specifying the attributes of the ink set containing the CMYK ink is performed. The acquired input image data Din indicates an input patch IP printed using the nozzle NZ in which the CMYK defect has not occurred.

According to this embodiment, since the defects of the nozzle NZ of CMYK are detected by using the scan data indicating the detection patch TP, the defects of these nozzles NZ can be detected with high accuracy. Since the input image data Din indicating the input patch IP is acquired by using the detection result of the defect of these nozzles, the input image data Din indicating the input patch IP printed using the nozzle NZ in which the defect does not occur is appropriate. Can be obtained in. Then, since the attributes of the ink set are specified using the acquired input image data Din, for example, the attributes of the ink set in which the color represented by the dots of the inks of a plurality of colors is different are specified with high accuracy. be able to. Therefore, the attributes of the ink set used in the printer 400 can be appropriately specified.

It will be explained in more detail. For example, the input patch IP does not include a monochromatic portion printed with one color of ink. Therefore, even if a problem occurs in the nozzle NZ that prints the input patch IP, white streaks are unlikely to appear in the input patch IP. Therefore, even if an attempt is made to detect a defect in the nozzle NZ using scan data indicating the input patch IP, it is difficult to detect the defect with high accuracy. In this embodiment, since the scan data indicating the detection patch TP including the monochromatic portions Pc, Pm, Py, and Pk is used to detect the defects of the nozzle NZ of CMYK, the defects of these nozzles NZ can be detected with high accuracy.

Since the components of the solvent and additives and the contents of the components are different between the genuine ink and the non-genuine ink, the characteristics of the ink such as the viscosity are different. If the monochromatic color is different between the genuine ink and the non-genuine ink, the difference between the genuine ink and the non-genuine ink may appear in the detection patch TP including the monochromatic portion. However, there is no big difference in the color tone of a single color between the genuine ink and the non-genuine ink, and there are many cases where there is a difference in characteristics other than the color tone such as viscosity. In this case, for example, only a slight difference in shade appears in the monochromatic portion. Even in this case, differences are likely to appear in the mixed color portions Pr, Pg, Pb, and Pbl printed using inks of two or more colors. For example, it is assumed that there is a difference in the amount of ink ejected from the nozzle NZ between genuine ink and non-genuine ink due to a difference in characteristics such as viscosity. In this case, for example, between the mixed color portion Pr printed using the genuine inks of M and Y and the mixed color portion Pr printed using the genuine ink of M and the non-genuine ink of C, Since the ratio of the area of C dots to the area of M dots is different, a difference appears in the overall color. Further, the genuine ink and the non-genuine ink may have different ways of mixing and spreading the ink when the dots of two colors overlap each other. Even in such a case, a difference that does not appear in the single color portion may appear in the mixed color portion. As described above, in the mixed color portion, in addition to the difference in the monochromatic tint between the genuine ink and the non-genuine ink, a difference due to a characteristic different from the tint appears. In this embodiment, since the attributes of the ink set are specified by using the input image data Din indicating the input patch IP including the mixed color portions Pr, Pg, Pb, and Plb, the attributes of the ink set are specified. Even when there is no difference in color, the attributes of the ink set can be accurately specified based on the difference caused by other characteristics.

However, if a nozzle NZ having a problem when printing the input patch IP is used, the dots to be formed by the nozzle NZ are not formed in the input patch IP. As a result, in the input patch IP, the difference due to the defect of the nozzle NZ and the difference between the genuine ink and the non-genuine ink are mixed. Since it is difficult to distinguish the difference caused by the defect of the nozzle NZ and the difference between the genuine ink and the non-genuine ink and specify the attribute of the ink set, in this case, the input indicating the input patch IP is input. Even if the image data Din is used, the accuracy of specifying the attributes of the ink set is lowered. In this embodiment, the defect of the nozzle NZ is detected with high accuracy, and the input image data Din indicating the input patch IP printed using the nozzle NZ in which the defect has not occurred is acquired. Therefore, the attributes of the ink set can be accurately specified by using the input image data DIN.

Further, according to this embodiment, the detection patch TP and the input patch IP are printed on one sheet S (FIG. 5). If the printing of the detection patch TP and the printing of the input patch IP are printed on different papers, the nozzle NZ will be printed between the time of printing the detection patch TP and the time of printing the input patch IP. The status of occurrence of defects may be different. In this case, there is a possibility that the input image data Din indicating the input patch IP printed using the nozzle NZ in which the defect has not occurred cannot be appropriately acquired. In this embodiment, such inconvenience can be suppressed, and the input image data Din indicating the input patch IP printed using the nozzle NZ in which no defect has occurred can be appropriately acquired.

Further, according to this embodiment, a plurality of input patch IPs are printed on the paper S (FIG. 5A), and the printed input patch IPs are printed using the detection result of the defect of the nozzle NZ. The input patch IP to be acquired is selected from the above (S310 to S320 in FIG. 8). Then, the input image data Din indicating one or more selected input patch IPs is acquired (S330, S340 in FIG. 8). As a result, it is possible to appropriately suppress the acquisition of the input image data Din indicating the input patch IP printed by using the defective nozzle NZ. Further, after the defect of the nozzle NZ is detected, the number of times of printing and reading required can be reduced as compared with the case of printing the input patch IP by using the nozzle NZ in which the defect has not occurred.

Further, according to the present embodiment, a plurality of detection patch TPs are printed on the paper S, and the plurality of input patch IPs are each associated with any of the plurality of detection patch TPs (FIG. 5). (A)). Then, in the input image data acquisition process (FIG. 8), the detection patch TP printed using the nozzle NZ in which no defect has occurred (detection determined to pass) from the plurality of printed detection patch TPs. Patch TP) is selected (S310 in FIG. 8), the input patch IP associated with the selected detection patch TP is identified (S320 in FIG. 8), and the input image data Din indicating the identified input patch IP is It is acquired (S330, S340 in FIG. 8). As a result, it is possible to easily acquire the input image data Din indicating the input patch IP printed using the nozzle NZ in which no defect has occurred.

Further, in this embodiment, in the test image TI, the plurality of input patch IPs are printed using the nozzles used to print the associated detection patch TPs. That is, the input patch IP and the detection patch TP corresponding to each other are patches printed using the same nozzle NZ. As a result, it is possible to appropriately acquire the input image data Din indicating the input patch IP printed using the nozzle NZ in which no defect has occurred.

Further, in the above embodiment, on the paper S, each input patch IP is printed at a position separated from the associated detection patch TP by a specific distance DS (FIG. 5 (A)) in the main scanning direction (FIG. 5). (A)). In the acquisition process of the input image data Din, the scan image SI is located at a position separated by a specific distance in the main scanning direction (+ X direction in FIG. 5A) from the detection patch TP selected using the defect detection result. The input patch IP is identified. As a result, appropriate input image data can be easily obtained from the scan data indicating one sheet S.

Further, in the above embodiment, as shown in FIG. 5 (A), the position of the printed input patch IP in the transport direction AR (Y direction in FIG. 5 (A)) is the transport direction of the input patch IP. It is deviated by a length ΔH shorter than the AR length Hi. As a result, a large number of input patch IPs can be prepared for the nozzle group of the print head 110. Therefore, for example, even when the number of defective nozzles NZ is relatively large, the probability that the input image data Din indicating the input patch IP printed using the defective nozzles NZ can be obtained can be obtained. Can be improved.

Further, in the above embodiment, the detection patch TP includes four monochromatic portions Pc, Pm, Py, and Pk that correspond one-to-one with the four color inks of CMYK used for printing (FIG. 5 (B)). Each of the four monochromatic portions Pc, Pm, Py, and Pk is a portion containing dots of the corresponding one color ink of the four color inks and not including dots of the uncorresponding three color inks. Then, the input patch IP includes four mixed color portions Pr, Pg, Pb, and Pbl, each of which corresponds to two or more color inks of the four color inks (FIG. 5 (C)). Each of the four color-mixed portions Pr, Pg, Pb, and Pbl is a portion containing dots of two or more corresponding inks of the four color inks and not including dots of inks of uncorresponding colors. Each of the four color inks of CMYK corresponds to at least one of M four mixed color portions Pr, Pg, Pb, and Pbl. As a result, when at least one of the four color inks of CMYK is a non-genuine ink, the feature due to the characteristics of the non-genuine ink becomes at least one of the four mixed color portions Pr, Pg, Pb, and Pbl. appear. Therefore, the attributes of the ink set composed of the four colors of CMYK ink can be appropriately specified.

Further, in the above embodiment, the four-color ink used for printing includes K ink, which is an achromatic ink, and three-color ink (CMYK ink), which is a chromatic ink. The detection patch TP includes a monochromatic portion Pk that expresses an achromatic color using dots of K ink (FIG. 5 (B)), and the input patch IP contains achromatic K ink and chromatic CMY ink. It includes a mixed color portion Pbl that is used to express an achromatic color (FIG. 5 (C)). For example, by analyzing the monochromatic portion Pk, it is possible to accurately determine whether or not a defect has occurred in the nozzle NZ that ejects K ink. In addition, if the CMYK ink contains non-genuine ink, differences such as the color tone of the mixed color portion Pbl deviating from the achromatic color appear in the input patch IP, so it is necessary to appropriately specify the attributes of the ink set. can. Therefore, the attributes of the ink set including the achromatic ink and the two or more chromatic inks can be appropriately specified.

Further, in the above embodiment, the input image data Din is input to the machine learning model DN, and the attributes of the ink set are specified based on the output data Dout of the machine learning model DN (S130 to S150 in FIG. 4). As a result, the attributes of the ink set can be appropriately specified using the machine learning model DN. For example, by visual inspection or a method that does not use the machine learning model DN (for example, histogram analysis), the attributes of the ink set can be specified even when it cannot be determined whether or not the ink set contains non-genuine ink.

Further, in the training process (FIG. 9) of the above embodiment, the training of the machine learning model DN is executed using the input image data Din acquired in the same manner as when the attributes of the ink set of FIG. 4 are specified. (S410, S420 in FIG. 9). As a result, the machine learning model DN can be trained using the input image data Din indicating the input patch IP printed using the nozzle NZ in which no defect has occurred. That is, the machine learning model DN can be trained by using the input image data Din indicating the input patch IP including the difference between the non-genuine ink and the genuine ink without including the difference due to the defect of the nozzle NZ. Therefore, the machine learning model DN can be trained so that the attributes of the ink set can be identified accurately.

As can be seen from the above description, for example, the C ink of this embodiment is an example of the first color ink, the M ink is an example of the second color ink, and the nozzle row NC is the first nozzle. An example of a group, the nozzle row NM is an example of a second nozzle group. The detection patch TP is an example of the first image, and the input patch IP is an example of the second image. Among the scan data, the data indicating the detection patch TP is an example of the first image data, and the input image data Din is an example of the second image data.

B. Modification example:
(1) The test image TI (FIG. 5 (A)) of the above embodiment is an example and is not limited thereto. FIG. 10 is a diagram showing an example of a test image TIb of a modified example. The test image TIb of FIG. 10 contains one detection patch TPb instead of the plurality of detection patch TPs (FIG. 5 (A)). Other configurations of the test image TIb are the same as those of the test image TI of FIG. 5 (A).

The detection patch TPb of FIG. 10 is a strip-shaped image extending in the transport direction AR, and is printed by using all the nozzles NZ of the print head 110 at the head position P1. Therefore, the length of the detection patch TPb in the transport direction AR is substantially equal to the nozzle length D (FIG. 2B). Similar to the detection patch TP of FIG. 5B, the detection patch TPb includes four monochromatic portions (not shown) corresponding to the ink of CMYK, and each monochromatic portion has a transport direction AR of the detection patch TPb. It is a band-shaped part that extends over the entire length of the ink.

When the test image TIb of FIG. 10 is used, the CPU 210 searches for white streaks in each monochromatic portion of the detection patch TPb in S115 of FIG. Detects whether or not. When the white streaks are detected, the CPU 210 stores the coordinates (Y coordinates) of the detected white streaks in the transport direction AR. In S120 of FIG. 4, the CPU 210 excludes the input patch IP arranged at the coordinates of the transport direction AR where the white streaks are located from the acquisition target, and arranges them at coordinates different from the coordinates of the transport direction AR where the white streaks are located. The input patch IP that has been specified is specified as the acquisition target. The CPU 210 acquires the input image data Din indicating the input patch IP to be acquired, and uses the input image data Din to specify the attribute of the ink set.

As in this modification, the detection patch and the input patch do not have to be associated with each other on a one-to-one basis. Further, the detection patch and the input patch do not have to be printed using exactly the same nozzle NZ. For example, as shown in FIG. 10, the input patch IP may be printed using some nozzles NZ among the nozzles NZ used for printing the detection patch TPb.

Further, in the test image TIb of FIG. 10, instead of the plurality of input patch IPs, one strip-shaped input image extending in the transport direction AR may be arranged as in the detection patch TPb. This one strip-shaped input image is printed using all the nozzles NZ of the print head 110 at the head position P1. As with the input patch IP of FIG. 5C, one strip-shaped input image includes four color mixing portions (not shown), and each color mixing portion covers the entire length of the input image in the transport direction AR. It is a strip-shaped part that extends over. In this case, the CPU 210 crops a rectangular input patch from one strip-shaped input image so as not to include the coordinates of the transport direction AR where the white streaks are located, and the input image data Din indicating the input patch. To get.

(2) In the above embodiment, the plurality of detection patch TPs and the plurality of input patch IPs are printed on one sheet S. Not limited to this, the plurality of detection patch TPs may be printed on the first paper, and the plurality of input patch IPs may be printed on the second paper. In this case, for example, the plurality of input patch IPs are each printed on the second paper using the nozzle NZ used when the corresponding detection patch TP was printed on the first paper. .. In this case, the CPU 210 acquires the first scan data indicating the detection patch TP and the second scan data indicating the input patch IP. The CPU 210 uses the first scan data to execute the nozzle clogging detection process of S115 in FIG. 4, and determines whether each detection patch TP passes or fails. The CPU 210 acquires the input image data Din indicating the input patch IP corresponding to the detection patch TP determined to pass from the second scan data.

(3) Further, the CPU 210 may detect the defective nozzle NZ by using the scan data showing only the band-shaped detection patch TPb shown in FIG. In this case, the CPU 210 prints the print data so as to print a plurality of input patch IPs using only the nozzle NZ in which the defect has not occurred, without using the nozzle NZ in which the detected defect has occurred. Generate. The CPU 210 prints a plurality of input patch IPs using the print data, and causes the scanner 500 to read the printed plurality of input patch IPs to generate scan data. The CPU 210 acquires a plurality of input image data Dins indicating a plurality of input patch IPs from the scan data.

(4) In the test image TI of FIG. 5A, the positions of the transport direction ARs of the plurality of input patch IPs are deviated by a length shorter than the length Hi of the transport direction ARs of the input patch IPs. Instead of this, the positions of the transport direction ARs of the plurality of input patch IPs may be deviated by a length equal to or greater than the length Hi of the transport direction ARs of the input patch IPs. In this case, the positions of the plurality of input patch IPs in the X direction may be the same as each other. The same applies to the positions of the transport direction ARs of the plurality of detection patch TPs.

The ink set used in the printer 400 of the above embodiment consists of four colors of CMYK ink. Not limited to this, the ink set used in the printer 400 may include only three color inks of CMY, or may contain five color inks including light cyan and light magenta in addition to CMYK. In general, the printer 400 is preferably a device that prints using ink of N colors (N is an integer of 3 or more). The detection patch TP preferably includes N monochromatic portions that correspond one-to-one with N-color inks. The input patch IP preferably includes M monochromatic portions (M is an integer of 2 or more) corresponding to two or more color inks of the N color inks.

(5) In the training process (FIG. 10) of the above embodiment, the machine learning model DN uses the input image data groups of two types of classes, the genuine ink set and the non-genuine ink set, to perform the two types of classes. Trained to identify. Not limited to this, the machine learning model DN may be trained to identify the three or more classes by using the input image data group of three or more classes. The three or more classes may be, for example, three or more ink sets from different manufacturers. In this case, in specifying the attributes of the ink set of FIG. 4, the attributes of the target ink set may be specified from three or more types of classes.

(6) In the above embodiment, the print medium on which the test image TI is printed is paper S, but the present invention is not limited to this. For example, when there is another printing medium in which the difference in ink characteristics is likely to appear, another printing medium (for example, a sheet or plate made of resin, glass, or the like) may be used.

(7) In the above embodiment, the image data showing the printed test image TI is scan data generated by using the scanner 500. Instead, the image data indicating the printed test image TI may be captured image data obtained by photographing the paper S using a digital camera provided with a two-dimensional image sensor.

(8) The configuration of the machine learning model DN (FIG. 3) of the above embodiment is an example, and is not limited to this. For example, in the machine learning model DN, the number of layers of the convolution layer and the fully connected layer may be changed as appropriate. In addition, the post-processing executed for the values output in each layer of the machine learning model DN can be changed as appropriate. For example, as the activation function used for post-processing, any function such as ReLU, LeakyReLU, PRELU, Softmax, and Sigmoid can be used. In addition, processes such as batch normalization and dropout can be added or omitted as post-processing as appropriate.

(9) The training of the machine learning model DN of the above embodiment (FIG. 10) is an example and can be changed as appropriate. For example, the mean square error is used for the error value EV, but other types of error values may be used instead. For example, a cross entropy error or a mean absolute error may be used for the error value EV.

(10) The hardware configuration of the data processing device 200 of FIG. 1 is an example, and is not limited thereto. For example, the processor of the data processing device 200 is not limited to the CPU, but may be a GPU (Graphics Processing Unit), an ASIC (application specific integrated circuit), or a combination of these and a CPU. Further, the data processing device 200 may be a plurality of computers (for example, a so-called cloud server) capable of communicating with each other via a network.

(10) In each of the above embodiments, a part of the configuration realized by the hardware may be replaced with software, and conversely, a part or all of the configuration realized by the software may be replaced with the hardware. You may do so. For example, the machine learning model DN may be realized by a hardware circuit such as an ASIC (Application Specific Integrated Circuit) instead of the program module.

Although the present invention has been described above based on Examples and Modifications, the above-described embodiments of the invention are for facilitating the understanding of the present invention and do not limit the present invention. The present invention can be modified and improved without departing from the spirit and claims, and the present invention includes equivalents thereof.

100 ... Printing mechanism, 1000 ... Ink identification system, 110 ... Printing head, 111 ... Nozzle forming surface, 120 ... Head drive unit, 130 ... Main scanning unit, 133 ... Carriage, 134 ... Sliding shaft, 140 ... Conveying unit, 141 ... Downstream roller pair, 142 ... Upstream roller pair, 145 ... Paper stand, 150 ... Cartridge mounting unit, 200 ... Data processing device, 210 ... CPU, 220 ... Volatile storage device, 230 ... Non-volatile storage device, 240 ... Operation unit , 250 ... Display unit, 270 ... Communication interface, 305 ... Input layer, 310 ... First convolution layer, 320 ... First pooling layer, 330 ... Second convolution layer, 340 ... Second pooling layer, 350 ... First Fully bonded layer, 360 ... 2nd fully bonded layer, 370 ... 3rd fully bonded layer, 400 ... Printer, 410 ... Control device, 420C, 420M, 430Y, 420K ... Ink cartridge, 500 ... Scanner, AR ... Transport direction, DS ... Distance, DN ... Machine learning model, Din ... Input image data, Dout ... Output data, EV ... Error value, IP ... Input patch, NC, NM, NY, NK ... Nozzle train, NZ ... Nozzle, PG ... Computer program, Pr, Pg, Pb, Pbl ... mixed color part, Pc, Pm, Py, Pk ... monochromatic part, S ... paper, SI ... scanned image, TD ... test image data, TI, TIb ... test image, TP, TPb ... detection patch

Claims (12)

The first image is printed using a printing apparatus including a printing head having a first nozzle group for ejecting ink of the first color and a second nozzle group for ejecting ink of a second color different from the first color. In the printing step, the first image includes a first portion containing the dots of the ink of the first color and not including the dots of the ink of the second color, and the dots of the ink of the second color. The printing process, which includes a second portion of the first color ink that does not contain dots.
A first image acquisition step of acquiring first image data indicating the printed first image using an image sensor, and
A detection step of detecting a defect of a nozzle of the first nozzle group and a defect of a nozzle of the second nozzle group using the first image data.
A second image acquisition step of acquiring second image data indicating a second image printed by the printing apparatus using the defect detection result, wherein the second image is of the first color. It is an image including dots of ink and dots of ink of the second color, the second image data is acquired by using an image sensor, and the acquired second image data is of the first nozzle group. The second image acquisition step of showing the second image printed by using the nozzle without the defect and the nozzle without the defect of the second nozzle group.
Using the acquired second image data, a specific step of specifying the attributes of the ink set including the first color ink and the second color ink, and
A method of identifying the attributes of an ink set. The specific method according to claim 1.
A specific method in which the first image and the second image are printed on one print medium. The specific method according to claim 1 or 2.
In the printing step, a plurality of the second images are further printed.
In the second image acquisition step,
Using the defect detection result, one or more of the second images are selected from the plurality of printed second images.
A specific method for acquiring the second image data showing the selected one or more second images. The specific method according to claim 3.
In the printing step, a plurality of the first images are printed.
Each of the plurality of second images is associated with any of the plurality of first images.
In the second image acquisition step,
From the plurality of printed first images using the defect detection result, the nozzle in which the defect does not occur in the first nozzle group and the nozzle in which the defect does not occur in the second nozzle group. Select one or more of the first images printed using and
Identify one or more of the second images associated with the selected one or more first images.
A specific method for acquiring the second image data showing the specified one or more second images. The specific method according to claim 4.
The plurality of second images are the nozzle used for printing the first portion of the associated first image among the plurality of nozzles included in the first nozzle group, and the first nozzle, respectively. A specific method of printing using, among a plurality of nozzles included in a group of two nozzles, the nozzle used for printing the second portion of the first image associated with the nozzle. The specific method according to claim 5.
The plurality of nozzles included in the first nozzle group have different positions in the first direction from each other.
The plurality of nozzles included in the second nozzle group have different positions in the first direction from each other.
The plurality of first images and the plurality of second images are printed on one printing medium.
In the one printing medium, the second image is printed at a position separated by a specific distance in the second direction intersecting the first direction from the position of the associated first image.
In the first image acquisition step, an image sensor is used to acquire specific image data indicating an image including the one print medium on which the plurality of first images and the plurality of second images are printed. ,
In the detection step, the defect of the nozzle of the first nozzle group and the defect of the nozzle of the second nozzle group are detected by using the first image data which is a part of the specific image data.
In the second image acquisition step,
In the image including the one print medium indicated by the specific image data, the image is located at a position separated by the specific distance in the second direction from the one or more first images selected by using the defect detection result. Identify one or more of the second images
A specific method for acquiring the second image data showing the specified one or more second images from the specific image data. The specific method according to any one of claims 3 to 6.
The plurality of nozzles included in the first nozzle group have different positions in the first direction from each other.
The plurality of nozzles included in the second nozzle group have different positions in the first direction from each other.
A specific method in which the positions of the plurality of printed second images in the first direction are deviated by a specific length shorter than the length of the second image in the first direction. The specific method according to any one of claims 1 to 7.
The printing device is a device that prints using N color ink (N is an integer of 3 or more) including the first color ink and the second color ink.
The print head is an N nozzle group including the first nozzle group and the second nozzle group, and has N nozzle groups for ejecting the N color ink.
The first image includes N parts including the first part and the second part, and includes the N parts corresponding to the N color ink on a one-to-one basis.
Each of the N portions is a portion containing dots of the corresponding one color ink of the N color inks and not including dots of the uncorresponding (N-1) color ink.
The second image includes M parts (M is an integer of 2 or more) corresponding to two or more colors of the N color inks.
Each of the M portions is a portion containing dots of two or more colors of the corresponding inks of the N colors of ink and not including dots of inks of uncorresponding colors.
A specific method in which each of the N color inks corresponds to at least one of the M portions. The specific method according to claim 8.
The N-color ink includes an achromatic ink and two or more chromatic inks.
The N parts of the first image include a part for expressing an achromatic color using the dots of the achromatic ink.
A specific method, wherein the M portions of the second image include a portion that expresses an achromatic color using the achromatic ink and two or more chromatic inks. The specific method according to any one of claims 1 to 9.
In the specific step, a specific method in which the second image data is input to a machine learning model and the attributes of the ink set are specified based on the output data of the machine learning model. A first image acquisition unit that acquires first image data indicating a first image printed using a printing device. The printing device includes a first nozzle group for ejecting first color ink and the first image. A print head having a second nozzle group for ejecting ink of a second color different from the color is provided, and the first image includes dots of the ink of the first color and dots of the ink of the second color. The first image data is generated by using an image sensor, including a first portion that does not include the first portion and a second portion that includes dots of the ink of the second color and does not include dots of the ink of the first color. The first image acquisition unit, which is the data to be printed, and
Using the first image data, a detection unit that detects a defect of the nozzle of the first nozzle group and a defect of the nozzle of the second nozzle group, and
A second image acquisition unit that acquires second image data indicating a second image printed by the printing apparatus using the defect detection result, wherein the second image is of the first color. It is an image including dots of ink and dots of ink of the second color, the second image data is acquired by using an image sensor, and the acquired second image data is of the first nozzle group. The second image printed by using the nozzle which did not have the defect and the nozzle which did not have the defect of the second nozzle group is shown, and the ink of the first color and the ink of the second color are mixed. The second image acquisition unit used to identify the attributes of the included ink set, and
An image processing device. A first image acquisition unit that acquires first image data indicating a first image printed using a printing device. The printing device includes a first nozzle group for ejecting first color ink and the first image. A print head having a second nozzle group for ejecting ink of a second color different from the color is provided, and the first image includes dots of the ink of the first color and dots of the ink of the second color. The first image data is generated by using an image sensor, including a first portion that does not include the first portion and a second portion that includes dots of the ink of the second color and does not include dots of the ink of the first color. The first image acquisition function, which is the data to be printed, and
Using the first image data, a detection function for detecting a defect of a nozzle of the first nozzle group and a defect of a nozzle of the second nozzle group, and
It is a second image acquisition function that acquires a second image data showing a mixed color image printed by the printing apparatus by using the defect detection result, and the second image is the ink of the first color. The second image data is an image including the dots of the above and the dots of the second color ink, the second image data is acquired by using an image sensor, and the acquired second image data is the said of the first nozzle group. The second image printed by using the non-defect-free nozzle and the non-defect-free nozzle of the second nozzle group is shown, and includes the first color ink and the second color ink. The second image acquisition function used to identify the attributes of the ink set, and
A computer program that makes a computer realize.

Images