CN111397539A - Multi-view visual detection system and method for ink-jet printing - Google Patents

Multi-view visual detection system and method for ink-jet printing Download PDF

Info

Publication number
CN111397539A
CN111397539A CN202010232618.6A CN202010232618A CN111397539A CN 111397539 A CN111397539 A CN 111397539A CN 202010232618 A CN202010232618 A CN 202010232618A CN 111397539 A CN111397539 A CN 111397539A
Authority
CN
China
Prior art keywords
ink
angle
camera
substrate
drop
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010232618.6A
Other languages
Chinese (zh)
Other versions
CN111397539B (en
Inventor
尹周平
陈建魁
欧闻
刘强强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN202010232618.6A priority Critical patent/CN111397539B/en
Publication of CN111397539A publication Critical patent/CN111397539A/en
Application granted granted Critical
Publication of CN111397539B publication Critical patent/CN111397539B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • G01B11/26Measuring arrangements characterised by the use of optical techniques for measuring angles or tapers; for testing the alignment of axes
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B41PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
    • B41JTYPEWRITERS; SELECTIVE PRINTING MECHANISMS, i.e. MECHANISMS PRINTING OTHERWISE THAN FROM A FORME; CORRECTION OF TYPOGRAPHICAL ERRORS
    • B41J2/00Typewriters or selective printing mechanisms characterised by the printing or marking process for which they are designed
    • B41J2/005Typewriters or selective printing mechanisms characterised by the printing or marking process for which they are designed characterised by bringing liquid or particles selectively into contact with a printing material
    • B41J2/01Ink jet
    • B41J2/07Ink jet characterised by jet control
    • B41J2/12Ink jet characterised by jet control testing or correcting charge or deflection
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B41PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
    • B41JTYPEWRITERS; SELECTIVE PRINTING MECHANISMS, i.e. MECHANISMS PRINTING OTHERWISE THAN FROM A FORME; CORRECTION OF TYPOGRAPHICAL ERRORS
    • B41J29/00Details of, or accessories for, typewriters or selective printing mechanisms not otherwise provided for
    • B41J29/38Drives, motors, controls or automatic cut-off devices for the entire printing mechanism
    • B41J29/393Devices for controlling or analysing the entire machine ; Controlling or analysing mechanical parameters involving printing of test patterns
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • G01B11/02Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N2015/0003Determining electric mobility, velocity profile, average speed or velocity of a plurality of particles

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Dispersion Chemistry (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Ink Jet (AREA)

Abstract

The invention belongs to the technical field related to inkjet printing visual inspection, and discloses a multi-view visual inspection system and method for inkjet printing, wherein the method mainly comprises the following steps: (1) firstly, detecting and determining the installation angle of a spray head; then, carrying out trial printing to detect and determine the volume, the speed and the angle of the flying ink drop; finally, detecting and determining the deflection angle of the jet hole array, the position and the angle of the substrate and the offset distance of the drop point of the ink drop; (2) and performing ink jet printing according to the obtained parameters, and performing liquid film detection on the printed substrate by adopting a predefined defect type after the substrate is printed. The invention can detect various control parameters required by the whole process of ink-jet printing, can effectively make up for the defect that the existing detection method of ink-jet printing can only detect a single module, and realizes the detection of the whole process of ink-jet printing with higher precision and higher efficiency.

Description

Multi-view visual detection system and method for ink-jet printing
Technical Field
The invention belongs to the technical field related to inkjet printing visual inspection, and particularly relates to a multi-view visual inspection system and method for inkjet printing.
Background
The ink-jet printing technology is applied to the fields of manufacturing display screens, flexible sensors and the like, and compared with the traditional printing technology, the ink-jet printing technology belongs to non-contact printing, and the alignment precision can reach the submicron level, so that the ink-jet printing technology is a high-efficiency, low-cost, green and friendly electronic manufacturing technology, and has the advantages of simple process, high printing resolution, less material waste and the like. In the ink-jet printing process, because the volume of the ink drop and the pixel pit area are both small, the requirement on the detection precision of the whole visual detection system is extremely high, the ink-jet printing technology is used for spray printing according to needs, the ink drop is sprayed to form a patterning device or a uniform and compact packaging layer, the positioning of a spray head and a substrate in the printing process, the measurement and control of the ink drop speed and the volume and the detection of a liquid film after the printing are all greatly influenced on the final spray printing effect, and therefore, a set of whole detection method is needed.
Some visual detection systems and methods related to the inkjet printing process have been proposed in the prior art, such as a visual detection device and method for detecting flying ink droplets, which are used to detect parameters such as the speed and volume of ink droplets ejected by a nozzle, a positioning method for nozzles of an inkjet printing device, which is used to accurately position the position of the nozzle, and a method for detecting a liquid film for inkjet printing, which is used to detect the thickness and defects of the liquid film after printing is completed, but these methods are all single modules, and have single functions and poor applicability.
Disclosure of Invention
Aiming at the defects or improvement requirements of the prior art, the invention provides a multi-view visual detection system and method for ink-jet printing, which can detect various control parameters required by the whole process of ink-jet printing, can effectively make up the defect that only a single module can be detected in the existing detection method of ink-jet printing, and realize the detection of the whole process of ink-jet printing with higher precision and higher efficiency.
To achieve the above object, according to one aspect of the present invention, there is provided a multi-view visual inspection method for inkjet printing, the multi-view visual inspection method mainly comprising the steps of:
(1) firstly, detecting and determining the installation angle of a spray head; then, carrying out trial printing to detect and determine the volume, the speed and the angle of the flying ink drop; finally, detecting and determining the deflection angle of the jet hole array, the position and the angle of the substrate and the offset distance of the drop point of the ink drop;
(2) and performing ink jet printing according to the obtained parameters, and performing liquid film detection on the printed substrate by adopting a predefined defect type after the substrate is printed.
Further, the detection of the installation angle of the spray head comprises the following steps: firstly, two positioning marks at the diagonal position below the spray head module are detected to obtain the centroid position coordinates (x) of the positioning marks1,y1) And (x)2,y2) (ii) a Then, the angle theta between the line formed by the centroids of the two marks and the horizontal axis is determinedjAnd the manufacturing included angle theta between the two positioning marks and the central line of the spray headmTo calculate the installation angle of the sprinkler:
Figure BDA0002429784510000021
in the formula, thetarotateIndicating the actual deflection angle of the sprinkler installation, with the sign representing either clockwise or counterclockwise rotation.
Further, the detection of the deflection angle of the jet hole array comprises the following steps: firstly, detecting the centers P of a plurality of spray holesiCoordinate (x) ofPi,yPi) Wherein i is 1,2, 3.; these points are then straightened using a least squares methodLine fitting to obtain a straight line L2
Figure BDA0002429784510000022
Order:
Figure BDA0002429784510000023
Figure BDA0002429784510000031
determining the parameters k and b of the straight line and the angle theta between the straight line and the horizontal lineh
Figure BDA0002429784510000032
Figure BDA0002429784510000033
Figure BDA0002429784510000034
Wherein N represents the number of all observed orifices, xpi、ypiRespectively represent the abscissa and ordinate of the i-th injection hole, k represents the slope of the fitted straight line, b represents the intercept of the fitted straight line, and thetahRepresenting the angle between the line and the horizontal axis.
Further, the detection of the volume of the flying ink drop comprises the steps of: detecting the projection images of the flying ink drop in the two camera directions respectively, fitting an ellipse, and then solving the height h of the flying ink drop and the maximum widths a and b in the two projection planes respectively; dispersing the flying ink drops into n slices with the thickness of delta h along the vertical direction, and fitting the cross section of each slice into an ellipse; finally, each of the ink droplets is calculated based on the maximum width a, b of the ink droplet projection and the ellipse to which the projection image of the ink droplet in the camera direction is fittedMajor and minor axes a of an ellipse in a slice planei、biWhere i is 1,2, …, n, the drop volume is calculated using the following equation:
Figure BDA0002429784510000035
wherein V represents the volume of the ink drop to be measured, h represents the height of the observed ink drop, S represents the cross-sectional area of the ink drop slice, ai、biThe major axis and the minor axis of the ellipse in the i-th cross section are shown, respectively, and n represents the number of ink droplet slices in the height direction.
Further, the detection of the velocity and angle of the flying ink drop comprises the following steps: two exposures are carried out in a preset time interval, and the flying ink drop is formed by P in the time interval delta t of two exposures of the ink drop observation camera6Point movement to P7A point, and P7The projection points of the points in the projection plane of the two cameras are respectively P7xAnd P7yThe height of the drop of the ink drop in the time t is calculated to be delta z through image processing, and when the sampling interval time delta t approaches to 0, the instantaneous speed and the angle of the ink drop are respectively calculated as follows:
Figure BDA0002429784510000041
Figure BDA0002429784510000042
wherein v isx、vy、vzRepresenting the velocity components, theta, of the ink drop in the x, y, and z directions, respectivelyx、θzRepresenting the angles of flight of the ink drops in the horizontal and vertical planes, respectively.
Further, the detection of the position and the angle of the substrate comprises the following steps: the three corners of the substrate are respectively provided with a cross positioning mark, firstly, a downward-looking high power camera is adopted to respectively detect the three positioning marks, and the central positions of the three cross positioning marks obtained through image processing calculation are respectively (x)s1,ys1)、(xs2,ys2)、(xs3,ys3) And if the first positioning mark and the third positioning mark are positioned on the diagonal line of the substrate, the position and the deflection angle of the central point of the substrate are calculated as follows:
Figure BDA0002429784510000043
Figure BDA0002429784510000044
wherein (x)s,ys) Position coordinates, theta, representing the center point of the substratesIndicating the deflection angle of the substrate.
Further, the detection of the drop landing offset distance comprises the following steps: first, coordinates (x) of expected landing positions of a plurality of ink droplets are preset by a headesti,yesti) The method includes the steps of firstly, obtaining a plurality of actual drop point position coordinates (x), wherein i is 1,2,3,4reai,yreai) Wherein, i is 1,2,3, 4.
Figure BDA0002429784510000045
Wherein x isbiasDenotes the average offset distance in the x-direction, ybiasRepresents the average offset distance in the y-direction and N represents the number of all observed actual landings.
Further, when liquid film detection is carried out, the whole substrate is divided into a plurality of rectangular grids, and each grid internally comprises a pixel pit; in image processing, each grid represents an ROI, an edge extraction operation is performed on a droplet in each ROI, and then the defect type of a pixel pit is determined by the droplet edge position.
According to another aspect of the present invention, a multi-view vision inspection system for inkjet printing is provided, which performs vision printing process and defect inspection by using the multi-view vision inspection method for inkjet printing as described above, and includes an upper view vision observation module, a lower view vision observation module, an ink droplet observation module, a liquid film observation module, and a nozzle control module, wherein the upper view vision observation module, the lower view vision detection module, the ink droplet observation module, and the liquid film observation module are respectively directly or indirectly connected to the nozzle control module;
the downward-looking visual observation module is used for collecting a substrate positioning mark image so as to determine the position and the angle of the substrate and feeding a detected result back to the spray head control module; the upward-looking visual observation module is used for detecting a positioning column of the sprayer so as to determine the position and the angle of the sprayer and feeding the result back to the sprayer control module; the ink drop observation module is used for collecting images of ink drops sprayed by the spray head and carrying out image processing so as to analyze the speed, the volume and the angle of the ink drops and feed back the result to the spray head control module; the downward-looking visual observation module is also used for acquiring a drop point image of the ink drop on the substrate, analyzing the drop point offset distance of the ink drop through image processing and feeding back the drop point offset distance to the spray head control module; the spray head control module is used for controlling the spray head module to perform ink-jet printing according to the received parameters; and the liquid film observation module is used for carrying out liquid film detection on the printed substrate.
Further, the upward vision detection module comprises an upward vision low power camera adopting a low power fixed focus lens and an upward vision high power camera adopting a high power fixed focus lens, the upward vision low power camera is arranged beside the ink drop observation module and can move along an X1 axis along with the ink drop observation module, the upward vision high power camera is arranged beside the substrate platform and can move along an X axis parallel to an X1 axis, the upward vision low power camera is used for detecting two positioning columns below the nozzle module to determine the angle of the nozzle, and the upward vision high power camera is used for detecting the angle of the nozzle array below the nozzle; the ink droplet observation module comprises a first ink droplet observation camera adopting a first stroboscopic light source and a second ink droplet observation camera adopting a second stroboscopic light source, wherein the first ink droplet observation camera reaches that the second ink droplet observation camera is adjacent to the upward viewing macroscopic low-power camera is arranged, and the first ink droplet observation camera reaches that the second ink droplet observation camera is arranged at the same height.
Further, the downward-looking visual observation module comprises a downward-looking low-power camera adopting a low-power fixed-focus lens and a downward-looking high-power camera adopting a high-power fixed-focus lens, the downward-looking low-power camera and the downward-looking high-power camera are both arranged on the spray head module, the downward-looking low-power camera is used for assisting the downward-looking high-power camera to carry out positioning, the downward-looking high-power camera is used for printing cross positioning marks for detecting three corners of the substrate before to determine the position and the deflection angle of the substrate, and the downward-looking high-power camera is used for observing the offset distance of a drop point during trial printing; the liquid film observation module comprises two high-resolution cameras adopting the same lens and a three-channel light source.
Generally, compared with the prior art, the multi-view visual inspection system and method for inkjet printing provided by the invention mainly have the following beneficial effects:
1. the method comprises a parameter calculation method of a plurality of visual detection systems, provides the realization of positioning and detection methods of a plurality of targets, and improves the detection precision of the whole visual system.
2. The multi-view visual detection system provided by the invention adopts a plurality of visual detection modules to carry out integral detection on the ink-jet printing process, can realize positioning, measurement and compensation before printing, can detect defects immediately after printing, integrates the whole visual detection process, and improves the printing efficiency of ink-jet printing.
3. The multi-view visual detection method is simple in process, easy to implement and beneficial to popularization and application.
Drawings
FIG. 1 is a schematic diagram of the present invention providing a multi-purpose visual inspection system for ink-jet printing;
FIG. 2 is a schematic view of a process of detecting a nozzle mounting angle and a nozzle array angle by a top view camera according to the present invention;
FIG. 3 is a schematic illustration of an ink drop volume measurement of an ink drop observation to which the present invention relates;
FIG. 4 is a schematic view of an ink drop observation camera measuring ink drop ejection speed and angle according to the present invention;
FIG. 5 is a schematic view of a downward-looking camera for detecting the position of the center point and the deflection angle of a substrate according to the present invention;
FIG. 6 is a schematic diagram of a downward-looking camera detecting a landing offset distance in accordance with the present disclosure;
FIG. 7 is a schematic view of a defect classification of a liquid film observation module according to the present invention;
FIG. 8 is a control block diagram of a closed-loop feedback control method for multi-vision inspection for inkjet printing according to the present invention;
fig. 9 is a schematic overall flow chart of the multi-purpose visual inspection method for inkjet printing according to the present invention.
The same reference numbers will be used throughout the drawings to refer to the same or like elements or structures, wherein: 1-upward-looking high power camera, 2-upward-looking low power camera, 3-downward-looking low power camera, 4-downward-looking high power camera, 5-first ink drop observation camera, 6-second ink drop observation camera, 7-first liquid film observation camera, 8-second liquid film observation camera, 9-substrate platform and 10-spray head module.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Referring to fig. 8 and 9, the multi-view visual inspection method for inkjet printing according to the present invention mainly includes the following steps:
firstly, detecting and determining a nozzle installation angle; then, carrying out trial printing to detect and determine the volume, the speed and the angle of the flying ink drop; finally, detecting and determining the deflection angle of the jet hole array, the position and the angle of the substrate and the offset distance of the drop point of the ink drop;
the multi-view visual detection system comprises an upper view visual observation module, a lower view visual observation module, an ink droplet observation module, a liquid film observation module and a sprayer control module, wherein the upper view visual detection module, the lower view visual detection module, the ink droplet observation module and the liquid film observation module are respectively and directly or indirectly connected with the sprayer control module. The head module 10 is disposed above a substrate stage 9, and the substrate stage 9 is formed with an X axis, an X1 axis, and a Z axis and a Z1 axis, which are parallel to each other, perpendicular to the Z axis.
Referring to fig. 1, the upward-view visual inspection module includes an upward-view high power camera 1 using a high power fixed focus lens and an upward-view low power camera 2 using a low power fixed focus lens, the upward-view low power camera 2 is disposed beside the ink droplet observation module and moves along with the ink droplet observation module along an X1 axis, the upward-view high power camera 1 is disposed beside the substrate platform 9 and can move along the X axis, and thus there is no definite position relationship between the upward-view high power camera 1 and the upward-view low power camera 2. The upward-looking low-power camera 2 is used for detecting two positioning columns below the sprayer module to determine the angle of the sprayer module, and the upward-looking high-power camera 1 is used for detecting the angle of the spray hole array below the sprayer.
The downward-looking vision observation module comprises a downward-looking low-power camera 3 adopting a low-power fixed-focus lens and a downward-looking high-power camera 4 adopting a high-power fixed-focus lens, the downward-looking low-power camera 3 and the downward-looking high-power camera 4 are all arranged on a sprayer module 10 and can follow the sprayer module 10 to move along a Y axis and a Z axis and the installation position needs to be ensured, the downward-looking low-power camera 3 and the downward-looking high-power camera 4 can simultaneously observe the same plane, wherein the downward-looking low-power camera 3 is used for assisting the downward-looking high-power camera 4 to position, the downward-looking high-power camera 4 is used for detecting the cross positioning mark of three corners of the substrate before printing to determine the position and the deflection angle of the substrate, and the downward-looking high-power camera is used for observing the offset distance of a drop point when the downward-looking high-power camera is.
The ink droplet observation module comprises a first ink droplet observation camera 5 adopting a first stroboscopic light source and a second ink droplet observation camera 6 adopting a second stroboscopic light source, wherein the first ink droplet observation camera 5 and the second ink droplet observation camera 6 are adjacent to the upward viewing macro camera 2, the first ink droplet observation camera 5 and the second ink droplet observation camera 6 are arranged at the same height, and the first ink droplet observation camera and the second ink droplet observation camera are kept at a certain angle. The first ink droplet observation camera 5 and the second ink droplet observation camera 6 respectively collect plane projection images of ink droplets ejected by the nozzles under two angles through a light path formed by the light path emission module, and distances from the lenses in the two directions to the ink droplet observation positions are kept equal.
The liquid film observation module comprises two high-resolution cameras which adopt the same lens and a three-channel light source, namely a first liquid film observation camera 7 and a second liquid film observation camera 8, wherein the first liquid film observation camera 7 and the second liquid film observation camera 8 are arranged at the same height and are kept parallel to each other downwards, and the first liquid film observation camera 7 and the second liquid film observation camera 8 are used for respectively collecting substrate pixel pit liquid film images formed after ink-jet printing. And the distances from the lens of the first liquid film camera 7 and the second liquid film camera 8 to the observation plane of the substrate are kept equal.
In other embodiments, the upward-looking and downward-looking visual observation modules respectively adopt two sets of visual units, and the two sets of visual units respectively comprise an area array industrial camera provided with a high-power lens and a white light point source and an area array industrial camera provided with a low-power lens and a white light point source; the ink drop observation module comprises two sets of visual units, and the two visual units respectively adopt two identical area array industrial cameras provided with high-power lenses, high-intensity high-frequency stroboscopic light sources and light path reflection modules; the liquid film observation module also comprises two sets of visual units, and the two visual units both adopt high-resolution industrial cameras and are matched with RGB three-channel light sources.
Before detection, the cameras in each group of visual detection modules need to be calibrated, and an image coordinate system of each camera, a conversion relation between the camera coordinate system and a world coordinate system, and an internal reference matrix and an external reference matrix of the camera are obtained.
Measuring the installation angle of the spray head: the opposite corners of the plane below the nozzle module 12 are respectively provided with a positioning column 11, and the positions of the two corner positioning columns 11 below the nozzle module 12 are detected by adopting the top-view low power camera 2 to obtain the coordinates (x) of the circle centers of the two positioning columns 111,y1) And (x)2,y2) (ii) a Then, according to an included angle theta between a straight line formed by the circles of the two positioning columns and a horizontal linejAnd the included angle theta between the two positioning positions of the nozzle and the central line of the nozzlemTo calculate the installation angle of the sprinkler:
Figure BDA0002429784510000091
in the formula, thetarotateIndicating the actual deflection angle of the sprinkler installation, with the sign representing either clockwise or counterclockwise rotation.
Referring to fig. 2, the angle of the nozzle array is detected: firstly, a top-view high power camera 1 is adopted to detect the centers P of a plurality of spray holesiCoordinate (x) ofPi,yPi) Where i 1,2,3, then a straight line is fitted to these points using a least squares method, a straight line L may be found2
Figure BDA0002429784510000101
Order:
Figure BDA0002429784510000102
Figure BDA0002429784510000103
determining the parameters k and b of the straight line and the angle theta between the straight line and the horizontal lineh
Figure BDA0002429784510000104
Figure BDA0002429784510000105
Figure BDA0002429784510000106
Wherein N represents the number of all observed orifices, xpi、ypiDenotes the abscissa and ordinate of the i-th nozzle hole, k denotes the slope of the fitted straight line, b denotes the intercept of the fitted straight line, and θhRepresenting the angle between the line and the horizontal axis.
Referring to fig. 3, the volume, velocity and angle of the flying ink drop are detected: after the images of the ink drops are simultaneously acquired by the two ink drop observation cameras, the projection images of the flying ink drops in the two camera directions are respectively detected, an ellipse can be fitted, then the height h of the flying ink drops and the maximum widths a and b in the two projection planes are solved, the flying ink drops are dispersed into n slices with the thickness of delta h in the vertical direction, the cross section of each slice is fitted into an ellipse, and finally the major axis and the minor axis a of the ellipse in each slice are calculated according to the maximum widths a and b of the ink drop projection and the ellipse fitted by the projection images of the ink drops in the camera directionsi、biWhere i is 1,2, …, n, the drop volume can be calculated using the following expression:
Figure BDA0002429784510000111
wherein V represents the volume of the ink drop to be measured, h represents the height of the observed ink drop, S represents the cross-sectional area of the ink drop slice, ai、biThe major axis and the minor axis of the ellipse in the i-th cross section are shown, respectively, and n represents the number of ink droplet slices in the height direction.
Referring to FIG. 4, the controller controls the droplet observing camera to expose twice in a certain time interval, and the flying droplet is controlled by P within the time interval Δ t of the two exposures of the droplet observing camera6Point movement to P7A point, and P7With dots in the projection plane of the two-phase machineProjection points are respectively P7xAnd P7yThe height of the drop of the ink drop in time t can be calculated as deltaz through image processing, and when the sampling interval time deltat approaches 0, the instantaneous speed and angle of the ink drop can be calculated as follows:
Figure BDA0002429784510000112
Figure BDA0002429784510000113
wherein v isx、vy、vzRepresenting the velocity components, theta, of the ink drop in the x, y, and z directions, respectivelyx、θzRepresenting the angles of flight of the ink drops in the horizontal and vertical planes, respectively.
Referring to fig. 5, the center position 15 and the angle of the substrate 14 are detected: the method comprises the steps that a cross positioning mark exists at each of three corners of a substrate, a downward-looking high power camera is adopted to detect the three positioning marks respectively, and the central positions of the three cross positioning marks obtained through image processing calculation are (x)s1,ys1)、(xs2,ys2)、(xs3,ys3) If the first and third positioning marks are located on the diagonal of the substrate, the position and deflection angle of the center point of the substrate can be obtained:
Figure BDA0002429784510000114
Figure BDA0002429784510000115
wherein (x)s,ys) Position coordinates, theta, representing the center point of the substratesIndicating the deflection angle of the substrate.
Referring to fig. 6, in the trial printing of the trial printing area of the substrate, the coordinates (x) of the expected landing positions of a plurality of ink droplets are preset by the nozzleesti,yesti) Wherein i 1,2,3,4Trial printing is carried out in a trial printing area of the substrate, after the trial printing is finished, actual drop points are respectively detected by adopting a downward-looking high-power camera, and a plurality of actual drop point position coordinates (x)reai,yreai) Wherein, i is 1,2,3,4, then respectively calculating the offset value of the actual landing point of each ink drop in the x and y directions and the preset landing point, and finally averaging, then calculating the final offset distance by the following expression:
Figure BDA0002429784510000121
wherein x isbiasDenotes the average offset distance in the x-direction, ybiasRepresents the average offset distance in the y-direction and N represents the number of all observed actual landings.
And step two, performing ink-jet printing according to the obtained parameters, and performing liquid film detection on the printed substrate by adopting a predefined defect type after the substrate is printed.
After all the parameters are detected, the substrate can be printed, after the printing is finished, two liquid film observation cameras are adopted to carry out liquid film detection on the printed substrate, as shown in fig. 7, the two uppermost rows represent pixel pits without defects, the rest pixel pits have defects, the defect types are mainly divided into ① scattered points 19, ② pixel connections 20, ③ missing spraying 21, ④ edge liquid drops are volatilized too fast 22, and the like.
The liquid film detection specifically comprises the following substeps: the whole substrate is divided into a plurality of rectangular grids, and each grid internally comprises a pixel pit. In image processing, each grid represents an ROI, an edge extraction operation is performed on a droplet in each ROI, and then the defect type of a pixel pit is determined by the droplet edge position. By this time, the whole process of visual observation of ink-jet printing is completed.
In one embodiment, the multi-view vision inspection method comprises the following steps of ① calibrating cameras in each group of vision modules, obtaining a conversion relation among an image coordinate system, a camera coordinate system and a world coordinate system of each camera and an internal reference matrix and an external reference matrix of the cameras, ② determining the position and the angle of a nozzle through detecting a positioning column of the nozzle by using an upper-view low-power lens camera and feeding back the result to a nozzle control module, ③ performing test printing on the nozzle in a test printing area, adopting an ink droplet observation camera to perform image processing on an ink droplet image sprayed by the nozzle, analyzing the speed, the volume and the angle of the ink droplet and feeding back the result to the nozzle control module, ④ detecting the angle of the nozzle array by using an upper-view high-power camera and feeding back the result to the nozzle control module, ⑤ positioning a mark image of a substrate by using a lower-view camera, determining the position and the angle of the substrate and feeding back the position and the angle to the nozzle control module, ⑥ nozzle performing test printing on a substrate in a test printing part, adopting a lower-view camera to acquire a landing point image of the ink droplet image on the substrate, analyzing the substrate, feeding back the mark image, feeding back the position and feeding back the defect control module, and defining the defect control module according to the ink droplet printing parameters of the nozzle control module, and defining the nozzle control module according to be printed by using a.
The multi-view visual detection system and method for ink-jet printing provided by the invention comprise an upper-view visual observation module, a lower-view visual observation module, an ink drop observation module and a liquid film observation module, can efficiently realize the visual observation process of each step in the whole ink-jet printing process, and is beneficial to improving the detection efficiency and the detection precision of the whole process.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (10)

1. A multi-purpose visual inspection method for inkjet printing, the method comprising the steps of:
(1) firstly, detecting and determining the installation angle of a spray head; then, carrying out trial printing to detect and determine the volume, the speed and the angle of the flying ink drop; finally, detecting and determining the deflection angle of the jet hole array, the position and the angle of the substrate and the offset distance of the drop point of the ink drop;
(2) and performing ink jet printing according to the obtained parameters, and performing liquid film detection on the printed substrate by adopting a predefined defect type after the substrate is printed.
2. The multi-purpose visual inspection method for inkjet printing according to claim 1, characterized in that: the detection of the installation angle of the spray head comprises the following steps: firstly, two positioning marks at the diagonal position below the spray head module are detected to obtain the centroid position coordinates (x) of the positioning marks1,y1) And (x)2,y2) (ii) a Then, the angle theta between the line formed by the centroids of the two marks and the horizontal axis is determinedjAnd the manufacturing included angle theta between the two positioning marks and the central line of the spray headmTo calculate the installation angle of the sprinkler:
Figure FDA0002429784500000011
in the formula, thetarotateIndicating the actual deflection angle of the sprinkler installation, with the sign representing either clockwise or counterclockwise rotation.
3. The multi-purpose visual inspection method for inkjet printing according to claim 1, characterized in that: the detection of the deflection angle of the jet hole array comprises the following steps: firstly, detecting the centers P of a plurality of spray holesiCoordinate (x) ofPi,yPi) Where i 1,2,3, then fitting a straight line to the points using a least squares method to obtain a straight line L2
Figure FDA0002429784500000012
Order:
Figure FDA0002429784500000021
Figure FDA0002429784500000022
determining the parameters k and b of the straight line and the angle theta between the straight line and the horizontal lineh
Figure FDA0002429784500000023
Figure FDA0002429784500000024
Figure FDA0002429784500000025
Wherein N represents the number of all observed orifices, xpi、ypiRespectively represent the abscissa and ordinate of the i-th injection hole, k represents the slope of the fitted straight line, b represents the intercept of the fitted straight line, and thetahRepresenting the angle between the line and the horizontal axis.
4. The multi-purpose visual inspection method for inkjet printing according to claim 1, characterized in that: the detection of the volume of the flying ink drop comprises the following steps: detecting the projection images of the flying ink drop in the two camera directions respectively, fitting an ellipse, and then solving the height h of the flying ink drop and the maximum widths a and b in the two projection planes respectively; dispersing the flying ink drops into n slices with the thickness of delta h along the vertical direction, and fitting the cross section of each slice into an ellipse; finally, the major and minor axes a of the ellipse in each slice plane are calculated according to the maximum width a and b of the ink drop projection and the ellipse fitted to the projection image of the ink drop in the camera directioni、biWhere i is 1,2, …, n, the drop volume is calculated using the following equation:
Figure FDA0002429784500000026
wherein V represents the volume of the ink drop to be measured, h represents the height of the observed ink drop, S represents the cross-sectional area of the ink drop slice, ai、biThe major axis and the minor axis of the ellipse in the i-th cross section are shown, respectively, and n represents the number of ink droplet slices in the height direction.
5. The multi-purpose visual inspection method for inkjet printing according to claim 4, characterized in that: the detection of the speed and angle of the flying ink drop comprises the following steps: two exposures are carried out in a preset time interval, and the flying ink drop is formed by P in the time interval delta t of two exposures of the ink drop observation camera6Point movement to P7A point, and P7The projection points of the points in the projection plane of the two cameras are respectively P7xAnd P7yThe height of the drop of the ink drop in the time t is calculated to be delta z through image processing, and when the sampling interval time delta t approaches to 0, the instantaneous speed and the angle of the ink drop are respectively calculated as follows:
Figure FDA0002429784500000031
Figure FDA0002429784500000032
wherein v isx、vy、vzRepresenting the velocity components, theta, of the ink drop in the x, y, and z directions, respectivelyx、θzRepresenting the angles of flight of the ink drops in the horizontal and vertical planes, respectively.
6. The multi-purpose visual inspection method for inkjet printing according to claim 1, characterized in that: the detection of the position and the angle of the substrate comprises the following steps: the method comprises the steps of respectively arranging a cross positioning mark at three corners of a substrate, respectively detecting the three positioning marks by using a down-looking high-power camera, and obtaining three positioning marks through image processing calculationThe central positions of the cross positioning marks are respectively (x)s1,ys1)、(xs2,ys2)、(xs3,ys3) And if the first positioning mark and the third positioning mark are positioned on the diagonal line of the substrate, the position and the deflection angle of the central point of the substrate are calculated as follows:
Figure FDA0002429784500000033
Figure FDA0002429784500000034
wherein (x)s,ys) Position coordinates, theta, representing the center point of the substratesIndicating the deflection angle of the substrate.
7. A multi-purpose visual inspection method for inkjet printing according to any one of claims 1 to 6, wherein: the detection of the drop offset distance of the ink drop comprises the following steps: first, coordinates (x) of expected landing positions of a plurality of ink droplets are preset by a headesti,yesti) The method includes the steps of firstly, obtaining a plurality of actual drop point position coordinates (x), wherein i is 1,2,3,4reai,yreai) Wherein, i is 1,2,3, 4.
Figure FDA0002429784500000041
Wherein x isbiasDenotes the average offset distance in the x-direction, ybiasRepresents the average offset distance in the y direction, and N represents the number of all observed actual landing points;
in addition, when the liquid film is detected, the whole substrate is divided into a plurality of rectangular grids, and each grid internally comprises a pixel pit; in image processing, each grid represents an ROI, an edge extraction operation is performed on a droplet in each ROI, and then the defect type of a pixel pit is determined by the droplet edge position.
8. A multi-purpose visual inspection system for inkjet printing, characterized by: the multi-view vision inspection system adopts the multi-view vision inspection method for ink-jet printing according to any one of claims 1 to 7 to perform a vision printing process and defect inspection, and comprises an upper-view vision observation module, a lower-view vision observation module, an ink droplet observation module, a liquid film observation module and a nozzle control module, wherein the upper-view vision observation module, the lower-view vision observation module, the ink droplet observation module and the liquid film observation module are respectively directly or indirectly connected with the nozzle control module;
the downward-looking visual observation module is used for collecting a substrate positioning mark image so as to determine the position and the angle of the substrate and feeding a detected result back to the spray head control module; the upward-looking visual observation module is used for detecting a positioning column of the sprayer so as to determine the position and the angle of the sprayer and feeding the result back to the sprayer control module; the ink drop observation module is used for collecting images of ink drops sprayed by the spray head and carrying out image processing so as to analyze the speed, the volume and the angle of the ink drops and feed back the result to the spray head control module; the downward-looking visual observation module is also used for acquiring a drop point image of the ink drop on the substrate, analyzing the drop point offset distance of the ink drop through image processing and feeding back the drop point offset distance to the spray head control module; the spray head control module is used for controlling the spray head module to perform ink-jet printing according to the received parameters; and the liquid film observation module is used for carrying out liquid film detection on the printed substrate.
9. A multi-purpose visual inspection system for inkjet printing according to claim 8, wherein: the upward-looking vision detection module comprises an upward-looking low-power camera adopting a low-power fixed-focus lens and an upward-looking high-power camera adopting a high-power fixed-focus lens, the upward-looking low-power camera is arranged beside the ink drop observation module and can move along with the ink drop observation module along an X1 axis, the upward-looking high-power camera is arranged beside the substrate platform and can move along an X axis parallel to an X1 axis, the upward-looking low-power camera is used for detecting two positioning columns below the spray head module to determine the angle of the spray head, and the upward-looking high-power camera is used for detecting the angle of a spray hole array below the spray head; the ink droplet observation module comprises a first ink droplet observation camera adopting a first stroboscopic light source and a second ink droplet observation camera adopting a second stroboscopic light source, wherein the first ink droplet observation camera reaches that the second ink droplet observation camera is adjacent to the upward viewing macroscopic low-power camera is arranged, and the first ink droplet observation camera reaches that the second ink droplet observation camera is arranged at the same height.
10. A multi-purpose visual inspection system for inkjet printing according to any of claims 8 to 9, wherein: the downward-looking visual observation module comprises a downward-looking low-power camera adopting a low-power fixed-focus lens and a downward-looking high-power camera adopting a high-power fixed-focus lens, the downward-looking low-power camera and the downward-looking high-power camera are both arranged on the spray head module, the downward-looking low-power camera is used for assisting the downward-looking high-power camera to carry out positioning, the downward-looking high-power camera is used for printing cross positioning marks for detecting three corners of the substrate before to determine the position and the deflection angle of the substrate, and the downward-looking high-power camera is used for observing the offset distance of a drop point during trial printing; the liquid film observation module comprises two high-resolution cameras adopting the same lens and a three-channel light source.
CN202010232618.6A 2020-03-28 2020-03-28 Multi-view visual detection system and method for ink-jet printing Active CN111397539B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010232618.6A CN111397539B (en) 2020-03-28 2020-03-28 Multi-view visual detection system and method for ink-jet printing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010232618.6A CN111397539B (en) 2020-03-28 2020-03-28 Multi-view visual detection system and method for ink-jet printing

Publications (2)

Publication Number Publication Date
CN111397539A true CN111397539A (en) 2020-07-10
CN111397539B CN111397539B (en) 2021-04-20

Family

ID=71436709

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010232618.6A Active CN111397539B (en) 2020-03-28 2020-03-28 Multi-view visual detection system and method for ink-jet printing

Country Status (1)

Country Link
CN (1) CN111397539B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111791589A (en) * 2020-09-10 2020-10-20 季华实验室 Positioning detection method and device based on ink-jet printer, electronic equipment and medium
CN112319046A (en) * 2020-09-10 2021-02-05 季华实验室 Positioning calibration device, ink-jet printer and jet printing point coordinate positioning calibration method
CN112757796A (en) * 2020-12-30 2021-05-07 华中科技大学 System and method for detecting quality of display device in whole spray printing manufacturing process
CN113752698A (en) * 2021-08-31 2021-12-07 华中科技大学 Ink drop point precision control method and system for ink jet printing
CN113959350A (en) * 2021-11-03 2022-01-21 Tcl华星光电技术有限公司 Ink-jet printing detection system and detection method
CN114536975A (en) * 2022-02-28 2022-05-27 华中科技大学 Method and device for detecting positioning precision of ink drop point in ink-jet printing
CN114994044A (en) * 2022-06-24 2022-09-02 杭州爱新凯科技有限公司 Ink-jet printing head tester
CN115008900A (en) * 2022-05-13 2022-09-06 华中科技大学 Flexible display jet printing film edge straightness control method and system
CN115071275A (en) * 2022-05-25 2022-09-20 复旦大学 Ink drop observer position calibration method and system
CN115384189A (en) * 2022-10-28 2022-11-25 季华实验室 Device and method for observing and counting drop points of ink drops of spray head
CN115682942A (en) * 2023-01-05 2023-02-03 季华实验室 Position detection system, position detection method, inkjet printing apparatus, and storage medium
CN116968449A (en) * 2023-07-31 2023-10-31 武汉国创科光电装备有限公司 Ink-jet printing device and method for small-size panel
CN117268738A (en) * 2023-11-20 2023-12-22 季华实验室 Nozzle detection method, device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1297402A (en) * 1998-04-23 2001-05-30 斯托克数字成像公司 Method and device for monitoring and/or correcting alignment of ink-jet printer
CN102653179A (en) * 2011-03-03 2012-09-05 株式会社理光 Image forming apparatus, droplet discharge detecting method in the image forming apparatus, and computer program product
US20160067959A1 (en) * 2014-09-10 2016-03-10 Seiko Epson Corporation Liquid discharge apparatus, control method of liquid discharge apparatus, and device driver, and printing system
CN109435473A (en) * 2018-09-11 2019-03-08 华中科技大学 A kind of flight ink droplet detection device and method suitable for inkjet printing
CN109703196A (en) * 2018-03-13 2019-05-03 广东聚华印刷显示技术有限公司 Print head ink droplet situation corrects result detection method and ink droplet titrimetry system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1297402A (en) * 1998-04-23 2001-05-30 斯托克数字成像公司 Method and device for monitoring and/or correcting alignment of ink-jet printer
CN102653179A (en) * 2011-03-03 2012-09-05 株式会社理光 Image forming apparatus, droplet discharge detecting method in the image forming apparatus, and computer program product
US20160067959A1 (en) * 2014-09-10 2016-03-10 Seiko Epson Corporation Liquid discharge apparatus, control method of liquid discharge apparatus, and device driver, and printing system
CN109703196A (en) * 2018-03-13 2019-05-03 广东聚华印刷显示技术有限公司 Print head ink droplet situation corrects result detection method and ink droplet titrimetry system
CN109435473A (en) * 2018-09-11 2019-03-08 华中科技大学 A kind of flight ink droplet detection device and method suitable for inkjet printing

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112319046A (en) * 2020-09-10 2021-02-05 季华实验室 Positioning calibration device, ink-jet printer and jet printing point coordinate positioning calibration method
CN112319046B (en) * 2020-09-10 2021-07-02 季华实验室 Positioning calibration device, ink-jet printer and jet printing point coordinate positioning calibration method
CN111791589A (en) * 2020-09-10 2020-10-20 季华实验室 Positioning detection method and device based on ink-jet printer, electronic equipment and medium
US11779946B2 (en) * 2020-12-30 2023-10-10 Huazhong University Of Science And Technology System and method for quality inspection on overall inkjet printing manufacturing process for display device
CN112757796A (en) * 2020-12-30 2021-05-07 华中科技大学 System and method for detecting quality of display device in whole spray printing manufacturing process
CN112757796B (en) * 2020-12-30 2022-04-08 华中科技大学 System and method for detecting quality of display device in whole spray printing manufacturing process
US20230415185A1 (en) * 2020-12-30 2023-12-28 Huazhong University Of Science And Technology Quality inspection system for entire inkjet printing manufacturing process for display device
US20220203700A1 (en) * 2020-12-30 2022-06-30 Huazhong University Of Science And Technology System and method for quality inspection on overall inkjet printing manufacturing process for display device
CN113752698A (en) * 2021-08-31 2021-12-07 华中科技大学 Ink drop point precision control method and system for ink jet printing
CN113752698B (en) * 2021-08-31 2022-07-12 华中科技大学 Ink drop point precision control method and system for ink jet printing
CN113959350A (en) * 2021-11-03 2022-01-21 Tcl华星光电技术有限公司 Ink-jet printing detection system and detection method
CN113959350B (en) * 2021-11-03 2023-11-28 Tcl华星光电技术有限公司 Ink jet printing detection system and detection method
CN114536975A (en) * 2022-02-28 2022-05-27 华中科技大学 Method and device for detecting positioning precision of ink drop point in ink-jet printing
CN115008900A (en) * 2022-05-13 2022-09-06 华中科技大学 Flexible display jet printing film edge straightness control method and system
CN115071275A (en) * 2022-05-25 2022-09-20 复旦大学 Ink drop observer position calibration method and system
CN115071275B (en) * 2022-05-25 2023-10-27 复旦大学 Ink drop observer position calibration method and system
CN114994044A (en) * 2022-06-24 2022-09-02 杭州爱新凯科技有限公司 Ink-jet printing head tester
CN115384189A (en) * 2022-10-28 2022-11-25 季华实验室 Device and method for observing and counting drop points of ink drops of spray head
CN115682942A (en) * 2023-01-05 2023-02-03 季华实验室 Position detection system, position detection method, inkjet printing apparatus, and storage medium
CN116968449A (en) * 2023-07-31 2023-10-31 武汉国创科光电装备有限公司 Ink-jet printing device and method for small-size panel
CN116968449B (en) * 2023-07-31 2024-03-26 武汉国创科光电装备有限公司 Ink-jet printing device and method for small-size panel
CN117268738A (en) * 2023-11-20 2023-12-22 季华实验室 Nozzle detection method, device, equipment and storage medium
CN117268738B (en) * 2023-11-20 2024-03-12 季华实验室 Nozzle detection method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN111397539B (en) 2021-04-20

Similar Documents

Publication Publication Date Title
CN111397539B (en) Multi-view visual detection system and method for ink-jet printing
CN111332020B (en) Multi-error compensation positioning control method and system for jet printing ink drops
CN1939730B (en) Methods and system for calibration of inkjet drop positioning
CN113752698B (en) Ink drop point precision control method and system for ink jet printing
CN109935741B (en) Inkjet printing alignment method and device for OLED display panel
US20210260892A1 (en) Manufacturing system and method for inkjet printing flexible display device
KR20070036727A (en) Methods and system for inkjet drop positioning
TWI784937B (en) Inkjet printing system and method for processing substrates
US20100238229A1 (en) Method for inspecting droplet discharge head, device for inspecting droplet discharge head, and droplet discharge device
WO2023226294A1 (en) Position calibration method and system for ink droplet observation device
US6702419B2 (en) System and method for delivering droplets
CN114536975B (en) Method and device for detecting positioning precision of ink drop point in ink-jet printing
CN113771518A (en) Ink jet printing method and ink jet printing apparatus
CN113858835A (en) Method and system for monitoring angle of ink-jet printing head in real time based on image processing
CN116381266A (en) Method and system for detecting ink drops ejected by jet printing nozzle array
JP2004141758A (en) Method of correcting dot position of droplet discharge device, alignment mask, droplet discharge method, electro-optical device and its production method, and an electronic equipment
JP2006258845A (en) Pattern forming device and head correcting method
JP2006130383A (en) Method and device for detection of dot shift
JP7442128B2 (en) Inkjet printing method and inkjet printing device
JP2008168207A (en) Inferior discharge detection device and its method
KR102611132B1 (en) Fringe information measuring apparatus and substrate treating system including the same
CN116968449B (en) Ink-jet printing device and method for small-size panel
KR20230022538A (en) Apparatus and method for processing substrate
KR102510910B1 (en) Maintenance unit and apparatus for treating substrate including the same
JP2011255292A (en) Positioning method, positioning device, droplet application method, liquid application apparatus, and reference plate

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant