JP7135586B2 - Image processing system, image processing method and program - Google Patents

Description

The present invention relates to an image processing system, an image processing method and a program.

Generally, in the field of cameras, an autofocus function is known that determines the in-focus state and moves the focus lens of the lens unit so as to bring the subject into focus. For example, Japanese Patent Application Laid-Open No. 2018-84701 (Patent Document 1) discloses a focus adjustment device that uses a contrast detection method or a phase difference detection method to determine focus within a limited range.

JP-A-2018-84701

For example, in industrial equipment for inspecting products, it is desirable to obtain an image in which the product is in focus for accurate inspection. When imaging products flowing through a manufacturing line, the distance from the camera may vary from product to product. The camera therefore needs to adjust its focus each time it images a product. In a system that captures an image of an object using an imaging system that can change the focus position and processes the image of the object, the focus is adjusted accurately and at high speed. is desired.

An object of the present invention is to enable accurate and/or high-speed focusing on an object in a system that images an object using an imaging system that can change the focus position and processes the image of the object. is to be

According to an example of the present disclosure, an image processing system includes an imaging unit that captures an image of an object and generates an image that includes the object; and an image processing unit that executes processing. The imaging unit can change the focus position. In the first mode in which the focus position of the object is set in advance, the image processing unit divides the image of the object into a reference area serving as a reference for the focus position and a target area to be subjected to predetermined processing by the image processing unit. to register the focus position of each of the reference area and the target area. In the second mode of executing the predetermined process, the image processing unit extracts the target region based on the focus position of the reference region in the image acquired by the imaging unit and the registered focus positions of the reference region and the target region. determines the focus position of the In the second mode, the imaging section images the target area at the determined focus position.

According to this disclosure, in the first mode, the image processing unit sets a reference area and a target area, and registers focus positions of each of the reference area and the target area. The reference area is an area that serves as a reference for the focus position. In the second mode, the image processing unit determines a focus position for imaging the target region based on the registered focus position and the focus position of the reference region in the image acquired by the imaging unit in the second mode. Ask for The focus position of the target area in the second mode can be determined from the relationship between the focus positions of the reference area in the first mode and the second mode and the focus position of the target area in the first mode. Therefore, the imaging unit can focus on the target area accurately and/or quickly.

In the above disclosure, the reference area and the target area are each defined so as to at least partially overlap each other.

According to this disclosure, the reference region and the target region may include common regions. Thereby, when determining the focus position of the target area in the second mode, it is possible to focus on the target area accurately and/or at high speed.

In the above disclosure, the image processing unit, in the second mode, compares the position of the reference area in the image captured by the imaging unit with the position of the reference area registered in the first mode, and Correct the position of the object in the image based on the result of

According to this disclosure, the reference area can also be used as a model for misregistration correction. Therefore, it is possible not only to obtain an image in which the object is in focus, but also to correct the displacement of the object area in the image.

In the above disclosure, the image processing unit uses the difference between the focus position of the registered reference region in the second mode and the focus position of the reference region acquired in the second mode to obtain the registered target The focus position of the target region is determined by correcting the focus position of the region.

According to this disclosure, in the second mode, the focus position of the region of interest can be determined by relatively simple calculations.

In the above disclosure, the target area includes plain areas.
According to this disclosure, the focus position of a target area can be determined even if the target area is a plain area (ie, an area without a distinct pattern). Solid areas are areas of low contrast. For this reason, it may not be easy to focus when imaging a plain area. On the other hand, according to this disclosure, the focus position in a reference region with a well-defined pattern can be used to determine the focus position in the target region. Therefore, the target area including the plain area can be imaged in focus.

In the above disclosure, in the first mode, the image processing unit determines the focus position of the target region based on the image obtained by the imaging unit capturing the pattern temporarily formed in the target region. .

According to this disclosure, a pattern can be formed in the region of interest that is advantageous for determining the focus position. For example, even if the target region itself does not have a clear pattern, the pattern temporarily formed in the target region can determine the focus position of the target region. Therefore, in the second mode, the focus position of the target area can be determined more accurately.

In the above disclosure, in the first mode, the image processing unit obtains a height difference between the reference region and the target region from information on the three-dimensional shape of the target, and calculates the height difference and the reference The focus position of the region is used to determine the focus position of the region of interest.

According to this disclosure, the focus position of the target area can be determined from the focus position and height information of the reference area. Therefore, the focus position of the region of interest can be determined by relatively simple calculations.

According to an example of the present disclosure, an image processing method includes an image capturing unit that captures an image of a target object and generates an image including the target object; An image processing method using an image processing system including an image processing unit that executes processing. The imaging unit can change the focus position. In the image processing method, in a first mode in which the focus position of the object is set in advance, the image processing unit creates a reference area serving as a reference for the focus position in the image of the object and a predetermined process performed by the image processing unit. In a step of setting a target region to be targeted and registering the focus position of each of the reference region and the target region, and in a second mode of executing a predetermined process, the image processing unit processes the image acquired by the imaging unit determining the focus position of the target area based on the focus position of the reference area and the registered focus positions of each of the reference area and the target area; capturing an image of the target area at the position; and performing the predetermined process using the image of the target captured at the determined focus position.

According to this disclosure, the focus position of the target area in the second mode is determined from the relationship between the focus positions of the reference area in the first mode and the second mode and the focus position of the target area in the first mode. can decide. Therefore, the imaging unit can focus on the target area accurately and/or quickly.

According to an example of the present disclosure, the program is a program for executing predetermined processing using an image obtained by imaging an object with an imaging device. The imaging device can change the focus position. The program sets a reference area serving as a reference for the focus position and a target area to be subjected to predetermined processing in the image of the object in a first mode in which the focus position of the object is preset in the computer. and registering the focus position of each of the reference region and the target region; determining the focus position of the target region based on each registered focus position; and executing a predetermined process using the image of the target object captured at the determined focus position. .

According to this disclosure, the focus position of the target area in the second mode is determined from the relationship between the focus positions of the reference area in the first mode and the second mode and the focus position of the target area in the first mode. can decide. Therefore, the image processing section can acquire an image of the target region in focus. As a result, the accuracy of processing by the image processing unit (for example, the accuracy of image recognition) can be improved.

According to the present invention, in a system for capturing an image of an object using an imaging system capable of changing the focus position and processing the image of the object, it is possible to focus on the object accurately and/or at high speed. become.

1 is a schematic diagram showing one application example of an image processing system according to an embodiment; FIG. It is a figure which shows an example of an internal structure of an imaging device. FIG. 5 is a schematic diagram for explaining focus position adjustment; It is a figure which shows an example of a structure of a lens. FIG. 10 is a diagram showing another example of a focus position adjusting lens; FIG. 4 is a diagram schematically showing imaging of a workpiece by an imaging device; FIG. 4 is a diagram showing an imaging target area in a work; FIG. 4 is a diagram showing areas for focus adjustment according to the present embodiment; FIG. 4 is a diagram for explaining registration of a focus position; FIG. FIG. 10 is a diagram for explaining focus position adjustment during inspection; FIG. 10 is a diagram for explaining an example of designation of a reference area; FIG. 4 is a diagram for explaining an example of designation of an inspection area; FIG. 4 is a flow chart showing a flow of a method for setting a focus position; It is an example of a jig for obtaining a focus position of an inspection area. FIG. 15 is a diagram showing a method of obtaining a focus position of an inspection area using the jig shown in FIG. 14; It is the flowchart which showed the flow of the inspection method.

§1 Application Example First, an example of a scene to which the present invention is applied will be described with reference to FIG. FIG. 1 is a schematic diagram showing one application example of the image processing system according to the present embodiment.

As shown in FIG. 1, an image processing system 1 according to this embodiment is implemented as, for example, a visual inspection system. The visual inspection system takes images of a plurality of positions to be inspected on a work W placed on a stage 90, for example, in a production line for industrial products, and uses the obtained images to perform visual inspection of the work W. . In the appearance inspection, the workpiece W is inspected for flaws, stains, presence of foreign matter, dimensions, and the like.

After completing the visual inspection of the work W placed on the stage 90 , the next work (not shown) is transferred onto the stage 90 . During imaging of the work W, the work W may stand still at a predetermined position on the stage 90 in a predetermined posture. Alternatively, the image of the work W may be captured while the work W is moving on the stage 90 .

As shown in FIG. 1, the image processing system 1 includes an imaging device 10 and an image processing device 20 as basic components. In this embodiment, the image processing system 1 further includes a PLC (Programmable Logic Controller) 50 and an input/display device 60 .

The imaging device 10 is connected to the image processing device 20 . The image capturing device 10 captures an image of a subject (work W) existing in the image capturing field of view according to a command from the image processing device 20, and generates image data including an image of the work W. FIG. In this embodiment, the imaging device 10 is an imaging system capable of changing the focus position. The imaging device 10 and the image processing device 20 may be integrated.

The imaging device 10 includes a lens module with a variable focus position. A focus position means a position where an image of the work W is formed. The focus position of the lens module is changed according to the distance between the imaging device 10 and the workpiece W. FIG. As a result, an image in which the workpiece W is in focus can be captured. Note that the imaging device 10 has an autofocus function, and can automatically focus on the work W. FIG.

The image processing device 20 acquires an image of the work W from the imaging device 10 . The image processing device 20 executes predetermined processing on the image. Image processing device 20 includes determination unit 21 , output unit 22 , storage unit 23 , and command generation unit 24 .

The determination unit 21 determines whether the appearance of the workpiece W is good or bad by executing predetermined processing on the image data generated by the imaging device 10 . The output unit 22 outputs the result of determination by the determination unit 21 . For example, the output unit 22 causes the input/display device 60 to display the determination result.

The storage unit 23 stores various data, programs, and the like. For example, the storage unit 23 stores image data acquired from the imaging device 10 and image data that has undergone predetermined processing. The storage unit 23 may store the determination result by the determination unit 21 . Further, the storage unit 23 stores programs for causing the image processing device 20 to execute various types of processing.

The command generation unit 24 receives a control command from the PLC 50 and outputs an imaging command (imaging trigger) to the imaging device 10 .

The image processing device 20 is connected to the PLC 50 . The PLC 50 controls the image processing device 20 . For example, the PLC 50 controls timing for the image processing device 20 to output an imaging command (imaging trigger) to the imaging device 10 .

An input/display device 60 is connected to the image processing device 20 . The input/display device 60 receives user input regarding various settings of the image processing system 1 . Further, the input/display device 60 displays information regarding the settings of the image processing system 1, the result of image processing of the workpiece W by the image processing device 20 (for example, the result of judging the appearance of the product), and other information.

In the present embodiment, in the first mode in which the focus position of the work W is set in advance, the image processing device 20 includes a reference region serving as a reference for the focus position in the image of the work W, and a predetermined region by the image processing device 20. , and register the focus position of each of the reference area and the target area. In the second mode of executing predetermined processing, the image processing device 20 performs the following operations based on the focus position of the reference region in the image acquired by the imaging device 10 and the registered focus positions of the reference region and the target region. , to determine the focus position of the region of interest. In the second mode, the imaging device 10 images the target area at the determined focus position.

The reference area is an area that serves as a reference for the focus position. The focus position of the target area in the second mode can be determined from the relationship between the focus positions of the reference area in the first mode and the second mode and the focus position of the target area in the first mode. This allows the imaging device 10 to accurately focus on the target region. In addition, the imaging device 10 can focus on the target area at high speed in the second mode.

FIG. 2 is a diagram showing an example of the internal configuration of the imaging device 10. As shown in FIG. As shown in FIG. 2, the imaging apparatus 10 includes an illumination unit 11, a lens module 12, an imaging device 13, an imaging device control unit 14, a lens control unit 16, registers 15 and 17, and a communication interface. and F section 18 .

The illumination unit 11 irradiates the work W with light. Light emitted from the illumination unit 11 is reflected by the surface of the work W and enters the lens module 12 . The lighting section 11 may be omitted.

The lens module 12 forms an image of the reflected light from the workpiece W on the imaging surface 13 a of the imaging device 13 . The lens module 12 has a lens 12a, a lens group 12b, a lens 12c, a movable portion 12d, and a focus adjustment portion 12e. The lens 12a is a lens mainly for changing the focus position. The focus adjustment unit 12e controls the lens 12a to change the focus position.

The lens group 12b is a lens group for changing the focal length. The zoom magnification is controlled by changing the focal length. The lens group 12b is installed in the movable portion 12d and is movable along the optical axis direction. The lens 12 c is a lens that is fixed at a predetermined position within the imaging device 10 .

The imaging element 13 is a photoelectric conversion element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and converts light from an imaging field into an image signal.

When the imaging device control unit 14 receives an imaging command from the image processing device 20 via the communication I/F unit 18 , the imaging device control unit 14 opens a shutter (not shown) for exposure, and generates image data based on the image signal from the imaging device 13 . do. At this time, the imaging device control unit 14 opens and closes the shutter so that the shutter speed (exposure time) corresponds to the imaging position, and generates image data with a preset resolution. Information indicating the shutter speed and resolution corresponding to the imaging position is stored in the register 15 in advance. The imaging device control section 14 outputs the generated image data to the image processing device 20 via the communication I/F section 18 .

The lens control unit 16 adjusts the focus of the imaging device 10 according to the instructions stored in the register 17 . Specifically, the lens control unit 16 controls the focus adjustment unit 12e so that the focus position changes according to the area of the work W to be imaged. The focus adjustment unit 12 e adjusts the position of the lens 12 a under the control of the lens control unit 16 . That is, the lens control unit 16 controls the lens 12a so that the imaging target area of the workpiece W is in focus. “In focus” means that an image of the imaging target area of the workpiece W is formed on the imaging surface 13 a of the imaging element 13 . Lens 12a will be described in detail later.

The lens control section 16 may control the movable section 12d to adjust the position of the lens group 12b so that the size of the area of the workpiece W included in the imaging field is substantially constant. In other words, the lens control section 16 can control the movable section 12d so that the size of the area of the workpiece W included in the imaging field falls within a predetermined range. The lens control unit 16 may adjust the position of the lens group 12b according to the distance between the imaging position and the workpiece W. Note that zoom adjustment is not essential in this embodiment.

§2 Concrete example <A. Configuration Example for Focus Adjustment>
FIG. 3 is a schematic diagram for explaining focus position adjustment. For simplicity of explanation, FIG. 3 shows only one lens (lens 12a).

As shown in FIG. 3, let a be the distance from the principal point O of the lens 12a to the target surface (surface of the workpiece W), b be the distance from the principal point O of the lens 12a to the imaging surface 13a, and b be the distance from the principal point O of the lens 12a. Let f be the distance (focal length) from the point O to the focal point F of the lens 12a. When the position of the image of the workpiece W is equal to the position of the imaging surface 13a, the following formula (1) holds.

1/a+1/b=1/f (1)
The working distance (WD) can change according to the height of the surface of the work W. FIG. The working distance is defined as the distance from the work W side surface of the lens 12a to the work W. When the formula (1) holds, an image in which the surface of the workpiece W is in focus can be captured. For example, the focus can be adjusted by changing the distance b.

The amount of change in the distance b can be represented as an offset from the reference distance. In the present embodiment, this offset is referred to as "lens extension amount". By associating the distance from the surface of the lens 12a on the work W side to the surface of the work W with the lens movement amount, the lens movement amount for obtaining an image in focus on the surface of the work W can be determined. . Note that the reference value of the distance b can be arbitrarily determined. For example, the reference value of the distance b may be the value of the focal length f.

As described above, the imaging device 10 has an autofocus function. Therefore, the imaging device 10 determines the degree of focus from the image of the workpiece W and adjusts the focus position. The configuration of the lens 12a for adjusting the focus position is not particularly limited. An example of the configuration of the lens 12a will be described below.

FIG. 4 is a diagram showing an example of the configuration of the lens 12a. In the configuration shown in FIG. 4, the focus adjustment unit 12e moves the lens 12a along the optical axis direction. By changing the position of the lens 12a, the extension amount of the lens 12a changes. Therefore, the lens 12a moves so that the image of the workpiece W is formed on the imaging surface 13a according to the change in the working distance WD.

3 and 4 show an example of one lens. Normally, FA lenses are often composed of a combination of a plurality of lenses. However, even in the combined lens, the combined focal length f and the position of the lens principal point can be obtained using the focal length f of each lens and the positional relationship between the lenses. WD can be calculated by using the characteristic value.

FIG. 4 shows an example in which the focus position is adjusted by a mechanical method. However, the method of adjusting the focus position is not limited to the mechanical method. FIG. 5 is a diagram showing another example of the focus position adjusting lens.

Illustrated in FIG. 5 is a liquid lens. The lens 12a includes a translucent container 70, electrodes 73a, 73b, 74a, 74b, insulators 75a, 75b, and insulating layers 76a, 76b.

A closed space in the translucent container 70 is filled with a conductive liquid 71 such as water and an insulating liquid 72 such as oil. The conductive liquid 71 and the insulating liquid 72 are not mixed and have different refractive indices.

Electrodes 73 a and 73 b are fixed between insulators 75 a and 75 b and translucent container 70 respectively and positioned in conductive liquid 71 .

Electrodes 74 a and 74 b are arranged near the edges of the interface between conductive liquid 71 and insulating liquid 72 . An insulating layer 76a is interposed between the electrode 74a and the conductive liquid 71 and insulating liquid 72 . An insulating layer 76b is interposed between the electrode 74b and the conductive liquid 71 and insulating liquid 72 . The electrodes 74a and 74b are arranged at symmetrical positions with respect to the optical axis of the lens 12a.

In the configuration shown in FIG. 5, the focus adjustment section 12e includes a voltage source 12e1 and a voltage source 12e2. The voltage source 12e1 applies a voltage Va between the electrodes 74a and 73a. Voltage source 12e2 applies voltage Vb between electrode 74b and electrode 73b.

When a voltage Va is applied between the electrodes 74a and 73a, the conductive liquid 71 is pulled by the electrodes 74a. Similarly, when voltage Vb is applied between electrodes 74b and 73b, conductive liquid 71 is pulled toward electrode 74b. This changes the curvature of the interface between the conductive liquid 71 and the insulating liquid 72 . Since the refractive indices of the conductive liquid 71 and the insulating liquid 72 are different, the curvature of the interface between the conductive liquid 71 and the insulating liquid 72 changes, thereby increasing the focal length of the lens 12a (focal length f ) changes. The curvature of the interface depends on the magnitude of the voltages Va and Vb. Therefore, by changing the voltages Va and Vb, the focus position of the lens 12a can be adjusted so that an image is formed on the imaging surface 13a even if the working distance WD changes.

Normally, voltage Va and voltage Vb are controlled to have the same value. As a result, the interface between the conductive liquid 71 and the insulating liquid 72 changes symmetrically with respect to the optical axis. However, voltage Va and voltage Vb may be controlled to different values. As a result, the interface between the conductive liquid 71 and the insulating liquid 72 becomes asymmetric with respect to the optical axis, and the direction of the imaging field of view of the imaging device 10 can be changed.

Furthermore, a liquid lens and a solid lens may be combined. In this case, the position of the principal point of the lens changes in addition to the focal length f. Therefore, since the distance b changes, the focus adjustment should be performed in consideration of the change.

<B. Image processing device>
The image processing device 20 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an auxiliary storage device, a communication I/F, etc., and performs information processing. The auxiliary storage device is composed of, for example, a hard disk drive, a solid state drive, etc., and stores programs and the like executed by the CPU.

The determination unit 21 of the image processing device 20 binarizes, for example, the difference image between the image of the non-defective work previously stored in the storage unit 23, and determines the number of pixels exceeding the threshold value and the reference value. The pass/fail of the position to be inspected may be determined by collation. The output unit 22 of the image processing device 20 may display the determination result on the input/display device 60 .

<C. Examples of workpieces and focus adjustment issues>
FIG. 6 is a diagram schematically showing imaging of the workpiece W by the imaging device. FIG. 7 is a diagram showing a region of the workpiece W to be imaged. In the examples shown in FIGS. 6 and 7, the work W has a region W1 and a region W2. Region W1 is, for example, the surface of a transparent body (such as glass). A region W2 is a region surrounding the region W1, and is, for example, the surface of the housing of the electronic device. An example of such a work W is an electronic device having a display (one example is a smart phone or a tablet). That is, the area W1 can be the display screen. Also, the region W1 does not have a distinct pattern. That is, the area W1 is a blank area.

An inspection area A1 is set within the area W1. The inspection area A1 is imaged by the imaging device 10 . In the appearance inspection, an area including the inspection area A1 is imaged by the imaging device 10, and the image of the inspection area A1 is subjected to predetermined processing by the image processing device 20 (not shown). The image processing device 20 uses the image of the inspection area A1 to inspect whether there is a scratch, stain, or foreign matter in the inspection area A1.

If an image of the inspection area A1 is captured while the inspection area A1 is out of focus, there is a possibility that a flaw or the like in the inspection area A1 cannot be detected from the acquired image. In order to perform a highly accurate appearance inspection, it is required that the imaging device 10 be focused on the inspection area A1. However, if the inspection area A1 is part of a blank area, as in the example above, no distinct pattern is formed in the inspection area A1. In such a case, since the difference in contrast within the inspection area A1 is small, there is a problem that it is difficult for the imaging device 10 to capture an image of the inspection area A1 that is in focus.

<D. Focus Adjustment According to the Present Embodiment>
According to this embodiment, focus adjustment has two modes. A first mode is a mode in which a focus position in a designated area is determined in advance. The designated areas are a reference area that serves as a reference for the focus position, and a target area that is subject to predetermined processing by the image processing unit. The second mode is the mode for determining the focus position in the actual inspection.

FIG. 8 is a diagram showing areas for focus adjustment according to this embodiment. In the present embodiment, an area (reference area B1) serving as a focus reference is set apart from the inspection area A1.

According to the example shown in FIG. 8, the reference area B1 includes part of the area W2. For example, the reference area is an area that includes the edge of area W2 and includes the outline of workpiece W. FIG. Therefore, the reference area B1 includes the boundary between the work W and its surroundings. This allows the reference area B1 to contain a distinct pattern. In this way, the image of the reference area B1 has a contrast that allows determination of the degree of focus.

As shown in FIG. 8, in one embodiment, the reference area B1 and the inspection area A1 are each defined so as to at least partially overlap each other. This allows the reference area B1 and the inspection area A1 to include a common area. Therefore, when determining the focus position of the inspection area A1 in the second mode, it is possible to focus on the inspection area A1 accurately and/or at high speed. Note that "at least a part of the reference area B1 and the inspection area A1 overlap each other" is not limited to the above example. The reference area B1 may be included in the inspection area A1. Conversely, the inspection area A1 may be included in the reference area B1. Further, in the present embodiment, at least a portion of the reference area B1 and the inspection area A1 do not necessarily have to overlap each other. That is, the reference area B1 and the inspection area A1 may not overlap at all.

In the example of FIG. 8, both the reference area B1 and the inspection area A1 are rectangular, but the shape of each of the reference area B1 and the inspection area A1 is not limited to being rectangular. For example, the shape of at least one of the reference area B1 and the inspection area A1 may be circular or any free shape that can form the area. Also, at least one of the reference area B1 and the inspection area A1 need not be limited to being a unified area. For example, at least one of the reference area B1 and the inspection area A1 may be a plurality of dispersed areas.

In this embodiment, prior to actual inspection, a reference area and an inspection area are set, and the focus positions of these areas are registered in the image processing device 20 . FIG. 9 is a diagram for explaining registration of the focus position. Referring to FIG. 9, an image of a sample workpiece W is captured by an imaging device 10 to obtain an image of the workpiece W. As shown in FIG. A reference area B1 and an inspection area A1 are respectively designated from the image of the workpiece W, and the images of the reference area B1 and the inspection area A1 are registered in the image processing device 20 .

Furthermore, the focus position when the reference region B1 is in focus and the focus position when the inspection region A1 is in focus are registered in the image processing device 20, respectively. Assume that the focus positions of the reference area B1 and the inspection area A1 are f0 and f1, respectively. The image processing device 20 registers the focus position values (f0, f1) inside the image processing device 20 . Alternatively, the image processing device 20 may convert the value of the focus position into a lens extension amount and store the value of the extension amount.

FIG. 10 is a diagram for explaining focus position adjustment during inspection. Referring to FIG. 10, an image of workpiece W to be inspected is captured by imaging device 10 . The image processing device 20 searches the reference area B1 from the image of the workpiece W and obtains the focus position of the reference area B1. Assume that the focus position is f2.

Next, the focus position of the inspection area A1 is calculated based on the focus position. The focus position f3 of the inspection area A1 at this time is expressed as f3=f1+(f2-f0). When the focus position f3 of the inspection area A1 is determined as described above, the imaging device 10 images the workpiece W according to the focus position f3. The image processing device 20 inspects the inspection area A1 using the image.

FIG. 11 is a diagram for explaining an example of designation of a reference area. FIG. 12 is a diagram for explaining an example of specifying an inspection area. As shown in FIGS. 11 and 12, input/display device 60 has a display. The display displays an image of the work W imaged by the imaging device 10 . The user designates the reference area B1 while looking at the image of the workpiece W (see FIG. 11). Next, the user designates an inspection area A1 while referring to the same image. For example, the user uses a pointing device such as a mouse to designate each of the reference area B1 and the inspection area A1. At this time, a frame line indicating the area designated by the user is displayed on the display of the input/display device 60 . Note that when specifying the inspection area A1, a frame line indicating the reference area B1 may be displayed at the same time.

FIG. 13 is a flow chart showing the flow of the focus position setting method. Note that the processing shown in FIG. 13 corresponds to the processing of the imaging device 10 and the image processing device 20 in the "first mode" described above. Further, the processing of the flowchart described below is executed by the determination unit 21 of the image processing device 20 reading out the program stored in the storage unit 23 . The same applies to the processing shown in FIG. 16, which will be described later.

Referring to FIG. 13, in step S11, imaging device 10 images a sample of an inspection object (hereinafter, "work sample"). The image processing device 20 acquires image data of the work sample from the imaging device 10 .

In step S12, a reference area B1 is set. An image of the work sample is displayed, for example, on the display of the input/display device 60, as described above. A partial area of the image is set as the reference area by the user operating an input device such as a pointing device. The image processing device 20 stores the image data of the reference region B1 in the storage unit 23. FIG.

In step S13, the focus position of the reference area B1 is set. For example, the imaging device 10 images the work sample while changing the focus position of the lens 12a. Thereby, a plurality of images of the reference area B1 are obtained. Each of the multiple images is determined and the most focused image is selected from among them. A focus position corresponding to the image is determined as the focus position f0 of the reference area B1. The value of f0 is stored in the storage unit 23 of the image processing device 20. FIG.

The method for selecting the most focused image from multiple images is not limited. A user may select one image from a plurality of images. The image processing device 20 may determine the most focused image among the plurality of images. For example, the image processing device 20 may select, from among the plurality of images, the image in which the edge of the work W has the greatest change in color, brightness, or the like.

In step S14, an inspection area A1 is set. As in the process of step S12, the display of the input/display device 60 displays the image of the work sample obtained by the process of step S11. A partial area of the image is set as the inspection area A1 by the user operating an input device such as a pointing device. The image processing device 20 stores the image data of the inspection area A1 in the storage unit 23 .

In step S15, the focus position f1 of the inspection area A1 is set. As a method of obtaining the focus position f1, for example, the following method can be exemplified.

A first method is to obtain the focus position f1 using a jig. FIG. 14 shows an example of a jig for obtaining the focus position f1 of the inspection area A1. FIG. 15 is a diagram showing a method of obtaining the focus position f1 of the inspection area A1 using the jig shown in FIG. As shown in FIGS. 14 and 15, the jig 100 is a transparent glass flat plate having a printing surface 101 on which a black and white pattern 102 is printed. The jig 100 is installed so that the pattern 102 of the jig 100 is in contact with the work sample region W1. As a result, a temporary pattern 102 is formed on the area W1 of the work sample as seen from the imaging device 10 . The temporarily formed pattern can accurately determine the focus position of the inspection area A1. Using this focus position, the focus position of the inspection area A1 is determined in the actual inspection (second mode). In the second mode, there is no distinct pattern in inspection area A1. However, a more accurate focus position of inspection area A1 can be determined.

The height of the printing surface 101 of the jig 100 and the height of the work sample region W1 are substantially the same. Therefore, with the pattern 102 as a clue, the focus position f1 can be obtained by the autofocus of the imaging device 10 . Note that the jig 100 is exaggerated in FIG. 14 in order to clearly show the pattern 102 of the jig 100 .

Note that the method for forming the temporary pattern 102 in the inspection area A1 is not limited to the method using the jig 100. FIG. For example, a projector may be used to project a specific pattern onto the inspection area A1 for a predetermined period of time.

A second method is a method of adjusting the focus of the imaging device 10 visually by the user. The inspection area A1 is imaged by the imaging device 10 while changing the focus of the imaging device 10 . This results in multiple images. The user selects the most focused image from multiple images. The focus position corresponding to the image is set to the focus position f1 of the inspection area A1. If there is a trigger for focusing, such as a faint pattern or flaw, in the inspection area A1, the second method can be used to set the focus position f1 of the inspection area A1. Alternatively, a plurality of candidate images may be selected in advance by autofocusing of the imaging device 10, and the user may select an appropriate image from among the candidate images.

A third method is to set the focus position f1 using information about the three-dimensional shape of the workpiece. When the three-dimensional shape of the workpiece is known in advance, the height difference between the reference area B1 and the inspection area A1 can be obtained from the three-dimensional shape information. A focus position f0 of the reference area B1 is converted into a working distance (WD). The working distance of the inspection area A1 is calculated by adding the height difference to the converted working distance. A focus position f1 is set by converting the working distance into a focus position. Conversion from the focus position to the working distance and conversion from the working distance to the focus position are performed by the image processing device 20, for example. With this method, the focus position of the inspection area A1 can be determined from the focus position and height information of the reference area B1. Therefore, the focus position of the inspection area A1 can be determined by a relatively simple calculation.

FIG. 16 is a flow chart showing the flow of the inspection method. The processing shown in FIG. 16 corresponds to the processing of the imaging device 10 and the image processing device 20 in the "second mode" described above. When the process is started as shown in FIG. 16, in step S21, the image processing system 1 (for example, the image processing device 20) searches for the reference area B1. In step S21, the reference area B1 is searched within the search space determined by the focus position when the workpiece W is imaged and the image of the workpiece W. FIG.

In step S22, the image processing device 20 obtains the focus position when the image including the reference area B1 obtained by the search in step S21 is captured. Thereby, the focus position f2 of the reference area B1 is obtained.

In step S23, the image processing device 20 obtains the XY positional deviation of the reference area B1. “XY positional deviation” means relative positional deviation of the image of the reference area obtained by the process of step S21 with respect to the image of the reference area obtained in step S12 (see FIG. 12). This positional deviation includes the amount of deviation along each of the two orthogonal axes (X-axis and Y-axis) in the plane and the rotation angle within the plane. The image processing device 20 obtains the XY positional deviation by comparing the images of the two reference areas. The reference area B1 can also be used as a model for correcting the positional deviation. Therefore, it is possible not only to obtain an image in which the workpiece W is in focus, but also to correct the displacement of the inspection area A1 in the image. Note that the process of step S23 is optional and not essential.

In step S24, the image processing device 20 obtains the focus position f3 of the inspection area A1. The focus position f3 is obtained according to the above formula f3=f1+(f2-f0). In this way, the focus position f3 of the inspection area A1 can be obtained by relatively simple calculation from the registered focus positions f0 and f1 and the focus position f2 obtained in step S22. Therefore, the focus position f3 can be determined in a short time.

In step S25, the imaging device 10 images the inspection area A1 of the workpiece W by adjusting the focus according to the focus position f3. The image processing device 20 acquires an image of the inspection area A1 from the imaging device 10 . This image is an image of the inspection area A1 with the inspection area A1 in focus.

In step S<b>26 , the determination unit 21 of the image processing device 20 processes the image captured by the imaging device 10 .

In step S27, the determination unit 21 determines whether the appearance of the workpiece W is good or bad based on the processed image. In step S28, the output unit 22 of the image processing device 20 outputs the inspection result (determination result). When the process of step S28 ends, the process returns to step S21 for inspection of the next workpiece.

According to this embodiment, the focus positions of the reference area and the inspection area are obtained when setting the focus position. Since the reference area is an area with a distinct pattern, it is easy to acquire a focused image. Therefore, the focus position can be obtained with high accuracy. During inspection, the focus position of the inspection area is corrected using the deviation of the focus position of the reference area. Even if the inspection area does not have a clear pattern (a pattern that is easy to focus), a focused image of the inspection area can be obtained. Therefore, the accuracy of inspection results can be enhanced.

<E. Note>
As described above, the present embodiment includes the following disclosures.

(Configuration 1)
an imaging unit (10) that captures an image of an object (W) and generates an image containing the object (W);
An image processing unit (20) that acquires an image of the object (W) from the imaging unit (10) and performs predetermined processing on the image,
The imaging unit (10) is capable of changing a focus position,
In the first mode in which the focus position of the object (W) is set in advance, the image processing section (20) includes a reference area serving as a reference for the focus position in the image of the object (W). (B1) and a target area (A1) to be subjected to the predetermined processing by the image processing unit (20) are set, and the focus of each of the reference area (B1) and the target area (A1) is set. Register your location and
In the second mode of executing the predetermined process, the image processing section (20) determines the focal position of the reference area (B1) in the image acquired by the imaging section (10) and the reference area (B1). and determining the focus position of the target area (A1) based on the registered focus position of each of the target area (A1);
An image processing system (1), wherein in the second mode, the imaging unit (10) images the target area (A1) at a determined focus position.

(Configuration 2)
The image processing system (1) according to configuration 1, wherein the reference area (B1) and the target area (A1) are each defined to at least partially overlap each other.

(Composition 3)
In the second mode, the image processing section (20) converts the position of the reference area (B1) in the image captured by the imaging section (10) to the position of the reference area (B1) registered in the first mode. An image processing system (1) according to configuration 1, wherein the position of the object (W) in the image is modified in comparison with the position of a reference area (B1) and based on the result of the comparison.

(Composition 4)
In the second mode, the image processing section (20) processes the registered focus position of the reference area (B1) and the focus position of the reference area (B1) acquired in the second mode. The focus position of the registered target area (A1) is corrected using the difference from the target area (A1) to determine the focus position of the target area (A1). An image processing system (1).

(Composition 5)
An image processing system (1) according to any of arrangements 1 to 4, wherein said target area (A1) comprises a plain area.

(Composition 6)
In the first mode, the image processing section (20), based on the image obtained by the imaging section (10) capturing the pattern temporarily formed in the target area (A1), 6. An image processing system (1) according to configuration 5, wherein said focus position of said target area (A1) is determined.

(Composition 7)
In the first mode, the image processing unit (20) calculates the height difference between the reference area (B1) and the target area (A1) from the three-dimensional shape information of the target object (W). and determining the focus position of the target area (A1) using the height difference and the focus position of the reference area (B1). The image processing system according to configuration 5 ( 1).

(Composition 8)
an imaging unit (10) for imaging an object (W) and generating an image including the object (W); obtaining an image of the object (W) from the imaging unit (10); An image processing method by an image processing system (1) comprising an image processing unit (20) for executing predetermined processing on an image,
The imaging unit (10) is capable of changing a focus position,
The image processing method includes
In a first mode for presetting the focus position of the object (W), the image processing unit (20) includes a reference area serving as a reference for the focus position in the image of the object (W). (B1) and a target area (A1) to be subjected to the predetermined processing by the image processing unit (20) are set, and the focus of each of the reference area (B1) and the target area (A1) is set. registering a location;
In the second mode for executing the predetermined process, the image processing section (20) determines the focus position of the reference area (B1) in the image acquired by the imaging section (10) and the reference area (B1). and determining the focus position of the target area (A1) based on the registered focus position of each of the target area (A1);
in the second mode, the imaging unit (10) imaging the target area (A1) at the determined focus position;
An image processing method, comprising: performing the predetermined processing by using the image of the object (W) imaged at the determined focus position by the image processing unit (20).

(Composition 9)
A program for executing a predetermined process using an image obtained by imaging an object (W) with an imaging device (10),
The imaging device (10) is capable of changing a focus position,
The program, in a computer (20),
In a first mode for presetting the focus position of the object (W), the image of the object (W) includes a reference area (B1) serving as a reference for the focus position and the predetermined process. A step of setting a target area (A1) to be the target of and registering the focus position of each of the reference area (B1) and the target area (A1);
In the second mode of executing the predetermined process, the focus position of the reference area (B1) in the image acquired by the imaging device (10), and each of the reference area (B1) and the target area (A1) determining the focus position of the target area (A1) based on the registered focus position of
and executing the predetermined process using the image of the object (W) captured at the determined focus position.

Although the embodiments of the present invention have been described, the embodiments disclosed this time should be considered as illustrative in all respects and not restrictive. The scope of the present invention is indicated by the scope of claims, and is intended to include all modifications within the meaning and scope of equivalence to the scope of claims.

Reference Signs List 1 image processing system, 10 imaging device, 11 illumination section, 12 lens module, 12a, 12c lenses, 12b lens group, 12d movable section, 12e1, 12e2 voltage source, 12e focus adjustment section, 13 imaging device, 13a imaging surface, 14 Imaging element control unit 15, 17 register 16 lens control unit 18 communication I/F unit 20 image processing device 21 determination unit 22 output unit 23 storage unit 24 command generation unit 60 display device 70 transparency Optical container 71 Conductive liquid 72 Insulating liquid 73a, 73b, 74a, 74b Electrode 75a, 75b Insulator 76a, 76b Insulating layer 90 Stage 100 Jig 101 Printing surface 102 Pattern A1 Inspection area, B1 reference area, F focus, O principal point, S11 to S15, S21 to S28 steps, W workpiece, W1, W2 areas, f0 to f3 focus position.

Claims (9)

an imaging unit that captures an image of an object and generates an image that includes the object;
an image processing unit that acquires an image of the object from the imaging unit and performs predetermined processing on the image;
The imaging unit is capable of changing a focus position,
In a first mode in which the focus position of the object is set in advance, the image processing unit includes a reference area serving as a reference for the focus position in the image of the object, and the predetermined focus position by the image processing unit. setting a target area to be processed in and registering the focus position of each of the reference area and the target area;
In the second mode of executing the predetermined process, the image processing unit controls the focus position of the reference area in the image acquired by the imaging unit, and the registered focus positions of each of the reference area and the target area. determining the focus position of the region of interest based on
The image processing system, wherein in the second mode, the imaging unit images the target area at a determined focus position. 2. The image processing system according to claim 1, wherein said reference area and said target area are each defined so as to at least partially overlap each other. In the second mode, the image processing unit compares the position of the reference area in the image captured by the imaging unit with the position of the reference area registered in the first mode, 2. The image processing system of claim 1, wherein the position of the object within the image is modified based on the result of the comparison. In the second mode, the image processing unit uses a difference between the registered focus position of the reference area and the focus position of the reference area acquired in the second mode to perform registration. 4. The image processing system according to any one of claims 1 to 3, wherein the focus position of the target area is determined by correcting the focus position of the target area. 5. An image processing system as claimed in any one of claims 1 to 4, wherein the region of interest comprises a solid color region. In the first mode, the image processing section adjusts the focus position of the target area based on the image obtained by the imaging section capturing an image of a pattern temporarily formed in the target area. 6. The image processing system of claim 5, wherein determining. In the first mode, the image processing unit obtains a difference in height between the reference region and the target region from information on the three-dimensional shape of the target, and obtains the difference in height; 6. The image processing system of claim 5, wherein the focus position of the target region is determined using the focus position of the reference region. an imaging unit that captures an image of an object and generates an image including the object; and an image processing unit that acquires the image of the object from the imaging unit and performs predetermined processing on the image. An image processing method by an image processing system comprising
The imaging unit is capable of changing a focus position,
The image processing method includes
In a first mode in which the focus position of the object is set in advance, the image processing unit includes a reference area serving as a reference for the focus position in the image of the object, and the predetermined focus position by the image processing unit. a step of setting a target region to be processed in and registering the focus position of each of the reference region and the target region;
In the second mode of executing the predetermined process, the image processing unit performs the focus position of the reference area in the image acquired by the imaging unit, and the registered focus positions of each of the reference area and the target area. determining the focus position of the region of interest based on
In the second mode, the imaging unit imaging the target area at the determined focus position;
An image processing method, wherein the image processing unit performs the predetermined processing using the image of the object imaged at the determined focus position. A program for executing a predetermined process using an image obtained by imaging an object with an imaging device,
The imaging device is capable of changing the focus position,
The program, in a computer,
In a first mode in which the focus position of the object is preset, a reference area serving as a reference for the focus position and a target area to be subjected to the predetermined processing are set in the image of the object. and registering the focus position of each of the reference area and the target area;
In the second mode of executing the predetermined process, based on the focus position of the reference area in the image acquired by the imaging device and the registered focus positions of each of the reference area and the target area, determining the focus position of a region of interest;
and executing the predetermined process using the image of the object captured at the determined focus position.

Images