WO2023162940A1 - Inspection device, inspection method, and inspection program - Google Patents

Inspection device, inspection method, and inspection program Download PDF

Info

Publication number
WO2023162940A1
WO2023162940A1 PCT/JP2023/006090 JP2023006090W WO2023162940A1 WO 2023162940 A1 WO2023162940 A1 WO 2023162940A1 JP 2023006090 W JP2023006090 W JP 2023006090W WO 2023162940 A1 WO2023162940 A1 WO 2023162940A1
Authority
WO
WIPO (PCT)
Prior art keywords
inspection
region
captured image
area
image
Prior art date
Application number
PCT/JP2023/006090
Other languages
French (fr)
Japanese (ja)
Inventor
麻理恵 神田
和久 大沼
貴之 藤堂
Original Assignee
ｉ－ＰＲＯ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ｉ－ＰＲＯ株式会社 filed Critical ｉ－ＰＲＯ株式会社
Publication of WO2023162940A1 publication Critical patent/WO2023162940A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • G01N21/956—Inspecting patterns on the surface of objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to an inspection device, an inspection method, and an inspection program.
  • Patent Literature 1 discloses a technique for obtaining an optimal illumination pattern for detecting so-called defects in an inspection object, such as damage and defective products, based on a group of images captured using a plurality of illumination patterns. is disclosed. Further, Patent Literature 2 discloses a technique of obtaining an illumination pattern for suppressing non-defective product variations and individual variations of inspection objects while emphasizing defects of the inspection objects.
  • Patent Documents 1 and 2 detect defects in an inspection object included in a group of captured images. It is valid.
  • an illumination pattern is obtained by applying Patent Documents 1 and 2 to the entire substrate as one inspection target.
  • the illumination pattern obtained when the entire board is treated as one inspection object is Therefore, the desired inspection accuracy may not be obtained.
  • the present invention has been made to solve such problems, and aims to improve the accuracy of appearance inspection for a plurality of inspection objects.
  • An inspection apparatus includes one or more processors, a memory, and a program stored in the memory, wherein the program stores a first inspection object and the first inspection object.
  • the program stores a first inspection object and the first inspection object.
  • an inspection target area in which a plurality of inspection objects including a second inspection object different from the and a camera that captures the inspection target area and outputs the captured image of the inspection target area as the captured image of the inspection target area according to a first imaging condition capturing a first captured image and a second captured image captured under a second imaging condition different from the first imaging condition; and including the first inspection object and the second inspection object.
  • an inspection method in an inspection device, in an inspection target area in which a plurality of inspection targets including a first inspection target and a second inspection target different from the first inspection target exist, and setting a first attention area for inspecting the first inspection object and a second attention area for inspecting the second inspection object, imaging the inspection object area, and performing the inspection.
  • a learning model for detecting an abnormality in a plurality of inspection objects including the first inspection object and the second inspection object; and the first inspection object corresponding to the first region of interest.
  • An inspection program provides an inspection target region in which a plurality of inspection targets including a first inspection target and a second inspection target different from the first inspection target exist, wherein the first inspection target is setting a first attention area for inspecting an inspection object and a second attention area for inspecting the second inspection object; a first captured image captured under a first imaging condition and a second captured image captured under a second imaging condition different from the first imaging condition, as captured images of the inspection target area, to a camera that outputs the captured images of and a learning model for detecting anomalies in a plurality of inspection objects including the first inspection object and the second inspection object, and the first corresponding to the first attention area inspecting the first inspection object based on the first area of the captured image; and performing the second inspection based on the learning model and a second area of the first captured image corresponding to the second attention area.
  • the accuracy of appearance inspection for multiple inspection objects is improved.
  • FIG. 1 is a block diagram showing a configuration example of an inspection system according to Embodiment 1.
  • FIG. 2 is a block diagram showing an example of the functional configuration of the inspection apparatus according to Embodiment 1.
  • FIG. 3 is a block diagram illustrating an example of a functional configuration of a management device according to Embodiment 1;
  • FIG. 4 is a diagram showing an example of an attention area set on a work according to the first embodiment.
  • FIG. 5 is a diagram showing an example of an imaging pattern according to Embodiment 1.
  • FIG. 6 is a diagram showing an example of an illumination pattern according to Embodiment 1.
  • FIG. FIG. 7 is a sequence chart showing pre-inspection operations in the inspection system according to the first embodiment.
  • FIG. 8 is a sequence chart showing inspection operations in the inspection system according to the first embodiment.
  • 9 is a flowchart illustrating an example of pre-examination processing according to Embodiment 1.
  • FIG. 10 is a flow chart showing a detailed example of the optimum condition determination process shown in FIG. 11 is a flowchart illustrating an example of inspection processing according to Embodiment 1.
  • FIG. 12 is a schematic diagram showing an example of a UI screen for generating and adjusting an imaging pattern and an illumination pattern according to Embodiment 1.
  • FIG. 13 is a schematic diagram showing an example of a UI screen for setting an attention area according to Embodiment 1.
  • FIG. 14 is a diagram showing a configuration example of attention area information according to Embodiment 1.
  • FIG. 15 is a schematic diagram showing an example of a UI screen for confirming a list of work inspection results according to the first embodiment.
  • FIG. 16 is a schematic diagram showing an example of a UI screen for confirming inspection results in detail according to the first embodiment.
  • FIG. 17 is a schematic diagram showing an example of reducing the size of the work image to the size of the image that can be input to the learning model.
  • FIG. 18 is a schematic diagram showing an example in which a plurality of cameras divides a workpiece into image sizes that can be input to a learning model and captures the images.
  • 19 is a block diagram illustrating an example of a functional configuration of an inspection apparatus according to Embodiment 2; FIG.
  • FIGS. 20A and 20B are diagrams illustrating an example of a first method for generating a synthesized image according to Embodiment 2.
  • FIG. 21A and 21B are diagrams illustrating an example of a second method for generating a synthesized image according to Embodiment 2.
  • FIG. 22 is a flowchart illustrating an example of pre-examination processing according to Embodiment 2.
  • FIG. 23 is a flow chart showing an example of the optimum condition determination process shown in FIG. 22.
  • FIG. 24 is a flowchart illustrating an example of inspection processing according to Embodiment 2.
  • FIG. FIG. 25 is a flowchart showing a modification of inspection processing according to the second embodiment.
  • FIG. 1 is a block diagram showing a configuration example of an inspection system 10 according to Embodiment 1. As shown in FIG. 1
  • the inspection system 10 is a system for automatically inspecting whether or not the component 2 is normally mounted on the board 1 .
  • the board 1 to be inspected, on which the component 2 is mounted is called a work 3 (see FIG. 4).
  • one component 2a is referred to as a first inspection object
  • the other component 2b is referred to as a second inspection object. good too.
  • the inspection system 10 includes an inspection device 20, an illumination device 30, a detection sensor 19, a management device 40, an input device 50, a speaker 52, a display device 60, and Patlite (registered trademark). ) 62.
  • the inspection device 20 captures an image of the workpiece 3 to be inspected, and inspects whether or not the component 2 is normally mounted on the board 1 based on the captured image.
  • a captured image of the workpiece 3 is hereinafter referred to as a workpiece image 4 (see FIG. 4).
  • the illumination device 30 illuminates the work 3 when the inspection device 20 images the work 3 .
  • the lighting device 30 is connected to the inspection device 20 via a predetermined cable 12 . Note that the illumination device 30 may be read as a light.
  • the management device 40 operates the inspection device 20 and displays inspection results.
  • the management device 40 is, for example, an information processing device represented by a PC (Personal Computer).
  • the management device 40 is connected to the inspection device 20 via a predetermined communication network 11 .
  • the communication network 11 may be either a wired LAN or a wireless LAN.
  • the input device 50 receives input operations from the user. Examples of the input device 50 include a keyboard, mouse, touchpad, microphone, or the like.
  • the input device 50 is connected to the management device 40 via a predetermined cable 13 or wireless communication (for example, Bluetooth (registered trademark)). Note that the input device 50 may be connected to the inspection device 20 .
  • the speaker 52 outputs audio related to the examination.
  • the speaker 52 is connected to the management device 40 via a predetermined cable 16.
  • the display device 60 displays a screen regarding inspection.
  • Examples of the display device 60 include an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display.
  • a display device 60 is connected to the management device 40 via a predetermined cable 14 .
  • the display device 60 may be connected to the inspection device 20 .
  • the input device 50, the speaker 52, and the display device 60 may be an integrated device (such as a tablet terminal, for example).
  • the management device 40, the input device 50, the speaker 52, and the display device 60 may be an integrated device.
  • the detection sensor 19 is a sensor for detecting the arrival of the workpiece 3 to be inspected.
  • the detection sensor 19 is connected to the inspection device 20 via a predetermined cable 15 . Note that the detection sensor 19 may be connected to the management device 40 .
  • the patrol light 62 is connected to the inspection device 20 via a predetermined cable 17.
  • the patrol light 62 blinks when receiving a predetermined signal transmitted according to the inspection result from the inspection device 20 .
  • the inspection apparatus 20 transmits a predetermined signal to the patrol light 62 when the abnormally attached component 2 is detected.
  • the patrol light 62 receives the signal and blinks. Accordingly, the user can immediately know that the inspection apparatus 20 has detected an abnormally mounted component 2 by seeing the flashing of the patrol light 62 .
  • inspection device 20 the lighting device 30, and the management device 40 are separate devices.
  • inspection device 20 may include illumination device 30 .
  • inspection device 20 may include management device 40 .
  • inspection device 20 may include illumination device 30 and management device 40 .
  • the inspection apparatus 20 includes a camera 21, one or more processors 22, a ROM (Read Only Memory) 23, a RAM (Random Access Memory) 24, a storage 25, a communication I/F (Interface) 26, and , and an input/output I/F 27 .
  • the camera 21, the processor 22, the ROM 23, the RAM 24, the storage 25, the communication I/F 26, and the input/output I/F 27 are connected via a bidirectional communicable bus (not shown).
  • the camera 21 includes, for example, a lens and an imaging device.
  • the camera 21 captures an image of the workpiece 3 to be inspected and generates a workpiece image 4 .
  • the camera 21 may be read as another term such as an imaging device or an imaging unit.
  • the processor 22 controls the operation of the inspection device 20 as a whole.
  • the processor 22 may be read as other terms such as computing means, CPU (Central Processing Unit), or controller.
  • the ROM 23 is a read-only non-volatile storage medium in which programs such as firmware are stored.
  • the RAM 24 is a volatile storage medium that enables high-speed reading and writing of information, and is used as a work area when the processor 22 processes information. Note that the RAM 24 may simply be read as a memory.
  • the storage 25 is a non-volatile storage medium from which information can be read and written, and stores various control programs, application programs, learning models 110 (see FIG. 2), and the like.
  • the storage 25 is configured by, for example, a flash memory or an SD card.
  • the communication I/F 26 is an interface for connecting the inspection device 20 to the communication network 11.
  • the communication I/F 26 may be an interface compatible with either a wired LAN or a wireless LAN.
  • the input/output I/F 27 is an interface for connecting the lighting device 30 and the detection sensor 19 .
  • the input device 50 and/or the display device 60 may be connected to the input/output I/F 27 .
  • Management device 40 includes one or more processors 41 , ROM 42 , RAM 43 , storage 44 , communication I/F 45 and input/output I/F 46 .
  • Processor 41, ROM 42, RAM 43, storage 44, communication I/F 45, and input/output I/F 46 are connected via a bus (not shown) capable of two-way communication.
  • the processor 41 controls the overall operation of the management device 40.
  • the ROM 42 is a read-only non-volatile storage medium in which programs such as firmware are stored.
  • the RAM 43 is a volatile storage medium that enables high-speed reading and writing of information, and is used as a work area when the processor 41 processes information. Note that the RAM 43 may simply be read as a memory.
  • the storage 44 is a non-volatile storage medium from which information can be read and written, and stores an OS (Operating System), various control programs, application programs, learning models 110 (see FIG. 3), and the like.
  • the storage 44 is configured by, for example, flash memory, SSD (Solid State Drive), or HDD (Hard Disk Drive).
  • the communication I/F 45 is an interface for connecting the management device 40 to the communication network 11.
  • the communication I/F 45 may be an interface compatible with either a wired LAN or a wireless LAN.
  • the input/output I/F 46 is an interface for connecting the input device 50 and/or the display device 60.
  • the management device 40 may include a GPU (Graphics Processing Unit) for processing image drawing at high speed.
  • GPU Graphics Processing Unit
  • the illumination device 30 includes an LED (Light Emitting Diode) light source 31 , an input/output I/F 32 and a dimming control circuit 33 .
  • LED Light Emitting Diode
  • the LED light source 31 includes a plurality of LEDs and is capable of emitting light.
  • the illumination device 30 may have a plurality of LED light sources 31 with different shapes.
  • the illumination device 30 may include a bar-shaped LED light source 31 , a multi-angle-shaped LED light source 31 , a dome-shaped LED light source 31 , and a backlight-shaped LED light source 31 .
  • the illumination device 30 may include an LED light source 31 that emits infrared rays.
  • the input/output I/F 32 is an interface for connecting the inspection device 20 .
  • the dimming control circuit 33 controls light emission of the LED light source 31 based on instructions from the inspection device 20 received through the input/output I/F 32 .
  • the dimming control circuit 33 controls which LED light source 31 is to emit light, the color and intensity of illumination of the LED light source 31, and the like.
  • FIG. 2 is a block diagram showing an example of the functional configuration of the inspection device 20 according to Embodiment 1. As shown in FIG.
  • the inspection apparatus 20 includes an imaging control unit 101, an optimum condition determination unit 102, an inspection execution unit 103, an attention area storage unit 104, an imaging condition storage unit 105, an optimum imaging condition storage unit 106, and a learning model storage unit. 107.
  • the functions of the imaging control unit 101, the optimum condition determination unit 102, and the inspection execution unit 103 may be realized by the processor 22 cooperating with the RAM 24 (memory) and the like to execute a computer program (inspection program).
  • Functions of the imaging condition storage unit 105 , the optimum imaging condition storage unit 106 , and the learning model storage unit 107 may be realized by the RAM 24 (memory) and/or the storage 25 .
  • the attention area storage unit 104 stores information (hereinafter referred to as attention area information) regarding a plurality of attention areas 5 (see FIG. 4) set on the workpiece 3 to be inspected.
  • the region of interest 5 is a region set surrounding the part 2, which is the object to be inspected, of the work 3, which is the object to be inspected.
  • a region in which a plurality of parts 5, which are objects to be inspected, exist on the workpiece 3 may be referred to as a region to be inspected. Details of the attention area 5 will be described later (see FIG. 4).
  • the imaging condition storage unit 105 stores information regarding a plurality of imaging conditions.
  • the imaging conditions include information about imaging by the camera 21 and information about lighting by the lighting device 30 . Details of the imaging conditions will be described later (see FIGS. 6 and 7).
  • the optimum imaging condition storage unit 106 stores the optimum imaging condition (hereinafter referred to as the optimum imaging condition) for the workpiece 3 to be inspected among the plurality of imaging conditions stored in the imaging condition storage unit 105 . Details of the optimum imaging conditions will be described later (see FIGS. 9 and 10).
  • the learning model storage unit 107 stores a learning model 110 used for detecting whether or not the component 2, which is the inspection object, is normally mounted in the attention area 5 of the work image 4 obtained by imaging the work 3. .
  • the imaging control unit 101 controls the camera 21 and the lighting device 30 to image the workpiece 3 to be inspected and generate the workpiece image 4 .
  • the imaging control unit 101 may adjust the image quality of the workpiece image 4 .
  • the optimum condition determining unit 102 determines the optimum imaging condition (optimum imaging condition) for each attention area 5 of the workpiece image 4 to be inspected from among the plurality of imaging conditions stored in the imaging condition storage unit 105 .
  • the optimal condition determination unit 102 stores the determined optimal imaging conditions in the optimal imaging condition storage unit 106 .
  • the inspection executing unit 103 uses the work image 4 captured by the imaging control unit 101 based on the optimal imaging conditions stored in the optimal imaging condition storage unit 106 to determine whether the component 2 is normal in the area corresponding to each attention area 5. Check whether it is attached to the The inspection executing unit 103 inputs the image of the area corresponding to the attention area 5 of the workpiece image 4 to the learning model 110 stored in the learning model storage unit 107, thereby executing the inspection.
  • FIG. 3 is a block diagram showing an example of functional configuration of the management device 40 according to the first embodiment.
  • the management device 40 has, as functions, an attention area setting unit 201, an imaging pattern generation unit 202, an illumination pattern generation unit 203, an imaging condition generation unit 204, a learning model generation unit 205, a UI control unit 206, an attention It has an area storage unit 207 , an imaging pattern storage unit 208 , an illumination pattern storage unit 209 , an imaging condition storage unit 210 , a learning model storage unit 211 , and an inspection result storage unit 212 .
  • the functions of the attention area setting unit 201, the imaging pattern generation unit 202, the illumination pattern generation unit 203, the imaging condition generation unit 204, the learning model generation unit 205, and the UI control unit 206 are performed by the processor 41 in cooperation with the RAM 43 (memory) and the like.
  • the functions of the region-of-interest storage unit 207, the imaging pattern storage unit 208, the illumination pattern storage unit 209, the imaging condition storage unit 210, the learning model storage unit 211, and the inspection result storage unit 212 are implemented by the RAM 43 (memory) and/or the storage 44. may be realized by
  • the region-of-interest storage unit 207 stores information (region-of-interest information) on a plurality of regions of interest 5 set for each work 3 .
  • the imaging pattern storage unit 208 stores a plurality of imaging patterns. Details of the imaging pattern will be described later (see FIG. 5).
  • the illumination pattern storage unit 209 stores a plurality of illumination patterns. Details of the illumination pattern will be described later (see FIG. 6).
  • the imaging condition storage unit 210 stores information regarding a plurality of imaging conditions. Details of the imaging conditions will be described later.
  • the learning model storage unit 211 stores the learning model 110 used to detect whether or not the component 2 is normally mounted in the attention area 5 of the work image 4 .
  • the inspection result storage unit 212 stores information indicating the inspection result of the work 3 inspected by the inspection device 20 (hereinafter referred to as work inspection result information).
  • the attention area setting unit 201 sets the attention area 5 so as to surround the part 2 which is the inspection object of the workpiece 3, and stores information (attention area information) on the attention area 5 in the attention area storage unit 207.
  • the attention area setting unit 201 also acquires attention area information associated with the workpiece 3 to be inspected from the attention area storage unit 207 and transmits the attention area information to the inspection apparatus 20 .
  • the inspection apparatus 20 stores the transmitted attention area information in the attention area storage unit 104 .
  • the imaging pattern generation unit 202 generates a plurality of imaging patterns and stores them in the imaging pattern storage unit 208 . Details of the imaging pattern will be described later (see FIG. 5).
  • the illumination pattern generation unit 203 generates a plurality of illumination patterns and stores them in the illumination pattern generation unit 203. Details of the illumination pattern will be described later (see FIG. 6).
  • the imaging condition generation unit 204 generates imaging conditions by combining the imaging pattern stored in the imaging pattern storage unit 208 and the illumination pattern stored in the illumination pattern storage unit 209 .
  • the imaging condition generation unit 204 stores the generated imaging conditions in the imaging condition storage unit 210 .
  • the imaging condition generation unit 204 also transmits a plurality of imaging conditions stored in the imaging condition storage unit 210 to the inspection device 20 .
  • the inspection device 20 stores the imaging conditions received from the management device 40 in the imaging condition storage unit 105 .
  • the learning model generation unit 205 generates and learns the learning model 110 used to check whether the component 2 is normally mounted in the attention area 5 .
  • the learning model generation unit 205 learns the learning model 110 using the feature amount of images (hereinafter referred to as attention area images) in the attention areas 5 of a plurality of work images 4 on which the parts 2 are normally mounted. I do.
  • attention area images the feature amount of images
  • the learning model 110 determines how much the feature amount of the input attention area image differs from the feature amount of the attention area image in which the part 2 is normally mounted. Output as an evaluation value.
  • the learning model 110 When a region-of-interest image is input, the learning model 110 outputs (that is, infers) a smaller evaluation value as the probability that the component 2 included in the region-of-interest image is normally mounted is higher. In other words, when the attention area image is input, the learning model 110 outputs (that is, infers) a larger evaluation value as the possibility that the part 2 included in the attention area image is abnormally mounted is higher. .
  • the learning model generation unit 205 stores the learned learning model 110 in the learning model storage unit 211 . Also, the learning model generation unit 205 transmits the learned learning model 110 to the inspection device 20 .
  • the inspection device 20 stores the learning model 110 received from the management device 40 in the learning model storage unit 107 .
  • the learning model 110 may be configured as a neural network for image analysis, a deep neural network, or a CNN (Convolutional Neural Network). However, the learning model 110 is not limited to these, and may be configured based on various artificial intelligence techniques or machine learning techniques. As described above, the present embodiment exemplifies the case of using the learning model 100 that outputs a larger evaluation value as the possibility that the component 2 is abnormally attached is higher. However, the present embodiment may be configured using the learning model 100 that outputs a larger evaluation value as the possibility that the component 2 is normally mounted is higher. Moreover, although the management device 40 has the learning model generation unit 205 in the present embodiment, the inspection device 20 may have the learning model generation unit 205 .
  • the UI control unit 206 generates a UI screen regarding examination and displays it on the display device 60 . Also, the UI control unit 206 receives input from the input device 50 and controls input and display of various information. For example, the UI control unit 206 generates a UI screen showing the inspection result of the work 3 based on the work inspection result information transmitted from the inspection device 20 and displays it on the display device 60 . Thereby, the user can visually recognize the inspection result for the workpiece 3 . Also, the UI control unit 206 stores workpiece inspection result information transmitted from the inspection apparatus 20 in the inspection result storage unit 212 . Details of the UI control unit 206 will be described later (see FIGS. 12, 13, 15, and 16).
  • FIG. 4 is a diagram showing an example of the attention area 5 set on the workpiece 3 according to the first embodiment.
  • the attention area setting unit 201 sets an attention area 5 (for example, attention areas 5a and 5b) so as to surround a part 2 (for example, parts 2a and 2b) that is an inspection target on the workpiece 3.
  • FIG. The part 2a may be read as the first inspection object, and the attention area 5a surrounding the part 2a may be read as the first attention area.
  • the part 2b may be read as the second inspection object, and the attention area 5b surrounding the part 2b may be read as the second attention area.
  • the shape of the attention area 5 may be rectangular. However, the shape of the attention area 5 is not limited to a rectangle, and may be polygonal, elliptical, or the like.
  • the attention area setting unit 201 sets the attention area 5 by, for example, one of the following methods (A1) and (A2).
  • the attention area setting unit 201 sets the attention area 5 based on the design information of the workpiece 3 .
  • the design information is information that determines the position of the component 2 that is the object to be inspected on the board 1 . Therefore, the attention area setting unit 201 automatically identifies the position on the board 1 where the inspection target component 2 is mounted based on the design information, and sets the attention area 5 at the identified position.
  • the user manually sets the attention area 5 on the work image 4 through the UI screen provided by the UI control unit 206 .
  • the user surrounds the part 2 of the inspection object in the workpiece image 4 through the UI screen.
  • the attention area setting unit 201 sets the enclosed area as the attention area 5 of the component 2 .
  • the attention area setting unit 201 generates attention area information indicating the attention area 5 set by the method (A1) or (A2) above, and stores it in the attention area storage unit 207 .
  • FIG. 5 is a diagram showing an example of an imaging pattern according to Embodiment 1.
  • FIG. 5 is a diagram showing an example of an imaging pattern according to Embodiment 1.
  • An imaging pattern is a combination of multiple different imaging parameters.
  • imaging parameters as shown in FIG. 5, are shutter speed, maximum exposure time, lens aperture value, maximum gain, camera sensitivity, brightness, white balance red volume, white balance blue volume, contrast intensity, and dark area correction. , bright area correction, and pedestal level.
  • the imaging pattern generation unit 202 generates a plurality of imaging patterns with at least one different imaging parameter, for example, by either method (B1) or (B2) below.
  • the user manually adjusts each imaging parameter through the UI screen provided by the UI control unit 206 to generate a plurality of imaging patterns. For example, the user generates a first imaging pattern in which each imaging parameter is adjusted so that the dark-colored component 2 attached to the first attention area 5 is appropriately imaged. Also, the user generates a second imaging pattern in which each imaging parameter is adjusted so that the bright-colored component 2 attached to the second attention area 5 is appropriately imaged.
  • the imaging pattern generation unit 202 stores the plurality of imaging patterns generated in this manner in the imaging pattern storage unit 208 .
  • the imaging pattern generation unit 202 analyzes the work image 4 captured by the camera 21 of the inspection device 20, automatically adjusts imaging parameters, and generates a plurality of imaging patterns. For example, the imaging pattern generation unit 202 analyzes the features of the region-of-interest image of the first region of interest 5 of the work image 4, determines each imaging parameter appropriate for the region of interest 5 from the analysis result, and Generate an imaging pattern. Examples of features of the attention area 5 include the color of the component 2, the reflectance of the component 2, the transmittance of the component 2, the material of the component 2, the height of the component 2, and the like.
  • the imaging pattern generation unit 202 analyzes the features of the attention area image of the second attention area 5 of the work image 4, determines each imaging parameter appropriate for the attention area 5 from the analysis result, Generate an imaging pattern.
  • the imaging pattern generation unit 202 stores the plurality of imaging patterns generated in this manner in the imaging pattern storage unit 208 .
  • the management device 40 has the imaging pattern generation unit 202
  • the inspection device 20 may have the imaging pattern generation unit 202 .
  • ⁇ Details of lighting pattern> 6 is a diagram showing an example of an illumination pattern according to Embodiment 1.
  • FIG. 1 is a diagram showing an example of an illumination pattern according to Embodiment 1.
  • a lighting pattern is a combination of multiple different lighting parameters.
  • lighting parameters include lighting shape, lighting method, lighting color, use of polarizing filter, use of infrared lighting, and lighting intensity, as shown in FIG.
  • Examples of lighting parameters related to lighting shape include bar, multi-angle, dome, and backlight.
  • the imaging control unit 101 captures an image of the workpiece 3 based on an illumination pattern in which the illumination parameter related to the shape of the illumination is “bar”, the imaging control unit 101 lights the bar-shaped LED light source 31 provided in the illumination device 30 .
  • Specular reflection, diffuse reflection, and transmission are examples of lighting parameters related to how to apply lighting.
  • Examples of lighting parameters related to lighting colors include blue, red, and green.
  • the illumination pattern generation unit 203 may generate a plurality of illumination patterns with at least one illumination parameter different from each other by the following method. That is, the user manually adjusts the lighting parameters through the UI screen provided by the UI control unit 206 to generate a plurality of lighting patterns.
  • the illumination pattern generation unit 203 stores the plurality of illumination patterns generated in this way in the illumination pattern storage unit 209 .
  • the management device 40 has the illumination pattern generation unit 203
  • the inspection device 20 may have the illumination pattern generation unit 203 .
  • the imaging condition generation unit 204 combines one imaging pattern stored in the imaging pattern storage unit 208 and one illumination pattern stored in the illumination pattern storage unit 209 to generate one imaging condition.
  • the imaging condition generation unit 204 generates a plurality of imaging conditions by combining imaging patterns and illumination patterns in different ways.
  • the imaging condition generation unit 204 transmits a plurality of imaging conditions stored in the imaging condition storage unit 210 to the inspection device 20 .
  • the inspection device 20 stores the plurality of imaging conditions received from the management device 40 in the imaging condition storage unit 105 .
  • FIG. 7 is a sequence chart showing pre-inspection operations in the inspection system 10 according to the first embodiment.
  • the management device 40 displays an inspection main UI screen (not shown) on the display device 60 (step S11).
  • the user inputs the user name on the examination main UI screen through the input device 50 (steps S12 and S13).
  • the management device 40 transmits a request for setting the input user name to the inspection device 20 (step S14).
  • the inspection device 20 receives the user name setting request, completes the setting of the user name, and transmits a completion response to the management device 40 (step S15).
  • the management device 40 receives the completion response from the inspection device 20 and displays on the display device 60 that the input of the user name has been completed (step S16).
  • the user selects the workpiece 3 to be inspected through the input device 50 (steps S17 and S18).
  • the management device 40 transmits a selection request for the selected work 3 to the inspection device 20 (step S19).
  • the inspection device 20 receives the selection request for the work 3, selects the work 3 to be inspected, and transmits a completion response to the management device 40 (step S20).
  • the management device 40 receives the completion response from the inspection device 20, and displays on the display device 60 that the selection of the work 3 to be inspected has been completed (step S21).
  • the management device 40 transmits to the inspection device 20 a request for distribution of the video stream being imaged by the camera 21 (step S22).
  • the inspection device 20 receives the video stream distribution request and transmits a response to the request to the management device 40 (step S23). Also, the inspection device 20 transmits the video stream being imaged by the camera 21 to the management device 40 (step S24).
  • the management device 40 displays the video stream received from the inspection device 20 in real time on the display device 60 (step S25).
  • the user can set the workpiece 3 to be inspected in the inspection device 20 . Also, the user can view the video stream being imaged by the camera 21 of the inspection device 20 in real time.
  • FIG. 8 is a sequence chart showing inspection operations in the inspection system 10 according to the first embodiment.
  • the user When the user starts the inspection, the user inputs inspection mode ON through the input device 50 (steps S31 and S32).
  • the management device 40 transmits the input inspection mode ON request to the inspection device 20 (step S33).
  • the inspection device 20 receives the request to turn on the inspection mode, changes the inspection mode to ON, and transmits a completion response to the management device 40 (step S34).
  • the management device 40 receives the completion response from the inspection device 20, and displays on the display device 60 that switching to inspection mode ON has been completed (step S35).
  • the management device 40 transmits a request for distribution of the video being captured by the camera 21 to the inspection device 20 (step S36).
  • the inspection device 20 receives the video distribution request and transmits an acknowledgment of the distribution request to the management device 40 (step S37). Also, the inspection device 20 transmits the video stream being imaged by the camera 21 to the management device 40 (step S38).
  • the management device 40 displays the video stream received from the inspection device 20 in real time on the display device 60 (step S39).
  • the detection sensor 19 transmits a work detection notification to the inspection device 20 (step S40).
  • the inspection apparatus 20 When the inspection apparatus 20 receives the workpiece detection notification, it transmits an inspection start notification for the workpiece 3 to the management apparatus 40 (step S41).
  • the management device 40 When the management device 40 receives the inspection start notification of the work 3 from the inspection device 20, it displays on the display device 60 that the inspection of the work 3 will start (step S42).
  • the inspection device 20 executes inspection processing of the workpiece 3 (step S43). The details of the inspection process for the workpiece 3 will be described later (see FIG. 11).
  • the inspection device 20 After completing the inspection process of the work 3, the inspection device 20 transmits work inspection result information including the inspection result of the work 3 to the management device 40 (step S44).
  • the management device 40 receives the workpiece inspection result information from the inspection device 20, and displays the content of the workpiece inspection result information on the display device 60 (step S45).
  • the inspection system 10 repeatedly performs the above-described processing of steps S40 to S45 on the workpieces 3 to be inspected that are sequentially conveyed.
  • the user When the user ends the inspection, the user inputs inspection mode OFF through the input device 50 (steps S46 and S47).
  • the management device 40 transmits the input request to turn off the inspection mode to the inspection device 20 (step S48).
  • the inspection device 20 receives the request to turn off the inspection mode, changes the inspection mode to OFF, and transmits a completion response to the management device 40 (step S49).
  • the management device 40 receives the completion response from the inspection device 20, and displays on the display device 60 that switching to inspection mode OFF has been completed (step S50).
  • pre-inspection processing performed before inspection processing of the workpiece 3 will be described with reference to FIGS. 9 and 10.
  • FIG. In the pre-inspection processing, setting of the attention area 5 for the workpiece 3 and determination of optimum imaging conditions are performed.
  • FIG. 9 is a flowchart showing an example of pre-examination processing according to the first embodiment.
  • the imaging control unit 101 controls the camera 21 to image the workpiece 3 and generate the workpiece image 4 (step S101).
  • the attention area setting unit 201 sets a plurality of attention areas 5 on the workpiece 4 (step S102).
  • the attention area setting unit 201 transmits information (attention area information) indicating a plurality of set attention areas 5 to the inspection device 20 .
  • the inspection apparatus 20 stores the received plural pieces of attention area information in the attention area storage unit 104 .
  • the imaging condition generation unit 204 generates a plurality of imaging conditions by combining a plurality of imaging patterns stored in the imaging pattern storage unit 208 and a plurality of illumination patterns stored in the illumination pattern storage unit 209 (step S103).
  • the imaging condition generation unit 204 transmits the plurality of generated imaging conditions to the inspection device 20 .
  • the inspection apparatus 20 stores the received multiple imaging conditions in the imaging condition storage unit 105 .
  • the optimum condition determining unit 102 selects one unselected imaging condition from among the imaging conditions stored in the imaging condition storage unit 105 (step S104).
  • the selected imaging conditions are referred to as selected imaging conditions.
  • the imaging control unit 101 controls the camera 21 and the lighting device 30 based on the selected imaging conditions to image the workpiece 3 and generate the workpiece image 4 (step S105).
  • the imaging control unit 101 stores the generated work image 4 in the RAM 24 or storage 25 .
  • the optimum condition determination unit 102 determines whether or not all imaging conditions stored in the imaging condition storage unit 105 have been selected (step S106).
  • step S106 NO
  • the inspection device 20 returns the process to step S104.
  • step S106 When all imaging conditions have been selected (step S106: YES), the optimum condition determination unit 102 executes optimum condition determination processing (step S107). Details of the optimum condition determination process will be described later (see FIG. 10). Then, the process ends.
  • FIG. 10 is a flow chart showing an example of the optimum condition determination process (step S107) shown in FIG.
  • the optimum condition determination unit 102 selects one unselected attention area 5 from among the plurality of attention areas 5 stored in the attention area storage unit 104 (step S201).
  • the selected attention area 5 is called a selected attention area 5 .
  • the optimum condition determination unit 102 acquires an image of the selected region of interest 5 from each of the plurality of work images 4 captured under different imaging conditions, stored in the RAM 24 or storage 25 in step S105. In the description of FIG. 10, this image is called a selected region-of-interest image.
  • the optimum condition determination unit 102 uses the learning model 110 stored in the learning model storage unit 107 to calculate an evaluation value for each of the acquired selected region-of-interest images (step S202).
  • the optimum condition determination unit 102 determines the optimum imaging conditions for the selected region of interest 5 based on the evaluation value calculated in step S202 (step S203). For example, the optimum condition determining unit 102 determines the imaging condition with the highest calculated evaluation value as the optimum imaging condition for the selected region of interest 5 .
  • the optimal condition determination unit 102 associates the selected attention area 5 stored in the attention area storage unit 104 with the optimal imaging condition determined in step S203 (step S204).
  • the optimum condition determination unit 102 determines whether or not all the attention areas 5 stored in the attention area storage unit 104 have been selected (step S205).
  • step S205 If an unselected region of interest 5 remains (step S205: NO), the inspection device 20 returns the process to step S201. If all the attention areas 5 have been selected (step S205: YES), the inspection device 20 terminates this process.
  • FIG. 11 is a flowchart showing an example of inspection processing according to the first embodiment.
  • the imaging control unit 101 selects the attention area storage unit 104 and the optimum imaging condition storage unit 106 associated with the workpiece 3 to be inspected (step S301).
  • the imaging control unit 101 determines whether or not the detection sensor 19 has detected the workpiece 3 (step S302). For example, the imaging control unit 101 determines whether or not a workpiece detection notification has been received from the detection sensor 19 .
  • step S302 If the detection sensor 19 has not detected the workpiece 3 (step S302: NO), the inspection device 20 returns the process to step S302. If the detection sensor 19 detects the workpiece 3 (step S302: YES), the inspection apparatus 20 advances the process to the next step S303.
  • the imaging control unit 101 selects one unselected optimum imaging condition from the optimum imaging condition storage unit 106 (step S303).
  • the selected optimum imaging conditions are referred to as selected optimum imaging conditions.
  • the imaging control unit 101 controls the camera 21 and the lighting device 30 based on the selected optimum imaging conditions to image the workpiece 3 and generate the workpiece image 4 (step S304).
  • the imaging control unit 101 stores the generated work image 4 in the RAM 43 or storage 44 .
  • the imaging control unit 101 determines whether or not all the optimum imaging conditions included in the optimum imaging condition storage unit 106 have been selected (step S305).
  • step S305: NO If unselected optimum imaging conditions remain (step S305: NO), the inspection apparatus 20 returns the process to step S303. If all the optimum imaging conditions have been selected (step S305: YES), the inspection apparatus 20 advances the process to the next step S306.
  • the inspection execution unit 103 selects one unselected attention area 5 from among the plurality of attention areas 5 stored in the attention area storage unit 104 (step S306).
  • the selected attention area 5 is called a selected attention area 5 .
  • the inspection executing unit 103 picks up images under the optimum imaging conditions associated with the selected region of interest 5 among the plurality of workpiece images 4 picked up under different optimum imaging conditions stored in the RAM 43 or the storage 44 in step S304.
  • the work image 4 that has been displayed is selected (step S307). In the description of FIG. 11, the selected work image 4 will be referred to as a selected work image 4.
  • FIG. 11 the selected work image 4 will be referred to as a selected work image 4.
  • the inspection executing unit 103 uses the learning model 110 stored in the learning model storage unit 107 to calculate the evaluation value of the image in the selected attention area 5 of the selected work image 4 (step S308).
  • the image is referred to as a selected region-of-interest image.
  • the selected workpiece image 4 is captured under optimum imaging conditions for imaging the selected attention area 5 . Therefore, the evaluation value calculated from the selected region-of-interest image obtained from the selected work image 4 is higher than the evaluation value calculated from the selected region-of-interest image obtained from the work image captured under a single imaging condition. , can be more accurate.
  • the inspection execution unit 103 determines the inspection result of the selected region of interest 5 based on the evaluation value calculated in step S308 (step S309). For example, when the evaluation value is less than a predetermined threshold value Th, the inspection execution unit 103 determines that the component 2 is abnormally attached (NG) in the selected attention area 5 . For example, when the evaluation value is equal to or greater than the threshold Th, the inspection execution unit 103 determines that the component 2 is normally mounted (OK) in the selected attention area 5 . The inspection execution unit 103 associates the inspection results with the selected region of interest 5 and stores them in the RAM 43 or the storage 44 .
  • the inspection execution unit 103 determines whether or not all the attention areas 5 stored in the attention area storage unit 104 have been selected (step S310). If there remains an unselected region of interest 5, the inspection apparatus 20 returns the process to step S306.
  • the inspection execution unit 103 collects the inspection results associated with each attention area 5 and stored in the RAM 43 or the storage 44 in step S309 to generate workpiece inspection result information. and transmitted to the management device 40 (step S311). The inspection apparatus 20 then returns the process to step S302 and inspects the work 3 that is next transported.
  • the UI control unit 206 of the management device 40 receives work inspection result information from the inspection device 20 and stores it in the inspection result storage unit 212 .
  • the UI control unit 206 also displays the content of the work inspection result Xu Yufeng on the display device 60 .
  • the user can see the contents of the workpiece inspection result information displayed on the display device 60 and can confirm whether or not the component 2 is normally mounted in each attention area 5 of the workpiece 3 .
  • ⁇ UI screen for patterns> 12 is a schematic diagram showing an example of a UI screen for generating and adjusting an imaging pattern and an illumination pattern according to Embodiment 1.
  • FIG. 12 is a schematic diagram showing an example of a UI screen for generating and adjusting an imaging pattern and an illumination pattern according to Embodiment 1.
  • the UI control unit 206 of the management device 40 displays, on the display device 60, a UI screen (hereinafter referred to as a pattern UI screen 300) for generating or adjusting the imaging pattern and the illumination pattern, as shown in FIG.
  • the pattern UI screen 300 includes an imaging pattern list area 301, an illumination pattern list area 302, an imaging condition-specific area 303, and a workpiece image confirmation area 304, as shown in FIG.
  • the UI control unit 206 lists the imaging patterns stored in the imaging pattern storage unit 208 in the imaging pattern list area 301 .
  • the user can generate a new imaging pattern by adding a new imaging pattern to the imaging pattern list area 301 . Further, the user can adjust the imaging parameters of the imaging patterns stored in the imaging pattern storage unit 208 by adjusting the imaging parameters displayed in the imaging pattern list area 301 .
  • the UI control unit 206 lists the illumination patterns stored in the illumination pattern storage unit 209 in the illumination pattern list area 302 .
  • a user can generate a new lighting pattern by adding a new lighting pattern to the lighting pattern list area 302 .
  • the user can adjust the illumination parameters of the illumination patterns stored in the illumination pattern storage unit 209 by adjusting the illumination parameters displayed in the illumination pattern list area 302 .
  • the UI control unit 206 displays a plurality of workpiece images 4 captured under different imaging conditions in the imaging condition-specific area 303 . Accordingly, the user can confirm what kind of work image 4 is captured for each imaging condition in the imaging condition-specific area 303 .
  • the UI control unit 206 displays the workpiece image 4 selected by the user from the imaging condition-specific area 303 in the workpiece image confirmation area 304 .
  • the UI control unit 206 enlarges or reduces and displays the work image 4 in the work image confirmation area 304 according to the user's operation. This allows the user to check the selected work image 4 in more detail.
  • FIG. 13 is a schematic diagram showing an example of a UI screen for setting the attention area 5 according to the first embodiment.
  • 14 is a diagram showing a configuration example of attention area information according to Embodiment 1.
  • the UI control unit 206 of the management device 40 displays a UI screen for inputting or correcting the attention area 5 (hereinafter referred to as an attention area UI screen 320) on the display device 60, as shown in FIG.
  • the attention area UI screen 320 includes an attention area list area 321 , an attention area confirmation area 322 , and an evaluation value confirmation area 323 .
  • the UI control unit 206 lists the attention area information stored in the attention area storage unit 207 in the attention area list area 321 .
  • the attention area information includes, as parameters, the part name, part number, inspection details, upper left coordinates of the attention area, lower right coordinates of the attention area, preprocessing position correction, imaging pattern, and illumination pattern. have.
  • Part name indicates the name of the part 2.
  • Part number indicates the part number of the part 2.
  • Inspection content indicates whether or not the component 2 is to be inspected. The inspection content also indicates how the part 2 may be mounted abnormally.
  • “Upper left coordinates of attention area” indicates the X and Y coordinates of the upper left point of the rectangular attention area 5 on the workpiece image 4 .
  • “Lower right coordinates of attention area” indicates the X and Y coordinates of the lower right point of the rectangular attention area 5 on the workpiece image 4.
  • FIG. "Pre-processing position correction” indicates whether or not the position of the attention area 5 is corrected in pre-processing.
  • a setting button 324 may be included in the “preprocessing position correction”.
  • a UI screen for setting a correction amount of the position of the attention area 5 by preprocessing and a threshold value Th to be compared with the evaluation value of the attention area 5 is displayed.
  • "Imaging pattern” indicates the imaging pattern of the region of interest 5 under the optimal imaging conditions.
  • "Illumination pattern” indicates the illumination pattern of the target area 5 under the optimum imaging conditions.
  • the user can set new attention area information by adding a new attention area 5 to the attention area list area 321 . Further, the user can adjust the parameters of the attention area information stored in the attention area storage unit 207 by adjusting the parameters of the attention area information displayed in the attention area list area 321 .
  • the UI control unit 206 displays the work image 4 in the attention area confirmation area 322 , and displays a rectangular frame indicating the attention area 5 corresponding to the attention area information selected in the attention area list area 321 on the work image 4 . Display superimposed. Thereby, the user can confirm that the attention area information being selected in the attention area list area 321 corresponds to the attention area 5 of the component 2 at which position on the workpiece image 4 .
  • the UI control unit 206 displays an evaluation value calculated from each attention area image of the attention area 5 captured under different imaging conditions for the attention area information selected in the attention area list area 321 in the evaluation value confirmation area 323 . , is displayed together with the attention area image. Thereby, the user can confirm the relationship between the imaging condition and the evaluation value for each attention area 5 .
  • the attention area setting unit 201 may store the contents input or modified in the attention area list area 321 in the attention area storage unit 207 as attention area information.
  • the attention area information may be configured as CSV data as shown in FIG. 14 .
  • FIG. 15 is a schematic diagram showing an example of a UI screen for checking inspection results of the workpiece 3 according to the first embodiment.
  • the UI control unit 206 uses a plurality of workpiece inspection results stored in the inspection result storage unit 212 to display a UI screen (hereinafter referred to as an inspection result list UI screen 340) for checking the inspection results of the workpiece 3 in a list. ) is displayed on the display device 60 .
  • a UI screen hereinafter referred to as an inspection result list UI screen 340
  • the inspection result list UI screen 340 includes a plurality of work areas 341 .
  • the inspection result list UI screen 340 also includes a work image 4 , an inspection result 342 , and a details button 343 for each work area 341 .
  • the UI control unit 206 displays a workpiece image 4 of one inspected workpiece 3 in one workpiece area 341 .
  • the UI control unit 206 When an abnormality is detected in at least one attention area 5 among the plurality of attention areas 5 in the work image 4 displayed in the work area 341 , the UI control unit 206 outputs an abnormality (NG) as the inspection result 342 . indicate. For the work image 4 displayed in the work area 341 , the UI control unit 206 displays normal (OK) as the inspection result 342 when no abnormality is detected in all of the plurality of attention areas 5 .
  • the UI control unit 206 displays the inspection result details UI screen 360 (see FIG. 16) for the work image 4 of the selected work area 341 .
  • FIG. 16 is a schematic diagram showing an example of a UI screen for confirming inspection results in detail according to the first embodiment.
  • the UI control unit 206 controls the work 3 for which the details button 343 is selected (depressed) in FIG. 15 (hereinafter referred to as the selected work 3 in the description of FIG. 16).
  • a UI screen (hereinafter referred to as an inspection result detail UI screen 360) is displayed.
  • the inspection result details UI screen 360 includes an explanation area 361 , a work area 362 and a parts list area 363 .
  • the UI control unit 206 displays, in the explanation area 361, the content explaining how the attention area 5 determined to be abnormal and the attention area 5 determined to be normal are displayed in a distinguishable manner.
  • the UI control unit 206 displays the work image 4 of the selected work 3 in the work area 362 .
  • the UI control unit 206 also superimposes the attention area 5 on each part 2 of the work image 4 .
  • the UI control unit 206 displays the attention area 5 of the part 2 determined to be abnormal and the attention area 5 of the part 2 determined to be normal in a distinguishable manner.
  • the UI control unit 206 displays the attention area 5 of the part 2 determined to be abnormal in red, and displays the attention area 5 of the part 2 determined to be normal in green. Thereby, the user can easily confirm which component 2 is determined to be abnormal.
  • the UI control unit 206 displays a list of the parts 2 attached to the selected workpiece 3 in the parts list area 363. In addition, the UI control unit 206 displays the parts 2 determined to be abnormal and the parts 2 determined to be normal in a distinguishable manner in the parts list area 363 . Thereby, the user can easily confirm which component 2 is determined to be abnormal.
  • Embodiment 1 (Summary of Embodiment 1) The contents of Embodiment 1 can be expressed as the following items.
  • Inspection device 20 includes one or more processors 22, memory (eg, RAM 24), and programs stored in the memory.
  • the program causes the processor 22 to do the following.
  • a first inspection target eg, a part 2a
  • a second inspection target eg, a component 2b
  • a first A first attention area 5 (5a) for inspecting an inspection object and a second attention area 5 (5b) for inspecting a second inspection object are set.
  • the program causes the camera 21, which captures an image of the inspection target area and outputs a captured image of the inspection target area (for example, the workpiece image 4), to take a first captured image captured under a first imaging condition as the captured image of the inspection target area, A second captured image captured under a second imaging condition different from the first imaging condition is captured.
  • the program includes a learning model 110 for detecting anomalies in a plurality of inspection objects including a first inspection object and a second inspection object, and a first region of a first captured image corresponding to a first region of interest. and inspects a second inspection object based on the learning model and the second region of the first captured image corresponding to the second region of interest.
  • the program inspects a first inspection object based on the learning model and a first area of a second captured image corresponding to the first attention area, and inspects the learning model and a second captured image corresponding to the second attention area.
  • a second inspection is performed to inspect the second inspection object based on the second area.
  • the program outputs the results of the first test and the results of the second test.
  • the inspection apparatus 20 inspects the first inspection object and the second inspection object using the first captured image captured under the first imaging condition, outputs the result of the first inspection, and outputs the result of the first inspection.
  • the first inspection object and the second inspection object are inspected using the second captured image captured under the two imaging conditions, and the result of the second inspection is output. Therefore, the inspection apparatus 20 can inspect each inspection target using captured images captured under different imaging conditions, and can obtain more accurate inspection results.
  • the program causes the processor 22 to execute the following.
  • the program defines a first imaging pattern including at least one imaging parameter as a first imaging condition based on a first region of the captured image corresponding to the first region of interest.
  • the program defines a second imaging pattern including at least one imaging parameter as a second imaging condition based on a second region of the captured image corresponding to the second region of interest.
  • each imaging condition is determined by an imaging pattern having different imaging parameters. Therefore, the inspection apparatus 20 can obtain more accurate inspection results by inspecting each inspection object using captured images captured under imaging conditions with different imaging parameters.
  • the inspection apparatus 20 described in item 2 includes an illumination device 30 that illuminates an inspection target area.
  • the program defines an illumination pattern including at least one illumination parameter for illumination device 30 to illuminate a region to be inspected. Thereby, the inspection device 20 can cause the illumination device 30 to illuminate the inspection target region with illumination patterns having different illumination parameters.
  • the program causes the processor 22 to perform the following.
  • the program causes the camera to output, as the first captured image, the captured image of the inspection target region captured by applying the first imaging condition for each at least one irradiation pattern.
  • the program causes the camera to output, as the second captured image, the captured image of the inspection target region captured by applying the second imaging condition for each at least one irradiation pattern.
  • the program causes the processor 22 (or processor 41) to execute the following.
  • the program generates the learning model 110 by learning based on the first captured image and the second captured image output by the camera. Thereby, the inspection apparatus 20 can generate the learning model 110 for inspecting the inspection object using the captured image.
  • the program causes the processor 22 (or processor 41) to execute the following.
  • the program sets the first region of interest and the second region of interest based on either design information defining the positions of a plurality of inspection objects or a captured image of the inspection target region. Thereby, the inspection apparatus 20 can set a plurality of attention areas in the inspection target area.
  • the inspection device 20 includes a camera 21 . Thereby, the inspection apparatus 20 can control the camera 21 to capture a captured image of the inspection target area.
  • the inspection device 20 implements the following inspection method.
  • the inspection apparatus 20 detects the first inspection object in an inspection object area in which a plurality of inspection objects including a first inspection object (for example, the part 2) and a second inspection object different from the first inspection object exist. and a second attention area 5 for inspecting a second inspection object are set.
  • the inspection apparatus 20 provides a camera 21 that captures an image of an inspection target area and outputs a captured image of the inspection target area (for example, the work image 4) as a captured image of the inspection target area, as a first image captured under a first imaging condition. An image and a second captured image captured under a second imaging condition different from the first imaging condition are captured.
  • the inspection apparatus 20 includes a learning model 110 for detecting anomalies in a plurality of inspection objects including a first inspection object and a second inspection object, and a first image of a first captured image corresponding to a first region of interest. and inspecting a second inspection object based on the learning model and a second area of the first captured image corresponding to the second attention area. Execute.
  • the inspection apparatus 20 inspects the first inspection object based on the learning model and the first area of the second captured image corresponding to the first attention area, and performs second imaging corresponding to the learning model and the second attention area.
  • a second inspection is performed to inspect a second inspection object based on a second region of the image.
  • the inspection device 20 outputs the result of the first inspection and the result of the second inspection.
  • the inspection apparatus 20 inspects the first inspection object and the second inspection object using the first captured image captured under the first imaging condition, outputs the result of the first inspection, and outputs the result of the first inspection.
  • the first inspection object and the second inspection object are inspected using the second captured image captured under the two imaging conditions, and the result of the second inspection is output. Therefore, the inspection apparatus 20 can inspect each inspection target using captured images captured under different imaging conditions, and can obtain more accurate inspection results.
  • the inspection program causes the processor 22 to do the following.
  • the inspection program includes a first inspection object (for example, a part 2) and a second inspection object different from the first inspection object in an inspection object area in which a plurality of inspection objects exist.
  • a first attention area 5 for inspection and a second attention area 5 for inspecting a second inspection object are set.
  • the inspection program causes the camera 21, which captures an image of the inspection target area and outputs a captured image of the inspection target area (for example, the workpiece image 4), to capture the first captured image captured under the first imaging condition as the captured image of the inspection target area. , and a second captured image captured under a second imaging condition different from the first imaging condition.
  • the inspection program comprises a learning model 110 for detecting anomalies in a plurality of inspection objects including a first inspection object and a second inspection object, and a first region of a first captured image corresponding to a first region of interest. and inspecting the second inspection object based on the learning model and the second region of the first captured image corresponding to the second region of interest. .
  • the inspection program inspects the first inspection object based on the learning model and a first area of a second captured image corresponding to the first attention area, and inspects the second captured image corresponding to the learning model and the second attention area.
  • a second inspection is performed for inspecting the second inspection object based on the second area of .
  • the inspection program outputs the result of the first inspection and the result of the second inspection.
  • the inspection program inspects the first inspection object and the second inspection object using the first captured image captured under the first imaging condition, outputs the result of the first inspection, and outputs the second inspection result.
  • the inspection program can inspect each inspection object using captured images captured under different imaging conditions, and can obtain more accurate inspection results.
  • FIG. 17 is a schematic diagram showing an example of reducing the size of the workpiece image 4 to the size of an image that can be input to the learning model 110.
  • FIG. 18 is a schematic diagram showing an example in which a plurality of cameras 21 divide the workpiece 3 into image sizes that can be input to the learning model 110 and capture the images.
  • the size of the image that can be input to the learning model 110 is smaller than the size of the work image 4 captured by the camera 21, the following countermeasure 1 or countermeasure 2 can be considered.
  • the inspection apparatus reduces the size of the workpiece image 4 to the size of an image that can be input to the learning model 110 to generate a reduced workpiece image 391 .
  • a possible countermeasure is to input and calculate the evaluation value.
  • the resolution of the part 2 is insufficient due to the reduction in the size of the workpiece image 4, and the problem arises that the learning model 110 cannot accurately calculate the evaluation value of the small part 2.
  • a countermeasure may be considered in which a plurality of cameras 21 divide and capture images of the workpiece 3, and the divided workpiece images 392 captured by each camera 21 are input to the learning model 110, respectively.
  • the present embodiment describes an inspection system 10 that suppresses an increase in processing time and cost while maintaining the accuracy of abnormality detection related to component mounting.
  • the hardware configuration of the inspection system 10 according to the second embodiment is the same as that of FIG. 1, and thus description thereof is omitted.
  • FIG. 19 is a block diagram showing an example of the functional configuration of the inspection device 20 according to the second embodiment.
  • the inspection apparatus 20 includes an imaging control unit 101, an optimum condition determination unit 102, an inspection execution unit 103, an attention area extraction unit 111, an attention area storage unit 104, an imaging condition storage unit 105, and an optimum imaging condition storage unit. 106 , a learning model storage unit 107 , and a synthetic image storage unit 112 .
  • the imaging control unit 101 controls the camera 21 and the lighting device 30 to image the workpiece 3 and generate the workpiece image 4 .
  • the region-of-interest extraction unit 111 extracts an image (region-of-interest image) of each region of interest 5 from the work image 4 .
  • the attention area extraction unit 111 combines the extracted attention area images to generate a composite image 400 (see FIGS. 20 and 21).
  • the attention area extraction unit 111 stores the generated synthetic image 400 in the synthetic image storage unit 112 . Details of the attention area extraction unit 111 and the synthesized image 400 will be described later (see FIGS. 20 and 21).
  • the optimum condition determination unit 102 determines the optimum imaging conditions for imaging each attention area 5 among a plurality of imaging conditions. In addition, the optimum condition determination unit 102 determines optimum extraction conditions (hereinafter referred to as optimum extraction conditions) when extracting an attention area image corresponding to the attention area 5 from the work image 4 .
  • optimum extraction conditions optimum extraction conditions
  • the inspection execution unit 103 uses the synthesized image 400 to inspect whether or not the component 2 is normally mounted in the area corresponding to the attention area 5 .
  • the inspection execution unit 103 inputs the image of the area corresponding to the attention area 5 of the synthesized image 400 to the learning model 110 stored in the learning model storage unit 107, thereby executing the inspection.
  • the inspection device 20 may generate the composite image 400 using either the first generation method or the second generation method.
  • FIG. 20 is a diagram showing an example of a first method for generating the synthesized image 400 according to the second embodiment.
  • the attention area extracting unit 111 extracts attention area images (pixels) corresponding to each attention area 5 (for example, attention areas 5a and 5b) extracted from the work image 4, to the attention area 5 (for example, attention areas 5a and 5b).
  • attention area images pixels
  • the attention area extracting unit 111 enlarges the attention area image of the part 2a for which the abnormality detection accuracy in the learning model 110 is improved by enlarging it.
  • the attention area extracting unit 111 reduces the attention area image of the part 2b for which the abnormality detection accuracy in the learning model 110 does not change much even if it is reduced.
  • the region-of-interest extraction unit 111 may adjust the surplus region in addition to the enlargement or reduction.
  • the surplus area indicates an area of peripheral pixels of the part 2 in the attention area image.
  • the region-of-interest extraction unit 111 extracts a region-of-interest image by setting a large surplus region for component 2, for which a larger surplus region (that is, peripheral pixels) increases the accuracy of calculation of the evaluation value of the learning model 110.
  • the region-of-interest extraction unit 111 extracts a region-of-interest image with a small surplus region for the part 2a for which the calculation accuracy of the evaluation value of the learning model 110 increases when the surplus region (that is, the peripheral pixels) is small. .
  • the region-of-interest extraction unit 111 synthesizes the region-of-interest image corresponding to each region-of-interest 5 obtained by enlarging or reducing and adjusting the surplus regions in one workpiece image 4 as described above, and obtains a single region-of-interest image as shown in FIG. generate two composite images 400;
  • the attention area extraction unit 111 stores the generated synthetic image 400 in the synthetic image storage unit 112 .
  • FIG. 21 is a diagram showing an example of a second method for generating the synthesized image 400 according to the second embodiment.
  • the attention area extraction unit 111 enlarges or reduces attention area images (pixels) corresponding to each attention area 5 (for example, attention areas 5a and 5b) extracted from the work image 4 so as to have a predetermined size. For example, the attention area extraction unit 111 enlarges the first attention area image corresponding to the first attention area 5a smaller than a predetermined size so as to have a predetermined size. For example, the attention area extraction unit 111 reduces the attention area image corresponding to the second attention area 5b larger than a predetermined size to a predetermined size.
  • the region-of-interest extraction unit 111 extracts peripheral pixels of the second region-of-interest 5 from the work image so that the pixels corresponding to the second region-of-interest 5b are the same as the pixels corresponding to the first region-of-interest 5a. Extract from 4.
  • the region-of-interest extraction unit 111 may adjust the surplus region in addition to the enlargement or reduction.
  • the attention area extracting unit 111 selects a large surplus area for the part 2a, for which the calculation accuracy of the evaluation value of the learning model 110 increases when the surplus area (that is, the surrounding pixels of the part 2) is large, and extracts the attention area image.
  • the attention area extracting unit 111 selects a small surplus area for the part 2b for which the calculation accuracy of the evaluation value of the learning model 110 increases when the surplus area (that is, the surrounding pixels of the part 2) is small, and extracts the attention area image.
  • the attention area extracting unit 111 synthesizes each attention area image of the same size by enlarging or reducing the one work image 4 and adjusting the surplus area in this way, and produces a result as shown in FIG. 21 .
  • a single composite image 400 is generated.
  • the attention area extraction unit 111 stores the generated synthetic image 400 in the synthetic image storage unit 112 .
  • the data amount of the image is smaller in this embodiment than in the above-described countermeasure 2, so the amount of processing and memory usage can be reduced.
  • the resolution of the small component 2 is no longer insufficient, so the inspection apparatus 20 according to the present embodiment can detect abnormalities with high accuracy.
  • ⁇ Pre-test processing flow> 22 is a flowchart illustrating an example of pre-examination processing according to Embodiment 2.
  • the imaging control unit 101 controls the camera 21 to image the workpiece 3 and generate the workpiece image 4 (step S401).
  • the attention area setting unit 201 sets a plurality of attention areas 5 on the workpiece 3 (step S402).
  • the attention area setting unit 201 transmits information (attention area information) indicating a plurality of set attention areas 5 to the inspection device 20 .
  • the inspection apparatus 20 stores the received plural pieces of attention area information in the attention area storage unit 104 .
  • the imaging condition generation unit 204 generates a plurality of imaging conditions by combining a plurality of imaging patterns stored in the imaging pattern storage unit 208 and a plurality of illumination patterns stored in the illumination pattern storage unit 209 (step S403).
  • the imaging condition generation unit 204 transmits the plurality of generated imaging conditions to the inspection device 20 .
  • the inspection apparatus 20 stores the received multiple imaging conditions in the imaging condition storage unit 105 .
  • the optimum condition determination unit 102 selects one unselected imaging condition from the plurality of imaging conditions stored in the imaging condition storage unit 105 (step S404).
  • the selected imaging conditions are referred to as selected imaging conditions.
  • the imaging control unit 101 controls the camera 21 and the lighting device 30 based on the selected imaging conditions to capture an image of the workpiece 3 and generate a workpiece image 4 (step S405).
  • the imaging control unit 101 stores the generated work image 4 in the RAM 43 or storage 44 .
  • the optimum condition determination unit 102 determines whether or not all imaging conditions stored in the imaging condition storage unit 105 have been selected (step S406).
  • the inspection apparatus 20 When unselected imaging conditions remain (S406: NO), the inspection apparatus 20 returns the process to step S404. If all imaging conditions have been selected (S406: YES), the inspection apparatus 20 advances the process to step S407.
  • the region-of-interest extraction unit 111 extracts each workpiece image stored in the RAM 43 or the storage 44 by the above-described first synthetic image generation method (see FIG. 20) or the second synthetic image generation method (see FIG. 21). 4, each region of interest 5 is extracted under various extraction conditions.
  • the extraction conditions include an enlargement rate or a reduction rate, a surplus area ratio, and the like.
  • the attention area extraction unit 111 extracts an image of an area corresponding to each attention area 5 in the work image 4, and enlarges or reduces it based on the extraction conditions. Further, the attention area extracting unit 111 extracts an image by increasing or decreasing the surplus area based on the extraction condition from the area corresponding to each attention area 5 in the work image 4 .
  • the attention area extracting unit 111 combines the attention area images corresponding to the extracted attention areas 5 to generate the composite image 400 (step S407).
  • the region-of-interest extraction unit 111 stores the plurality of generated synthetic images 400 in the synthetic image storage unit 112 .
  • the composite image storage unit 112 stores a plurality of composite images 400 obtained by combining attention area images extracted under various extraction conditions from the work images 4 captured under various imaging conditions. .
  • the inspection device 20 executes optimum condition determination processing (step S408). Details of the optimum condition determination process will be described later (see FIG. 23). Then, the process ends.
  • FIG. 23 is a flow chart showing an example of the optimum condition determination process (step S408) shown in FIG.
  • the optimum condition determination unit 102 selects one unselected attention area 5 from among the plurality of attention areas 5 stored in the attention area storage unit 104 (step S501).
  • the selected attention area 5 is called a selected attention area 5 .
  • the optimum condition determination unit 102 selects an image in the selected region of interest 5 (hereinafter referred to as a selection (referred to as a region-of-interest image).
  • the optimum condition determination unit 102 uses the learning model 110 stored in the learning model storage unit 107 to calculate an evaluation value for each of the acquired selected region-of-interest images (step S502).
  • the optimum condition determination unit 102 determines the optimum imaging conditions and optimum extraction conditions for the selected region of interest 5 based on the evaluation values calculated in step S502 (step S503). For example, the optimum condition determining unit 102 determines the imaging condition with the highest calculated evaluation value as the optimum imaging condition for the selected region of interest 5 . The optimum condition determination unit 102 also determines the extraction condition with the highest calculated evaluation value as the optimum extraction condition for the selected region of interest 5 .
  • the optimum condition determination unit 102 associates the selected attention area 5 with the optimum imaging conditions and optimum extraction conditions determined in step S503, and stores them in the attention area storage unit 104 (step S504).
  • the optimal condition determination unit 102 stores the optimal imaging conditions determined in step S503 in the optimal imaging condition storage unit 106 (step S505).
  • the optimum condition determination unit 102 determines whether or not all the attention areas 5 stored in the attention area storage unit 104 have been selected (step S506).
  • step S506 If an unselected region of interest 5 remains (step S506: NO), the inspection device 20 returns the process to step S501. If all the attention areas 5 have been selected (step S506: YES), the inspection device 20 terminates this process.
  • ⁇ Inspection processing flow> 24 is a flowchart illustrating an example of inspection processing according to Embodiment 2.
  • FIG. 24 is a flowchart illustrating an example of inspection processing according to Embodiment 2.
  • the imaging control unit 101 selects the attention area storage unit 104 and the optimum imaging condition storage unit 106 associated with the workpiece 3 to be inspected (step S601).
  • the imaging control unit 101 determines whether or not the detection sensor 19 has detected the workpiece 3 (step S602). For example, the imaging control unit 101 determines whether or not a workpiece detection notification has been received from the detection sensor 19 .
  • step S602 NO
  • the inspection device 20 returns the process to step S602. If the detection sensor 19 detects the workpiece 3 (step S602: YES), the inspection apparatus 20 advances the process to the next step S603.
  • the imaging control unit 101 selects one unselected optimum imaging condition from the optimum imaging condition storage unit 106 (step S603).
  • the selected optimum imaging conditions are referred to as selected optimum imaging conditions.
  • the imaging control unit 101 controls the camera 21 and the illumination device 30 based on the selected optimum imaging conditions to image the workpiece 3 and generate the workpiece image 4 (step S604).
  • the region-of-interest extraction unit 111 generates a composite image 400 from the work image 4 (step S605). Specifically, the attention area extracting unit 111 extracts each attention area 5 of the workpiece image 4 under the optimum extraction condition associated with the attention area 5, and synthesizes the extracted attention area images to form a synthesized image 400. to generate The attention area extraction unit 111 stores the generated synthetic image 400 in the synthetic image storage unit 112 . As a result, the amount of memory used can be reduced compared to the first embodiment in which the work image 4 is stored as it is.
  • the imaging control unit 101 determines whether or not all the optimum imaging conditions included in the optimum imaging condition storage unit 106 have been selected (step S606).
  • step S606: NO If unselected optimum imaging conditions remain (step S606: NO), the inspection device 20 returns the process to step S603. If all the optimum imaging conditions have been selected (step S606: YES), the inspection apparatus 20 advances the process to the next step S607.
  • the inspection execution unit 103 selects one unselected attention area 5 from among the plurality of attention areas 5 stored in the attention area storage unit 104 (step S607).
  • the selected attention area 5 is called a selected attention area 5 .
  • the examination executing unit 103 selects a synthesized image 400 generated based on the optimum imaging condition and the optimum extraction condition associated with the selected region of interest 5 among the plurality of synthesized images 400 stored in the synthesized image storage unit 112. is selected (step S608).
  • the selected synthetic image 400 will be referred to as the selected synthetic image 400 .
  • the inspection execution unit 103 uses the learning model 110 stored in the learning model storage unit 107 to calculate the image evaluation value of the area corresponding to the selected attention area 5 in the selected combined image 400 (step S609).
  • the selected synthesized image 400 is obtained by synthesizing an attention area image extracted under optimum imaging conditions for extracting the selected attention area 5 from the work image 4 imaged under the optimum imaging conditions for imaging the selected attention area 5. It is generated. Therefore, the evaluation value calculated from the selected region-of-interest image obtained from the selected composite image 400 is higher than the evaluation value calculated from the selected region-of-interest image obtained from the work image captured under a single imaging condition. , can be more accurate.
  • the inspection execution unit 103 determines the inspection result of the selected region of interest 5 based on the evaluation value calculated in step S609 (step S610). For example, when the evaluation value is less than a predetermined threshold value Th, the inspection execution unit 103 determines that the component 2 is abnormally attached (NG) in the selected attention area 5 . For example, when the evaluation value is equal to or greater than the threshold Th, the inspection execution unit 103 determines that the component 2 is normally mounted (OK) in the selected attention area 5 . The inspection execution unit 103 associates the inspection results with the selected region of interest 5 and stores them in the RAM 34 or the storage 44 .
  • the inspection execution unit 103 determines whether or not all the attention areas 5 stored in the attention area storage unit 104 have been selected (step S611). If there remains an unselected region of interest 5, the inspection apparatus 20 returns the process to step S607.
  • the inspection execution unit 103 collects the inspection results associated with each attention area 5 and stored in the RAM 43 or the storage 44 in step S610 to generate workpiece inspection result information. and transmitted to the management device 40 (step S612). The inspection apparatus 20 then returns the process to step S602 and inspects the work 3 that is next transported.
  • the UI control unit 206 of the management device 40 receives work inspection result information from the inspection device 20 and stores it in the inspection result storage unit 212 . Also, the UI control unit 206 displays the content of the work inspection result information on the display device 60 . The user can see the contents of the workpiece inspection result information displayed on the display device 60 and can confirm whether or not the component 2 is normally mounted in each attention area 5 of the workpiece 3 .
  • FIG. 25 is a flowchart showing a modification of inspection processing according to the second embodiment.
  • the imaging control unit 101 selects the attention area storage unit 104 and the optimum imaging condition storage unit 106 associated with the workpiece 3 to be inspected (step S701).
  • the imaging control unit 101 determines whether or not the detection sensor 19 has detected the workpiece 3 (step S702). For example, the imaging control unit 101 determines whether or not a workpiece detection notification has been received from the detection sensor 19 .
  • step S702 If the detection sensor 19 has not detected the workpiece 3 (step S702: NO), the inspection device 20 returns the process to step S702. If the detection sensor 19 detects the workpiece 3 (step S702: YES), the inspection apparatus 20 advances the process to the next step S703.
  • the imaging control unit 101 controls the camera 21 and the lighting device 30 based on predetermined imaging conditions to capture an image of the workpiece 3 and generate a workpiece image 4 (step S704).
  • the region-of-interest extraction unit 111 generates a composite image 400 from the work image 4 (step S705). Specifically, the attention area extracting unit 111 extracts each attention area 5 of the workpiece image 4 under the optimum extraction condition associated with the attention area 5, and synthesizes the extracted attention area images to form a composite image 400. to generate The attention area extraction unit 111 stores the generated synthetic image 400 in the synthetic image storage unit 112 . As a result, the amount of memory used can be reduced compared to the first embodiment in which the work image 4 is stored as it is.
  • the inspection execution unit 103 selects one unselected attention area 5 from among the plurality of attention areas 5 stored in the attention area storage unit 104 (step S705).
  • the selected attention area 5 is called a selected attention area 5 .
  • the test execution unit 103 uses the learning model 110 stored in the learning model storage unit 107 to calculate the evaluation value of the image of the region corresponding to the selected attention region 5 in the synthesized image 400 generated in step S705. (Step S706).
  • the synthesized image 400 is generated by synthesizing attention area images extracted under optimum imaging conditions for extracting the selected attention area 5 . Therefore, the evaluation value calculated from the selected region-of-interest image obtained from the selected composite image 400 is higher than the evaluation value calculated from the selected region-of-interest image obtained from the work image captured under a single imaging condition. , can be more accurate.
  • the inspection execution unit 103 determines the inspection result of the selected region of interest 5 based on the evaluation value calculated in step S706 (step S707). For example, when the evaluation value is less than a predetermined threshold value Th, the inspection execution unit 103 determines that the component 2 is abnormally attached (NG) in the selected attention area 5 . For example, when the evaluation value is equal to or greater than the threshold Th, the inspection execution unit 103 determines that the component 2 is normally mounted (OK) in the selected attention area 5 . The inspection execution unit 103 associates the inspection results with the selected region of interest 5 and stores them in the RAM 34 or the storage 44 .
  • the inspection execution unit 103 determines whether or not all the attention areas 5 stored in the attention area storage unit 104 have been selected (step S708). If an unselected region of interest 5 remains, the inspection apparatus 20 returns the process to step S705.
  • the inspection execution unit 103 collects the inspection results associated with each attention area 5 stored in the RAM 43 or the storage 44 in step S707 to generate workpiece inspection result information. and transmitted to the management device 40 (step S709).
  • the inspection apparatus 20 then returns the process to step S702 and inspects the work 3 that is next transported.
  • Inspection device 20 includes one or more processors 22, memory (eg, RAM 24), and programs stored in the memory.
  • the program causes the processor 22 to do the following.
  • the program performs a first inspection in an inspection area including a plurality of inspection objects including a first inspection object (for example, part 2a) and a second inspection object (for example, part 2b) different from the first inspection object.
  • a first attention area 5 (5a) for inspecting an object and a second attention area 5 (5b) for inspecting a second inspection object are set.
  • the program causes the camera 21 that captures an image of the inspection target area and outputs a captured image of the inspection target area (for example, the workpiece image 4) to capture the inspection target area.
  • the program extracts a first image area corresponding to the first attention area and a second image area corresponding to the second attention area from the captured image.
  • the program executes a first inspection for inspecting a first inspection object based on a learning model 110 for detecting anomalies in a plurality of inspection objects and a first image region.
  • the program performs a second inspection that inspects a second inspection object based on the learning model and the second image region.
  • the program outputs the results of the first test and the results of the second test.
  • the inspection apparatus 20 inspects the first inspection object using the first image area extracted from the captured image, outputs the result of the first inspection, and uses the second image area extracted from the captured image. to output the result of the second inspection of the second inspection object.
  • the inspection apparatus 20 can inspect each inspection object using the appropriately extracted image area, and can obtain a more accurate inspection result.
  • the inspection apparatus 20 can reduce the processing load of the processor 22 and the amount of memory (for example, RAM 23) usage compared to the case where the first inspection object and the second inspection object are inspected directly from the captured image.
  • the program causes the processor 22 to execute the following.
  • the program generates a first composite image 400 that includes a first image region and a second image region.
  • the program uses the first composite image and the learning model to perform the first test and the second test.
  • the inspection apparatus 20 can perform the first inspection and the second inspection using the first synthesized image. Therefore, the inspection apparatus 20 can reduce the processing load of the processor 22 and the amount of memory used compared to the case where the first inspection object and the second inspection object are inspected directly from the captured image.
  • the program causes the processor 22 to perform the following.
  • the program extracts the first image area by enlarging or reducing the pixels corresponding to the first attention area from the captured image.
  • the program enlarges or reduces pixels corresponding to the second attention area from the captured image to extract the second image area.
  • the inspection device 20 can generate the first synthetic image of a size that can be input to the learning model.
  • the program causes the processor 22 to perform the following.
  • the program extracts the first region of interest and peripheral pixels of the first region of interest from the captured image as pixels corresponding to the first region of interest.
  • the program extracts the second region of interest and peripheral pixels of the second region of interest from the captured image as pixels corresponding to the second region of interest. In this way, the inspection apparatus 20 can improve the inspection accuracy in the learning model by also extracting the peripheral pixels of each attention area.
  • the program causes the processor 22 to perform the following.
  • the program extracts peripheral pixels of the second region of interest from the captured image so that pixels corresponding to the second region of interest have the same size as pixels corresponding to the first region of interest.
  • the inspection apparatus 20 can combine images extracted in the same size for each attention area of the captured image to generate a combined image.
  • the program causes the processor 22 to execute the following.
  • the program generates a learning model 110 by learning based on the first image region and the second image region.
  • the inspection apparatus 20 can generate the learning model 110 for inspecting the inspection object using the captured image.
  • the program causes the processor 22 to execute the following.
  • the program defines first imaging conditions including a first imaging pattern including a plurality of imaging parameters based on a first region of the captured image corresponding to the first region of interest.
  • the program defines second imaging conditions including a second imaging pattern including a plurality of imaging parameters based on a second region of the captured image corresponding to the second region of interest.
  • each imaging condition is determined by an imaging pattern having different imaging parameters. Therefore, the inspection apparatus 20 can obtain more accurate inspection results by inspecting each inspection object using captured images captured under imaging conditions with different imaging parameters.
  • the inspection apparatus 20 includes an illumination device 30 that illuminates an inspection target area.
  • the program causes the processor 22 to do the following.
  • the program defines an illumination pattern including a plurality of illumination parameters for illuminating an area to be inspected by the illuminator. Thereby, the inspection device 20 can cause the illumination device 30 to illuminate the inspection target region with illumination patterns having different illumination parameters.
  • the program causes the processor 22 to perform the following.
  • the program captures an image of an inspection target area obtained by applying a first imaging condition for each of a plurality of irradiation conditions and an image by applying a second imaging condition for each of a plurality of irradiation conditions as an image of the inspection object.
  • the camera 21 is caused to output the picked-up image of the inspection target area.
  • the inspection apparatus 20 can inspect each inspection object using captured images captured with different illumination patterns, and can obtain more accurate inspection results.
  • the program causes the processor 22 to execute the following.
  • the program sets the first region of interest and the second region of interest based on either design information that defines the positions of a plurality of inspection objects or a captured image of the inspection object region. Thereby, the inspection apparatus 20 can set a plurality of attention areas in the inspection target area.
  • the inspection device 20 includes a camera 21 . Thereby, the inspection apparatus 20 can control the camera 21 to capture a captured image of the inspection target area.
  • the device (for example, inspection device 20) implements the following image processing method.
  • the apparatus inspects a first inspection object in an inspection object area including a plurality of inspection objects including a first inspection object (eg, part 2) and a second inspection object different from the first inspection object.
  • a first attention area for performing inspection and a second attention area for performing inspection of a second inspection object are set.
  • the apparatus causes the camera 21 that captures an image of the inspection target area and outputs a captured image of the inspection target area (for example, the workpiece image 4) to capture the inspection target area.
  • the device extracts a first image region corresponding to the first region of interest 5 and a second image region corresponding to the second region of interest 5 from the captured image.
  • the apparatus performs a first inspection for inspecting a first inspection object based on a learning model 110 for detecting anomalies in a plurality of inspection objects and a first image region.
  • the apparatus performs a second inspection that inspects a second inspection object based on the learned model and the second image region.
  • the device outputs the result of the first test and the result of said second test.
  • the apparatus inspects the first inspection object using the first image area extracted from the captured image, outputs the result of the first inspection, and uses the second image area extracted from the captured image to A result of the second inspection of the second inspection object is output. Therefore, the apparatus can reduce the processing load of the processor 22 and the usage of the memory (for example, the RAM 23) as compared with the case of directly inspecting the first inspection object and the second inspection object from the captured image.
  • the image processing program causes the processor 22 to do the following.
  • the image processing program in an inspection area including a plurality of inspection objects including a first inspection object (for example, part 2) and a second inspection object different from the first inspection object, A first attention area for inspection and a second attention area for inspecting a second inspection object are set.
  • the image processing program causes the camera 21 that captures an image of the inspection target area and outputs a captured image of the inspection target area (for example, the work image 4) to capture the inspection target area.
  • the image processing program extracts a first image area corresponding to the first attention area and a second image area corresponding to the second attention area from the captured image.
  • the image processing program executes a first inspection for inspecting a first inspection object based on a learning model 110 for detecting anomalies in a plurality of inspection objects and a first image region.
  • the image processing program performs a second inspection for inspecting a second inspection object based on the learning model and the second image region.
  • the image processing program outputs the result of the first inspection and the result of the second inspection.
  • the image processing program inspects the first inspection object using the first image area extracted from the captured image, outputs the result of the first inspection, and uses the second image area extracted from the captured image. to output the result of the second inspection of the second inspection object. Therefore, the image processing program can reduce the processing load of the processor 22 and the usage of the memory (for example, the RAM 23) as compared with the case of directly inspecting the first inspection object and the second inspection object from the captured image.
  • steps included in the processing disclosed in this specification do not necessarily have to be executed in chronological order according to the order described in the sequence diagrams and/or flowcharts.
  • steps in a process may be performed out of order and/or in parallel from the order illustrated in the sequence diagrams and/or flowcharts. It is also possible to omit some of the steps included in the process and/or add additional steps to the process.
  • an apparatus or a module thereof may be provided.
  • an imaging control module for example, an imaging control module, an optimum condition determination module, an inspection execution module, and/or a region of interest extraction module
  • a processor and/or a processing element equivalent to the processor for example, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, CPLD: Complex Programmable Logic Device, etc.
  • a recording medium computer-readable non-temporary recording medium
  • the technology of the present disclosure is useful for devices and the like that inspect whether or not each component is normally mounted on a board.

Abstract

An inspection device according to the present invention: executes a first inspection in which a first inspection object is inspected on the basis of a training model for detecting abnormalities in inspection objects, and a first region in a first captured image, the first region corresponding to the first region of interest, and in which a second inspection object is inspected on the basis of the training model and a second region in the first captured image, the second region corresponding to the second region of interest; executes a second inspection in which the first inspection object is inspected on the basis of the training model and a first region in a second captured image, the first region corresponding to the first region of interest, and in which the second inspection object is inspected on the basis of the training model and a second region in the second captured image, the second region corresponding to the second region of interest; and outputs the results of the first inspection and the second inspection.

Description

検査装置、検査方法、及び検査プログラムInspection device, inspection method, and inspection program
　本発明は、検査装置、検査方法、及び検査プログラムに関する。 The present invention relates to an inspection device, an inspection method, and an inspection program.
　近年、製造現場における検査の自動化を目的として、カメラで撮像した検査対象物の画像に基づいた製品の外観検査が行われている（例えば、特許文献１及び特許文献２参照）。 In recent years, with the aim of automating inspections at manufacturing sites, product appearance inspections have been performed based on images of inspection objects captured by cameras (see Patent Documents 1 and 2, for example).
　特許文献１には、複数の照明パターンを用いて撮像された画像群に基づいて、検査対象物の損傷や悪品等、いわゆる検査対象物の欠陥を検出するための最適な照明パターンを得る技術が開示されている。また、特許文献２には、検査対象物の欠陥を強調しつつ、検査対象物の良品ばらつきや個体ばらつきを抑制するための照明パターンを得る技術が開示されている。 Patent Literature 1 discloses a technique for obtaining an optimal illumination pattern for detecting so-called defects in an inspection object, such as damage and defective products, based on a group of images captured using a plurality of illumination patterns. is disclosed. Further, Patent Literature 2 discloses a technique of obtaining an illumination pattern for suppressing non-defective product variations and individual variations of inspection objects while emphasizing defects of the inspection objects.
欧州特許出願公開第２８８７０５５号明細書EP-A-2887055 日本国特開２０２１－５０７１３１号公報Japanese Patent Application Laid-Open No. 2021-507131
　特許文献１及び特許文献２に開示されている技術は、撮像された画像群に含まれる検査対象物の欠陥を検出するため、撮像された画像群に含まれる検査対象物の数が少ない画像に対して有効である。ところで、製造現場では、たくさんの部品を実装した基板に対する検査の自動化が望まれている。ここで、このような要望に対して、特許文献１及び特許文献２を適用して基板全体を一つの検査対象物とした照明パターンを取得すると仮定する。このような場合、基板に実装されている部品は色や素材、形状などが様々であることから、基板全体を一つの検査対象物としたときに得られる照明パターンが基板に実装されている各々の部品の検査に好適であるとは限らないため、所望の検査精度を得られない可能性がある。 The techniques disclosed in Patent Documents 1 and 2 detect defects in an inspection object included in a group of captured images. It is valid. By the way, at the manufacturing site, it is desired to automate the inspection of boards on which many components are mounted. Here, it is assumed that, in response to such a request, an illumination pattern is obtained by applying Patent Documents 1 and 2 to the entire substrate as one inspection target. In such a case, since the components mounted on the board vary in color, material, shape, etc., the illumination pattern obtained when the entire board is treated as one inspection object is Therefore, the desired inspection accuracy may not be obtained.
　本発明は、このような課題を解決するためになされたものであり、複数の検査対象物に対する外観検査の精度を向上することを目的とする。 The present invention has been made to solve such problems, and aims to improve the accuracy of appearance inspection for a plurality of inspection objects.
　本発明の一態様に係る検査装置は、１つ以上のプロセッサと、メモリと、前記メモリに保存されているプログラムと、を備え、前記プログラムは、第１検査対象物と前記第１検査対象物とは異なる第２検査対象物とを含む複数の検査対象物が存在する検査対象領域において、前記第１検査対象物の検査を行うための第１注目領域と、前記第２検査対象物の検査を行うための第２注目領域とを設定することと、前記検査対象領域を撮像して前記検査対象領域の撮像画像を出力するカメラに、前記検査対象領域の撮像画像として、第１撮像条件により撮像した第１撮像画像と、前記第１撮像条件とは異なる第２撮像条件により撮像した第２撮像画像とを撮像させることと、前記第１検査対象物と前記第２検査対象物とを含む複数の検査対象物の異常を検知するための学習モデルと、前記第１注目領域に対応する前記第１撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第１撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第１検査を実行することと、前記学習モデルと前記第１注目領域に対応する前記第２撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第２撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第２検査を実行することと、前記第１検査の結果と前記第２検査の結果とを出力することと、を、前記１つ以上のプロセッサに実行させる。 An inspection apparatus according to an aspect of the present invention includes one or more processors, a memory, and a program stored in the memory, wherein the program stores a first inspection object and the first inspection object. In an inspection target area in which a plurality of inspection objects including a second inspection object different from the and a camera that captures the inspection target area and outputs the captured image of the inspection target area as the captured image of the inspection target area according to a first imaging condition capturing a first captured image and a second captured image captured under a second imaging condition different from the first imaging condition; and including the first inspection object and the second inspection object. inspecting the first inspection object based on a learning model for detecting anomalies in a plurality of inspection objects and a first region of the first captured image corresponding to the first region of interest; executing a first inspection for inspecting the second inspection object based on a model and a second region of the first captured image corresponding to the second region of interest; and the learning model and the first region of interest. and inspecting the first inspection object based on the first region of the second captured image corresponding to the learning model and the second region of the second captured image corresponding to the second region of interest and outputting results of the first inspection and results of the second inspection to the one or more processors. Let
　本発明の一態様に係る検査方法は、検査装置において、第１検査対象物と前記第１検査対象物とは異なる第２検査対象物とを含む複数の検査対象物が存在する検査対象領域において、前記第１検査対象物の検査を行うための第１注目領域と、前記第２検査対象物の検査を行うための第２注目領域とを設定し、前記検査対象領域を撮像して前記検査対象領域の撮像画像を出力するカメラに、前記検査対象領域の撮像画像として、第１撮像条件により撮像した第１撮像画像と、前記第１撮像条件とは異なる第２撮像条件により撮像した第２撮像画像とを撮像させ、前記第１検査対象物と前記第２検査対象物とを含む複数の検査対象物の異常を検知するための学習モデルと、前記第１注目領域に対応する前記第１撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第１撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第１検査を実行し、前記学習モデルと前記第１注目領域に対応する前記第２撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第２撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第２検査を実行し、前記第１検査の結果と前記第２検査の結果とを出力する。 In an inspection method according to an aspect of the present invention, in an inspection device, in an inspection target area in which a plurality of inspection targets including a first inspection target and a second inspection target different from the first inspection target exist, and setting a first attention area for inspecting the first inspection object and a second attention area for inspecting the second inspection object, imaging the inspection object area, and performing the inspection. A first captured image captured under a first imaging condition and a second captured image captured under a second imaging condition different from the first imaging condition, as captured images of the inspection target region, to a camera that outputs a captured image of the target region. a learning model for detecting an abnormality in a plurality of inspection objects including the first inspection object and the second inspection object; and the first inspection object corresponding to the first region of interest. inspecting the first inspection object based on the first area of the captured image; and performing the second inspection based on the learning model and a second area of the first captured image corresponding to the second attention area. performing a first inspection for inspecting an object, inspecting the first inspection object based on the learning model and a first region of the second captured image corresponding to the first region of interest, and performing the learning; performing a second inspection for inspecting the second inspection object based on the model and a second region of the second captured image corresponding to the second region of interest, and performing the second inspection with the result of the first inspection; and output the result of
　本発明の一態様に係る検査プログラムは、第１検査対象物と前記第１検査対象物とは異なる第２検査対象物とを含む複数の検査対象物が存在する検査対象領域において、前記第１検査対象物の検査を行うための第１注目領域と、前記第２検査対象物の検査を行うための第２注目領域とを設定することと、前記検査対象領域を撮像して前記検査対象領域の撮像画像を出力するカメラに、前記検査対象領域の撮像画像として、第１撮像条件により撮像した第１撮像画像と、前記第１撮像条件とは異なる第２撮像条件により撮像した第２撮像画像とを撮像させることと、前記第１検査対象物と前記第２検査対象物とを含む複数の検査対象物の異常を検知するための学習モデルと、前記第１注目領域に対応する前記第１撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第１撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第１検査を実行することと、前記学習モデルと前記第１注目領域に対応する前記第２撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第２撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第２検査を実行することと、前記第１検査の結果と前記第２検査の結果とを出力することと、を、プロセッサに実行させる。 An inspection program according to an aspect of the present invention provides an inspection target region in which a plurality of inspection targets including a first inspection target and a second inspection target different from the first inspection target exist, wherein the first inspection target is setting a first attention area for inspecting an inspection object and a second attention area for inspecting the second inspection object; a first captured image captured under a first imaging condition and a second captured image captured under a second imaging condition different from the first imaging condition, as captured images of the inspection target area, to a camera that outputs the captured images of and a learning model for detecting anomalies in a plurality of inspection objects including the first inspection object and the second inspection object, and the first corresponding to the first attention area inspecting the first inspection object based on the first area of the captured image; and performing the second inspection based on the learning model and a second area of the first captured image corresponding to the second attention area. performing a first inspection for inspecting an object; inspecting the first inspection object based on the learning model and a first region of the second captured image corresponding to the first region of interest; executing a second inspection for inspecting the second inspection object based on the learning model and a second region of the second captured image corresponding to the second region of interest; and a result of the first inspection. and outputting the result of the second test.
　なお、これらの包括的又は具体的な態様は、システム、装置、方法、集積回路、コンピュータプログラム又は記録媒体で実現されてもよく、システム、装置、方法、集積回路、コンピュータプログラム及び記録媒体の任意な組み合わせで実現されてもよい。 In addition, these generic or specific aspects may be realized by a system, device, method, integrated circuit, computer program or recording medium, and any of the system, device, method, integrated circuit, computer program and recording medium may be implemented. may be implemented in any combination.
　本発明によれば、複数の検査対象物に対する外観検査の精度が向上する。 According to the present invention, the accuracy of appearance inspection for multiple inspection objects is improved.
図１は、実施の形態１に係る検査システムの構成例を示すブロック図である。FIG. 1 is a block diagram showing a configuration example of an inspection system according to Embodiment 1. FIG. 図２は、実施の形態１に係る検査装置の機能構成の一例を示すブロック図である。FIG. 2 is a block diagram showing an example of the functional configuration of the inspection apparatus according to Embodiment 1. FIG. 図３は、実施の形態１に係る管理装置の機能構成の一例を示すブロック図である。3 is a block diagram illustrating an example of a functional configuration of a management device according to Embodiment 1; FIG. 図４は、実施の形態１に係るワークに設定される注目領域の一例を示す図である。FIG. 4 is a diagram showing an example of an attention area set on a work according to the first embodiment. 図５は、実施の形態１に係る撮像パターンの一例を示す図である。FIG. 5 is a diagram showing an example of an imaging pattern according to Embodiment 1. FIG. 図６は、実施の形態１に係る照明パターンの一例を示す図である。6 is a diagram showing an example of an illumination pattern according to Embodiment 1. FIG. 図７は、実施の形態１に係る検査システムにおける検査前の動作を示すシーケンスチャートである。FIG. 7 is a sequence chart showing pre-inspection operations in the inspection system according to the first embodiment. 図８は、実施の形態１に係る検査システムにおける検査の動作を示すシーケンスチャートである。FIG. 8 is a sequence chart showing inspection operations in the inspection system according to the first embodiment. 図９は、実施の形態１に係る検査前処理の一例を示すフローチャートである。9 is a flowchart illustrating an example of pre-examination processing according to Embodiment 1. FIG. 図１０は、図９に示す最適条件決定処理の詳細例を示すフローチャートである。FIG. 10 is a flow chart showing a detailed example of the optimum condition determination process shown in FIG. 図１１は、実施の形態１に係る検査処理の一例を示すフローチャートである。11 is a flowchart illustrating an example of inspection processing according to Embodiment 1. FIG. 図１２は、実施の形態１に係る撮像パターン及び照明パターンを生成及び調整するためのＵＩ画面の一例を示す模式図である。12 is a schematic diagram showing an example of a UI screen for generating and adjusting an imaging pattern and an illumination pattern according to Embodiment 1. FIG. 図１３は、実施の形態１に係る注目領域を設定するためのＵＩ画面の一例を示す模式図である。13 is a schematic diagram showing an example of a UI screen for setting an attention area according to Embodiment 1. FIG. 図１４は、実施の形態１に係る注目領域情報の構成例を示す図である。14 is a diagram showing a configuration example of attention area information according to Embodiment 1. FIG. 図１５は、実施の形態１に係るワークの検査結果を一覧で確認するためのＵＩ画面の一例を示す模式図である。FIG. 15 is a schematic diagram showing an example of a UI screen for confirming a list of work inspection results according to the first embodiment. 図１６は、実施の形態１に係る検査結果を詳細に確認するためのＵＩ画面の一例を示す模式図である。FIG. 16 is a schematic diagram showing an example of a UI screen for confirming inspection results in detail according to the first embodiment. 図１７は、ワーク画像のサイズを、学習モデルに入力可能な画像のサイズに縮小する例を示す模式図である。FIG. 17 is a schematic diagram showing an example of reducing the size of the work image to the size of the image that can be input to the learning model. 図１８は、複数のカメラでワークを、学習モデルに入力可能な画像のサイズに分割して撮像する例を示す模式図である。FIG. 18 is a schematic diagram showing an example in which a plurality of cameras divides a workpiece into image sizes that can be input to a learning model and captures the images. 図１９は、実施の形態２に係る検査装置の機能構成の一例を示すブロック図である。19 is a block diagram illustrating an example of a functional configuration of an inspection apparatus according to Embodiment 2; FIG. 図２０は、実施の形態２に係る合成画像の第１の生成方法の一例を示す図である。20A and 20B are diagrams illustrating an example of a first method for generating a synthesized image according to Embodiment 2. FIG. 図２１は、実施の形態２に係る合成画像の第２の生成方法の一例を示す図である。21A and 21B are diagrams illustrating an example of a second method for generating a synthesized image according to Embodiment 2. FIG. 図２２は、実施の形態２に係る検査前処理の一例を示すフローチャートである。22 is a flowchart illustrating an example of pre-examination processing according to Embodiment 2. FIG. 図２３は、図２２に示す最適条件決定処理の一例を示すフローチャートである。23 is a flow chart showing an example of the optimum condition determination process shown in FIG. 22. FIG. 図２４は、実施の形態２に係る検査処理の一例を示すフローチャートである。24 is a flowchart illustrating an example of inspection processing according to Embodiment 2. FIG. 図２５は、実施の形態２に係る検査処理の変形例を示すフローチャートである。FIG. 25 is a flowchart showing a modification of inspection processing according to the second embodiment.
　以下、図面を適宜参照して、本開示の実施の形態について、詳細に説明する。ただし、必要以上に詳細な説明は省略する場合がある。例えば、すでによく知られた事項の詳細説明及び実質的に同一の構成に対する重複説明を省略する場合がある。これは、以下の説明が不必要に冗長になるのを避け、当業者の理解を容易にするためである。なお、添付図面及び以下の説明は、当業者が本開示を十分に理解するために提供されるのであって、これらにより特許請求の記載の主題を限定することは意図されていない。 Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially the same configurations may be omitted. This is to avoid unnecessary verbosity in the following description and to facilitate understanding by those skilled in the art. It should be noted that the accompanying drawings and the following description are provided to allow those skilled in the art to fully understand the present disclosure and are not intended to limit the subject matter of the claims.
（実施の形態１）
＜検査システムの構成＞
　図１は、実施の形態１に係る検査システム１０の構成例を示すブロック図である。 (Embodiment 1)
<Configuration of inspection system>
FIG. 1 is a block diagram showing a configuration example of an inspection system 10 according to Embodiment 1. As shown in FIG.
　従来、基板１（図４参照）に部品２（図４参照）が正常に装着されているか否かを検査員が目視で検査する外観検査が行われている。しかし、検査員による目視での外観検査は、検査員の負担が大きく、検査員によるばらつきも発生し、時間もかかる。そのため、外観検査の自動化が求められている。本実施の形態に係る検査システム１０は、基板１に部品２が正常に装着されているか否かを自動的に検査するためのシステムである。本実施の形態において、検査対象である、部品２が装着された基板１をワーク３（図４参照）と称する。なお、基板１に装着された検査対象物である複数の部品２のうち、１つの部品２ａを、第１検査対象物と称し、他の１つの部品２ｂを、第２検査対象物と称してもよい。 Conventionally, an appearance inspection is performed by an inspector to visually check whether the component 2 (see FIG. 4) is normally mounted on the board 1 (see FIG. 4). However, visual inspection by an inspector places a heavy burden on the inspector, causes variation among inspectors, and takes time. Therefore, automation of appearance inspection is required. The inspection system 10 according to the present embodiment is a system for automatically inspecting whether or not the component 2 is normally mounted on the board 1 . In the present embodiment, the board 1 to be inspected, on which the component 2 is mounted, is called a work 3 (see FIG. 4). Among the plurality of components 2 mounted on the substrate 1 and serving as inspection objects, one component 2a is referred to as a first inspection object, and the other component 2b is referred to as a second inspection object. good too.
　図１に示すように、検査システム１０は、検査装置２０と、照明装置３０と、検出センサ１９と、管理装置４０と、入力装置５０と、スピーカ５２と、表示装置６０と、パトライト（登録商標）６２とを含む。 As shown in FIG. 1, the inspection system 10 includes an inspection device 20, an illumination device 30, a detection sensor 19, a management device 40, an input device 50, a speaker 52, a display device 60, and Patlite (registered trademark). ) 62.
　検査装置２０は、検査対象のワーク３を撮像し、その撮像画像に基づいて、基板１に部品２が正常に装着されているか否かを検査する。以下、ワーク３を撮像した撮像画像を、ワーク画像４（図４参照）と称する。 The inspection device 20 captures an image of the workpiece 3 to be inspected, and inspects whether or not the component 2 is normally mounted on the board 1 based on the captured image. A captured image of the workpiece 3 is hereinafter referred to as a workpiece image 4 (see FIG. 4).
　照明装置３０は、検査装置２０がワーク３を撮像する際に当該ワーク３を照射する。照明装置３０は、所定のケーブル１２を介して、検査装置２０に接続される。なお、照明装置３０は、ライトと読み替えられてもよい。 The illumination device 30 illuminates the work 3 when the inspection device 20 images the work 3 . The lighting device 30 is connected to the inspection device 20 via a predetermined cable 12 . Note that the illumination device 30 may be read as a light.
　管理装置４０は、検査装置２０の操作及び検査結果の表示等を行う。管理装置４０は、例えば、ＰＣ（Ｐｅｒｓｏｎａｌ　Ｃｏｍｐｕｔｅｒ）に代表される情報処理装置である。管理装置４０は、所定の通信ネットワーク１１を介して検査装置２０に接続される。通信ネットワーク１１は、有線ＬＡＮ又は無線ＬＡＮのいずれであってもよい。 The management device 40 operates the inspection device 20 and displays inspection results. The management device 40 is, for example, an information processing device represented by a PC (Personal Computer). The management device 40 is connected to the inspection device 20 via a predetermined communication network 11 . The communication network 11 may be either a wired LAN or a wireless LAN.
　入力装置５０は、ユーザから入力操作を受け付ける。入力装置５０の例として、キーボード、マウス、タッチパッド、又は、マイク等が挙げられる。入力装置５０は、所定のケーブル１３又は無線通信（例えばＢｌｕｅｔｏｏｔｈ（登録商標））を介して管理装置４０に接続される。なお、入力装置５０は、検査装置２０に接続されてもよい。 The input device 50 receives input operations from the user. Examples of the input device 50 include a keyboard, mouse, touchpad, microphone, or the like. The input device 50 is connected to the management device 40 via a predetermined cable 13 or wireless communication (for example, Bluetooth (registered trademark)). Note that the input device 50 may be connected to the inspection device 20 .
　スピーカ５２は、検査に関する音声を出力する。スピーカ５２は、所定のケーブル１６を介して管理装置４０に接続される。なお、スピーカ５２は、検査装置２０に接続されてもよい。 The speaker 52 outputs audio related to the examination. The speaker 52 is connected to the management device 40 via a predetermined cable 16. FIG. Note that the speaker 52 may be connected to the inspection device 20 .
　表示装置６０は、検査に関する画面を表示する。表示装置６０の例として、ＬＣＤ（Ｌｉｑｕｉｄ　Ｃｒｙｓｔａｌ　Ｄｉｓｐｌａｙ）、又は、有機ＥＬ（Ｅｌｅｃｔｒｏ　Ｌｕｍｉｎｅｓｃｅｎｃｅ）ディスプレイ等が挙げられる。表示装置６０は、所定のケーブル１４を介して管理装置４０に接続される。なお、表示装置６０は、検査装置２０に接続されてもよい。なお、入力装置５０、スピーカ５２及び表示装置６０は、一体の装置（例えばタブレット端末等）であってもよい。あるいは、管理装置４０、入力装置５０、スピーカ５２、及び、表示装置６０は、一体の装置であってもよい。 The display device 60 displays a screen regarding inspection. Examples of the display device 60 include an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display. A display device 60 is connected to the management device 40 via a predetermined cable 14 . Note that the display device 60 may be connected to the inspection device 20 . Note that the input device 50, the speaker 52, and the display device 60 may be an integrated device (such as a tablet terminal, for example). Alternatively, the management device 40, the input device 50, the speaker 52, and the display device 60 may be an integrated device.
　検出センサ１９は、検査対象のワーク３の到着を検出するためのセンサである。検出センサ１９は、所定のケーブル１５を介して検査装置２０に接続される。なお、検出センサ１９は、管理装置４０に接続されてもよい。 The detection sensor 19 is a sensor for detecting the arrival of the workpiece 3 to be inspected. The detection sensor 19 is connected to the inspection device 20 via a predetermined cable 15 . Note that the detection sensor 19 may be connected to the management device 40 .
　パトライト６２は、所定のケーブル１７を介して検査装置２０に接続される。パトライト６２は、検査装置２０から検査結果に応じて送信される所定の信号を受信した場合、点滅する。例えば、検査装置２０は、異常に装着されている部品２を検出した場合、所定の信号をパトライト６２に送信する。パトライト６２は、当該信号を受信し、点滅する。これにより、ユーザは、パトライト６２の点滅を見て、検査装置２０において異常に装着されている部品２が検出されたことを直ちに知ることができる。 The patrol light 62 is connected to the inspection device 20 via a predetermined cable 17. The patrol light 62 blinks when receiving a predetermined signal transmitted according to the inspection result from the inspection device 20 . For example, the inspection apparatus 20 transmits a predetermined signal to the patrol light 62 when the abnormally attached component 2 is detected. The patrol light 62 receives the signal and blinks. Accordingly, the user can immediately know that the inspection apparatus 20 has detected an abnormally mounted component 2 by seeing the flashing of the patrol light 62 .
　なお、本実施の形態では、検査装置２０、照明装置３０、及び、管理装置４０を別の装置としている。しかし、検査装置２０は照明装置３０を含んでもよい。あるいは、検査装置２０は管理装置４０を含んでもよい。あるいは、検査装置２０は、照明装置３０及び管理装置４０を含んでもよい。 Note that, in the present embodiment, the inspection device 20, the lighting device 30, and the management device 40 are separate devices. However, inspection device 20 may include illumination device 30 . Alternatively, inspection device 20 may include management device 40 . Alternatively, inspection device 20 may include illumination device 30 and management device 40 .
＜検査装置のハードウェア構成＞
　検査装置２０は、カメラ２１と、１つ以上のプロセッサ２２と、ＲＯＭ（Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ）２３と、ＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）２４と、ストレージ２５と、通信Ｉ／Ｆ（Ｉｎｔｅｒｆａｃｅ）２６と、及び、入出力Ｉ／Ｆ２７とを含む。カメラ２１と、プロセッサ２２と、ＲＯＭ２３と、ＲＡＭ２４と、ストレージ２５と、通信Ｉ／Ｆ２６と、入出力Ｉ／Ｆ２７とは、双方向通信可能なバス（図示しない）を介して接続される。 <Hardware configuration of inspection device>
The inspection apparatus 20 includes a camera 21, one or more processors 22, a ROM (Read Only Memory) 23, a RAM (Random Access Memory) 24, a storage 25, a communication I/F (Interface) 26, and , and an input/output I/F 27 . The camera 21, the processor 22, the ROM 23, the RAM 24, the storage 25, the communication I/F 26, and the input/output I/F 27 are connected via a bidirectional communicable bus (not shown).
　カメラ２１は、例えばレンズと撮像素子とを含む。カメラ２１は、検査対象のワーク３を撮像し、ワーク画像４を生成する。カメラ２１は、撮像装置又は撮像部といった他の用語に読み替えられてもよい。 The camera 21 includes, for example, a lens and an imaging device. The camera 21 captures an image of the workpiece 3 to be inspected and generates a workpiece image 4 . The camera 21 may be read as another term such as an imaging device or an imaging unit.
　プロセッサ２２は、検査装置２０全体の動作を制御する。プロセッサ２２は、演算手段、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）、又は、コントローラといった他の用語に読み替えられてもよい。 The processor 22 controls the operation of the inspection device 20 as a whole. The processor 22 may be read as other terms such as computing means, CPU (Central Processing Unit), or controller.
　ＲＯＭ２３は、読み出し専用の不揮発性の記憶媒体であり、ファームウェア等のプログラムが格納される。 The ROM 23 is a read-only non-volatile storage medium in which programs such as firmware are stored.
　ＲＡＭ２４は、情報の高速な読み書きが可能な揮発性の記憶媒体であり、プロセッサ２２が情報を処理する際の作業領域として用いられる。なお、ＲＡＭ２４は、単にメモリと読み替えられてもよい。 The RAM 24 is a volatile storage medium that enables high-speed reading and writing of information, and is used as a work area when the processor 22 processes information. Note that the RAM 24 may simply be read as a memory.
　ストレージ２５は、情報の読み書きが可能な不揮発性の記憶媒体であり、各種の制御プログラム、アプリケーションプログラム、及び、学習モデル１１０（図２参照）等が格納される。ストレージ２５は、例えば、フラッシュメモリ、又は、ＳＤカード等によって構成される。 The storage 25 is a non-volatile storage medium from which information can be read and written, and stores various control programs, application programs, learning models 110 (see FIG. 2), and the like. The storage 25 is configured by, for example, a flash memory or an SD card.
　通信Ｉ／Ｆ２６は、検査装置２０を通信ネットワーク１１に接続するためのインタフェースである。通信Ｉ／Ｆ２６は、有線ＬＡＮ又は無線ＬＡＮのいずれに対応するインタフェースであってもよい。 The communication I/F 26 is an interface for connecting the inspection device 20 to the communication network 11. The communication I/F 26 may be an interface compatible with either a wired LAN or a wireless LAN.
　入出力Ｉ／Ｆ２７は、照明装置３０及び検出センサ１９を接続するためのインタフェースである。なお、入出力Ｉ／Ｆ２７には、入力装置５０及び／又は表示装置６０が接続されてもよい。 The input/output I/F 27 is an interface for connecting the lighting device 30 and the detection sensor 19 . In addition, the input device 50 and/or the display device 60 may be connected to the input/output I/F 27 .
　プロセッサ２２が、ＲＯＭ２３に格納されたプログラム、又は、ストレージ２５からＲＡＭ２４にロードされたプログラムを実行することにより、後述する、検査装置２０が有する各種機能が実現される。 By the processor 22 executing the program stored in the ROM 23 or the program loaded from the storage 25 to the RAM 24, various functions of the inspection device 20, which will be described later, are realized.
＜管理装置のハードウェア構成＞
　管理装置４０は、１つ以上のプロセッサ４１と、ＲＯＭ４２と、ＲＡＭ４３と、ストレージ４４と、通信Ｉ／Ｆ４５と、入出力Ｉ／Ｆ４６とを含む。プロセッサ４１と、ＲＯＭ４２と、ＲＡＭ４３と、ストレージ４４と、通信Ｉ／Ｆ４５と、入出力Ｉ／Ｆ４６とは、双方向通信可能なバス（図示しない）を介して接続される。 <Hardware configuration of management device>
Management device 40 includes one or more processors 41 , ROM 42 , RAM 43 , storage 44 , communication I/F 45 and input/output I/F 46 . Processor 41, ROM 42, RAM 43, storage 44, communication I/F 45, and input/output I/F 46 are connected via a bus (not shown) capable of two-way communication.
　プロセッサ４１は、管理装置４０全体の動作を制御する。 The processor 41 controls the overall operation of the management device 40.
　ＲＯＭ４２は、読み出し専用の不揮発性の記憶媒体であり、ファームウェア等のプログラムが格納されている。 The ROM 42 is a read-only non-volatile storage medium in which programs such as firmware are stored.
　ＲＡＭ４３は、情報の高速な読み書きが可能な揮発性の記憶媒体であり、プロセッサ４１が情報を処理する際の作業領域として用いられる。なお、ＲＡＭ４３は、単にメモリと読み替えられてもよい。 The RAM 43 is a volatile storage medium that enables high-speed reading and writing of information, and is used as a work area when the processor 41 processes information. Note that the RAM 43 may simply be read as a memory.
　ストレージ４４は、情報の読み書きが可能な不揮発性の記憶媒体であり、ＯＳ（Ｏｐｅｒａｔｉｎｇ　Ｓｙｓｔｅｍ）、各種の制御プログラム、アプリケーションプログラム、及び、学習モデル１１０（図３参照）等が格納される。ストレージ４４は、例えば、フラッシュメモリ、ＳＳＤ（Ｓｏｌｉｄ　Ｓｔａｔｅ　Ｄｒｉｖｅ）、又は、ＨＤＤ（Ｈａｒｄ　Ｄｉｓｋ　Ｄｒｉｖｅ）等によって構成される。 The storage 44 is a non-volatile storage medium from which information can be read and written, and stores an OS (Operating System), various control programs, application programs, learning models 110 (see FIG. 3), and the like. The storage 44 is configured by, for example, flash memory, SSD (Solid State Drive), or HDD (Hard Disk Drive).
　通信Ｉ／Ｆ４５は、管理装置４０を通信ネットワーク１１に接続するためのインタフェースである。通信Ｉ／Ｆ４５は、有線ＬＡＮ又は無線ＬＡＮのいずれに対応するインタフェースであってもよい。 The communication I/F 45 is an interface for connecting the management device 40 to the communication network 11. The communication I/F 45 may be an interface compatible with either a wired LAN or a wireless LAN.
　入出力Ｉ／Ｆ４６は、入力装置５０及び／又は表示装置６０を接続するためのインタフェースである。 The input/output I/F 46 is an interface for connecting the input device 50 and/or the display device 60.
　プロセッサ４１が、ＲＯＭ４２に格納されたプログラム、又は、ストレージ４４からＲＡＭ４３にロードされたプログラムを実行することにより、後述する、管理装置４０が有する各種機能が実現される。なお、管理装置４０は、画像描画を高速に処理するためのＧＰＵ（Ｇｒａｐｈｉｃｓ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）を備えてもよい。 By the processor 41 executing a program stored in the ROM 42 or a program loaded from the storage 44 to the RAM 43, various functions of the management device 40, which will be described later, are realized. Note that the management device 40 may include a GPU (Graphics Processing Unit) for processing image drawing at high speed.
＜照明装置のハードウェア構成＞
　照明装置３０は、ＬＥＤ（Ｌｉｇｈｔ　Ｅｍｉｔｔｉｎｇ　Ｄｉｏｄｅ）光源３１と、入出力Ｉ／Ｆ３２と、調光制御回路３３とを含む。 <Hardware Configuration of Lighting Device>
The illumination device 30 includes an LED (Light Emitting Diode) light source 31 , an input/output I/F 32 and a dimming control circuit 33 .
　ＬＥＤ光源３１は、複数のＬＥＤを含んで構成され、発光可能である。照明装置３０は、互いに形状の異なる複数のＬＥＤ光源３１を有してよい。例えば、照明装置３０は、バー形状のＬＥＤ光源３１と、マルチアングル形状のＬＥＤ光源３１と、ドーム形状のＬＥＤ光源３１と、バックライト形状のＬＥＤ光源３１とを備えてよい。さらに、照明装置３０は、赤外線を発光するＬＥＤ光源３１を備えてもよい。 The LED light source 31 includes a plurality of LEDs and is capable of emitting light. The illumination device 30 may have a plurality of LED light sources 31 with different shapes. For example, the illumination device 30 may include a bar-shaped LED light source 31 , a multi-angle-shaped LED light source 31 , a dome-shaped LED light source 31 , and a backlight-shaped LED light source 31 . Furthermore, the illumination device 30 may include an LED light source 31 that emits infrared rays.
　入出力Ｉ／Ｆ３２は、検査装置２０を接続するためのインタフェースである。 The input/output I/F 32 is an interface for connecting the inspection device 20 .
　調光制御回路３３は、入出力Ｉ／Ｆ３２を通じて受信した検査装置２０からの指示に基づいて、ＬＥＤ光源３１の発光を制御する。例えば、調光制御回路３３は、いずれのＬＥＤ光源３１を発光させるか、ＬＥＤ光源３１の照明の色及び強度等を制御する。 The dimming control circuit 33 controls light emission of the LED light source 31 based on instructions from the inspection device 20 received through the input/output I/F 32 . For example, the dimming control circuit 33 controls which LED light source 31 is to emit light, the color and intensity of illumination of the LED light source 31, and the like.
＜検査装置の機能構成＞
　図２は、実施の形態１に係る検査装置２０の機能構成の一例を示すブロック図である。 <Functional configuration of inspection device>
FIG. 2 is a block diagram showing an example of the functional configuration of the inspection device 20 according to Embodiment 1. As shown in FIG.
　検査装置２０は、撮像制御部１０１と、最適条件決定部１０２と、検査実行部１０３と、注目領域格納部１０４と、撮像条件格納部１０５と、最適撮像条件格納部１０６と、学習モデル格納部１０７とを有する。撮像制御部１０１、最適条件決定部１０２、及び、検査実行部１０３の機能は、プロセッサ２２がＲＡＭ２４（メモリ）等と協働してコンピュータプログラム（検査プログラム）を実行することにより、実現されてよい。撮像条件格納部１０５、最適撮像条件格納部１０６、及び、学習モデル格納部１０７の機能は、ＲＡＭ２４（メモリ）及び／又はストレージ２５によって実現されてよい。 The inspection apparatus 20 includes an imaging control unit 101, an optimum condition determination unit 102, an inspection execution unit 103, an attention area storage unit 104, an imaging condition storage unit 105, an optimum imaging condition storage unit 106, and a learning model storage unit. 107. The functions of the imaging control unit 101, the optimum condition determination unit 102, and the inspection execution unit 103 may be realized by the processor 22 cooperating with the RAM 24 (memory) and the like to execute a computer program (inspection program). . Functions of the imaging condition storage unit 105 , the optimum imaging condition storage unit 106 , and the learning model storage unit 107 may be realized by the RAM 24 (memory) and/or the storage 25 .
　注目領域格納部１０４は、検査対象のワーク３に設定される複数の注目領域５（図４参照）に関する情報（以下、注目領域情報と称する）を格納する。注目領域５は、検査対象であるワーク３の検査対象物である部品２を囲んで設定される領域である。ワーク３における検査対象物である複数の部品５が存在する領域を、検査対象領域と称してもよい。なお、注目領域５の詳細については後述する（図４参照）。 The attention area storage unit 104 stores information (hereinafter referred to as attention area information) regarding a plurality of attention areas 5 (see FIG. 4) set on the workpiece 3 to be inspected. The region of interest 5 is a region set surrounding the part 2, which is the object to be inspected, of the work 3, which is the object to be inspected. A region in which a plurality of parts 5, which are objects to be inspected, exist on the workpiece 3 may be referred to as a region to be inspected. Details of the attention area 5 will be described later (see FIG. 4).
　撮像条件格納部１０５は、複数の撮像条件に関する情報を格納する。撮像条件には、カメラ２１の撮像に関する情報と、照明装置３０の照明に関する情報とが含まれる。なお、撮像条件の詳細については後述する（図６及び図７参照）。 The imaging condition storage unit 105 stores information regarding a plurality of imaging conditions. The imaging conditions include information about imaging by the camera 21 and information about lighting by the lighting device 30 . Details of the imaging conditions will be described later (see FIGS. 6 and 7).
　最適撮像条件格納部１０６は、撮像条件格納部１０５に格納されている複数の撮像条件のうち、検査対象のワーク３に最適な撮像条件（以下、最適撮像条件と称する）を格納する。なお、最適撮像条件の詳細については後述する（図９及び図１０参照）。 The optimum imaging condition storage unit 106 stores the optimum imaging condition (hereinafter referred to as the optimum imaging condition) for the workpiece 3 to be inspected among the plurality of imaging conditions stored in the imaging condition storage unit 105 . Details of the optimum imaging conditions will be described later (see FIGS. 9 and 10).
　学習モデル格納部１０７は、ワーク３を撮像したワーク画像４の注目領域５において、検査対象物である部品２が正常に装着されているか否かを検出するために用いられる学習モデル１１０を格納する。 The learning model storage unit 107 stores a learning model 110 used for detecting whether or not the component 2, which is the inspection object, is normally mounted in the attention area 5 of the work image 4 obtained by imaging the work 3. .
　撮像制御部１０１は、カメラ２１及び照明装置３０を制御して、検査対象のワーク３を撮像し、ワーク画像４を生成する。撮像制御部１０１は、ワーク画像４の画質調整を行ってもよい。 The imaging control unit 101 controls the camera 21 and the lighting device 30 to image the workpiece 3 to be inspected and generate the workpiece image 4 . The imaging control unit 101 may adjust the image quality of the workpiece image 4 .
　最適条件決定部１０２は、撮像条件格納部１０５に格納されている複数の撮像条件のうち、検査対象のワーク画像４の各注目領域５に最適な撮像条件（最適撮像条件）を決定する。最適条件決定部１０２は、決定した最適撮像条件を最適撮像条件格納部１０６に格納する。 The optimum condition determining unit 102 determines the optimum imaging condition (optimum imaging condition) for each attention area 5 of the workpiece image 4 to be inspected from among the plurality of imaging conditions stored in the imaging condition storage unit 105 . The optimal condition determination unit 102 stores the determined optimal imaging conditions in the optimal imaging condition storage unit 106 .
　検査実行部１０３は、最適撮像条件格納部１０６に格納されている最適撮像条件に基づいて撮像制御部１０１が撮像したワーク画像４を用いて、各注目領域５に対応する領域において部品２が正常に装着されているか否かを検査する。検査実行部１０３は、学習モデル格納部１０７に格納されている学習モデル１１０に対して、ワーク画像４の注目領域５に対応する領域の画像を入力することにより、当該検査を実行する。 The inspection executing unit 103 uses the work image 4 captured by the imaging control unit 101 based on the optimal imaging conditions stored in the optimal imaging condition storage unit 106 to determine whether the component 2 is normal in the area corresponding to each attention area 5. Check whether it is attached to the The inspection executing unit 103 inputs the image of the area corresponding to the attention area 5 of the workpiece image 4 to the learning model 110 stored in the learning model storage unit 107, thereby executing the inspection.
＜管理装置の機能構成＞
　図３は、実施の形態１に係る管理装置４０の機能構成の一例を示すブロック図である。 <Functional configuration of management device>
FIG. 3 is a block diagram showing an example of functional configuration of the management device 40 according to the first embodiment.
　管理装置４０は、機能として、注目領域設定部２０１と、撮像パターン生成部２０２と、照明パターン生成部２０３と、撮像条件生成部２０４と、学習モデル生成部２０５と、ＵＩ制御部２０６と、注目領域格納部２０７と、撮像パターン格納部２０８と、照明パターン格納部２０９と、撮像条件格納部２１０と、学習モデル格納部２１１と、検査結果格納部２１２とを有する。注目領域設定部２０１、撮像パターン生成部２０２、照明パターン生成部２０３、撮像条件生成部２０４、学習モデル生成部２０５、及び、ＵＩ制御部２０６の機能は、プロセッサ４１がＲＡＭ４３（メモリ）等と協働してコンピュータプログラム（検査プログラム）を実行することにより、実現されてよい。注目領域格納部２０７、撮像パターン格納部２０８、照明パターン格納部２０９、撮像条件格納部２１０、学習モデル格納部２１１、及び、検査結果格納部２１２の機能は、ＲＡＭ４３（メモリ）及び／又はストレージ４４によって実現されてよい。 The management device 40 has, as functions, an attention area setting unit 201, an imaging pattern generation unit 202, an illumination pattern generation unit 203, an imaging condition generation unit 204, a learning model generation unit 205, a UI control unit 206, an attention It has an area storage unit 207 , an imaging pattern storage unit 208 , an illumination pattern storage unit 209 , an imaging condition storage unit 210 , a learning model storage unit 211 , and an inspection result storage unit 212 . The functions of the attention area setting unit 201, the imaging pattern generation unit 202, the illumination pattern generation unit 203, the imaging condition generation unit 204, the learning model generation unit 205, and the UI control unit 206 are performed by the processor 41 in cooperation with the RAM 43 (memory) and the like. It may be realized by working and executing a computer program (inspection program). The functions of the region-of-interest storage unit 207, the imaging pattern storage unit 208, the illumination pattern storage unit 209, the imaging condition storage unit 210, the learning model storage unit 211, and the inspection result storage unit 212 are implemented by the RAM 43 (memory) and/or the storage 44. may be realized by
　注目領域格納部２０７は、各ワーク３に設定される複数の注目領域５に関する情報（注目領域情報）を格納する。 The region-of-interest storage unit 207 stores information (region-of-interest information) on a plurality of regions of interest 5 set for each work 3 .
　撮像パターン格納部２０８は、複数の撮像パターンを格納する。なお、撮像パターンの詳細については後述する（図５参照）。 The imaging pattern storage unit 208 stores a plurality of imaging patterns. Details of the imaging pattern will be described later (see FIG. 5).
　照明パターン格納部２０９は、複数の照明パターンを格納する。なお、照明パターンの詳細については後述する（図６参照）。 The illumination pattern storage unit 209 stores a plurality of illumination patterns. Details of the illumination pattern will be described later (see FIG. 6).
　撮像条件格納部２１０は、複数の撮像条件に関する情報を格納する。なお、撮像条件の詳細については後述する。 The imaging condition storage unit 210 stores information regarding a plurality of imaging conditions. Details of the imaging conditions will be described later.
　学習モデル格納部２１１は、ワーク画像４の注目領域５において部品２が正常に装着されているか否かを検出するために用いられる学習モデル１１０を格納する。 The learning model storage unit 211 stores the learning model 110 used to detect whether or not the component 2 is normally mounted in the attention area 5 of the work image 4 .
　検査結果格納部２１２は、検査装置２０によって検査されたワーク３の検査結果を示す情報（以下、ワーク検査結果情報と称する）を格納する。 The inspection result storage unit 212 stores information indicating the inspection result of the work 3 inspected by the inspection device 20 (hereinafter referred to as work inspection result information).
　注目領域設定部２０１は、ワーク３の検査対象物である部品２を囲むように注目領域５を設定し、その注目領域５に関する情報（注目領域情報）を注目領域格納部２０７に格納する。また、注目領域設定部２０１は、注目領域格納部２０７から、検査対象のワーク３に対応付けられている注目領域情報を取得し、検査装置２０に送信する。検査装置２０は、送信された注目領域情報を注目領域格納部１０４に格納する。 The attention area setting unit 201 sets the attention area 5 so as to surround the part 2 which is the inspection object of the workpiece 3, and stores information (attention area information) on the attention area 5 in the attention area storage unit 207. The attention area setting unit 201 also acquires attention area information associated with the workpiece 3 to be inspected from the attention area storage unit 207 and transmits the attention area information to the inspection apparatus 20 . The inspection apparatus 20 stores the transmitted attention area information in the attention area storage unit 104 .
　撮像パターン生成部２０２は、複数の撮像パターンを生成し、撮像パターン格納部２０８に格納する。なお、撮像パターンの詳細については後述する（図５参照）。 The imaging pattern generation unit 202 generates a plurality of imaging patterns and stores them in the imaging pattern storage unit 208 . Details of the imaging pattern will be described later (see FIG. 5).
　照明パターン生成部２０３は、複数の照明パターンを生成し、照明パターン生成部２０３に格納する。なお、照明パターンの詳細については後述する（図６参照）。 The illumination pattern generation unit 203 generates a plurality of illumination patterns and stores them in the illumination pattern generation unit 203. Details of the illumination pattern will be described later (see FIG. 6).
　撮像条件生成部２０４は、撮像パターン格納部２０８に格納されている撮像パターンと、照明パターン格納部２０９に格納されている照明パターンとを組み合わせて撮像条件を生成する。撮像条件生成部２０４は、生成した複数の撮像条件を撮像条件格納部２１０に格納する。また、撮像条件生成部２０４は、撮像条件格納部２１０に格納されている複数の撮像条件を、検査装置２０に送信する。検査装置２０は、管理装置４０から受信した撮像条件を撮像条件格納部１０５に格納する。 The imaging condition generation unit 204 generates imaging conditions by combining the imaging pattern stored in the imaging pattern storage unit 208 and the illumination pattern stored in the illumination pattern storage unit 209 . The imaging condition generation unit 204 stores the generated imaging conditions in the imaging condition storage unit 210 . The imaging condition generation unit 204 also transmits a plurality of imaging conditions stored in the imaging condition storage unit 210 to the inspection device 20 . The inspection device 20 stores the imaging conditions received from the management device 40 in the imaging condition storage unit 105 .
　学習モデル生成部２０５は、注目領域５において部品２が正常に装着されているか否かを検査するために用いられる学習モデル１１０の生成及び学習を行う。例えば、学習モデル生成部２０５は、部品２が正常に装着されている複数のワーク画像４の注目領域５における画像（以下、注目領域画像と称する）の特徴量を用いて、学習モデル１１０の学習を行う。学習モデル１１０は、注目領域画像が入力された場合、その入力された注目領域画像の特徴量が、部品２が正常に装着されている注目領域画像の特徴量と比較してどのくらい異なっているかを評価値として出力する。学習モデル１１０は、注目領域画像が入力されると、当該注目領域画像に含まれる部品２が正常に装着されている可能性が高いほど、小さな評価値を出力（つまり推論）する。別言すると、学習モデル１１０は、注目領域画像が入力されると、当該注目領域画像に含まれる部品２が異常に装着されている可能性が高いほど、大きな評価値を出力（つまり推論）する。学習モデル生成部２０５は、学習済みの学習モデル１１０を、学習モデル格納部２１１に格納する。また、学習モデル生成部２０５は、学習済みの学習モデル１１０を検査装置２０に送信する。検査装置２０は、管理装置４０から受信した学習モデル１１０を学習モデル格納部１０７に格納する。学習モデル１１０は、画像解析に係るニューラルネットワーク、ディープニューラルネットワーク、又は、ＣＮＮ（Ｃｏｎｖｏｌｕｔｉｏｎａｌ　Ｎｅｕｒａｌ　Ｎｅｔｗｏｒｋ）として構成されてよい。ただし、学習モデル１１０は、これらに限られず、様々な人工知能技術又は機械学習技術に基づいて構成されてよい。なお、本実施の形態では、上述のとおり、部品２が異常に装着されている可能性が高いほど大きな評価値を出力する学習モデル１００を用いる場合を例示している。しかし、本実施の形態は、部品２が正常に装着されている可能性が高いほど大きな評価値を出力する学習モデル１００を用いて構成されてもよい。また、本実施の形態では、管理装置４０が学習モデル生成部２０５を有しているが、検査装置２０が学習モデル生成部２０５を有してもよい。 The learning model generation unit 205 generates and learns the learning model 110 used to check whether the component 2 is normally mounted in the attention area 5 . For example, the learning model generation unit 205 learns the learning model 110 using the feature amount of images (hereinafter referred to as attention area images) in the attention areas 5 of a plurality of work images 4 on which the parts 2 are normally mounted. I do. When the attention area image is input, the learning model 110 determines how much the feature amount of the input attention area image differs from the feature amount of the attention area image in which the part 2 is normally mounted. Output as an evaluation value. When a region-of-interest image is input, the learning model 110 outputs (that is, infers) a smaller evaluation value as the probability that the component 2 included in the region-of-interest image is normally mounted is higher. In other words, when the attention area image is input, the learning model 110 outputs (that is, infers) a larger evaluation value as the possibility that the part 2 included in the attention area image is abnormally mounted is higher. . The learning model generation unit 205 stores the learned learning model 110 in the learning model storage unit 211 . Also, the learning model generation unit 205 transmits the learned learning model 110 to the inspection device 20 . The inspection device 20 stores the learning model 110 received from the management device 40 in the learning model storage unit 107 . The learning model 110 may be configured as a neural network for image analysis, a deep neural network, or a CNN (Convolutional Neural Network). However, the learning model 110 is not limited to these, and may be configured based on various artificial intelligence techniques or machine learning techniques. As described above, the present embodiment exemplifies the case of using the learning model 100 that outputs a larger evaluation value as the possibility that the component 2 is abnormally attached is higher. However, the present embodiment may be configured using the learning model 100 that outputs a larger evaluation value as the possibility that the component 2 is normally mounted is higher. Moreover, although the management device 40 has the learning model generation unit 205 in the present embodiment, the inspection device 20 may have the learning model generation unit 205 .
　ＵＩ制御部２０６は、検査に関するＵＩ画面を生成し、表示装置６０に表示する。また、ＵＩ制御部２０６は、入力装置５０からの入力を受け付けて、各種情報の入力及び表示を制御する。例えば、ＵＩ制御部２０６は、検査装置２０から送信されるワーク検査結果情報に基づいて、ワーク３の検査結果を示すＵＩ画面を生成し、表示装置６０に表示する。これにより、ユーザは、ワーク３に対する検査結果を視認できる。また、ＵＩ制御部２０６は、検査装置２０から送信されるワーク検査結果情報を、検査結果格納部２１２に格納する。なお、ＵＩ制御部２０６の詳細については後述する（図１２、図１３、図１５、図１６参照）。 The UI control unit 206 generates a UI screen regarding examination and displays it on the display device 60 . Also, the UI control unit 206 receives input from the input device 50 and controls input and display of various information. For example, the UI control unit 206 generates a UI screen showing the inspection result of the work 3 based on the work inspection result information transmitted from the inspection device 20 and displays it on the display device 60 . Thereby, the user can visually recognize the inspection result for the workpiece 3 . Also, the UI control unit 206 stores workpiece inspection result information transmitted from the inspection apparatus 20 in the inspection result storage unit 212 . Details of the UI control unit 206 will be described later (see FIGS. 12, 13, 15, and 16).
＜注目領域の詳細＞
　図４は、実施の形態１に係るワーク３に設定される注目領域５の一例を示す図である。 <Details of attention area>
FIG. 4 is a diagram showing an example of the attention area 5 set on the workpiece 3 according to the first embodiment.
　図４に示すように、基板１には複数の部品２が装着される。注目領域設定部２０１は、ワーク３における検査対象物である部品２（例えば部品２ａ、２ｂ）を囲むように注目領域５（例えば注目領域５ａ、５ｂ）を設定する。部品２ａは第１検査対象物と読み替え、部品２ａを囲む注目領域５ａは第１注目領域と読み替えられてもよい。部品２ｂは第２検査対象物と読み替え、部品２ｂを囲む注目領域５ｂは第２注目領域と読み替えられてもよい。注目領域５の形状は矩形であってよい。ただし、注目領域５の形状は矩形に限られず、多角形又は楕円形等であってもよい。注目領域設定部２０１は、例えば、次の（Ａ１）又は（Ａ２）のいずれかの方法によって注目領域５を設定する。 As shown in FIG. 4, a plurality of components 2 are mounted on the substrate 1. The attention area setting unit 201 sets an attention area 5 (for example, attention areas 5a and 5b) so as to surround a part 2 (for example, parts 2a and 2b) that is an inspection target on the workpiece 3. FIG. The part 2a may be read as the first inspection object, and the attention area 5a surrounding the part 2a may be read as the first attention area. The part 2b may be read as the second inspection object, and the attention area 5b surrounding the part 2b may be read as the second attention area. The shape of the attention area 5 may be rectangular. However, the shape of the attention area 5 is not limited to a rectangle, and may be polygonal, elliptical, or the like. The attention area setting unit 201 sets the attention area 5 by, for example, one of the following methods (A1) and (A2).
（Ａ１）注目領域設定部２０１は、ワーク３の設計情報に基づいて注目領域５を設定する。設計情報は、基板１における検査対象物である部品２の位置を定める情報である。そこで、注目領域設定部２０１は、設計情報に基づいて、検査対象の部品２が装着される基板１上の位置を自動的に特定し、その特定した位置に注目領域５を設定する。 (A1) The attention area setting unit 201 sets the attention area 5 based on the design information of the workpiece 3 . The design information is information that determines the position of the component 2 that is the object to be inspected on the board 1 . Therefore, the attention area setting unit 201 automatically identifies the position on the board 1 where the inspection target component 2 is mounted based on the design information, and sets the attention area 5 at the identified position.
（Ａ２）ユーザは、ＵＩ制御部２０６が提供するＵＩ画面を通じて、ワーク画像４に対して手動で注目領域５を設定する。例えば、ユーザが、ＵＩ画面を通じて、ワーク画像４における検査対象物の部品２を囲む。注目領域設定部２０１は、囲まれた領域を当該部品２の注目領域５に設定する。 (A2) The user manually sets the attention area 5 on the work image 4 through the UI screen provided by the UI control unit 206 . For example, the user surrounds the part 2 of the inspection object in the workpiece image 4 through the UI screen. The attention area setting unit 201 sets the enclosed area as the attention area 5 of the component 2 .
　注目領域設定部２０１は、上記の（Ａ１）又は（Ａ２）の方法によって設定した注目領域５を示す注目領域情報を生成し、注目領域格納部２０７に格納する。 The attention area setting unit 201 generates attention area information indicating the attention area 5 set by the method (A1) or (A2) above, and stores it in the attention area storage unit 207 .
＜撮像パターンの詳細＞
　図５は、実施の形態１に係る撮像パターンの一例を示す図である。 <Details of imaging pattern>
FIG. 5 is a diagram showing an example of an imaging pattern according to Embodiment 1. FIG.
　撮像パターンは、複数の異なる撮像パラメータを組み合わせたものである。撮像パラメータの例として、図５に示すように、シャッタースピート、最長露光時間、レンズの絞り値、最大ゲイン、カメラ感度、明度、ホワイトバランスのＲｅｄボリューム、ホワイトバランスのＢｌｕｅボリューム、コントラスト強度、暗部補正、明部補正、及び、ペデスタルレベルが挙げられる。 An imaging pattern is a combination of multiple different imaging parameters. Examples of imaging parameters, as shown in FIG. 5, are shutter speed, maximum exposure time, lens aperture value, maximum gain, camera sensitivity, brightness, white balance red volume, white balance blue volume, contrast intensity, and dark area correction. , bright area correction, and pedestal level.
　撮像パターン生成部２０２は、例えば、次の（Ｂ１）又は（Ｂ２）のいずれかの方法により、少なくとも１つの撮像パラメータが異なる複数の撮像パターンを生成する。 The imaging pattern generation unit 202 generates a plurality of imaging patterns with at least one different imaging parameter, for example, by either method (B1) or (B2) below.
（Ｂ１）ユーザは、ＵＩ制御部２０６が提供するＵＩ画面を通じて、手動で各撮像パラメータを調整し、複数の撮像パターンを生成する。例えば、ユーザは、第１の注目領域５に装着される暗い色の部品２が適切に撮像されるよう各撮像パラメータを調整した第１の撮像パターンを生成する。また、ユーザは、第２の注目領域５に装着される明るい色の部品２が適切に撮像されるよう各撮像パラメータを調整した第２の撮像パターンを生成する。撮像パターン生成部２０２は、このように生成された複数の撮像パターンを、撮像パターン格納部２０８に格納する。 (B1) The user manually adjusts each imaging parameter through the UI screen provided by the UI control unit 206 to generate a plurality of imaging patterns. For example, the user generates a first imaging pattern in which each imaging parameter is adjusted so that the dark-colored component 2 attached to the first attention area 5 is appropriately imaged. Also, the user generates a second imaging pattern in which each imaging parameter is adjusted so that the bright-colored component 2 attached to the second attention area 5 is appropriately imaged. The imaging pattern generation unit 202 stores the plurality of imaging patterns generated in this manner in the imaging pattern storage unit 208 .
（Ｂ２）撮像パターン生成部２０２は、検査装置２０のカメラ２１が撮像したワーク画像４を分析して撮像パラメータを自動的に調整し、複数の撮像パターンを生成する。例えば、撮像パターン生成部２０２は、ワーク画像４の第１の注目領域５の注目領域画像の特徴を分析し、その分析結果から当該注目領域５に適切な各撮像パラメータを決定し、第１の撮像パターンを生成する。注目領域５の特徴の例として、部品２の色、部品２の反射率、部品２の透過率、部品２の材質、及び、部品２の高さ等が挙げられる。また、撮像パターン生成部２０２は、ワーク画像４の第２の注目領域５の注目領域画像の特徴を分析し、その分析結果から当該注目領域５に適切な各撮像パラメータを決定し、第２の撮像パターンを生成する。撮像パターン生成部２０２は、このように生成した複数の撮像パターンを、撮像パターン格納部２０８に格納する。 (B2) The imaging pattern generation unit 202 analyzes the work image 4 captured by the camera 21 of the inspection device 20, automatically adjusts imaging parameters, and generates a plurality of imaging patterns. For example, the imaging pattern generation unit 202 analyzes the features of the region-of-interest image of the first region of interest 5 of the work image 4, determines each imaging parameter appropriate for the region of interest 5 from the analysis result, and Generate an imaging pattern. Examples of features of the attention area 5 include the color of the component 2, the reflectance of the component 2, the transmittance of the component 2, the material of the component 2, the height of the component 2, and the like. In addition, the imaging pattern generation unit 202 analyzes the features of the attention area image of the second attention area 5 of the work image 4, determines each imaging parameter appropriate for the attention area 5 from the analysis result, Generate an imaging pattern. The imaging pattern generation unit 202 stores the plurality of imaging patterns generated in this manner in the imaging pattern storage unit 208 .
　なお、本実施の形態では、管理装置４０が撮像パターン生成部２０２を有する例を説明するが、検査装置２０が撮像パターン生成部２０２を有してもよい。 In this embodiment, an example in which the management device 40 has the imaging pattern generation unit 202 will be described, but the inspection device 20 may have the imaging pattern generation unit 202 .
＜照明パターンの詳細＞
　図６は、実施の形態１に係る照明パターンの一例を示す図である。 <Details of lighting pattern>
6 is a diagram showing an example of an illumination pattern according to Embodiment 1. FIG.
　照明パターンは、複数の異なる照明パラメータを組み合わせたものである。照明パラメータの例として、図６に示すように、照明の形状、照明の当て方、照明の色、偏光フィルタの使用有無、赤外線照明の使用有無、及び、照明の強度が挙げられる。 A lighting pattern is a combination of multiple different lighting parameters. Examples of lighting parameters include lighting shape, lighting method, lighting color, use of polarizing filter, use of infrared lighting, and lighting intensity, as shown in FIG.
　照明の形状に関する照明パラメータの例として、バー、マルチアングル、ドーム、及び、バックライトが挙げられる。例えば、撮像制御部１０１は、照明の形状に関する照明パラメータが「バー」の照明パターンに基づいてワーク３を撮像する場合、照明装置３０が備えるバー形状のＬＥＤ光源３１を点灯させる。 Examples of lighting parameters related to lighting shape include bar, multi-angle, dome, and backlight. For example, when the imaging control unit 101 captures an image of the workpiece 3 based on an illumination pattern in which the illumination parameter related to the shape of the illumination is “bar”, the imaging control unit 101 lights the bar-shaped LED light source 31 provided in the illumination device 30 .
　照明の当て方に関する照明パラメータの例として、正反射、拡散反射、及び、透過が挙げられる。 　Specular reflection, diffuse reflection, and transmission are examples of lighting parameters related to how to apply lighting.
　照明の色に関する照明パラメータの例として、青色、赤色、及び、緑色が挙げられる。 Examples of lighting parameters related to lighting colors include blue, red, and green.
　照明パターン生成部２０３は、次の方法により、少なくとも１つの照明パラメータが互いに異なる複数の照明パターンを生成してよい。すなわち、ユーザは、ＵＩ制御部２０６が提供するＵＩ画面を通じて、手動で照明パラメータを調整し、複数の照明パターンを生成する。照明パターン生成部２０３は、このように生成された複数の照明パターンを、照明パターン格納部２０９に格納する。 The illumination pattern generation unit 203 may generate a plurality of illumination patterns with at least one illumination parameter different from each other by the following method. That is, the user manually adjusts the lighting parameters through the UI screen provided by the UI control unit 206 to generate a plurality of lighting patterns. The illumination pattern generation unit 203 stores the plurality of illumination patterns generated in this way in the illumination pattern storage unit 209 .
　なお、本実施の形態では、管理装置４０が照明パターン生成部２０３を有する例を説明するが、検査装置２０が照明パターン生成部２０３を有してもよい。 In this embodiment, an example in which the management device 40 has the illumination pattern generation unit 203 will be described, but the inspection device 20 may have the illumination pattern generation unit 203 .
＜撮像条件の詳細＞
　撮像条件生成部２０４は、撮像パターン格納部２０８に格納されている１つの撮像パターンと、照明パターン格納部２０９に格納されている１つの照明パターンとを組み合せて、１つの撮像条件を生成する。撮像条件生成部２０４は、撮像パターンと照明パターンとを互いに異なるように組み合わせて複数の撮像条件を生成する。撮像条件生成部２０４は、生成した複数の撮像条件を、撮像条件格納部２１０に格納する。例えば、撮像パターン格納部２０８に４つの撮像パターンが格納されており、照明パターン格納部２０９に４つの照明パターンが格納されている場合、撮像条件生成部２０４は、４×４＝１６個の撮像条件を生成する。撮像条件生成部２０４は、撮像条件格納部２１０に格納されている複数の撮像条件を検査装置２０に送信する。検査装置２０は、管理装置４０から受信した複数の撮像条件を撮像条件格納部１０５に格納する。 <Details of imaging conditions>
The imaging condition generation unit 204 combines one imaging pattern stored in the imaging pattern storage unit 208 and one illumination pattern stored in the illumination pattern storage unit 209 to generate one imaging condition. The imaging condition generation unit 204 generates a plurality of imaging conditions by combining imaging patterns and illumination patterns in different ways. The imaging condition generation unit 204 stores the generated imaging conditions in the imaging condition storage unit 210 . For example, when four imaging patterns are stored in the imaging pattern storage unit 208 and four illumination patterns are stored in the illumination pattern storage unit 209, the imaging condition generation unit 204 performs 4×4=16 imaging. Generate conditions. The imaging condition generation unit 204 transmits a plurality of imaging conditions stored in the imaging condition storage unit 210 to the inspection device 20 . The inspection device 20 stores the plurality of imaging conditions received from the management device 40 in the imaging condition storage unit 105 .
＜検査前シーケンス＞
　図７は、実施の形態１に係る検査システム１０における検査前の動作を示すシーケンスチャートである。 <Pre-examination sequence>
FIG. 7 is a sequence chart showing pre-inspection operations in the inspection system 10 according to the first embodiment.
　管理装置４０は、表示装置６０に検査メインＵＩ画面（図示しない）を表示する（ステップＳ１１）。 The management device 40 displays an inspection main UI screen (not shown) on the display device 60 (step S11).
　ユーザは、入力装置５０を通じて、検査メインＵＩ画面にユーザ名を入力する（ステップＳ１２、ステップＳ１３）。 The user inputs the user name on the examination main UI screen through the input device 50 (steps S12 and S13).
　管理装置４０は、入力されたユーザ名の設定要求を検査装置２０に送信する（ステップＳ１４）。 The management device 40 transmits a request for setting the input user name to the inspection device 20 (step S14).
　検査装置２０は、ユーザ名の設定要求を受信し、当該ユーザ名の設定を完了し、完了応答を管理装置４０へ送信する（ステップＳ１５）。 The inspection device 20 receives the user name setting request, completes the setting of the user name, and transmits a completion response to the management device 40 (step S15).
　管理装置４０は、検査装置２０から完了応答を受信し、ユーザ名の入力が完了した旨を表示装置６０に表示する（ステップＳ１６）。 The management device 40 receives the completion response from the inspection device 20 and displays on the display device 60 that the input of the user name has been completed (step S16).
　ユーザは、入力装置５０を通じて、検査対象とするワーク３を選択する（ステップＳ１７、ステップＳ１８）。 The user selects the workpiece 3 to be inspected through the input device 50 (steps S17 and S18).
　管理装置４０は、選択されたワーク３の選択要求を検査装置２０に送信する（ステップＳ１９）。 The management device 40 transmits a selection request for the selected work 3 to the inspection device 20 (step S19).
　検査装置２０は、ワーク３の選択要求を受信し、検査対象とするワーク３を選択し、完了応答を管理装置４０へ送信する（ステップＳ２０）。 The inspection device 20 receives the selection request for the work 3, selects the work 3 to be inspected, and transmits a completion response to the management device 40 (step S20).
　管理装置４０は、検査装置２０から完了応答を受信し、検査対象とするワーク３の選択が完了した旨を表示装置６０に表示する（ステップＳ２１）。 The management device 40 receives the completion response from the inspection device 20, and displays on the display device 60 that the selection of the work 3 to be inspected has been completed (step S21).
　また、管理装置４０は、カメラ２１が撮像中の映像ストリームの配信要求を検査装置２０に送信する（ステップＳ２２）。 In addition, the management device 40 transmits to the inspection device 20 a request for distribution of the video stream being imaged by the camera 21 (step S22).
　検査装置２０は、映像ストリームの配信要求を受信し、当該要求の受領応答を管理装置４０に送信する（ステップＳ２３）。また、検査装置２０は、カメラ２１が撮像中の映像ストリームを管理装置４０に送信する（ステップＳ２４）。 The inspection device 20 receives the video stream distribution request and transmits a response to the request to the management device 40 (step S23). Also, the inspection device 20 transmits the video stream being imaged by the camera 21 to the management device 40 (step S24).
　管理装置４０は、検査装置２０から受信した映像ストリームをリアルタイムで表示装置６０に表示する（ステップＳ２５）。 The management device 40 displays the video stream received from the inspection device 20 in real time on the display device 60 (step S25).
　これにより、ユーザは、検査対象とするワーク３を、検査装置２０に設定できる。また、ユーザは、検査装置２０のカメラ２１が撮像中の映像ストリームをリアルタイムに視聴できる。 Thereby, the user can set the workpiece 3 to be inspected in the inspection device 20 . Also, the user can view the video stream being imaged by the camera 21 of the inspection device 20 in real time.
＜検査シーケンス＞
　図８は、実施の形態１に係る検査システム１０における検査の動作を示すシーケンスチャートである。 <Inspection sequence>
FIG. 8 is a sequence chart showing inspection operations in the inspection system 10 according to the first embodiment.
　ユーザは、検査を開始する場合、入力装置５０を通じて、検査モードＯＮを入力する（ステップＳ３１、ステップＳ３２）。 When the user starts the inspection, the user inputs inspection mode ON through the input device 50 (steps S31 and S32).
　管理装置４０は、入力された検査モードＯＮの要求を検査装置２０に送信する（ステップＳ３３）。 The management device 40 transmits the input inspection mode ON request to the inspection device 20 (step S33).
　検査装置２０は、検査モードＯＮの要求を受信し、検査モードをＯＮに変更し、完了応答を管理装置４０へ送信する（ステップＳ３４）。 The inspection device 20 receives the request to turn on the inspection mode, changes the inspection mode to ON, and transmits a completion response to the management device 40 (step S34).
　管理装置４０は、検査装置２０から完了応答を受信し、検査モードＯＮへの切り替えが完了した旨を表示装置６０に表示する（ステップＳ３５）。 The management device 40 receives the completion response from the inspection device 20, and displays on the display device 60 that switching to inspection mode ON has been completed (step S35).
　また、管理装置４０は、カメラ２１が撮像中の映像の配信要求を検査装置２０へ送信する（ステップＳ３６）。 Also, the management device 40 transmits a request for distribution of the video being captured by the camera 21 to the inspection device 20 (step S36).
　検査装置２０は、映像の配信要求を受信し、当該配信要求の受領応答を管理装置４０に送信する（ステップＳ３７）。また、検査装置２０は、カメラ２１が撮像中の映像ストリームを管理装置４０に送信する（ステップＳ３８）。 The inspection device 20 receives the video distribution request and transmits an acknowledgment of the distribution request to the management device 40 (step S37). Also, the inspection device 20 transmits the video stream being imaged by the camera 21 to the management device 40 (step S38).
　管理装置４０は、検査装置２０から受信した映像ストリームをリアルタイムで表示装置６０に表示する（ステップＳ３９）。 The management device 40 displays the video stream received from the inspection device 20 in real time on the display device 60 (step S39).
　検出センサ１９は、ワーク３を検出した場合、ワーク検出通知を検査装置２０に送信する（ステップＳ４０）。 When detecting the work 3, the detection sensor 19 transmits a work detection notification to the inspection device 20 (step S40).
　検査装置２０は、ワーク検出通知を受信した場合、ワーク３の検査開始通知を管理装置４０に送信する（ステップＳ４１）。 When the inspection apparatus 20 receives the workpiece detection notification, it transmits an inspection start notification for the workpiece 3 to the management apparatus 40 (step S41).
　管理装置４０は、検査装置２０からワーク３の検査開始通知を受信した場合、ワーク３の検査を開始する旨を表示装置６０に表示する（ステップＳ４２）。 When the management device 40 receives the inspection start notification of the work 3 from the inspection device 20, it displays on the display device 60 that the inspection of the work 3 will start (step S42).
　また、検査装置２０は、ワーク３の検査処理を実行する（ステップＳ４３）。当該ワーク３の検査処理の詳細については後述する（図１１参照）。 Also, the inspection device 20 executes inspection processing of the workpiece 3 (step S43). The details of the inspection process for the workpiece 3 will be described later (see FIG. 11).
　検査装置２０は、ワーク３の検査処理を完了した後、当該ワーク３の検査結果を含むワーク検査結果情報を管理装置４０に送信する（ステップＳ４４）。 After completing the inspection process of the work 3, the inspection device 20 transmits work inspection result information including the inspection result of the work 3 to the management device 40 (step S44).
　管理装置４０は、ワーク検査結果情報を検査装置２０から受信し、当該ワーク検査結果情報の内容を表示装置６０に表示する（ステップＳ４５）。 The management device 40 receives the workpiece inspection result information from the inspection device 20, and displays the content of the workpiece inspection result information on the display device 60 (step S45).
　検査システム１０は、順次搬送されてくる検査対象のワーク３に対して、上述したステップＳ４０～ステップＳ４５の処理を繰り返し行う。 The inspection system 10 repeatedly performs the above-described processing of steps S40 to S45 on the workpieces 3 to be inspected that are sequentially conveyed.
　ユーザは、検査を終了する場合、入力装置５０を通じて、検査モードＯＦＦを入力する
（ステップＳ４６、ステップＳ４７）。 When the user ends the inspection, the user inputs inspection mode OFF through the input device 50 (steps S46 and S47).
　管理装置４０は、入力された検査モードＯＦＦの要求を検査装置２０に送信する（ステップＳ４８）。 The management device 40 transmits the input request to turn off the inspection mode to the inspection device 20 (step S48).
　検査装置２０は、検査モードＯＦＦの要求を受信し、検査モードをＯＦＦに変更し、完了応答を管理装置４０へ送信する（ステップＳ４９）。 The inspection device 20 receives the request to turn off the inspection mode, changes the inspection mode to OFF, and transmits a completion response to the management device 40 (step S49).
　管理装置４０は、完了応答を検査装置２０から受信し、検査モードＯＦＦへの切り替えが完了した旨を表示装置６０に表示する（ステップＳ５０）。 The management device 40 receives the completion response from the inspection device 20, and displays on the display device 60 that switching to inspection mode OFF has been completed (step S50).
＜検査前処理フロー＞
　次に、図９及び図１０を参照して、ワーク３の検査処理の前に行われる検査前処理について説明する。検査前処理では、ワーク３に対する注目領域５の設定及び最適撮像条件の決定が行われる。 <Pre-test processing flow>
Next, pre-inspection processing performed before inspection processing of the workpiece 3 will be described with reference to FIGS. 9 and 10. FIG. In the pre-inspection processing, setting of the attention area 5 for the workpiece 3 and determination of optimum imaging conditions are performed.
　図９は、実施の形態１に係る検査前処理の一例を示すフローチャートである。 FIG. 9 is a flowchart showing an example of pre-examination processing according to the first embodiment.
　撮像制御部１０１は、カメラ２１を制御してワーク３を撮像し、ワーク画像４を生成する（ステップＳ１０１）。 The imaging control unit 101 controls the camera 21 to image the workpiece 3 and generate the workpiece image 4 (step S101).
　注目領域設定部２０１は、ワーク４に複数の注目領域５を設定する（ステップＳ１０２）。注目領域設定部２０１は、設定した複数の注目領域５を示す情報（注目領域情報）を検査装置２０に送信する。検査装置２０は、受信した複数の注目領域情報を注目領域格納部１０４に格納する。 The attention area setting unit 201 sets a plurality of attention areas 5 on the workpiece 4 (step S102). The attention area setting unit 201 transmits information (attention area information) indicating a plurality of set attention areas 5 to the inspection device 20 . The inspection apparatus 20 stores the received plural pieces of attention area information in the attention area storage unit 104 .
　撮像条件生成部２０４は、撮像パターン格納部２０８に格納されている複数の撮像パターンと、照明パターン格納部２０９に格納されている複数の照明パターンとを組み合わせて、複数の撮像条件を生成する（ステップＳ１０３）。撮像条件生成部２０４は、生成した複数の撮像条件を検査装置２０に送信する。検査装置２０は、受信した複数の撮像条件を撮像条件格納部１０５に格納する。 The imaging condition generation unit 204 generates a plurality of imaging conditions by combining a plurality of imaging patterns stored in the imaging pattern storage unit 208 and a plurality of illumination patterns stored in the illumination pattern storage unit 209 ( step S103). The imaging condition generation unit 204 transmits the plurality of generated imaging conditions to the inspection device 20 . The inspection apparatus 20 stores the received multiple imaging conditions in the imaging condition storage unit 105 .
　最適条件決定部１０２は、撮像条件格納部１０５に格納されている撮像条件のうち、未選択の撮像条件を１つ選択する（ステップＳ１０４）。図１０の説明において、当該選択された撮像条件を、選択撮像条件と称する。 The optimum condition determining unit 102 selects one unselected imaging condition from among the imaging conditions stored in the imaging condition storage unit 105 (step S104). In the description of FIG. 10, the selected imaging conditions are referred to as selected imaging conditions.
　撮像制御部１０１は、選択撮像条件に基づいてカメラ２１及び照明装置３０を制御してワーク３を撮像し、ワーク画像４を生成する（ステップＳ１０５）。撮像制御部１０１は、生成したワーク画像４を、ＲＡＭ２４又はストレージ２５に格納する。 The imaging control unit 101 controls the camera 21 and the lighting device 30 based on the selected imaging conditions to image the workpiece 3 and generate the workpiece image 4 (step S105). The imaging control unit 101 stores the generated work image 4 in the RAM 24 or storage 25 .
　最適条件決定部１０２は、撮像条件格納部１０５に格納されているすべての撮像条件を選択したか否かを判定する（ステップＳ１０６）。 The optimum condition determination unit 102 determines whether or not all imaging conditions stored in the imaging condition storage unit 105 have been selected (step S106).
　未選択の撮像条件が残っている場合（ステップＳ１０６：ＮＯ）、検査装置２０は、処理をステップＳ１０４に戻す。 If unselected imaging conditions remain (step S106: NO), the inspection device 20 returns the process to step S104.
　すべての撮像条件を選択した場合（ステップＳ１０６：ＹＥＳ）、最適条件決定部１０２は、最適条件決定処理を実行する（ステップＳ１０７）。なお、最適条件決定処理の詳細については後述する（図１０参照）。そして、本処理は終了する。 When all imaging conditions have been selected (step S106: YES), the optimum condition determination unit 102 executes optimum condition determination processing (step S107). Details of the optimum condition determination process will be described later (see FIG. 10). Then, the process ends.
　図１０は、図９に示す最適条件決定処理（ステップＳ１０７）の一例を示すフローチャートである。 FIG. 10 is a flow chart showing an example of the optimum condition determination process (step S107) shown in FIG.
　最適条件決定部１０２は、注目領域格納部１０４に格納されている複数の注目領域５のうち、未選択の注目領域５を１つ選択する（ステップＳ２０１）。図１０の説明において、当該選択された注目領域５を、選択注目領域５と称する。 The optimum condition determination unit 102 selects one unselected attention area 5 from among the plurality of attention areas 5 stored in the attention area storage unit 104 (step S201). In the description of FIG. 10 , the selected attention area 5 is called a selected attention area 5 .
　最適条件決定部１０２は、ステップＳ１０５においてＲＡＭ２４又はストレージ２５に格納された、異なる撮像条件で撮像された複数のワーク画像４のそれぞれから、選択注目領域５における画像を取得する。図１０の説明において、当該画像を選択注目領域画像と称する。最適条件決定部１０２は、学習モデル格納部１０７に格納されている学習モデル１１０を用いて、取得した複数の選択注目領域画像のそれぞれについて評価値を算出する（ステップＳ２０２）。 The optimum condition determination unit 102 acquires an image of the selected region of interest 5 from each of the plurality of work images 4 captured under different imaging conditions, stored in the RAM 24 or storage 25 in step S105. In the description of FIG. 10, this image is called a selected region-of-interest image. The optimum condition determination unit 102 uses the learning model 110 stored in the learning model storage unit 107 to calculate an evaluation value for each of the acquired selected region-of-interest images (step S202).
　最適条件決定部１０２は、ステップＳ２０２で算出した評価値に基づいて、選択注目領域５の最適撮像条件を決定する（ステップＳ２０３）。例えば、最適条件決定部１０２は、評価値が最も高く算出された撮像条件を、選択注目領域５の最適撮像条件に決定する。 The optimum condition determination unit 102 determines the optimum imaging conditions for the selected region of interest 5 based on the evaluation value calculated in step S202 (step S203). For example, the optimum condition determining unit 102 determines the imaging condition with the highest calculated evaluation value as the optimum imaging condition for the selected region of interest 5 .
　最適条件決定部１０２は、注目領域格納部１０４に格納されている選択注目領域５に、ステップＳ２０３で決定した最適撮像条件を対応付ける（ステップＳ２０４）。 The optimal condition determination unit 102 associates the selected attention area 5 stored in the attention area storage unit 104 with the optimal imaging condition determined in step S203 (step S204).
　最適条件決定部１０２は、注目領域格納部１０４に格納されているすべての注目領域５を選択したか否かを判定する（ステップＳ２０５）。 The optimum condition determination unit 102 determines whether or not all the attention areas 5 stored in the attention area storage unit 104 have been selected (step S205).
　未選択の注目領域５が残っている場合（ステップＳ２０５：ＮＯ）、検査装置２０は、処理をステップＳ２０１に戻す。すべての注目領域５を選択した場合（ステップＳ２０５：ＹＥＳ）、検査装置２０は、本処理を終了する。 If an unselected region of interest 5 remains (step S205: NO), the inspection device 20 returns the process to step S201. If all the attention areas 5 have been selected (step S205: YES), the inspection device 20 terminates this process.
＜検査処理フロー＞
　次に、図１１を参照して、ワーク３の検査処理について説明する。 <Inspection processing flow>
Next, referring to FIG. 11, inspection processing of the workpiece 3 will be described.
　図１１は、実施の形態１に係る検査処理の一例を示すフローチャートである。 FIG. 11 is a flowchart showing an example of inspection processing according to the first embodiment.
　撮像制御部１０１は、検査対象のワーク３に対応付けられている注目領域格納部１０４及び最適撮像条件格納部１０６を選択する（ステップＳ３０１）。 The imaging control unit 101 selects the attention area storage unit 104 and the optimum imaging condition storage unit 106 associated with the workpiece 3 to be inspected (step S301).
　撮像制御部１０１は、検出センサ１９がワーク３を検出したか否かを判定する（ステップＳ３０２）。例えば、撮像制御部１０１は、検出センサ１９からワーク検出通知を受信したか否かを判定する。 The imaging control unit 101 determines whether or not the detection sensor 19 has detected the workpiece 3 (step S302). For example, the imaging control unit 101 determines whether or not a workpiece detection notification has been received from the detection sensor 19 .
　検出センサ１９がワーク３を未検出である場合（ステップＳ３０２：ＮＯ）、検査装置２０は、処理をステップＳ３０２に戻す。検出センサ１９がワーク３を検出した場合（ステップＳ３０２：ＹＥＳ）、検査装置２０は、処理を次のステップＳ３０３に進める。 If the detection sensor 19 has not detected the workpiece 3 (step S302: NO), the inspection device 20 returns the process to step S302. If the detection sensor 19 detects the workpiece 3 (step S302: YES), the inspection apparatus 20 advances the process to the next step S303.
　撮像制御部１０１は、最適撮像条件格納部１０６から、未選択の最適撮像条件を１つ選択する（ステップＳ３０３）。図１１の説明において、当該選択された最適撮像条件を、選択最適撮像条件と称する。 The imaging control unit 101 selects one unselected optimum imaging condition from the optimum imaging condition storage unit 106 (step S303). In the description of FIG. 11, the selected optimum imaging conditions are referred to as selected optimum imaging conditions.
　撮像制御部１０１は、選択最適撮像条件に基づいてカメラ２１及び照明装置３０を制御してワーク３を撮像し、ワーク画像４を生成する（ステップＳ３０４）。撮像制御部１０１は、生成したワーク画像４をＲＡＭ４３又はストレージ４４に格納する。 The imaging control unit 101 controls the camera 21 and the lighting device 30 based on the selected optimum imaging conditions to image the workpiece 3 and generate the workpiece image 4 (step S304). The imaging control unit 101 stores the generated work image 4 in the RAM 43 or storage 44 .
　撮像制御部１０１は、最適撮像条件格納部１０６に含まれるすべての最適撮像条件を選択したか否かを判定する（ステップＳ３０５）。 The imaging control unit 101 determines whether or not all the optimum imaging conditions included in the optimum imaging condition storage unit 106 have been selected (step S305).
　未選択の最適撮像条件が残っている場合（ステップＳ３０５：ＮＯ）、検査装置２０は、処理をステップＳ３０３に戻す。すべての最適撮像条件を選択した場合（ステップＳ３０５：ＹＥＳ）、検査装置２０は、処理を次のステップＳ３０６に進める。 If unselected optimum imaging conditions remain (step S305: NO), the inspection apparatus 20 returns the process to step S303. If all the optimum imaging conditions have been selected (step S305: YES), the inspection apparatus 20 advances the process to the next step S306.
　検査実行部１０３は、注目領域格納部１０４に格納されている複数の注目領域５のうち、未選択の注目領域５を１つ選択する（ステップＳ３０６）。図１１の説明において、当該選択された注目領域５を、選択注目領域５と称する。 The inspection execution unit 103 selects one unselected attention area 5 from among the plurality of attention areas 5 stored in the attention area storage unit 104 (step S306). In the description of FIG. 11 , the selected attention area 5 is called a selected attention area 5 .
　検査実行部１０３は、ステップＳ３０４においてＲＡＭ４３又はストレージ４４に格納された、異なる最適撮像条件で撮像された複数のワーク画像４のうち、選択注目領域５に対応付けられている最適撮像条件にて撮像されたワーク画像４を選択する（ステップＳ３０７）。図１１の説明において、当該選択されたワーク画像４を、選択ワーク画像４と称する。 The inspection executing unit 103 picks up images under the optimum imaging conditions associated with the selected region of interest 5 among the plurality of workpiece images 4 picked up under different optimum imaging conditions stored in the RAM 43 or the storage 44 in step S304. The work image 4 that has been displayed is selected (step S307). In the description of FIG. 11, the selected work image 4 will be referred to as a selected work image 4. FIG.
　検査実行部１０３は、学習モデル格納部１０７に格納されている学習モデル１１０を用いて、選択ワーク画像４の選択注目領域５における画像の評価値を算出する（ステップＳ３０８）。図１１の説明において、当該画像を選択注目領域画像と称する。選択ワーク画像４は、選択注目領域５を撮像するに最適な撮像条件にて撮像されたものである。よって、当該選択ワーク画像４から取得された選択注目領域画像から算出される評価値は、単一の撮像条件で撮像されたワーク画像から取得された選択注目領域画像から算出される評価値よりも、精度が高くなり得る。 The inspection executing unit 103 uses the learning model 110 stored in the learning model storage unit 107 to calculate the evaluation value of the image in the selected attention area 5 of the selected work image 4 (step S308). In the description of FIG. 11, the image is referred to as a selected region-of-interest image. The selected workpiece image 4 is captured under optimum imaging conditions for imaging the selected attention area 5 . Therefore, the evaluation value calculated from the selected region-of-interest image obtained from the selected work image 4 is higher than the evaluation value calculated from the selected region-of-interest image obtained from the work image captured under a single imaging condition. , can be more accurate.
　検査実行部１０３は、ステップＳ３０８で算出した評価値に基づいて、選択注目領域５の検査結果を決定する（ステップＳ３０９）。例えば、検査実行部１０３は、評価値が所定の閾値Ｔｈ未満である場合、選択注目領域５において部品２が異常に装着されている（ＮＧ）と決定する。例えば、検査実行部１０３は、評価値が閾値Ｔｈ以上である場合、選択注目領域５において部品２が正常に装着されている（ＯＫ）と決定する。検査実行部１０３は、選択注目領域５に検査結果を対応付けてＲＡＭ４３又はストレージ４４に格納する。 The inspection execution unit 103 determines the inspection result of the selected region of interest 5 based on the evaluation value calculated in step S308 (step S309). For example, when the evaluation value is less than a predetermined threshold value Th, the inspection execution unit 103 determines that the component 2 is abnormally attached (NG) in the selected attention area 5 . For example, when the evaluation value is equal to or greater than the threshold Th, the inspection execution unit 103 determines that the component 2 is normally mounted (OK) in the selected attention area 5 . The inspection execution unit 103 associates the inspection results with the selected region of interest 5 and stores them in the RAM 43 or the storage 44 .
　検査実行部１０３は、注目領域格納部１０４に格納されているすべての注目領域５を選択したか否かを判定する（ステップＳ３１０）。未選択の注目領域５が残っている場合、検査装置２０は、処理をステップＳ３０６に戻す。 The inspection execution unit 103 determines whether or not all the attention areas 5 stored in the attention area storage unit 104 have been selected (step S310). If there remains an unselected region of interest 5, the inspection apparatus 20 returns the process to step S306.
　すべての注目領域５を選択した場合、検査実行部１０３は、ステップＳ３０９にてＲＡＭ４３又はストレージ４４に格納された、各注目領域５に対応付けられている検査結果をまとめてワーク検査結果情報を生成し、管理装置４０へ送信する（ステップＳ３１１）。そして、検査装置２０は、処理をステップＳ３０２に戻し、次に搬送されてくるワーク３の検査を行う。 When all the attention areas 5 have been selected, the inspection execution unit 103 collects the inspection results associated with each attention area 5 and stored in the RAM 43 or the storage 44 in step S309 to generate workpiece inspection result information. and transmitted to the management device 40 (step S311). The inspection apparatus 20 then returns the process to step S302 and inspects the work 3 that is next transported.
　なお、管理装置４０のＵＩ制御部２０６は、検査装置２０からワーク検査結果情報を受信し、検査結果格納部２１２に格納する。また、ＵＩ制御部２０６は、ワーク検査結果徐由豊の内容を表示装置６０に表示する。ユーザは、表示装置６０に表示されたワーク検査結果情報の内容を見て、ワーク３の各注目領域５において部品２が正常に装着されているか否かを確認できる。 The UI control unit 206 of the management device 40 receives work inspection result information from the inspection device 20 and stores it in the inspection result storage unit 212 . The UI control unit 206 also displays the content of the work inspection result Xu Yufeng on the display device 60 . The user can see the contents of the workpiece inspection result information displayed on the display device 60 and can confirm whether or not the component 2 is normally mounted in each attention area 5 of the workpiece 3 .
＜パターン用ＵＩ画面＞
　図１２は、実施の形態１に係る撮像パターン及び照明パターンを生成及び調整するためのＵＩ画面の一例を示す模式図である。 <UI screen for patterns>
12 is a schematic diagram showing an example of a UI screen for generating and adjusting an imaging pattern and an illumination pattern according to Embodiment 1. FIG.
　管理装置４０のＵＩ制御部２０６は、図１２に示すように、撮像パターン及び照明パターンを生成又は調整するためのＵＩ画面（以下、パターン用ＵＩ画面３００と称する）を表示装置６０に表示する。 The UI control unit 206 of the management device 40 displays, on the display device 60, a UI screen (hereinafter referred to as a pattern UI screen 300) for generating or adjusting the imaging pattern and the illumination pattern, as shown in FIG.
　パターン用ＵＩ画面３００は、図１２に示すように、撮像パターンリスト領域３０１と、照明パターンリスト領域３０２と、撮像条件別領域３０３と、ワーク画像確認領域３０４とを含む。 The pattern UI screen 300 includes an imaging pattern list area 301, an illumination pattern list area 302, an imaging condition-specific area 303, and a workpiece image confirmation area 304, as shown in FIG.
　ＵＩ制御部２０６は、撮像パターンリスト領域３０１に、撮像パターン格納部２０８に格納されている撮像パターンをリスト表示する。ユーザは、撮像パターンリスト領域３０１に、新たな撮像パターンを追加することにより、新たな撮像パターンを生成できる。また、ユーザは、撮像パターンリスト領域３０１に表示されている撮像パラメータを調整することにより、撮像パターン格納部２０８に格納されている撮像パターンの撮像パラメータを調整できる。 The UI control unit 206 lists the imaging patterns stored in the imaging pattern storage unit 208 in the imaging pattern list area 301 . The user can generate a new imaging pattern by adding a new imaging pattern to the imaging pattern list area 301 . Further, the user can adjust the imaging parameters of the imaging patterns stored in the imaging pattern storage unit 208 by adjusting the imaging parameters displayed in the imaging pattern list area 301 .
　ＵＩ制御部２０６は、照明パターンリスト領域３０２に、照明パターン格納部２０９に格納されている照明パターンをリスト表示する。ユーザは、照明パターンリスト領域３０２に、新たな照明パターンを追加することにより、新たな照明パターンを生成できる。また、ユーザは、照明パターンリスト領域３０２に表示されている照明パラメータを調整することにより、照明パターン格納部２０９に格納されている照明パターンの照明パラメータを調整できる。 The UI control unit 206 lists the illumination patterns stored in the illumination pattern storage unit 209 in the illumination pattern list area 302 . A user can generate a new lighting pattern by adding a new lighting pattern to the lighting pattern list area 302 . Also, the user can adjust the illumination parameters of the illumination patterns stored in the illumination pattern storage unit 209 by adjusting the illumination parameters displayed in the illumination pattern list area 302 .
　ＵＩ制御部２０６は、異なる撮像条件で撮像した複数のワーク画像４を撮像条件別領域３０３に表示する。これにより、ユーザは、撮像条件別領域３０３において、撮像条件毎にどのようなワーク画像４が撮像されるのかを確認できる。 The UI control unit 206 displays a plurality of workpiece images 4 captured under different imaging conditions in the imaging condition-specific area 303 . Accordingly, the user can confirm what kind of work image 4 is captured for each imaging condition in the imaging condition-specific area 303 .
　ＵＩ制御部２０６は、ユーザが撮像条件別領域３０３から選択したワーク画像４を、ワーク画像確認領域３０４に表示する。ＵＩ制御部２０６は、ユーザの操作に応じて、ワーク画像確認領域３０４におけるワーク画像４を拡大又は縮小して表示する。これにより、ユーザは、選択したワーク画像４をより詳細に確認できる。 The UI control unit 206 displays the workpiece image 4 selected by the user from the imaging condition-specific area 303 in the workpiece image confirmation area 304 . The UI control unit 206 enlarges or reduces and displays the work image 4 in the work image confirmation area 304 according to the user's operation. This allows the user to check the selected work image 4 in more detail.
＜注目領域用ＵＩ画面＞
　図１３は、実施の形態１に係る注目領域５を設定するためのＵＩ画面の一例を示す模式図である。図１４は、実施の形態１に係る注目領域情報の構成例を示す図である。 <UI screen for attention area>
FIG. 13 is a schematic diagram showing an example of a UI screen for setting the attention area 5 according to the first embodiment. 14 is a diagram showing a configuration example of attention area information according to Embodiment 1. FIG.
　管理装置４０のＵＩ制御部２０６は、図１３に示すように、注目領域５を入力又は修正するためのＵＩ画面（以下、注目領域用ＵＩ画面３２０と称する）を表示装置６０に表示する。 The UI control unit 206 of the management device 40 displays a UI screen for inputting or correcting the attention area 5 (hereinafter referred to as an attention area UI screen 320) on the display device 60, as shown in FIG.
　注目領域用ＵＩ画面３２０は、注目領域リスト領域３２１と、注目領域確認領域３２２と、評価値確認領域３２３とを含む。 The attention area UI screen 320 includes an attention area list area 321 , an attention area confirmation area 322 , and an evaluation value confirmation area 323 .
　ＵＩ制御部２０６は、注目領域リスト領域３２１に、注目領域格納部２０７に格納されている注目領域情報をリスト表示する。注目領域情報は、例えば、各部品２について、パラメータとして、部品名、部品品番、検査内容、注目領域の左上座標、注目領域の右下座標、前処理位置補正、撮像パターン、及び、照明パターンを有する。 The UI control unit 206 lists the attention area information stored in the attention area storage unit 207 in the attention area list area 321 . For example, for each component 2, the attention area information includes, as parameters, the part name, part number, inspection details, upper left coordinates of the attention area, lower right coordinates of the attention area, preprocessing position correction, imaging pattern, and illumination pattern. have.
・『部品名』は、部品２の名称を示す。
・『部品品番』は、部品２の品番を示す。
・『検査内容』は、部品２を検査対象とするか否かを示す。また、検査内容は、部品２がどのように異常に装着される可能性があるかを示す。
・『注目領域の左上座標』は、矩形の注目領域５の左上の点のワーク画像４上でのＸ座標及びＹ座標を示す。
・『注目領域の右下座標』は、矩形の注目領域５の右下の点のワーク画像４上でのＸ座標及びＹ座標を示す。
・『前処理位置補正』は、前処理で注目領域５の位置を補正するか否かを示す。また、『前処理位置補正』には、設定ボタン３２４が含まれてよい。設定ボタン３２４が選択（押下）されると、前処理による注目領域５の位置の補正量、及び、注目領域５の評価値と比較される閾値Ｔｈなどを設定するためのＵＩ画面が表示されてもよい。
・『撮像パターン』は、注目領域５の最適撮像条件の撮像パターンを示す。
・『照明パターン』は、注目領域５の最適撮像条件の照明パターンを示す。 - "Part name" indicates the name of the part 2.
- "Part number" indicates the part number of the part 2.
- "Inspection content" indicates whether or not the component 2 is to be inspected. The inspection content also indicates how the part 2 may be mounted abnormally.
"Upper left coordinates of attention area" indicates the X and Y coordinates of the upper left point of the rectangular attention area 5 on the workpiece image 4 .
"Lower right coordinates of attention area" indicates the X and Y coordinates of the lower right point of the rectangular attention area 5 on the workpiece image 4. FIG.
"Pre-processing position correction" indicates whether or not the position of the attention area 5 is corrected in pre-processing. In addition, a setting button 324 may be included in the “preprocessing position correction”. When the setting button 324 is selected (pressed), a UI screen for setting a correction amount of the position of the attention area 5 by preprocessing and a threshold value Th to be compared with the evaluation value of the attention area 5 is displayed. good too.
"Imaging pattern" indicates the imaging pattern of the region of interest 5 under the optimal imaging conditions.
- "Illumination pattern" indicates the illumination pattern of the target area 5 under the optimum imaging conditions.
　ユーザは、注目領域リスト領域３２１に、新たな注目領域５を追加することにより、新たな注目領域情報を設定できる。また、ユーザは、注目領域リスト領域３２１に表示されている注目領域情報のパラメータを調整することにより、注目領域格納部２０７に格納されている注目領域情報のパラメータを調整できる。 The user can set new attention area information by adding a new attention area 5 to the attention area list area 321 . Further, the user can adjust the parameters of the attention area information stored in the attention area storage unit 207 by adjusting the parameters of the attention area information displayed in the attention area list area 321 .
　ＵＩ制御部２０６は、注目領域確認領域３２２に、ワーク画像４を表示すると共に、注目領域リスト領域３２１にて選択された注目領域情報に対応する注目領域５を示す矩形枠を、ワーク画像４に重畳表示する。これにより、ユーザは、注目領域リスト領域３２１にて選択中の注目領域情報が、ワーク画像４のいずれの位置の部品２の注目領域５に対応しているかを確認できる。 The UI control unit 206 displays the work image 4 in the attention area confirmation area 322 , and displays a rectangular frame indicating the attention area 5 corresponding to the attention area information selected in the attention area list area 321 on the work image 4 . Display superimposed. Thereby, the user can confirm that the attention area information being selected in the attention area list area 321 corresponds to the attention area 5 of the component 2 at which position on the workpiece image 4 .
　ＵＩ制御部２０６は、評価値確認領域３２３に、注目領域リスト領域３２１にて選択された注目領域情報について、異なる撮像条件にて撮像した当該注目領域５の各注目領域画像から算出した評価値を、当該注目領域画像と共に表示する。これにより、ユーザは、注目領域５ごとに、撮像条件と評価値との関係を確認できる。 The UI control unit 206 displays an evaluation value calculated from each attention area image of the attention area 5 captured under different imaging conditions for the attention area information selected in the attention area list area 321 in the evaluation value confirmation area 323 . , is displayed together with the attention area image. Thereby, the user can confirm the relationship between the imaging condition and the evaluation value for each attention area 5 .
　注目領域設定部２０１は、注目領域リスト領域３２１にて入力又は修正された内容を注目領域情報として注目領域格納部２０７に格納してよい。なお、注目領域情報は、図１４に示すように、ＣＳＶデータとして構成されてもよい。 The attention area setting unit 201 may store the contents input or modified in the attention area list area 321 in the attention area storage unit 207 as attention area information. Note that the attention area information may be configured as CSV data as shown in FIG. 14 .
＜検査結果一覧用ＵＩ画面＞
　図１５は、実施の形態１に係るワーク３の検査結果を一覧で確認するためのＵＩ画面の一例を示す模式図である。 <UI screen for inspection result list>
FIG. 15 is a schematic diagram showing an example of a UI screen for checking inspection results of the workpiece 3 according to the first embodiment.
　ＵＩ制御部２０６は、検査結果格納部２１２に格納されている複数のワーク検査結果を用いて、ワーク３の検査結果を一覧で確認するためのＵＩ画面（以下、検査結果一覧用ＵＩ画面３４０と称する）を表示装置６０に表示する。 The UI control unit 206 uses a plurality of workpiece inspection results stored in the inspection result storage unit 212 to display a UI screen (hereinafter referred to as an inspection result list UI screen 340) for checking the inspection results of the workpiece 3 in a list. ) is displayed on the display device 60 .
　検査結果一覧用ＵＩ画面３４０は、複数のワーク領域３４１を含む。また、検査結果一覧用ＵＩ画面３４０は、ワーク領域３４１ごとに、ワーク画像４と、検査結果３４２と、詳細ボタン３４３とを含む。 The inspection result list UI screen 340 includes a plurality of work areas 341 . The inspection result list UI screen 340 also includes a work image 4 , an inspection result 342 , and a details button 343 for each work area 341 .
　ＵＩ制御部２０６は、１つのワーク領域３４１に、検査を行った１つのワーク３のワーク画像４を表示する。 The UI control unit 206 displays a workpiece image 4 of one inspected workpiece 3 in one workpiece area 341 .
　ＵＩ制御部２０６は、ワーク領域３４１に表示されているワーク画像４について、複数の注目領域５のうち、少なくとも１つの注目領域５において異常が検出された場合、検査結果３４２として異常（ＮＧ）を表示する。ＵＩ制御部２０６は、ワーク領域３４１に表示されているワーク画像４について、複数の注目領域５のすべてにおいて異常が検出されなかった場合、検査結果３４２として正常（ＯＫ）を表示する。 When an abnormality is detected in at least one attention area 5 among the plurality of attention areas 5 in the work image 4 displayed in the work area 341 , the UI control unit 206 outputs an abnormality (NG) as the inspection result 342 . indicate. For the work image 4 displayed in the work area 341 , the UI control unit 206 displays normal (OK) as the inspection result 342 when no abnormality is detected in all of the plurality of attention areas 5 .
　ＵＩ制御部２０６は、ワーク領域３４１の詳細ボタン３４３が選択（押下）された場合、選択されたワーク領域３４１のワーク画像４についての検査結果詳細用ＵＩ画面３６０（図１６参照）を表示する。 When the details button 343 of the work area 341 is selected (depressed), the UI control unit 206 displays the inspection result details UI screen 360 (see FIG. 16) for the work image 4 of the selected work area 341 .
＜検査結果詳細用ＵＩ画面＞
　図１６は、実施の形態１に係る検査結果を詳細に確認するためのＵＩ画面の一例を示す模式図である。 <UI screen for inspection result details>
FIG. 16 is a schematic diagram showing an example of a UI screen for confirming inspection results in detail according to the first embodiment.
　ＵＩ制御部２０６は、図１５で詳細ボタン３４３が選択（押下）されたワーク３（以下、図１６の説明において選択ワーク３と称する）について、当該ワーク３の検査結果を詳細に確認するためのＵＩ画面（以下、検査結果詳細用ＵＩ画面３６０と称する）を表示する。 The UI control unit 206 controls the work 3 for which the details button 343 is selected (depressed) in FIG. 15 (hereinafter referred to as the selected work 3 in the description of FIG. 16). A UI screen (hereinafter referred to as an inspection result detail UI screen 360) is displayed.
　検査結果詳細用ＵＩ画面３６０は、説明領域３６１と、ワーク領域３６２と、部品リスト領域３６３とを含む。 The inspection result details UI screen 360 includes an explanation area 361 , a work area 362 and a parts list area 363 .
　ＵＩ制御部２０６は、説明領域３６１に、異常と判定された注目領域５と、正常と判定された注目領域５とをどのように区別可能に表示しているかを説明する内容を表示する。 The UI control unit 206 displays, in the explanation area 361, the content explaining how the attention area 5 determined to be abnormal and the attention area 5 determined to be normal are displayed in a distinguishable manner.
　ＵＩ制御部２０６は、ワーク領域３６２に、選択ワーク３のワーク画像４を表示する。また、ＵＩ制御部２０６は、当該ワーク画像４の各部品２上に注目領域５を重畳表示する。このとき、ＵＩ制御部２０６は、異常と判定された部品２の注目領域５と、正常と判定された部品２の注目領域５とを区別可能に表示する。例えば、ＵＩ制御部２０６は、異常と判定された部品２の注目領域５を赤色で表示し、正常と判定された部品２の注目領域５を緑色で表示する。これにより、ユーザは、いずれの部品２が異常と判定されたかを容易に確認できる。 The UI control unit 206 displays the work image 4 of the selected work 3 in the work area 362 . The UI control unit 206 also superimposes the attention area 5 on each part 2 of the work image 4 . At this time, the UI control unit 206 displays the attention area 5 of the part 2 determined to be abnormal and the attention area 5 of the part 2 determined to be normal in a distinguishable manner. For example, the UI control unit 206 displays the attention area 5 of the part 2 determined to be abnormal in red, and displays the attention area 5 of the part 2 determined to be normal in green. Thereby, the user can easily confirm which component 2 is determined to be abnormal.
　ＵＩ制御部２０６は、部品リスト領域３６３に、選択ワーク３に装着されている部品２のリストを表示する。また、ＵＩ制御部２０６は、部品リスト領域３６３において、異常と判定された部品２と、正常と判定された部品２とを区別可能に表示する。これにより、ユーザは、いずれの部品２が異常と判定されたかを容易に確認できる。 The UI control unit 206 displays a list of the parts 2 attached to the selected workpiece 3 in the parts list area 363. In addition, the UI control unit 206 displays the parts 2 determined to be abnormal and the parts 2 determined to be normal in a distinguishable manner in the parts list area 363 . Thereby, the user can easily confirm which component 2 is determined to be abnormal.
（実施の形態１のまとめ）
　実施の形態１の内容は、以下の項目のように表現できる。 (Summary of Embodiment 1)
The contents of Embodiment 1 can be expressed as the following items.
＜項目１＞
　検査装置２０は、１つ以上のプロセッサ２２と、メモリ（例えばＲＡＭ２４）と、メモリに保存されているプログラムと、を備える。プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、第１検査対象物（例えば部品２ａ）と第１検査対象物とは異なる第２検査対象物（例えば部品２ｂ）とを含む複数の検査対象物が存在する検査対象領域において、第１検査対象物の検査を行うための第１注目領域５（５ａ）と、第２検査対象物の検査を行うための第２注目領域５（５ｂ）とを設定する。
　プログラムは、検査対象領域を撮像して検査対象領域の撮像画像（例えばワーク画像４）を出力するカメラ２１に、検査対象領域の撮像画像として、第１撮像条件により撮像した第１撮像画像と、第１撮像条件とは異なる第２撮像条件により撮像した第２撮像画像とを撮像させる。
　プログラムは、第１検査対象物と第２検査対象物とを含む複数の検査対象物の異常を検知するための学習モデル１１０と、第１注目領域に対応する第１撮像画像の第１領域とに基づいて第１検査対象物を検査し、かつ学習モデルと第２注目領域に対応する第１撮像画像の第２領域とに基づいて第２検査対象物を検査する第１検査を実行する。
　プログラムは、学習モデルと第１注目領域に対応する第２撮像画像の第１領域とに基づいて第１検査対象物を検査し、かつ学習モデルと第２注目領域に対応する第２撮像画像の第２領域とに基づいて第２検査対象物を検査する第２検査を実行する。
　プログラムは、第１検査の結果と第２検査の結果とを出力する。
　これにより、検査装置２０は、第１撮像条件で撮像された第１撮像画像を用いて、第１検査対象物と第２検査対象物とを検査して第１検査の結果を出力し、第２撮像条件で撮像された第２撮像画像を用いて、第１検査対象物と第２検査対象物とを検査して第２検査の結果を出力する。よって、検査装置２０は、異なる撮像条件で撮像した撮像画像を用いて、各検査対象物を検査することができ、より精度の高い検査結果を得ることができる。 <Item 1>
Inspection device 20 includes one or more processors 22, memory (eg, RAM 24), and programs stored in the memory. The program causes the processor 22 to do the following.
In an inspection target area in which a plurality of inspection targets including a first inspection target (eg, a part 2a) and a second inspection target (eg, a component 2b) different from the first inspection target exist, a first A first attention area 5 (5a) for inspecting an inspection object and a second attention area 5 (5b) for inspecting a second inspection object are set.
The program causes the camera 21, which captures an image of the inspection target area and outputs a captured image of the inspection target area (for example, the workpiece image 4), to take a first captured image captured under a first imaging condition as the captured image of the inspection target area, A second captured image captured under a second imaging condition different from the first imaging condition is captured.
The program includes a learning model 110 for detecting anomalies in a plurality of inspection objects including a first inspection object and a second inspection object, and a first region of a first captured image corresponding to a first region of interest. and inspects a second inspection object based on the learning model and the second region of the first captured image corresponding to the second region of interest.
The program inspects a first inspection object based on the learning model and a first area of a second captured image corresponding to the first attention area, and inspects the learning model and a second captured image corresponding to the second attention area. A second inspection is performed to inspect the second inspection object based on the second area.
The program outputs the results of the first test and the results of the second test.
Thereby, the inspection apparatus 20 inspects the first inspection object and the second inspection object using the first captured image captured under the first imaging condition, outputs the result of the first inspection, and outputs the result of the first inspection. The first inspection object and the second inspection object are inspected using the second captured image captured under the two imaging conditions, and the result of the second inspection is output. Therefore, the inspection apparatus 20 can inspect each inspection target using captured images captured under different imaging conditions, and can obtain more accurate inspection results.
＜項目２＞
　項目１に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、第１注目領域に対応する撮像画像の第１領域に基づいて、第１撮像条件として少なくとも１つの撮像パラメータを含む第１撮像パターンを定める。
　プログラムは、第２注目領域に対応する撮像画像の第２領域に基づいて、第２撮像条件として少なくとも１つの撮像パラメータを含む第２撮像パターンを定める。
　これにより、各撮像条件は、撮像パラメータの異なる撮像パターンによって定められる。よって、検査装置２０は、撮像パラメータの異なる撮像条件で撮像した撮像画像を用いて、各検査対象物を検査することにより、より精度の高い検査結果を得ることができる。 <Item 2>
In the inspection device 20 described in item 1, the program causes the processor 22 to execute the following.
The program defines a first imaging pattern including at least one imaging parameter as a first imaging condition based on a first region of the captured image corresponding to the first region of interest.
The program defines a second imaging pattern including at least one imaging parameter as a second imaging condition based on a second region of the captured image corresponding to the second region of interest.
Thereby, each imaging condition is determined by an imaging pattern having different imaging parameters. Therefore, the inspection apparatus 20 can obtain more accurate inspection results by inspecting each inspection object using captured images captured under imaging conditions with different imaging parameters.
＜項目３＞
　項目２に記載の検査装置２０は、検査対象領域を照射する照明装置３０を備える。
　プログラムは、照明装置３０が検査対象領域を照射するための少なくとも１つの照射パラメータを含む照射パターンを定める。
　これにより、検査装置２０は、照明装置３０に、照明パラメータの異なる照明パターンによって検査対象領域を照射させることができる。 <Item 3>
The inspection apparatus 20 described in item 2 includes an illumination device 30 that illuminates an inspection target area.
The program defines an illumination pattern including at least one illumination parameter for illumination device 30 to illuminate a region to be inspected.
Thereby, the inspection device 20 can cause the illumination device 30 to illuminate the inspection target region with illumination patterns having different illumination parameters.
＜項目４＞
　項目３に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、第１撮像画像として、少なくとも１つの照射パターンごとに第１撮像条件を適用して撮像した検査対象領域の撮像画像をカメラに出力させる。
　プログラムは、第２撮像画像として、少なくとも１つの照射パターンごとに第２撮像条件を適用して撮像した検査対象領域の撮像画像をカメラに出力させる。
　これにより、検査装置２０は、異なる照明パターンで撮像した撮像画像を用いて、各検査対象物を検査することができ、より精度の高い検査結果を得ることができる。 <Item 4>
In the inspection device 20 described in item 3, the program causes the processor 22 to perform the following.
The program causes the camera to output, as the first captured image, the captured image of the inspection target region captured by applying the first imaging condition for each at least one irradiation pattern.
The program causes the camera to output, as the second captured image, the captured image of the inspection target region captured by applying the second imaging condition for each at least one irradiation pattern.
As a result, the inspection apparatus 20 can inspect each inspection object using captured images captured with different illumination patterns, and can obtain more accurate inspection results.
＜項目５＞
　項目１から４のいずれか１項に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２（又はプロセッサ４１）に実行させる。
　プログラムは、カメラが出力した第１撮像画像と第２撮像画像とに基づいた学習により、学習モデル１１０を生成する。
　これにより、検査装置２０は、撮像画像を用いて、検査対象物を検査するための学習モデル１１０を生成できる。 <Item 5>
In the inspection apparatus 20 according to any one of items 1 to 4, the program causes the processor 22 (or processor 41) to execute the following.
The program generates the learning model 110 by learning based on the first captured image and the second captured image output by the camera.
Thereby, the inspection apparatus 20 can generate the learning model 110 for inspecting the inspection object using the captured image.
＜項目６＞
　項目１から５のいずれか１項に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２（又はプロセッサ４１）に実行させる。
　プログラムは、複数の検査対象物の位置を定めた設計情報、又は検査対象領域の撮像画像のいずれかに基づいて第１注目領域と第２注目領域とを設定する。
　これにより、検査装置２０は、検査対象領域において、複数の注目領域を設定できる。 <Item 6>
In the inspection apparatus 20 according to any one of items 1 to 5, the program causes the processor 22 (or processor 41) to execute the following.
The program sets the first region of interest and the second region of interest based on either design information defining the positions of a plurality of inspection objects or a captured image of the inspection target region.
Thereby, the inspection apparatus 20 can set a plurality of attention areas in the inspection target area.
＜項目７＞
　項目１から６のいずれか１項に記載の検査装置２０は、カメラ２１を備える。
　これにより、検査装置２０は、カメラ２１を制御して、検査対象領域の撮像画像を撮像できる。 <Item 7>
The inspection device 20 according to any one of items 1 to 6 includes a camera 21 .
Thereby, the inspection apparatus 20 can control the camera 21 to capture a captured image of the inspection target area.
＜項目８＞
　検査装置２０は、次の検査方法を実施する。
　検査装置２０は、第１検査対象物（例えば部品２）と第１検査対象物とは異なる第２検査対象物とを含む複数の検査対象物が存在する検査対象領域において、第１検査対象物の検査を行うための第１注目領域５と、第２検査対象物の検査を行うための第２注目領域５とを設定する。
　検査装置２０は、検査対象領域を撮像して前記検査対象領域の撮像画像（例えばワーク画像４）を出力するカメラ２１に、検査対象領域の撮像画像として、第１撮像条件により撮像した第１撮像画像と、第１撮像条件とは異なる第２撮像条件により撮像した第２撮像画像とを撮像させる。
　検査装置２０は、第１検査対象物と第２検査対象物とを含む複数の検査対象物の異常を検知するための学習モデル１１０と、第１注目領域に対応する第１撮像画像の第１領域とに基づいて第１検査対象物を検査し、かつ前記学習モデルと第２注目領域に対応する第１撮像画像の第２領域とに基づいて第２検査対象物を検査する第１検査を実行する。
　検査装置２０は、学習モデルと第１注目領域に対応する第２撮像画像の第１領域とに基づいて第１検査対象物を検査し、かつ学習モデルと第２注目領域に対応する第２撮像画像の第２領域とに基づいて第２検査対象物を検査する第２検査を実行する。
　検査装置２０は、第１検査の結果と前記第２検査の結果とを出力する。
　これにより、検査装置２０は、第１撮像条件で撮像された第１撮像画像を用いて、第１検査対象物と第２検査対象物とを検査して第１検査の結果を出力し、第２撮像条件で撮像された第２撮像画像を用いて、第１検査対象物と第２検査対象物とを検査して第２検査の結果を出力する。よって、検査装置２０は、異なる撮像条件で撮像した撮像画像を用いて、各検査対象物を検査することができ、より精度の高い検査結果を得ることができる。 <Item 8>
The inspection device 20 implements the following inspection method.
The inspection apparatus 20 detects the first inspection object in an inspection object area in which a plurality of inspection objects including a first inspection object (for example, the part 2) and a second inspection object different from the first inspection object exist. and a second attention area 5 for inspecting a second inspection object are set.
The inspection apparatus 20 provides a camera 21 that captures an image of an inspection target area and outputs a captured image of the inspection target area (for example, the work image 4) as a captured image of the inspection target area, as a first image captured under a first imaging condition. An image and a second captured image captured under a second imaging condition different from the first imaging condition are captured.
The inspection apparatus 20 includes a learning model 110 for detecting anomalies in a plurality of inspection objects including a first inspection object and a second inspection object, and a first image of a first captured image corresponding to a first region of interest. and inspecting a second inspection object based on the learning model and a second area of the first captured image corresponding to the second attention area. Execute.
The inspection apparatus 20 inspects the first inspection object based on the learning model and the first area of the second captured image corresponding to the first attention area, and performs second imaging corresponding to the learning model and the second attention area. A second inspection is performed to inspect a second inspection object based on a second region of the image.
The inspection device 20 outputs the result of the first inspection and the result of the second inspection.
Thereby, the inspection apparatus 20 inspects the first inspection object and the second inspection object using the first captured image captured under the first imaging condition, outputs the result of the first inspection, and outputs the result of the first inspection. The first inspection object and the second inspection object are inspected using the second captured image captured under the two imaging conditions, and the result of the second inspection is output. Therefore, the inspection apparatus 20 can inspect each inspection target using captured images captured under different imaging conditions, and can obtain more accurate inspection results.
＜項目９＞
　検査プログラムは、次のことをプロセッサ２２に実行させる。
　検査プログラムは、第１検査対象物（例えば部品２）と第１検査対象物とは異なる第２検査対象物とを含む複数の検査対象物が存在する検査対象領域において、第１検査対象物の検査を行うための第１注目領域５と、第２検査対象物の検査を行うための第２注目領域５とを設定する。
　検査プログラムは、検査対象領域を撮像して検査対象領域の撮像画像（例えばワーク画像４）を出力するカメラ２１に、検査対象領域の撮像画像として、第１撮像条件により撮像した第１撮像画像と、第１撮像条件とは異なる第２撮像条件により撮像した第２撮像画像とを撮像させる。
　検査プログラムは、第１検査対象物と第２検査対象物とを含む複数の検査対象物の異常を検知するための学習モデル１１０と、第１注目領域に対応する第１撮像画像の第１領域とに基づいて第１検査対象物を検査し、かつ学習モデルと第２注目領域に対応する第１撮像画像の第２領域とに基づいて第２検査対象物を検査する第１検査を実行する。
　検査プログラムは、学習モデルと第１注目領域に対応する第２撮像画像の第１領域とに基づいて第１検査対象物を検査し、かつ学習モデルと第２注目領域に対応する第２撮像画像の第２領域とに基づいて第２検査対象物を検査する第２検査を実行する。
　検査プログラムは、第１検査の結果と第２検査の結果とを出力する。
　これにより、検査プログラムは、第１撮像条件で撮像された第１撮像画像を用いて、第１検査対象物と第２検査対象物とを検査して第１検査の結果を出力し、第２撮像条件で撮像された第２撮像画像を用いて、第１検査対象物と第２検査対象物とを検査して第２検査の結果を出力する。よって、検査プログラムは、異なる撮像条件で撮像した撮像画像を用いて、各検査対象物を検査することができ、より精度の高い検査結果を得ることができる。 <Item 9>
The inspection program causes the processor 22 to do the following.
The inspection program includes a first inspection object (for example, a part 2) and a second inspection object different from the first inspection object in an inspection object area in which a plurality of inspection objects exist. A first attention area 5 for inspection and a second attention area 5 for inspecting a second inspection object are set.
The inspection program causes the camera 21, which captures an image of the inspection target area and outputs a captured image of the inspection target area (for example, the workpiece image 4), to capture the first captured image captured under the first imaging condition as the captured image of the inspection target area. , and a second captured image captured under a second imaging condition different from the first imaging condition.
The inspection program comprises a learning model 110 for detecting anomalies in a plurality of inspection objects including a first inspection object and a second inspection object, and a first region of a first captured image corresponding to a first region of interest. and inspecting the second inspection object based on the learning model and the second region of the first captured image corresponding to the second region of interest. .
The inspection program inspects the first inspection object based on the learning model and a first area of a second captured image corresponding to the first attention area, and inspects the second captured image corresponding to the learning model and the second attention area. A second inspection is performed for inspecting the second inspection object based on the second area of .
The inspection program outputs the result of the first inspection and the result of the second inspection.
Accordingly, the inspection program inspects the first inspection object and the second inspection object using the first captured image captured under the first imaging condition, outputs the result of the first inspection, and outputs the second inspection result. Using the second captured image captured under the imaging conditions, the first inspection object and the second inspection object are inspected, and the result of the second inspection is output. Therefore, the inspection program can inspect each inspection object using captured images captured under different imaging conditions, and can obtain more accurate inspection results.
（実施の形態２）
　実施の形態２では、実施の形態１で説明済みの構成要素については共通の参照符号を付し、説明を省略する場合がある。 (Embodiment 2)
In the second embodiment, the same reference numerals are given to the constituent elements that have already been explained in the first embodiment, and the explanation may be omitted.
＜本実施の形態に至る経緯＞
　本実施の形態に至る経緯について、図１７及び図１８を参照しながら説明する。図１７は、ワーク画像４のサイズを、学習モデル１１０に入力可能な画像のサイズに縮小する例を示す模式図である。図１８は、複数のカメラ２１でワーク３を、学習モデル１１０に入力可能な画像のサイズに分割して撮像する例を示す模式図である。 <Background leading up to the present embodiment>
The background to the present embodiment will be described with reference to FIGS. 17 and 18. FIG. FIG. 17 is a schematic diagram showing an example of reducing the size of the workpiece image 4 to the size of an image that can be input to the learning model 110. As shown in FIG. FIG. 18 is a schematic diagram showing an example in which a plurality of cameras 21 divide the workpiece 3 into image sizes that can be input to the learning model 110 and capture the images.
　学習モデル１１０に入力可能な画像のサイズが、カメラ２１によって撮像されるワーク画像４のサイズよりも小さい場合、次の対策１又は対策２が考えらえる。 If the size of the image that can be input to the learning model 110 is smaller than the size of the work image 4 captured by the camera 21, the following countermeasure 1 or countermeasure 2 can be considered.
＜＜対策１＞＞
　図１７に示すように、検査装置が、ワーク画像４のサイズを、学習モデル１１０に入力可能な画像のサイズに縮小して縮小ワーク画像３９１を生成し、当該縮小ワーク画像３９１を学習モデル１１０に入力して評価値を算出する対策が考えられる。しかし、この場合、ワーク画像４のサイズの縮小によって部品２の解像度が不足し、学習モデル１１０が、小さな部品２の評価値を精度良く算出できないという問題が生じる。 <<Countermeasure 1>>
As shown in FIG. 17 , the inspection apparatus reduces the size of the workpiece image 4 to the size of an image that can be input to the learning model 110 to generate a reduced workpiece image 391 . A possible countermeasure is to input and calculate the evaluation value. However, in this case, the resolution of the part 2 is insufficient due to the reduction in the size of the workpiece image 4, and the problem arises that the learning model 110 cannot accurately calculate the evaluation value of the small part 2. FIG.
＜＜対策２＞＞
　図１８に示すように、複数のカメラ２１でワーク３を分割して撮像し、各カメラ２１が撮像した分割ワーク画像３９２をそれぞれ学習モデル１１０に入力する対策が考えられる。しかし、この場合、検査装置２０に複数のカメラ２１を搭載する必要があり、検査システム１０の構成が煩雑になる。 <<Countermeasure 2>>
As shown in FIG. 18, a countermeasure may be considered in which a plurality of cameras 21 divide and capture images of the workpiece 3, and the divided workpiece images 392 captured by each camera 21 are input to the learning model 110, respectively. However, in this case, it is necessary to mount a plurality of cameras 21 on the inspection apparatus 20, and the configuration of the inspection system 10 becomes complicated.
　あるいは、１台のカメラ２１でワーク３全体を撮像し、その撮像したワーク画像４を複数の分割ワーク画像３９２に分割し、分割ワーク画像３９２をそれぞれ学習モデル１１０に入力する対策が考えられる。しかし、この場合、検査装置は、複数の分割ワーク画像３９２を処理する必要があるため、処理量及びメモリ使用量が大きくなり、検査装置２０の処理時間及びコストの増大を招く。 Alternatively, it is conceivable to take an image of the entire work 3 with one camera 21, divide the imaged work image 4 into a plurality of divided work images 392, and input each of the divided work images 392 to the learning model 110. However, in this case, since the inspection apparatus needs to process a plurality of divided work images 392, the amount of processing and the amount of memory used are increased, and the processing time and cost of the inspection apparatus 20 are increased.
　このような問題に鑑みて、本実施の形態は、部品装着に係る異常検出の精度を保ちつつ、処理時間及びコストの増大を抑制する検査システム１０について説明する。なお、実施の形態２に係る検査システム１０のハードウェア構成については、図１と同様であるため、説明を省略する。 In view of such problems, the present embodiment describes an inspection system 10 that suppresses an increase in processing time and cost while maintaining the accuracy of abnormality detection related to component mounting. Note that the hardware configuration of the inspection system 10 according to the second embodiment is the same as that of FIG. 1, and thus description thereof is omitted.
＜検査装置の機能構成＞
　図１９は、実施の形態２に係る検査装置２０の機能構成の一例を示すブロック図である。 <Functional configuration of inspection device>
FIG. 19 is a block diagram showing an example of the functional configuration of the inspection device 20 according to the second embodiment.
　検査装置２０は、撮像制御部１０１と、最適条件決定部１０２と、検査実行部１０３と、注目領域抽出部１１１と、注目領域格納部１０４と、撮像条件格納部１０５と、最適撮像条件格納部１０６と、学習モデル格納部１０７と、合成画像格納部１１２とを有する。 The inspection apparatus 20 includes an imaging control unit 101, an optimum condition determination unit 102, an inspection execution unit 103, an attention area extraction unit 111, an attention area storage unit 104, an imaging condition storage unit 105, and an optimum imaging condition storage unit. 106 , a learning model storage unit 107 , and a synthetic image storage unit 112 .
　撮像制御部１０１は、カメラ２１及び照明装置３０を制御してワーク３を撮像し、ワーク画像４を生成する。 The imaging control unit 101 controls the camera 21 and the lighting device 30 to image the workpiece 3 and generate the workpiece image 4 .
　注目領域抽出部１１１は、ワーク画像４から各注目領域５の画像（注目領域画像）を抽出する。注目領域抽出部１１１は、抽出した各注目領域画像を合成し、合成画像４００（図２０、図２１参照）を生成する。注目領域抽出部１１１は、生成した合成画像４００を合成画像格納部１１２に格納する。なお、注目領域抽出部１１１及び合成画像４００の詳細については後述する（図２０及び図２１参照）。 The region-of-interest extraction unit 111 extracts an image (region-of-interest image) of each region of interest 5 from the work image 4 . The attention area extraction unit 111 combines the extracted attention area images to generate a composite image 400 (see FIGS. 20 and 21). The attention area extraction unit 111 stores the generated synthetic image 400 in the synthetic image storage unit 112 . Details of the attention area extraction unit 111 and the synthesized image 400 will be described later (see FIGS. 20 and 21).
　最適条件決定部１０２は、複数の撮像条件のうち、各注目領域５の撮像に最適な最適撮像条件を決定する。加えて、最適条件決定部１０２は、ワーク画像４から注目領域５に対応する注目領域画像を抽出する際に最適な抽出条件（以下、最適抽出条件と称する）を決定する。なお、抽出条件及び最適抽出条件については後述する。 The optimum condition determination unit 102 determines the optimum imaging conditions for imaging each attention area 5 among a plurality of imaging conditions. In addition, the optimum condition determination unit 102 determines optimum extraction conditions (hereinafter referred to as optimum extraction conditions) when extracting an attention area image corresponding to the attention area 5 from the work image 4 . The extraction conditions and optimum extraction conditions will be described later.
　検査実行部１０３は、合成画像４００を用いて、注目領域５に対応する領域において部品２が正常に装着されているか否かを検査する。検査実行部１０３は、学習モデル格納部１０７に格納されている学習モデル１１０に対して、合成画像４００の注目領域５に対応する領域の画像を入力することにより、当該検査を実行する。 The inspection execution unit 103 uses the synthesized image 400 to inspect whether or not the component 2 is normally mounted in the area corresponding to the attention area 5 . The inspection execution unit 103 inputs the image of the area corresponding to the attention area 5 of the synthesized image 400 to the learning model 110 stored in the learning model storage unit 107, thereby executing the inspection.
＜合成画像の詳細＞
　次に、合成画像４００の第１の生成方法、及び、第２の生成方法について説明する。検査装置２０は、第１の生成方法、及び、第２の生成方法のいずれで合成画像４００を生成してもよい。 <Details of composite image>
Next, a first generation method and a second generation method of the synthetic image 400 will be described. The inspection device 20 may generate the composite image 400 using either the first generation method or the second generation method.
＜＜合成画像の第１の生成方法＞＞
　図２０は、実施の形態２に係る合成画像４００の第１の生成方法の一例を示す図である。 <<First Synthetic Image Generation Method>>
FIG. 20 is a diagram showing an example of a first method for generating the synthesized image 400 according to the second embodiment.
　注目領域抽出部１１１は、ワーク画像４から抽出した各注目領域５（例えば、注目領域５ａ、５ｂ）に対応する注目領域画像（画素）を、当該注目領域５（例えば注目領域５ａ、５ｂ）に含まれる部品２（例えば部品２ａ、２ｂ）に応じて、最適なサイズに拡大又は縮小する。例えば、注目領域抽出部１１１は、拡大した方が学習モデル１１０での異常検出精度が高くなる部品２ａの注目領域画像を拡大する。例えば、注目領域抽出部１１１は、縮小しても学習モデル１１０での異常検出精度があまり変わらない部品２ｂの注目領域画像を縮小する。 The attention area extracting unit 111 extracts attention area images (pixels) corresponding to each attention area 5 (for example, attention areas 5a and 5b) extracted from the work image 4, to the attention area 5 (for example, attention areas 5a and 5b). Depending on the part 2 involved (eg part 2a, 2b), it is scaled up or down to an optimal size. For example, the attention area extracting unit 111 enlarges the attention area image of the part 2a for which the abnormality detection accuracy in the learning model 110 is improved by enlarging it. For example, the attention area extracting unit 111 reduces the attention area image of the part 2b for which the abnormality detection accuracy in the learning model 110 does not change much even if it is reduced.
　注目領域抽出部１１１は、拡大又は縮小に加えて、余剰領域も調整してよい。余剰領域とは、注目領域画像における部品２の周辺画素の領域を示す。例えば、注目領域抽出部１１１は、余剰領域（つまり周辺画素）を大きくとった方が学習モデル１１０の評価値の算出精度が高くなる部品２については余剰領域を大きくとって注目領域画像を抽出する。例えば、注目領域抽出部１１１は、余剰領域（つまり周辺画素）を小さくとった方が学習モデル１１０の評価値の算出精度が高くなる部品２ａについては余剰領域を小さくとって注目領域画像を抽出する。 The region-of-interest extraction unit 111 may adjust the surplus region in addition to the enlargement or reduction. The surplus area indicates an area of peripheral pixels of the part 2 in the attention area image. For example, the region-of-interest extraction unit 111 extracts a region-of-interest image by setting a large surplus region for component 2, for which a larger surplus region (that is, peripheral pixels) increases the accuracy of calculation of the evaluation value of the learning model 110. . For example, the region-of-interest extraction unit 111 extracts a region-of-interest image with a small surplus region for the part 2a for which the calculation accuracy of the evaluation value of the learning model 110 increases when the surplus region (that is, the peripheral pixels) is small. .
　注目領域抽出部１１１は、１つのワーク画像４について、このように拡大又は縮小、並びに、余剰領域を調整した各注目領域５に対応する注目領域画像を合成して、図２０に示すような１つの合成画像４００を生成する。注目領域抽出部１１１は、生成した合成画像４００を、合成画像格納部１１２に格納する。 The region-of-interest extraction unit 111 synthesizes the region-of-interest image corresponding to each region-of-interest 5 obtained by enlarging or reducing and adjusting the surplus regions in one workpiece image 4 as described above, and obtains a single region-of-interest image as shown in FIG. generate two composite images 400; The attention area extraction unit 111 stores the generated synthetic image 400 in the synthetic image storage unit 112 .
＜＜合成画像の第２の生成方法＞＞
　図２１は、実施の形態２に係る合成画像４００の第２の生成方法の一例を示す図である。 <<Second Generation Method of Synthetic Image>>
FIG. 21 is a diagram showing an example of a second method for generating the synthesized image 400 according to the second embodiment.
　注目領域抽出部１１１は、ワーク画像４から抽出した各注目領域５（例えば、注目領域５ａ、５ｂ）に対応する注目領域画像（画素）について、所定のサイズとなるように、拡大又は縮小する。例えば、注目領域抽出部１１１は、所定のサイズよりも小さな第１の注目領域５ａに対応する第１の注目領域画像については所定のサイズとなるように拡大する。例えば、注目領域抽出部１１１は、所定のサイズよりも大きな第２の注目領域５ｂに対応する注目領域画像については所定のサイズとなるように縮小する。例えば、注目領域抽出部１１１は、第２の注目領域５ｂに対応する画素が、第１の注目領域５ａに対応する画素と同じになるように、第２の注目領域５の周辺画素をワーク画像４から抽出する。 The attention area extraction unit 111 enlarges or reduces attention area images (pixels) corresponding to each attention area 5 (for example, attention areas 5a and 5b) extracted from the work image 4 so as to have a predetermined size. For example, the attention area extraction unit 111 enlarges the first attention area image corresponding to the first attention area 5a smaller than a predetermined size so as to have a predetermined size. For example, the attention area extraction unit 111 reduces the attention area image corresponding to the second attention area 5b larger than a predetermined size to a predetermined size. For example, the region-of-interest extraction unit 111 extracts peripheral pixels of the second region-of-interest 5 from the work image so that the pixels corresponding to the second region-of-interest 5b are the same as the pixels corresponding to the first region-of-interest 5a. Extract from 4.
　注目領域抽出部１１１は、拡大又は縮小に加えて、余剰領域も調整してよい。例えば、注目領域抽出部１１１は、余剰領域（つまり部品２の周辺画素）を大きくとった方が学習モデル１１０の評価値の算出精度が高くなる部品２ａについては余剰領域を大きくとって注目領域画像を抽出する。例えば、注目領域抽出部１１１は、余剰領域（つまり部品２の周辺画素）を小さくとった方が学習モデル１１０の評価値の算出精度が高くなる部品２ｂについては余剰領域を小さくとって注目領域画像を抽出する。 The region-of-interest extraction unit 111 may adjust the surplus region in addition to the enlargement or reduction. For example, the attention area extracting unit 111 selects a large surplus area for the part 2a, for which the calculation accuracy of the evaluation value of the learning model 110 increases when the surplus area (that is, the surrounding pixels of the part 2) is large, and extracts the attention area image. to extract For example, the attention area extracting unit 111 selects a small surplus area for the part 2b for which the calculation accuracy of the evaluation value of the learning model 110 increases when the surplus area (that is, the surrounding pixels of the part 2) is small, and extracts the attention area image. to extract
　そして、注目領域抽出部１１１は、１つのワーク画像４について、このように拡大又は縮小、並びに、余剰領域を調整して同じサイズにした各注目領域画像を合成して、図２１に示すような１つの合成画像４００を生成する。注目領域抽出部１１１は、生成した合成画像４００を、合成画像格納部１１２に格納する。 Then, the attention area extracting unit 111 synthesizes each attention area image of the same size by enlarging or reducing the one work image 4 and adjusting the surplus area in this way, and produces a result as shown in FIG. 21 . A single composite image 400 is generated. The attention area extraction unit 111 stores the generated synthetic image 400 in the synthetic image storage unit 112 .
　このように合成画像４００を生成することにより、上述した対策２と比較して、本実施の形態は、画像のデータ量が小さくなるため、処理量及びメモリ使用量を低減できる。また、上述した対策１と比較して、小さな部品２の解像度が不足しなくなるので、本実施の形態に係る検査装置２０は、高い精度で異常を検出できる。 By generating the composite image 400 in this way, the data amount of the image is smaller in this embodiment than in the above-described countermeasure 2, so the amount of processing and memory usage can be reduced. In addition, as compared with measure 1 described above, the resolution of the small component 2 is no longer insufficient, so the inspection apparatus 20 according to the present embodiment can detect abnormalities with high accuracy.
＜検査前処理フロー＞
　図２２は、実施の形態２に係る検査前処理の一例を示すフローチャートである。 <Pre-test processing flow>
22 is a flowchart illustrating an example of pre-examination processing according to Embodiment 2. FIG.
　撮像制御部１０１は、カメラ２１を制御してワーク３を撮像し、ワーク画像４を生成する（ステップＳ４０１）。 The imaging control unit 101 controls the camera 21 to image the workpiece 3 and generate the workpiece image 4 (step S401).
　注目領域設定部２０１は、ワーク３に複数の注目領域５を設定する（ステップＳ４０２）。注目領域設定部２０１は、設定した複数の注目領域５を示す情報（注目領域情報）を検査装置２０に送信する。検査装置２０は、受信した複数の注目領域情報を注目領域格納部１０４に格納する。 The attention area setting unit 201 sets a plurality of attention areas 5 on the workpiece 3 (step S402). The attention area setting unit 201 transmits information (attention area information) indicating a plurality of set attention areas 5 to the inspection device 20 . The inspection apparatus 20 stores the received plural pieces of attention area information in the attention area storage unit 104 .
　撮像条件生成部２０４は、撮像パターン格納部２０８に格納されている複数の撮像パターンと、照明パターン格納部２０９に格納されている複数の照明パターンとを組み合わせて、複数の撮像条件を生成する（ステップＳ４０３）。撮像条件生成部２０４は、生成した複数の撮像条件を検査装置２０に送信する。検査装置２０は、受信した複数の撮像条件を撮像条件格納部１０５に格納する。 The imaging condition generation unit 204 generates a plurality of imaging conditions by combining a plurality of imaging patterns stored in the imaging pattern storage unit 208 and a plurality of illumination patterns stored in the illumination pattern storage unit 209 ( step S403). The imaging condition generation unit 204 transmits the plurality of generated imaging conditions to the inspection device 20 . The inspection apparatus 20 stores the received multiple imaging conditions in the imaging condition storage unit 105 .
　最適条件決定部１０２は、撮像条件格納部１０５に格納されている複数の撮像条件のうち、未選択の撮像条件を１つ選択する（ステップＳ４０４）。図２２の説明において、当該選択された撮像条件を、選択撮像条件と称する。 The optimum condition determination unit 102 selects one unselected imaging condition from the plurality of imaging conditions stored in the imaging condition storage unit 105 (step S404). In the description of FIG. 22, the selected imaging conditions are referred to as selected imaging conditions.
　撮像制御部１０１は、選択撮像条件に基づいてカメラ２１及び照明装置３０を制御してワーク３を撮像し、ワーク画像４を生成する（ステップＳ４０５）。撮像制御部１０１は、生成したワーク画像４をＲＡＭ４３又はストレージ４４に格納する。 The imaging control unit 101 controls the camera 21 and the lighting device 30 based on the selected imaging conditions to capture an image of the workpiece 3 and generate a workpiece image 4 (step S405). The imaging control unit 101 stores the generated work image 4 in the RAM 43 or storage 44 .
　最適条件決定部１０２は、撮像条件格納部１０５に格納されているすべての撮像条件を選択したか否かを判定する（ステップＳ４０６）。 The optimum condition determination unit 102 determines whether or not all imaging conditions stored in the imaging condition storage unit 105 have been selected (step S406).
　未選択の撮像条件が残っている場合（Ｓ４０６：ＮＯ）、検査装置２０は、処理をステップＳ４０４に戻す。すべての撮像条件を選択した場合（Ｓ４０６：ＹＥＳ）、検査装置２０は、処理をステップＳ４０７に進める。 When unselected imaging conditions remain (S406: NO), the inspection apparatus 20 returns the process to step S404. If all imaging conditions have been selected (S406: YES), the inspection apparatus 20 advances the process to step S407.
　注目領域抽出部１１１は、上述した合成画像の第１の生成方法（図２０参照）又は合成画像の第２の生成方法（図２１参照）によって、ＲＡＭ４３又はストレージ４４に格納されている各ワーク画像４から、様々な抽出条件で各注目領域５を抽出する。抽出条件には、拡大率又は縮小率、並びに、余剰領域の割合等が含まれている。例えば、注目領域抽出部１１１は、ワーク画像４内の各注目領域５に対応する領域の画像を抽出し、抽出条件に基づいて拡大又は縮小する。また、注目領域抽出部１１１は、ワーク画像４内の各注目領域５に対応する領域から、抽出条件に基づいて余剰領域を増加又は減少させて画像を抽出する。注目領域抽出部１１１は、抽出した各注目領域５に対応する注目領域画像を合成して合成画像４００を生成する（ステップＳ４０７）。注目領域抽出部１１１は、生成した複数の合成画像４００を、合成画像格納部１１２に格納する。これにより、合成画像格納部１１２には、様々な撮像条件にて撮像された各ワーク画像４から、様々な抽出条件にて抽出された注目領域画像を合成した複数の合成画像４００が格納される。 The region-of-interest extraction unit 111 extracts each workpiece image stored in the RAM 43 or the storage 44 by the above-described first synthetic image generation method (see FIG. 20) or the second synthetic image generation method (see FIG. 21). 4, each region of interest 5 is extracted under various extraction conditions. The extraction conditions include an enlargement rate or a reduction rate, a surplus area ratio, and the like. For example, the attention area extraction unit 111 extracts an image of an area corresponding to each attention area 5 in the work image 4, and enlarges or reduces it based on the extraction conditions. Further, the attention area extracting unit 111 extracts an image by increasing or decreasing the surplus area based on the extraction condition from the area corresponding to each attention area 5 in the work image 4 . The attention area extracting unit 111 combines the attention area images corresponding to the extracted attention areas 5 to generate the composite image 400 (step S407). The region-of-interest extraction unit 111 stores the plurality of generated synthetic images 400 in the synthetic image storage unit 112 . As a result, the composite image storage unit 112 stores a plurality of composite images 400 obtained by combining attention area images extracted under various extraction conditions from the work images 4 captured under various imaging conditions. .
　検査装置２０は、最適条件決定処理を実行する（ステップＳ４０８）。なお、最適条件決定処理の詳細については後述する（図２３参照）。そして、本処理は終了する。 The inspection device 20 executes optimum condition determination processing (step S408). Details of the optimum condition determination process will be described later (see FIG. 23). Then, the process ends.
　図２３は、図２２に示す最適条件決定処理（ステップＳ４０８）の一例を示すフローチャートである。 FIG. 23 is a flow chart showing an example of the optimum condition determination process (step S408) shown in FIG.
　最適条件決定部１０２は、注目領域格納部１０４に格納されている複数の注目領域５のうち、未選択の注目領域５を１つ選択する（ステップＳ５０１）。図２３の説明において、当該選択された注目領域５を、選択注目領域５と称する。 The optimum condition determination unit 102 selects one unselected attention area 5 from among the plurality of attention areas 5 stored in the attention area storage unit 104 (step S501). In the description of FIG. 23 , the selected attention area 5 is called a selected attention area 5 .
　最適条件決定部１０２は、合成画像格納部１１２に格納された、異なる撮像条件で撮像され、異なる抽出条件で抽出された複数の合成画像４００のそれぞれから、選択注目領域５における画像（以下、選択注目領域画像と称する）を取得する。最適条件決定部１０２は、学習モデル格納部１０７に格納されている学習モデル１１０を用いて、取得した複数の選択注目領域画像のそれぞれについて評価値を算出する（ステップＳ５０２）。 The optimum condition determination unit 102 selects an image in the selected region of interest 5 (hereinafter referred to as a selection (referred to as a region-of-interest image). The optimum condition determination unit 102 uses the learning model 110 stored in the learning model storage unit 107 to calculate an evaluation value for each of the acquired selected region-of-interest images (step S502).
　最適条件決定部１０２は、ステップＳ５０２で算出した評価値に基づいて、選択注目領域５の最適撮像条件及び最適抽出条件を決定する（ステップＳ５０３）。例えば、最適条件決定部１０２は、評価値が最も高く算出された撮像条件を、選択注目領域５の最適撮像条件に決定する。また、最適条件決定部１０２は、評価値が最も高く算出された抽出条件を、選択注目領域５の最適抽出条件に決定する。 The optimum condition determination unit 102 determines the optimum imaging conditions and optimum extraction conditions for the selected region of interest 5 based on the evaluation values calculated in step S502 (step S503). For example, the optimum condition determining unit 102 determines the imaging condition with the highest calculated evaluation value as the optimum imaging condition for the selected region of interest 5 . The optimum condition determination unit 102 also determines the extraction condition with the highest calculated evaluation value as the optimum extraction condition for the selected region of interest 5 .
　最適条件決定部１０２は、選択注目領域５に、ステップＳ５０３で決定した最適撮像条件及び最適抽出条件を対応付けて、注目領域格納部１０４に格納する（ステップＳ５０４）。 The optimum condition determination unit 102 associates the selected attention area 5 with the optimum imaging conditions and optimum extraction conditions determined in step S503, and stores them in the attention area storage unit 104 (step S504).
　最適条件決定部１０２は、ステップＳ５０３で決定した最適撮像条件を、最適撮像条件格納部１０６に格納する（ステップＳ５０５）。 The optimal condition determination unit 102 stores the optimal imaging conditions determined in step S503 in the optimal imaging condition storage unit 106 (step S505).
　最適条件決定部１０２は、注目領域格納部１０４に格納されているすべての注目領域５を選択したか否かを判定する（ステップＳ５０６）。 The optimum condition determination unit 102 determines whether or not all the attention areas 5 stored in the attention area storage unit 104 have been selected (step S506).
　未選択の注目領域５が残っている場合（ステップＳ５０６：ＮＯ）、検査装置２０は、処理をステップＳ５０１に戻す。すべての注目領域５を選択した場合（ステップＳ５０６：ＹＥＳ）、検査装置２０は、本処理を終了する。 If an unselected region of interest 5 remains (step S506: NO), the inspection device 20 returns the process to step S501. If all the attention areas 5 have been selected (step S506: YES), the inspection device 20 terminates this process.
＜検査処理フロー＞
　図２４は、実施の形態２に係る検査処理の一例を示すフローチャートである。 <Inspection processing flow>
24 is a flowchart illustrating an example of inspection processing according to Embodiment 2. FIG.
　撮像制御部１０１は、検査対象のワーク３に対応付けられている注目領域格納部１０４及び最適撮像条件格納部１０６を選択する（ステップＳ６０１）。 The imaging control unit 101 selects the attention area storage unit 104 and the optimum imaging condition storage unit 106 associated with the workpiece 3 to be inspected (step S601).
　撮像制御部１０１は、検出センサ１９がワーク３を検出したか否かを判定する（ステップＳ６０２）。例えば、撮像制御部１０１は、検出センサ１９からワーク検出通知を受信したか否かを判定する。 The imaging control unit 101 determines whether or not the detection sensor 19 has detected the workpiece 3 (step S602). For example, the imaging control unit 101 determines whether or not a workpiece detection notification has been received from the detection sensor 19 .
　検出センサ１９がワーク３を未検出である場合（ステップＳ６０２：ＮＯ）、検査装置２０は、処理をステップＳ６０２に戻す。検出センサ１９がワーク３を検出した場合（ステップＳ６０２：ＹＥＳ）、検査装置２０は、処理を次のステップＳ６０３に進める。 When the detection sensor 19 has not detected the workpiece 3 (step S602: NO), the inspection device 20 returns the process to step S602. If the detection sensor 19 detects the workpiece 3 (step S602: YES), the inspection apparatus 20 advances the process to the next step S603.
　撮像制御部１０１は、最適撮像条件格納部１０６から、未選択の最適撮像条件を１つ選択する（ステップＳ６０３）。図２４の説明において、当該選択された最適撮像条件を、選択最適撮像条件と称する。 The imaging control unit 101 selects one unselected optimum imaging condition from the optimum imaging condition storage unit 106 (step S603). In the description of FIG. 24, the selected optimum imaging conditions are referred to as selected optimum imaging conditions.
　撮像制御部１０１は、選択最適撮像条件に基づいてカメラ２１及び照明装置３０を制御してワーク３を撮像し、ワーク画像４を生成する（ステップＳ６０４）。 The imaging control unit 101 controls the camera 21 and the illumination device 30 based on the selected optimum imaging conditions to image the workpiece 3 and generate the workpiece image 4 (step S604).
　注目領域抽出部１１１は、上述した合成画像の第１の生成方法（図２０参照）又は合成画像の第２の生成方法（図２１参照）により、ワーク画像４から合成画像４００を生成する（ステップＳ６０５）。具体的には、注目領域抽出部１１１は、ワーク画像４の各注目領域５を当該注目領域５に対応付けられている最適抽出条件で抽出し、抽出した注目領域画像を合成して合成画像４００を生成する。注目領域抽出部１１１は、生成した合成画像４００を、合成画像格納部１１２に格納する。これにより、ワーク画像４をそのまま格納する実施の形態１と比較して、メモリの使用量を削減できる。 The region-of-interest extraction unit 111 generates a composite image 400 from the work image 4 (step S605). Specifically, the attention area extracting unit 111 extracts each attention area 5 of the workpiece image 4 under the optimum extraction condition associated with the attention area 5, and synthesizes the extracted attention area images to form a synthesized image 400. to generate The attention area extraction unit 111 stores the generated synthetic image 400 in the synthetic image storage unit 112 . As a result, the amount of memory used can be reduced compared to the first embodiment in which the work image 4 is stored as it is.
　撮像制御部１０１は、最適撮像条件格納部１０６に含まれるすべての最適撮像条件を選択したか否かを判定する（ステップＳ６０６）。 The imaging control unit 101 determines whether or not all the optimum imaging conditions included in the optimum imaging condition storage unit 106 have been selected (step S606).
　未選択の最適撮像条件が残っている場合（ステップＳ６０６：ＮＯ）、検査装置２０は、処理をステップＳ６０３に戻す。すべての最適撮像条件を選択した場合（ステップＳ６０６：ＹＥＳ）、検査装置２０は、処理を次のステップＳ６０７に進める。 If unselected optimum imaging conditions remain (step S606: NO), the inspection device 20 returns the process to step S603. If all the optimum imaging conditions have been selected (step S606: YES), the inspection apparatus 20 advances the process to the next step S607.
　検査実行部１０３は、注目領域格納部１０４に格納されている複数の注目領域５のうち、未選択の注目領域５を１つ選択する（ステップＳ６０７）。図２４の説明において、当該選択された注目領域５を、選択注目領域５と称する。 The inspection execution unit 103 selects one unselected attention area 5 from among the plurality of attention areas 5 stored in the attention area storage unit 104 (step S607). In the description of FIG. 24 , the selected attention area 5 is called a selected attention area 5 .
　検査実行部１０３は、合成画像格納部１１２に格納されている複数の合成画像４００のうち、選択注目領域５に対応付けられている最適撮像条件及び最適抽出条件に基づいて生成された合成画像４００を選択する（ステップＳ６０８）。図２４の説明において、当該選択された合成画像４００を、選択合成画像４００と称する。 The examination executing unit 103 selects a synthesized image 400 generated based on the optimum imaging condition and the optimum extraction condition associated with the selected region of interest 5 among the plurality of synthesized images 400 stored in the synthesized image storage unit 112. is selected (step S608). In the description of FIG. 24 , the selected synthetic image 400 will be referred to as the selected synthetic image 400 .
　検査実行部１０３は、学習モデル格納部１０７に格納されている学習モデル１１０を用いて、選択合成画像４００における選択注目領域５に対応する領域の画像の評価値を算出する（ステップＳ６０９）。選択合成画像４００は、選択注目領域５を撮像するに最適な撮像条件で撮像されたワーク画像４から、選択注目領域５を抽出するに最適な撮像条件で抽出された注目領域画像を合成して生成されたものである。よって、当該選択合成画像４００から取得された選択注目領域画像から算出される評価値は、単一の撮像条件で撮像されたワーク画像から取得された選択注目領域画像から算出される評価値よりも、精度が高くなり得る。 The inspection execution unit 103 uses the learning model 110 stored in the learning model storage unit 107 to calculate the image evaluation value of the area corresponding to the selected attention area 5 in the selected combined image 400 (step S609). The selected synthesized image 400 is obtained by synthesizing an attention area image extracted under optimum imaging conditions for extracting the selected attention area 5 from the work image 4 imaged under the optimum imaging conditions for imaging the selected attention area 5. It is generated. Therefore, the evaluation value calculated from the selected region-of-interest image obtained from the selected composite image 400 is higher than the evaluation value calculated from the selected region-of-interest image obtained from the work image captured under a single imaging condition. , can be more accurate.
　検査実行部１０３は、ステップＳ６０９で算出した評価値に基づいて、選択注目領域５の検査結果を決定する（ステップＳ６１０）。例えば、検査実行部１０３は、評価値が所定の閾値Ｔｈ未満である場合、選択注目領域５において部品２が異常に装着されている（ＮＧ）と決定する。例えば、検査実行部１０３は、評価値が閾値Ｔｈ以上である場合、選択注目領域５において部品２が正常に装着されている（ＯＫ）と決定する。検査実行部１０３は、選択注目領域５に検査結果を対応付けてＲＡＭ３４又はストレージ４４に格納する。 The inspection execution unit 103 determines the inspection result of the selected region of interest 5 based on the evaluation value calculated in step S609 (step S610). For example, when the evaluation value is less than a predetermined threshold value Th, the inspection execution unit 103 determines that the component 2 is abnormally attached (NG) in the selected attention area 5 . For example, when the evaluation value is equal to or greater than the threshold Th, the inspection execution unit 103 determines that the component 2 is normally mounted (OK) in the selected attention area 5 . The inspection execution unit 103 associates the inspection results with the selected region of interest 5 and stores them in the RAM 34 or the storage 44 .
　検査実行部１０３は、注目領域格納部１０４に格納されているすべての注目領域５を選択したか否かを判定する（ステップＳ６１１）。未選択の注目領域５が残っている場合、検査装置２０は、処理をステップＳ６０７に戻す。 The inspection execution unit 103 determines whether or not all the attention areas 5 stored in the attention area storage unit 104 have been selected (step S611). If there remains an unselected region of interest 5, the inspection apparatus 20 returns the process to step S607.
　すべての注目領域５を選択した場合、検査実行部１０３は、ステップＳ６１０にてＲＡＭ４３又はストレージ４４に格納された、各注目領域５に対応付けられている検査結果をまとめてワーク検査結果情報を生成し、管理装置４０へ送信する（ステップＳ６１２）。そして、検査装置２０は、処理をステップＳ６０２に戻し、次に搬送されてくるワーク３の検査を行う。 When all the attention areas 5 are selected, the inspection execution unit 103 collects the inspection results associated with each attention area 5 and stored in the RAM 43 or the storage 44 in step S610 to generate workpiece inspection result information. and transmitted to the management device 40 (step S612). The inspection apparatus 20 then returns the process to step S602 and inspects the work 3 that is next transported.
　なお、管理装置４０のＵＩ制御部２０６は、検査装置２０からワーク検査結果情報を受信し、検査結果格納部２１２に格納する。また、ＵＩ制御部２０６は、ワーク検査結果情報の内容を表示装置６０に表示する。ユーザは、表示装置６０に表示されたワーク検査結果情報の内容を見て、ワーク３の各注目領域５において部品２が正常に装着されているか否かを確認できる。 The UI control unit 206 of the management device 40 receives work inspection result information from the inspection device 20 and stores it in the inspection result storage unit 212 . Also, the UI control unit 206 displays the content of the work inspection result information on the display device 60 . The user can see the contents of the workpiece inspection result information displayed on the display device 60 and can confirm whether or not the component 2 is normally mounted in each attention area 5 of the workpiece 3 .
＜検査処理フローの変形例＞
　次に、検査処理の変形例として、検査装置２０が、単一の撮像条件によって撮像されたワーク画像から合成画像４００を生成して検査を行う場合の処理について説明する。図２５は、実施の形態２に係る検査処理の変形例を示すフローチャートである。 <Modified example of inspection processing flow>
Next, as a modified example of the inspection process, a process in which the inspection apparatus 20 generates a composite image 400 from a workpiece image captured under a single imaging condition and performs inspection will be described. FIG. 25 is a flowchart showing a modification of inspection processing according to the second embodiment.
　撮像制御部１０１は、検査対象のワーク３に対応付けられている注目領域格納部１０４及び最適撮像条件格納部１０６を選択する（ステップＳ７０１）。 The imaging control unit 101 selects the attention area storage unit 104 and the optimum imaging condition storage unit 106 associated with the workpiece 3 to be inspected (step S701).
　撮像制御部１０１は、検出センサ１９がワーク３を検出したか否かを判定する（ステップＳ７０２）。例えば、撮像制御部１０１は、検出センサ１９からワーク検出通知を受信したか否かを判定する。 The imaging control unit 101 determines whether or not the detection sensor 19 has detected the workpiece 3 (step S702). For example, the imaging control unit 101 determines whether or not a workpiece detection notification has been received from the detection sensor 19 .
　検出センサ１９がワーク３を未検出である場合（ステップＳ７０２：ＮＯ）、検査装置２０は、処理をステップＳ７０２に戻す。検出センサ１９がワーク３を検出した場合（ステップＳ７０２：ＹＥＳ）、検査装置２０は、処理を次のステップＳ７０３に進める。 If the detection sensor 19 has not detected the workpiece 3 (step S702: NO), the inspection device 20 returns the process to step S702. If the detection sensor 19 detects the workpiece 3 (step S702: YES), the inspection apparatus 20 advances the process to the next step S703.
　撮像制御部１０１は、所定の撮像条件に基づいてカメラ２１及び照明装置３０を制御してワーク３を撮像し、ワーク画像４を生成する（ステップＳ７０４）。 The imaging control unit 101 controls the camera 21 and the lighting device 30 based on predetermined imaging conditions to capture an image of the workpiece 3 and generate a workpiece image 4 (step S704).
　注目領域抽出部１１１は、上述した合成画像の第１の生成方法（図２０参照）又は合成画像の第２の生成方法（図２１参照）により、ワーク画像４から合成画像４００を生成する（ステップＳ７０５）。具体的には、注目領域抽出部１１１は、ワーク画像４の各注目領域５を当該注目領域５に対応付けられている最適抽出条件で抽出し、抽出した注目領域画像を合成して合成画像４００を生成する。注目領域抽出部１１１は、生成した合成画像４００を、合成画像格納部１１２に格納する。これにより、ワーク画像４をそのまま格納する実施の形態１と比較して、メモリの使用量を削減できる。 The region-of-interest extraction unit 111 generates a composite image 400 from the work image 4 (step S705). Specifically, the attention area extracting unit 111 extracts each attention area 5 of the workpiece image 4 under the optimum extraction condition associated with the attention area 5, and synthesizes the extracted attention area images to form a composite image 400. to generate The attention area extraction unit 111 stores the generated synthetic image 400 in the synthetic image storage unit 112 . As a result, the amount of memory used can be reduced compared to the first embodiment in which the work image 4 is stored as it is.
　検査実行部１０３は、注目領域格納部１０４に格納されている複数の注目領域５のうち、未選択の注目領域５を１つ選択する（ステップＳ７０５）。図２５の説明において、当該選択された注目領域５を、選択注目領域５と称する。 The inspection execution unit 103 selects one unselected attention area 5 from among the plurality of attention areas 5 stored in the attention area storage unit 104 (step S705). In the description of FIG. 25 , the selected attention area 5 is called a selected attention area 5 .
　検査実行部１０３は、学習モデル格納部１０７に格納されている学習モデル１１０を用いて、ステップＳ７０５にて生成された合成画像４００における選択注目領域５に対応する領域の画像の評価値を算出する（ステップＳ７０６）。当該合成画像４００は、選択注目領域５を抽出するに最適な撮像条件で抽出された注目領域画像を合成して生成されたものである。よって、当該選択合成画像４００から取得された選択注目領域画像から算出される評価値は、単一の撮像条件で撮像されたワーク画像から取得された選択注目領域画像から算出される評価値よりも、精度が高くなり得る。 The test execution unit 103 uses the learning model 110 stored in the learning model storage unit 107 to calculate the evaluation value of the image of the region corresponding to the selected attention region 5 in the synthesized image 400 generated in step S705. (Step S706). The synthesized image 400 is generated by synthesizing attention area images extracted under optimum imaging conditions for extracting the selected attention area 5 . Therefore, the evaluation value calculated from the selected region-of-interest image obtained from the selected composite image 400 is higher than the evaluation value calculated from the selected region-of-interest image obtained from the work image captured under a single imaging condition. , can be more accurate.
　検査実行部１０３は、ステップＳ７０６で算出した評価値に基づいて、選択注目領域５の検査結果を決定する（ステップＳ７０７）。例えば、検査実行部１０３は、評価値が所定の閾値Ｔｈ未満である場合、選択注目領域５において部品２が異常に装着されている（ＮＧ）と決定する。例えば、検査実行部１０３は、評価値が閾値Ｔｈ以上である場合、選択注目領域５において部品２が正常に装着されている（ＯＫ）と決定する。検査実行部１０３は、選択注目領域５に検査結果を対応付けてＲＡＭ３４又はストレージ４４に格納する。 The inspection execution unit 103 determines the inspection result of the selected region of interest 5 based on the evaluation value calculated in step S706 (step S707). For example, when the evaluation value is less than a predetermined threshold value Th, the inspection execution unit 103 determines that the component 2 is abnormally attached (NG) in the selected attention area 5 . For example, when the evaluation value is equal to or greater than the threshold Th, the inspection execution unit 103 determines that the component 2 is normally mounted (OK) in the selected attention area 5 . The inspection execution unit 103 associates the inspection results with the selected region of interest 5 and stores them in the RAM 34 or the storage 44 .
　検査実行部１０３は、注目領域格納部１０４に格納されているすべての注目領域５を選択したか否かを判定する（ステップＳ７０８）。未選択の注目領域５が残っている場合、検査装置２０は、処理をステップＳ７０５に戻す。 The inspection execution unit 103 determines whether or not all the attention areas 5 stored in the attention area storage unit 104 have been selected (step S708). If an unselected region of interest 5 remains, the inspection apparatus 20 returns the process to step S705.
　すべての注目領域５を選択した場合、検査実行部１０３は、ステップＳ７０７にてＲＡＭ４３又はストレージ４４に格納された、各注目領域５に対応付けられている検査結果をまとめてワーク検査結果情報を生成し、管理装置４０へ送信する（ステップＳ７０９）。そして、検査装置２０は、処理をステップＳ７０２に戻し、次に搬送されてくるワーク３の検査を行う。 When all the attention areas 5 are selected, the inspection execution unit 103 collects the inspection results associated with each attention area 5 stored in the RAM 43 or the storage 44 in step S707 to generate workpiece inspection result information. and transmitted to the management device 40 (step S709). The inspection apparatus 20 then returns the process to step S702 and inspects the work 3 that is next transported.
＜実施の形態２のまとめ＞
　実施の形態２の内容は、以下の項目のように表現できる。 <Summary of Embodiment 2>
The contents of the second embodiment can be expressed as the following items.
＜項目１＞
　検査装置２０は、１つ以上のプロセッサ２２と、メモリ（例えばＲＡＭ２４）と、メモリに保存されているプログラムと、を備える。プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、第１検査対象物（例えば部品２ａ）と第１検査対象物とは異なる第２検査対象物（例えば部品２ｂ）とを含む複数の検査対象物を含む検査対象領域において、第１検査対象物の検査を行うための第１注目領域５（５ａ）と、第２検査対象物の検査を行うための第２注目領域５（５ｂ）とを設定する。
　プログラムは、検査対象領域を撮像して検査対象領域の撮像画像（例えばワーク画像４）を出力するカメラ２１に検査対象領域を撮像させる。
　プログラムは、撮像画像から、第１注目領域に対応する第１画像領域と、第２注目領域に対応する第２画像領域とを抽出する。
　プログラムは、複数の検査対象物の異常を検知するための学習モデル１１０と、第１画像領域とに基づいて第１検査対象物を検査する第１検査を実行する。
　プログラムは、学習モデルと、第２画像領域とに基づいて第２検査対象物を検査する第２検査を実行する。
　プログラムは、第１検査の結果と、第２検査の結果とを出力する。
　これにより、検査装置２０は、撮像画像から抽出した第１画像領域を用いて、第１検査対象物を検査して第１検査の結果を出力し、撮像画像から抽出した第２画像領域を用いて、第２検査対象物を検査した第２検査の結果を出力する。よって、検査装置２０は、適切に抽出した画像領域を用いて、各検査対象物を検査することができ、より精度の高い検査結果を得ることができる。加えて、検査装置２０は、撮像画像からそのまま第１検査対象物及び第２検査対象物を検査する場合と比較して、プロセッサ２２の処理負荷及びメモリ（例えばＲＡＭ２３）の使用量を軽減できる。 <Item 1>
Inspection device 20 includes one or more processors 22, memory (eg, RAM 24), and programs stored in the memory. The program causes the processor 22 to do the following.
The program performs a first inspection in an inspection area including a plurality of inspection objects including a first inspection object (for example, part 2a) and a second inspection object (for example, part 2b) different from the first inspection object. A first attention area 5 (5a) for inspecting an object and a second attention area 5 (5b) for inspecting a second inspection object are set.
The program causes the camera 21 that captures an image of the inspection target area and outputs a captured image of the inspection target area (for example, the workpiece image 4) to capture the inspection target area.
The program extracts a first image area corresponding to the first attention area and a second image area corresponding to the second attention area from the captured image.
The program executes a first inspection for inspecting a first inspection object based on a learning model 110 for detecting anomalies in a plurality of inspection objects and a first image region.
The program performs a second inspection that inspects a second inspection object based on the learning model and the second image region.
The program outputs the results of the first test and the results of the second test.
As a result, the inspection apparatus 20 inspects the first inspection object using the first image area extracted from the captured image, outputs the result of the first inspection, and uses the second image area extracted from the captured image. to output the result of the second inspection of the second inspection object. Therefore, the inspection apparatus 20 can inspect each inspection object using the appropriately extracted image area, and can obtain a more accurate inspection result. In addition, the inspection apparatus 20 can reduce the processing load of the processor 22 and the amount of memory (for example, RAM 23) usage compared to the case where the first inspection object and the second inspection object are inspected directly from the captured image.
＜項目２＞
　項目１に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、第１画像領域と第２画像領域とを含む第１合成画像４００を生成する。
　プログラムは、第１合成画像と学習モデルとを用いて第１検査と前記第２検査とを実行する。
　これにより、検査装置２０は、第１合成画像を用いて、第１検査及び第２検査を実行できる。よって、検査装置２０は、撮像画像からそのまま第１検査対象物及び第２検査対象物を検査する場合と比較して、プロセッサ２２の処理負荷及びメモリの使用量を軽減できる。 <Item 2>
In the inspection device 20 described in item 1, the program causes the processor 22 to execute the following.
The program generates a first composite image 400 that includes a first image region and a second image region.
The program uses the first composite image and the learning model to perform the first test and the second test.
Thereby, the inspection apparatus 20 can perform the first inspection and the second inspection using the first synthesized image. Therefore, the inspection apparatus 20 can reduce the processing load of the processor 22 and the amount of memory used compared to the case where the first inspection object and the second inspection object are inspected directly from the captured image.
＜項目３＞
　項目１又は２に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、撮像画像から第１注目領域に対応する画素を拡大又は縮小して第１画像領域を抽出する。
　プログラムは、撮像画像から第２注目領域に対応する画素を拡大又は縮小して第２画像領域を抽出する。
　これにより、検査装置２０は、学習モデルに入力可能なサイズの第１合成画像を生成できる。 <Item 3>
In the inspection device 20 according to item 1 or 2, the program causes the processor 22 to perform the following.
The program extracts the first image area by enlarging or reducing the pixels corresponding to the first attention area from the captured image.
The program enlarges or reduces pixels corresponding to the second attention area from the captured image to extract the second image area.
As a result, the inspection device 20 can generate the first synthetic image of a size that can be input to the learning model.
＜項目４＞
　項目３に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、第１注目領域に対応する画素として、第１注目領域と、第１注目領域の周辺画素とを撮像画像から抽出する。
　プログラムは、第２注目領域に対応する画素として、第２注目領域と、第２注目領域の周辺画素とを撮像画像から抽出する。
　このように、検査装置２０は、各注目領域の周辺画素も抽出することにより、学習モデルにおける検査精度を高めることができる。 <Item 4>
In the inspection device 20 described in item 3, the program causes the processor 22 to perform the following.
The program extracts the first region of interest and peripheral pixels of the first region of interest from the captured image as pixels corresponding to the first region of interest.
The program extracts the second region of interest and peripheral pixels of the second region of interest from the captured image as pixels corresponding to the second region of interest.
In this way, the inspection apparatus 20 can improve the inspection accuracy in the learning model by also extracting the peripheral pixels of each attention area.
＜項目５＞
　項目４に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、第２注目領域に対応する画素が第１注目領域に対応する画素と同じ大きさになるように第２注目領域の周辺画素を撮像画像から抽出する。
　これにより、検査装置２０は、撮像画像の各注目領域について同じサイズで抽出した画像を合成して、合成画像を生成できる。 <Item 5>
In the inspection device 20 described in item 4, the program causes the processor 22 to perform the following.
The program extracts peripheral pixels of the second region of interest from the captured image so that pixels corresponding to the second region of interest have the same size as pixels corresponding to the first region of interest.
As a result, the inspection apparatus 20 can combine images extracted in the same size for each attention area of the captured image to generate a combined image.
＜項目６＞
　項目１から５のいずれか１項に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、第１画像領域と、第２画像領域とに基づいた学習により、学習モデル１１０を生成する。
　これにより、検査装置２０は、撮像画像を用いて、検査対象物を検査するための学習モデル１１０を生成できる。 <Item 6>
In the inspection apparatus 20 according to any one of items 1 to 5, the program causes the processor 22 to execute the following.
The program generates a learning model 110 by learning based on the first image region and the second image region.
Thereby, the inspection apparatus 20 can generate the learning model 110 for inspecting the inspection object using the captured image.
＜項目７＞
　項目１から６のいずれか１項に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、第１注目領域に対応する撮像画像の第１領域に基づいて、複数の撮像パラメータを含む第１撮像パターンを含む第１撮像条件を定める。
　プログラムは、第２注目領域に対応する撮像画像の第２領域に基づいて、複数の撮像パラメータを含む第２撮像パターンを含む第２撮像条件を定める。
　これにより、各撮像条件は、撮像パラメータの異なる撮像パターンによって定められる。よって、検査装置２０は、撮像パラメータの異なる撮像条件で撮像した撮像画像を用いて、各検査対象物を検査することにより、より精度の高い検査結果を得ることができる。 <Item 7>
In the inspection device 20 according to any one of items 1 to 6, the program causes the processor 22 to execute the following.
The program defines first imaging conditions including a first imaging pattern including a plurality of imaging parameters based on a first region of the captured image corresponding to the first region of interest.
The program defines second imaging conditions including a second imaging pattern including a plurality of imaging parameters based on a second region of the captured image corresponding to the second region of interest.
Thereby, each imaging condition is determined by an imaging pattern having different imaging parameters. Therefore, the inspection apparatus 20 can obtain more accurate inspection results by inspecting each inspection object using captured images captured under imaging conditions with different imaging parameters.
＜項目８＞
　項目７に記載の検査装置２０は、検査対象領域を照射する照明装置３０を備える。
　検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、照明装置が検査対象領域を照射するための複数の照射パラメータを含む照射パターンを定める。
　これにより、検査装置２０は、照明装置３０に、照明パラメータの異なる照明パターンによって検査対象領域を照射させることができる。 <Item 8>
The inspection apparatus 20 according to item 7 includes an illumination device 30 that illuminates an inspection target area.
In the inspection device 20, the program causes the processor 22 to do the following.
The program defines an illumination pattern including a plurality of illumination parameters for illuminating an area to be inspected by the illuminator.
Thereby, the inspection device 20 can cause the illumination device 30 to illuminate the inspection target region with illumination patterns having different illumination parameters.
＜項目９＞
　項目８に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、検査対象物の撮像画像として、複数の照射条件ごとに第１撮像条件を適用して撮像した検査対象領域の撮像画像と、複数の照射条件ごとに第２撮像条件を適用して撮像した検査対象領域の撮像画像とをカメラ２１に出力させる。
　これにより、検査装置２０は、異なる照明パターンで撮像した撮像画像を用いて、各検査対象物を検査することができ、より精度の高い検査結果を得ることができる。 <Item 9>
In the inspection device 20 according to item 8, the program causes the processor 22 to perform the following.
The program captures an image of an inspection target area obtained by applying a first imaging condition for each of a plurality of irradiation conditions and an image by applying a second imaging condition for each of a plurality of irradiation conditions as an image of the inspection object. The camera 21 is caused to output the picked-up image of the inspection target area.
As a result, the inspection apparatus 20 can inspect each inspection object using captured images captured with different illumination patterns, and can obtain more accurate inspection results.
＜項目１０＞
　項目１から１０のいずれか１項に記載の検査装置２０において、プログラムは、次のことをプロセッサ２２に実行させる。
　プログラムは、複数の検査対象物の位置を定めた設計情報、又は検査対象物領域の撮像画像のいずれかに基づいて第１注目領域と第２注目領域とを設定する。
　これにより、検査装置２０は、検査対象領域において、複数の注目領域を設定できる。 <Item 10>
In the inspection apparatus 20 according to any one of items 1 to 10, the program causes the processor 22 to execute the following.
The program sets the first region of interest and the second region of interest based on either design information that defines the positions of a plurality of inspection objects or a captured image of the inspection object region.
Thereby, the inspection apparatus 20 can set a plurality of attention areas in the inspection target area.
＜項目１１＞
　項目１から１０のいずれか１項に記載の検査装置２０は、カメラ２１を備える。
　これにより、検査装置２０は、カメラ２１を制御して、検査対象領域の撮像画像を撮像できる。 <Item 11>
The inspection device 20 according to any one of items 1 to 10 includes a camera 21 .
Thereby, the inspection apparatus 20 can control the camera 21 to capture a captured image of the inspection target area.
＜項目１２＞
　装置（例えば検査装置２０）は、次の画像処理方法を実施する。
　装置は、第１検査対象物（例えば部品２）と第１検査対象物とは異なる第２検査対象物とを含む複数の検査対象物を含む検査対象領域において、第１検査対象物の検査を行うための第１注目領域と、第２検査対象物の検査を行うための第２注目領域とを設定する。
　装置は、検査対象領域を撮像して検査対象領域の撮像画像（例えばワーク画像４）を出力するカメラ２１に検査対象領域を撮像させる。
　装置は、撮像画像から、第１注目領域５に対応する第１画像領域と、第２注目領域５に対応する第２画像領域とを抽出する。
　装置は、複数の検査対象物の異常を検知するための学習モデル１１０と、第１画像領域とに基づいて第１検査対象物を検査する第１検査を実行する。
　装置は、学習モデルと、第２画像領域とに基づいて第２検査対象物を検査する第２検査を実行する。
　装置は、第１検査の結果と、前記第２検査の結果とを出力する。
　これにより、装置は、撮像画像から抽出した第１画像領域を用いて、第１検査対象物を検査して第１検査の結果を出力し、撮像画像から抽出した第２画像領域を用いて、第２検査対象物を検査した第２検査の結果を出力する。よって、装置は、撮像画像からそのまま第１検査対象物及び第２検査対象物を検査する場合と比較して、プロセッサ２２の処理負荷及びメモリ（例えばＲＡＭ２３）の使用量を軽減できる。 <Item 12>
The device (for example, inspection device 20) implements the following image processing method.
The apparatus inspects a first inspection object in an inspection object area including a plurality of inspection objects including a first inspection object (eg, part 2) and a second inspection object different from the first inspection object. A first attention area for performing inspection and a second attention area for performing inspection of a second inspection object are set.
The apparatus causes the camera 21 that captures an image of the inspection target area and outputs a captured image of the inspection target area (for example, the workpiece image 4) to capture the inspection target area.
The device extracts a first image region corresponding to the first region of interest 5 and a second image region corresponding to the second region of interest 5 from the captured image.
The apparatus performs a first inspection for inspecting a first inspection object based on a learning model 110 for detecting anomalies in a plurality of inspection objects and a first image region.
The apparatus performs a second inspection that inspects a second inspection object based on the learned model and the second image region.
The device outputs the result of the first test and the result of said second test.
As a result, the apparatus inspects the first inspection object using the first image area extracted from the captured image, outputs the result of the first inspection, and uses the second image area extracted from the captured image to A result of the second inspection of the second inspection object is output. Therefore, the apparatus can reduce the processing load of the processor 22 and the usage of the memory (for example, the RAM 23) as compared with the case of directly inspecting the first inspection object and the second inspection object from the captured image.
＜項目１３＞
　画像処理プログラムは、次のことをプロセッサ２２に実行させる。
　画像処理プログラムは、第１検査対象物（例えば部品２）と第１検査対象物とは異なる第２検査対象物とを含む複数の検査対象物を含む検査対象領域において、第１検査対象物の検査を行うための第１注目領域と、第２検査対象物の検査を行うための第２注目領域とを設定する。
　画像処理プログラムは、検査対象領域を撮像して前記検査対象領域の撮像画像（例えばワーク画像４）を出力するカメラ２１に検査対象領域を撮像させる。
　画像処理プログラムは、撮像画像から、第１注目領域に対応する第１画像領域と、第２注目領域に対応する第２画像領域とを抽出する。
　画像処理プログラムは、複数の検査対象物の異常を検知するための学習モデル１１０と、第１画像領域とに基づいて第１検査対象物を検査する第１検査を実行する。
　画像処理プログラムは、学習モデルと、第２画像領域とに基づいて第２検査対象物を検査する第２検査を実行する。
　画像処理プログラムは、第１検査の結果と、第２検査の結果とを出力する。
　これにより、画像処理プログラムは、撮像画像から抽出した第１画像領域を用いて、第１検査対象物を検査して第１検査の結果を出力し、撮像画像から抽出した第２画像領域を用いて、第２検査対象物を検査した第２検査の結果を出力する。よって、画像処理プログラムは、撮像画像からそのまま第１検査対象物及び第２検査対象物を検査する場合と比較して、プロセッサ２２の処理負荷及びメモリ（例えばＲＡＭ２３）の使用量を軽減できる。 <Item 13>
The image processing program causes the processor 22 to do the following.
The image processing program, in an inspection area including a plurality of inspection objects including a first inspection object (for example, part 2) and a second inspection object different from the first inspection object, A first attention area for inspection and a second attention area for inspecting a second inspection object are set.
The image processing program causes the camera 21 that captures an image of the inspection target area and outputs a captured image of the inspection target area (for example, the work image 4) to capture the inspection target area.
The image processing program extracts a first image area corresponding to the first attention area and a second image area corresponding to the second attention area from the captured image.
The image processing program executes a first inspection for inspecting a first inspection object based on a learning model 110 for detecting anomalies in a plurality of inspection objects and a first image region.
The image processing program performs a second inspection for inspecting a second inspection object based on the learning model and the second image region.
The image processing program outputs the result of the first inspection and the result of the second inspection.
Thereby, the image processing program inspects the first inspection object using the first image area extracted from the captured image, outputs the result of the first inspection, and uses the second image area extracted from the captured image. to output the result of the second inspection of the second inspection object. Therefore, the image processing program can reduce the processing load of the processor 22 and the usage of the memory (for example, the RAM 23) as compared with the case of directly inspecting the first inspection object and the second inspection object from the captured image.
　以上、説明した本発明の実施形態は、当業者であれば、本発明が上述した実施形態に限定されるものではないこと、上述した実施形態は例示にすぎないということ、及び、本発明の目的から逸脱しない範囲において様々な変形が可能であるということは理解される。 The above-described embodiments of the present invention will allow those skilled in the art to understand that the present invention is not limited to the above-described embodiments, that the above-described embodiments are merely exemplary, and that It is understood that various modifications are possible without departing from the intended purpose.
　例えば、本明細書にて開示される処理に含まれるステップは、必ずしもシーケンス図及び／又はフローチャートに記載された順序に従って時系列に実行せずともよい。例えば、処理に含まれるステップは、シーケンス図及び／又はフローチャートに記載した順序と異なる順序で実行すること、及び／又は、並列的に実行することができる。また、処理に含まれるステップの一部を削除すること、及び／又は、さらなるステップを処理に追加することも可能である。 For example, the steps included in the processing disclosed in this specification do not necessarily have to be executed in chronological order according to the order described in the sequence diagrams and/or flowcharts. For example, steps in a process may be performed out of order and/or in parallel from the order illustrated in the sequence diagrams and/or flowcharts. It is also possible to omit some of the steps included in the process and/or add additional steps to the process.
　また、本明細書において説明した検査装置２０の構成要素（例えば、撮像制御部１０１、最適条件決定部１０２、検査実行部１０３、及び／又は、注目領域抽出部１１１）を備える装置又はそのモジュール（例えば、撮像制御モジュール、最適条件決定モジュール、検査実行モジュール、及び／又は、注目領域抽出モジュール）が提供されてもよい。さらに、上述した構成要素の処理をプロセッサ、及び／又は、プロセッサに相当する処理要素
（例えば、ＡＳＩＣ：Ａｐｐｌｉｃａｔｉｏｎ　Ｓｐｅｃｉｆｉｃ　Ｉｎｔｅｇｒａｔｅｄ　Ｃｉｒｃｕｉｔ，ＦＰＧＡ：Ｆｉｅｌｄ　Ｐｒｏｇｒａｍｍａｂｌｅ　Ｇａｔｅ　Ａｒｒａｙ，ＣＰＬＤ：Ｃｏｍｐｌｅｘ　Ｐｒｏｇｒａｍｍａｂｌｅ　Ｌｏｇｉｃ　Ｄｅｖｉｃｅ等）に実行させるためのプログラムが提供されてもよい。また、当該プログラムを記録した記録媒体（コンピュータに読み取り可能な非一時的記録媒体）が提供されてもよい。このような装置、モジュール、方法、プログラム及び記録媒体も本発明に含まれる。 Also, an apparatus or a module thereof ( For example, an imaging control module, an optimum condition determination module, an inspection execution module, and/or a region of interest extraction module) may be provided. Furthermore, to cause a processor and/or a processing element equivalent to the processor (for example, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, CPLD: Complex Programmable Logic Device, etc.) to execute the processing of the above-described components program may be provided. A recording medium (computer-readable non-temporary recording medium) recording the program may also be provided. Such devices, modules, methods, programs and recording media are also included in the present invention.
　なお、本出願は、２０２２年２月２５日出願の日本特許出願（特願２０２２－０２８５５７）に基づくものであり、その内容は本出願の中に参照として援用される。 This application is based on a Japanese patent application (Japanese Patent Application No. 2022-028557) filed on February 25, 2022, the content of which is incorporated herein by reference.
　本開示の技術は、基板に各部品が正常に装着されているか否かを検査する装置等に有用である。 The technology of the present disclosure is useful for devices and the like that inspect whether or not each component is normally mounted on a board.
１　基板
２、２ａ、２ｂ　部品
３　ワーク
４　ワーク画像
５、５ａ、５ｂ　注目領域
１０　検査システム
１１　通信ネットワーク
１２、１３、１４、１５、１６、１７　ケーブル
１９　検出センサ
２０　検査装置
２１　カメラ
２２　プロセッサ
２３　ＲＯＭ
２４　ＲＡＭ
２５　ストレージ
２６　通信Ｉ／Ｆ
２７　入出力Ｉ／Ｆ
３０　照明装置
３１　ＬＥＤ光源
３２　入出力Ｉ／Ｆ
３３　調光制御回路
４０　管理装置
４１　プロセッサ
４２　ＲＯＭ
４３　ＲＡＭ
４４　ストレージ
４５　通信Ｉ／Ｆ
４６　入出力Ｉ／Ｆ
５０　入力装置
５２　スピーカ
６０　表示装置
６２　パトライト
１０１　撮像制御部
１０２　最適条件決定部
１０３　検査実行部
１０４　注目領域格納部
１０５　撮像条件格納部
１０６　最適撮像条件格納部１
１０７　学習モデル格納部
１１０　学習モデル
１１１　注目領域抽出部
１１２　合成画像格納部
２０１　注目領域設定部
２０２　撮像パターン生成部
２０３　照明パターン生成部
２０４　撮像条件生成部
２０５　学習モデル生成部
２０６　ＵＩ制御部
２０７　注目領域格納部
２０８　撮像パターン格納部
２０９　照明パターン格納部
２１０　撮像条件格納部
２１１　学習モデル格納部
２１２　検査結果格納部
３００　パターン用ＵＩ画面
３０１　撮像パターンリスト領域
３０２　照明パターンリスト領域
３０３　撮像条件別領域
３０４　ワーク画像確認領域
３２０　注目領域用ＵＩ画面
３２１　注目領域リスト領域
３２２　注目領域確認領域
３２３　評価値確認領域
３２４　設定ボタン
３４０　検査結果一覧用ＵＩ画面
３４１　ワーク領域
３４２　検査結果
３４３　詳細ボタン
３６０　検査結果詳細用ＵＩ画面
３６１　説明領域
３６２　ワーク領域
３６３　部品リスト領域
３９１　縮小ワーク画像
３９２　分割ワーク画像
４００　合成画像
  1 board 2, 2a, 2b part 3 work 4 work image 5, 5a, 5b attention area 10 inspection system 11 communication network 12, 13, 14, 15, 16, 17 cable 19 detection sensor 20 inspection device 21 camera 22 processor 23 ROM
24 RAMs
25 Storage 26 Communication I/F
27 input/output I/F
30 lighting device 31 LED light source 32 input/output I/F
33 dimming control circuit 40 management device 41 processor 42 ROM
43 RAM
44 Storage 45 Communication I/F
46 input/output I/F
50 input device 52 speaker 60 display device 62 patrol light 101 imaging control unit 102 optimum condition determination unit 103 inspection execution unit 104 attention area storage unit 105 imaging condition storage unit 106 optimum imaging condition storage unit 1
107 learning model storage unit 110 learning model 111 attention area extraction unit 112 synthetic image storage unit 201 attention area setting unit 202 imaging pattern generation unit 203 illumination pattern generation unit 204 imaging condition generation unit 205 learning model generation unit 206 UI control unit 207 attention area Storage unit 208 Imaging pattern storage unit 209 Illumination pattern storage unit 210 Imaging condition storage unit 211 Learning model storage unit 212 Inspection result storage unit 300 Pattern UI screen 301 Imaging pattern list area 302 Illumination pattern list area 303 Imaging condition-specific area 304 Work image Confirmation area 320 Attention area UI screen 321 Attention area list area 322 Attention area confirmation area 323 Evaluation value confirmation area 324 Setting button 340 Inspection result list UI screen 341 Work area 342 Inspection result 343 Details button 360 Inspection result details UI screen 361 Description area 362 Work area 363 Parts list area 391 Reduced work image 392 Divided work image 400 Composite image

Claims (9)

  1. 　１つ以上のプロセッサと、
    　メモリと、
    　前記メモリに保存されているプログラムと、を備え、
    　前記プログラムは、
    　第１検査対象物と前記第１検査対象物とは異なる第２検査対象物とを含む複数の検査対象物が存在する検査対象領域において、前記第１検査対象物の検査を行うための第１注目領域と、前記第２検査対象物の検査を行うための第２注目領域とを設定することと、
    　前記検査対象領域を撮像して前記検査対象領域の撮像画像を出力するカメラに、前記検査対象領域の撮像画像として、第１撮像条件により撮像した第１撮像画像と、前記第１撮像条件とは異なる第２撮像条件により撮像した第２撮像画像とを撮像させることと、
    　前記第１検査対象物と前記第２検査対象物とを含む複数の検査対象物の異常を検知するための学習モデルと、前記第１注目領域に対応する前記第１撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第１撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第１検査を実行することと、
    　前記学習モデルと前記第１注目領域に対応する前記第２撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第２撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第２検査を実行することと、
    　前記第１検査の結果と前記第２検査の結果とを出力することと、
    　を、前記１つ以上のプロセッサに実行させる、
    　検査装置。     one or more processors;
    memory;
    a program stored in the memory;
    Said program
    In an inspection area where a plurality of inspection objects including a first inspection object and a second inspection object different from the first inspection object exist, a first inspection object for inspecting the first inspection object setting a region of interest and a second region of interest for inspecting the second inspection object;
    A first captured image captured under a first imaging condition as a captured image of the inspection target area to a camera that captures the inspection target area and outputs the captured image of the inspection target area, and the first imaging condition Capturing a second captured image captured under a different second imaging condition;
    a learning model for detecting anomalies in a plurality of inspection objects including the first inspection object and the second inspection object; and a first region of the first captured image corresponding to the first region of interest. and inspecting the second inspection object based on the learning model and a second region of the first captured image corresponding to the second region of interest. performing an inspection;
    inspecting the first inspection object based on the learning model and a first area of the second captured image corresponding to the first attention area, and inspecting the first inspection object corresponding to the learning model and the second attention area; 2 performing a second inspection for inspecting the second inspection object based on a second area of the captured image;
    outputting the result of the first inspection and the result of the second inspection;
    causes the one or more processors to execute
    inspection equipment.
  2. 　前記プログラムは、
    　前記第１注目領域に対応する前記撮像画像の第１領域に基づいて、前記第１撮像条件として少なくとも１つの撮像パラメータを含む第１撮像パターンを定めることと、
    　前記第２注目領域に対応する前記撮像画像の第２領域に基づいて、前記第２撮像条件として少なくとも１つの撮像パラメータを含む第２撮像パターンを定めることと、
    　を、前記１つ以上のプロセッサに実行させる、
    　請求項１に記載の検査装置。     Said program
    Determining a first imaging pattern including at least one imaging parameter as the first imaging condition based on a first region of the captured image corresponding to the first region of interest;
    Determining a second imaging pattern including at least one imaging parameter as the second imaging condition based on a second region of the captured image corresponding to the second region of interest;
    causes the one or more processors to execute
    The inspection device according to claim 1.
  3. 　前記検査対象領域を照射する照明装置を備え、
    　前記プログラムは、
    　前記照明装置が前記検査対象領域を照射するための少なくとも１つの照射パラメータを含む照射パターンを定めること、
    　を、前記１つ以上のプロセッサに実行させる、
    　請求項２に記載の検査装置。     A lighting device that irradiates the inspection target area,
    Said program
    defining an illumination pattern including at least one illumination parameter for the illumination device to illuminate the inspection target area;
    causes the one or more processors to execute
    The inspection device according to claim 2.
  4. 　前記プログラムは、
    　前記第１撮像画像として、前記少なくとも１つの照射パターンごとに前記第１撮像条件を適用して撮像した前記検査対象領域の撮像画像を前記カメラに出力させることと、
    　前記第２撮像画像として、前記少なくとも１つの照射パターンごとに前記第２撮像条件を適用して撮像した前記検査対象領域の撮像画像を前記カメラに出力させることと、
    　を前記１つ以上のプロセッサに実行させる、
    　請求項３に記載の検査装置。     Said program
    causing the camera to output, as the first captured image, a captured image of the inspection target region captured by applying the first imaging condition for each of the at least one irradiation pattern;
    causing the camera to output, as the second captured image, a captured image of the inspection target region captured by applying the second imaging condition for each of the at least one irradiation pattern;
    causes the one or more processors to execute
    The inspection device according to claim 3.
  5. 　前記プログラムは、
    　前記カメラが出力した前記第１撮像画像と前記第２撮像画像とに基づいた学習により、
    前記学習モデルを生成することと、
    　を、前記１つ以上のプロセッサに実行させる、
    　請求項１から４のいずれか１項に記載の検査装置。     Said program
    By learning based on the first captured image and the second captured image output by the camera,
    generating the learning model;
    causes the one or more processors to execute
    The inspection device according to any one of claims 1 to 4.
  6. 　前記プログラムは、
    　前記複数の検査対象物の位置を定めた設計情報、又は前記検査対象領域の撮像画像のいずれかに基づいて前記第１注目領域と前記第２注目領域とを設定すること、
    　を前記１つ以上のプロセッサに実行させる、
    　請求項１から５のいずれか１項に記載の検査装置。     Said program
    setting the first region of interest and the second region of interest based on either design information defining the positions of the plurality of inspection objects or a captured image of the inspection target region;
    causes the one or more processors to execute
    The inspection device according to any one of claims 1 to 5.
  7. 　前記カメラを備える、
    　請求項１から６のいずれか１項に記載の検査装置。     comprising the camera;
    The inspection device according to any one of claims 1 to 6.
  8. 　検査装置において、
    　第１検査対象物と前記第１検査対象物とは異なる第２検査対象物とを含む複数の検査対象物が存在する検査対象領域において、前記第１検査対象物の検査を行うための第１注目領域と、前記第２検査対象物の検査を行うための第２注目領域とを設定し、
    　前記検査対象領域を撮像して前記検査対象領域の撮像画像を出力するカメラに、前記検査対象領域の撮像画像として、第１撮像条件により撮像した第１撮像画像と、前記第１撮像条件とは異なる第２撮像条件により撮像した第２撮像画像とを撮像させ、
    　前記第１検査対象物と前記第２検査対象物とを含む複数の検査対象物の異常を検知するための学習モデルと、前記第１注目領域に対応する前記第１撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第１撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第１検査を実行し、
    　前記学習モデルと前記第１注目領域に対応する前記第２撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第２撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第２検査を実行し、
    　前記第１検査の結果と前記第２検査の結果とを出力する、
    　検査方法。     In the inspection device,
    In an inspection area where a plurality of inspection objects including a first inspection object and a second inspection object different from the first inspection object exist, a first inspection object for inspecting the first inspection object setting a region of interest and a second region of interest for inspecting the second inspection object;
    A first captured image captured under a first imaging condition as a captured image of the inspection target area to a camera that captures the inspection target area and outputs the captured image of the inspection target area, and the first imaging condition Capturing a second captured image captured under a different second imaging condition,
    a learning model for detecting anomalies in a plurality of inspection objects including the first inspection object and the second inspection object; and a first region of the first captured image corresponding to the first region of interest. and inspecting the second inspection object based on the learning model and a second region of the first captured image corresponding to the second region of interest. run the inspection,
    inspecting the first inspection object based on the learning model and a first area of the second captured image corresponding to the first attention area, and inspecting the first inspection object corresponding to the learning model and the second attention area; 2 performing a second inspection for inspecting the second inspection object based on the second area of the captured image;
    outputting the results of the first inspection and the results of the second inspection;
    Inspection method.
  9. 　第１検査対象物と前記第１検査対象物とは異なる第２検査対象物とを含む複数の検査対象物が存在する検査対象領域において、前記第１検査対象物の検査を行うための第１注目領域と、前記第２検査対象物の検査を行うための第２注目領域とを設定することと、
    　前記検査対象領域を撮像して前記検査対象領域の撮像画像を出力するカメラに、前記検査対象領域の撮像画像として、第１撮像条件により撮像した第１撮像画像と、前記第１撮像条件とは異なる第２撮像条件により撮像した第２撮像画像とを撮像させることと、
    　前記第１検査対象物と前記第２検査対象物とを含む複数の検査対象物の異常を検知するための学習モデルと、前記第１注目領域に対応する前記第１撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第１撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第１検査を実行することと、
    　前記学習モデルと前記第１注目領域に対応する前記第２撮像画像の第１領域とに基づいて前記第１検査対象物を検査し、かつ前記学習モデルと前記第２注目領域に対応する前記第２撮像画像の第２領域とに基づいて前記第２検査対象物を検査する第２検査を実行することと、
    　前記第１検査の結果と前記第２検査の結果とを出力することと、
    　を、プロセッサに実行させる、
    　検査プログラム。
          In an inspection area where a plurality of inspection objects including a first inspection object and a second inspection object different from the first inspection object exist, a first inspection object for inspecting the first inspection object setting a region of interest and a second region of interest for inspecting the second inspection object;
    A first captured image captured under a first imaging condition as a captured image of the inspection target area to a camera that captures the inspection target area and outputs the captured image of the inspection target area, and the first imaging condition Capturing a second captured image captured under a different second imaging condition;
    a learning model for detecting anomalies in a plurality of inspection objects including the first inspection object and the second inspection object; and a first region of the first captured image corresponding to the first region of interest. and inspecting the second inspection object based on the learning model and a second region of the first captured image corresponding to the second region of interest. performing an inspection;
    inspecting the first inspection object based on the learning model and a first area of the second captured image corresponding to the first attention area, and inspecting the first inspection object corresponding to the learning model and the second attention area; 2 performing a second inspection for inspecting the second inspection object based on a second area of the captured image;
    outputting the result of the first inspection and the result of the second inspection;
    causes the processor to execute
    inspection program.
PCT/JP2023/006090 2022-02-25 2023-02-20 Inspection device, inspection method, and inspection program WO2023162940A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022028557 2022-02-25
JP2022-028557 2022-02-25

Publications (1)

Publication Number Publication Date
WO2023162940A1 true WO2023162940A1 (en) 2023-08-31

Family

ID=87765849

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/006090 WO2023162940A1 (en) 2022-02-25 2023-02-20 Inspection device, inspection method, and inspection program

Country Status (1)

Country Link
WO (1) WO2023162940A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016045019A (en) * 2014-08-20 2016-04-04 オムロン株式会社 Teaching device for substrate inspection device, and teaching method
JP2019100917A (en) * 2017-12-05 2019-06-24 パナソニックＩｐマネジメント株式会社 Inspection program generation system, generation method of inspection program and generation program of inspection program
JP2021120631A (en) * 2020-01-30 2021-08-19 株式会社デンソーテン Image generator and image generation method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016045019A (en) * 2014-08-20 2016-04-04 オムロン株式会社 Teaching device for substrate inspection device, and teaching method
JP2019100917A (en) * 2017-12-05 2019-06-24 パナソニックＩｐマネジメント株式会社 Inspection program generation system, generation method of inspection program and generation program of inspection program
JP2021120631A (en) * 2020-01-30 2021-08-19 株式会社デンソーテン Image generator and image generation method

Similar Documents

Publication Publication Date Title
US10489900B2 (en) Inspection apparatus, inspection method, and program
US20050207655A1 (en) Inspection system and method for providing feedback
JP2019100917A (en) Inspection program generation system, generation method of inspection program and generation program of inspection program
TWI484164B (en) Optical re - inspection system and its detection method
JP2015143656A (en) Inspection apparatus and inspection method
JP5239561B2 (en) Substrate appearance inspection method and substrate appearance inspection apparatus
KR20100124653A (en) Apparatus and method for visual inspection
KR20140091916A (en) Inspection Method For Display Panel
US11682113B2 (en) Multi-camera visual inspection appliance and method of use
JP2013079880A (en) Image quality inspection method, image quality inspection device, and image quality inspection program
KR20120048748A (en) Pattern image transmitter for inspecting display and method of inspecting display using the same
JP2014055915A (en) Appearance inspection device, appearance inspection method, and program
WO2023162940A1 (en) Inspection device, inspection method, and inspection program
WO2023162941A1 (en) Inspection device, image processing method, and image processing program
CN106845466A (en) A kind of product identification method and its system based on image
JP2007033126A (en) Substrate inspection device, parameter adjusting method thereof and parameter adjusting device
JP2009080004A (en) Inspection device
KR100675766B1 (en) Image sensor inspection system
CN112213081A (en) Screen body detection equipment
JP2003216930A (en) Method and apparatus for inspecting discoloration
KR200386330Y1 (en) System for testing a electronic scales using vision system
JP2002031605A (en) Defect-confirming apparatus and automatic visual inspection apparatus
JP2005003474A (en) Method, apparatus and program for evaluating color display panel
JP5947168B2 (en) Appearance inspection apparatus, control method and program for appearance inspection apparatus
WO2020184567A1 (en) Image inspection device and image inspection method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23759941

Country of ref document: EP

Kind code of ref document: A1