JP2017156161A - Illumination condition setting device, illumination condition setting method, and computer for illumination condition setting - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an illumination condition setting device that allows lighting conditions to be set appropriately.SOLUTION: The illumination condition setting device comprises: a storage unit for storing, for a first illumination light source in which the luminance value of one of pixels of an image is saturated even when an object is illuminated with a minimum set value of light emission luminance, two reference images corresponding to mutually different light emission luminance, and storing, for a second illumination light source other than the first illumination light source, one reference image corresponding to a prescribed light emission luminance; a virtual image generation unit 32 for generating, for each illumination condition, a virtual image for each illumination light source when the illumination light source illuminates the object with light emission luminance that corresponds to the illumination condition; a composite image generation unit 33 for generating, for each illumination condition, a composite image from the virtual image of each illumination light source; and an illumination condition setting unit 35 for selecting an illumination condition on the basis of the composite image of each illumination condition. The virtual image generation unit 32 generates a virtual image for the first illumination light source from the two reference images, and generates a virtual image for the second illumination light source from the one reference image.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an illumination condition setting device, an illumination condition setting method, and an illumination condition setting computer program for setting an illumination condition when illuminating an object using a plurality of illumination light sources, for example.

  Conventionally, by performing image recognition processing on images obtained by photographing products or parts storage, etc., inspection of assembled products or confirmation of the presence of parts during product assembly has been performed. Yes. In order to execute such inspection based on image recognition processing with high accuracy, a product or a part to be photographed (hereinafter simply referred to as an object) is represented on the image so that it can be easily seen. preferable. For that purpose, it is preferable that the object is illuminated under appropriate illumination conditions such that the luminance value of the pixel in which the object is reflected in the image is not saturated and no shadow is generated. Thus, a technique for creating an illumination state under an arbitrary illumination condition has been proposed (see, for example, Patent Document 1).

  In the illumination condition extraction method disclosed in Patent Document 1, a light that causes the movable illumination unit to irradiate a sample work with a certain intensity of illumination light is disposed at an illumination position that is a certain distance away from the sample work, and the illumination light is emitted from the illumination position. In a state where is irradiated, the imaging unit images the sample work. Then, the image composition unit synthesizes a plurality of captured images having different illumination positions by calculating an average luminance or a weighted average for each pixel. Thereby, an image in a state in which a plurality of illumination lights are irradiated from an arbitrary illumination position is displayed on the composite image display unit. In this illumination condition extraction method, for example, in the composite image inspection region, a composite image in which the area of the defective portion of the sample work is large and the luminance difference between the defective portion and the normal portion is large is extracted. The synthesis condition is selected as the illumination condition.

JP 2002-310935 A

  In the technique disclosed in Patent Document 1, in order to obtain a composite image in which the intensity of illumination light emitted from each light is virtually changed, the brightness of each pixel is determined between a plurality of captured images having different illumination positions. A weighted average is calculated. However, a part of the surface of the object may have a high reflectance like a mirror surface. In such a case, the illumination light reflected by a part of the surface becomes too bright, and the luminance value of the pixel corresponding to the part of the surface on the image is saturated. Some luminance cannot be expressed accurately. As a result, in the synthesized image obtained by performing the weighted average of the luminance for each pixel, the luminance of part of the surface is not accurately reproduced, and the optimal illumination condition may not be selected.

  In one aspect, an object of the present invention is to provide an illumination condition setting device that can appropriately set illumination conditions.

  According to one embodiment, an illumination condition setting device is provided. This illumination condition setting device is an area corresponding to an object on an image obtained by imaging the object by an imaging unit when the object is illuminated with a minimum settable emission luminance among a plurality of illumination light sources. A first reference image obtained by illuminating an object with a first light emission luminance and picking up an image by an image pickup unit, and a first illumination light source including a pixel whose luminance value is saturated A second reference image obtained by illuminating an object with a second light emission luminance different from the light emission luminance and picked up by an image pickup unit is stored. On the other hand, among the plurality of illumination light sources, the first illumination light source For the second illumination light source other than the above, a storage unit that illuminates the object with the third emission luminance and stores the third reference image obtained by imaging by the imaging unit, and each of the plurality of illumination conditions For each of the multiple illumination sources. Generated for each of a plurality of illumination light sources for each of a plurality of illumination conditions and a virtual image generation unit that generates a virtual image of the object when the light source illuminates the object with light emission luminance according to the illumination conditions A composite image generating unit that generates a composite image by calculating a sum of luminance values between corresponding pixels for the virtual image as a luminance value of the corresponding pixel of the composite image, and a composite image for each of a plurality of illumination conditions And an illumination condition setting unit that selects any one of a plurality of illumination conditions. The virtual image generation unit generates a virtual image for the first illumination light source based on the first reference image and the second reference image, while a virtual image for the second illumination light source is generated by the third reference image. Based on the reference image.

  Lighting conditions can be set appropriately.

It is a schematic block diagram of the illumination condition setting apparatus by one embodiment. It is a hardware block diagram of a control apparatus. It is a functional block diagram of the process part regarding an illumination condition setting process. (A) represents an example of a luminance value profile in a predetermined pixel row of the reference image having the light emission luminance of the minimum luminance Pa when two reference images are generated. (B) represents an example of a profile of luminance values in the same pixel column as (a) in the other reference image whose emission luminance is the luminance Pb. (C) is a diagram showing the relationship between the luminance value of each reference image and the luminance value of the corresponding pixel of the virtual image when two reference images are generated. It is a figure which shows an example of a synthesized image. It is an operation | movement flowchart of an illumination condition setting process.

  Hereinafter, an illumination condition setting device will be described with reference to the drawings. This illumination condition setting device illuminates an object with illumination light having a predetermined emission luminance from each of the plurality of illumination light sources whose emission luminance can be adjusted, and images the object with an imaging unit. The obtained image is set as a reference image used for determining the illumination condition. At this time, even if the light emission luminance is set to the minimum luminance value, for an illumination light source having a pixel whose luminance value is saturated on the obtained image, this illumination condition setting device has two types of light emission luminances different from each other. A reference image is generated. Then, the illumination condition setting device creates a virtual image that is an image of the object when the emission luminance is virtually set to an arbitrary value for each illumination light source based on the reference image, and for each illumination light source created By synthesizing the virtual image, a composite image of the object under an arbitrary illumination condition is obtained. And this illumination condition setting apparatus calculates | requires the illumination condition from which the recognition accuracy of the target object by image recognition becomes the best based on the synthesized image for every illumination condition. At this time, the illumination condition setting device calculates the luminance value of each pixel of the virtual image by linearly interpolating the luminance value of the corresponding pixel in each reference image for the illumination light source from which two reference images are generated. .

  FIG. 1 is a schematic configuration diagram of an illumination condition setting device according to one embodiment. The illumination condition setting device 1 includes a plurality of illumination light sources 11-1 to 11-n (where n is an integer of 2 or more), a light source driving unit 12, an imaging unit 13, and a control device 14. The illumination condition setting device 1 illuminates the object 16 placed on the stage 15 with a plurality of illumination light sources 11-1 to 11-n and images the object 16 obtained by photographing with the imaging unit 13. Based on the above, the illumination condition is set so that the accuracy of the image recognition processing for the object 16 is improved. In the present embodiment, the illumination condition includes a set value of light emission luminance for each of the plurality of illumination light sources 11-1 to 11-n.

  In the present embodiment, the object 16 is a component tray and a component placed on the component tray. However, the object 16 is not limited to this, and may be an arbitrary article.

  Each of the plurality of illumination light sources 11-1 to 11-n includes a light emitting element capable of adjusting light emission luminance, such as a fluorescent lamp, a light emitting diode, or an organic electroluminescence light source. Note that the illumination light from the light emitting element included in each illumination light source may be white light, or monochromatic light or infrared light. Alternatively, each of the plurality of illumination light sources 11-1 to 11-n may have different types of light emitting elements. Each of the plurality of illumination light sources 11-1 to 11-n emits illumination light having a light emission luminance set by the control device 14 in accordance with the power supplied from the light source driving unit 12.

  Each of the plurality of illumination light sources 11-1 to 11-n is attached to a support member (not shown) facing the lower side so that the surface of the stage 15 can be illuminated from different directions above the stage 15. For example, each of the plurality of illumination light sources 11-1 to 11-n is mounted so as to be equidistant with respect to the center of the stage 15 and at equal intervals from the adjacent illumination light sources. Or each of the some illumination light sources 11-1 to 11-n may be attached so that the distance from the center of the stage 15 may mutually differ.

  The light source driving unit 12 switches on / off of each of the plurality of illumination light sources 11-1 to 11-n and adjusts the light emission luminance according to the control signal from the control device 14 received via the signal line. Therefore, the light source driving unit 12 includes a circuit for driving the illumination light source for each of the plurality of illumination light sources 11-1 to 11-n. Then, the light source driving unit 12 supplies, for each of the plurality of illumination light sources 11-1 to 11-n, power for each illumination light source to emit illumination light having light emission luminance according to a control signal from the control device 14. It supplies according to the drive system of each illumination light source.

  The imaging unit 13 is attached to a support member (not shown) vertically above the stage 15 toward the stage 15. The imaging unit 13 includes an image sensor and an imaging optical system that forms an image of the object 16 on the image sensor, and the object placed on the stage 15 in accordance with a shooting instruction signal from the control device 14. 16 is photographed, and an image showing the object 16 is generated. Then, the imaging unit 13 outputs the generated image to the control device 14 via a signal line.

  Note that the image generated by the imaging unit 13 may be a color image or a monochrome image.

  The control device 14 controls each part of the illumination condition setting device 1. Moreover, the control apparatus 14 sets an illumination condition based on the reference | standard image of the target object 16 about each of several illumination light source 11-1 to 11-n obtained by the imaging part 13 as an example of a calibration process. To do. Furthermore, the control device 14 performs image recognition processing on an image obtained by photographing the object 16 by the imaging unit 13 with each illumination light source turned on or off according to the set illumination condition. The presence / absence determination or pass / fail determination of the object 16 is executed.

  FIG. 2 is a hardware configuration diagram of the control device 14. The control device 14 includes a communication unit 21, a storage unit 22, and a processing unit 23. The control device 14 may further include a display.

  The communication unit 21 includes an interface for connecting the control device 14 to other devices such as the light source driving unit 12 and the imaging unit 13 and a control circuit for the interface. For example, the communication unit 21 outputs the control signal received from the processing unit 23 to either the light source driving unit 12 or the imaging unit 13. Alternatively, the communication unit 21 passes the image received from the imaging unit 13 to the processing unit 23.

  The storage unit 22 includes, for example, a volatile or non-volatile semiconductor memory circuit. Furthermore, the storage unit 22 may have a hard disk device or the like. The storage unit 22 stores various data used in the illumination condition setting process. In addition, the storage unit 22 stores a computer program for executing various processes such as an illumination condition setting process executed by the processing unit 23. Furthermore, the storage unit 22 stores the image received from the imaging unit 13.

The processing unit 23 includes one or a plurality of processors and their peripheral circuits. And the process part 23 controls the illumination condition setting apparatus 1 whole.
Moreover, the process part 23 selects the optimal illumination condition from the several illumination conditions prepared previously by performing an illumination condition setting process as an example of a calibration process.

  In each illumination condition, whether or not to turn on each of the illumination light sources 11-1 to 11-n and the light emission luminance in the case of turning on are designated. For each illumination light source, the light emission luminance is specified as one of a predetermined number of values set within the light emission luminance adjustment range, for example. And what is necessary is just to prepare several illumination conditions so that the combination of the light emission luminance value of each illumination light source and the presence or absence of lighting may differ for every illumination condition.

  FIG. 3 is a functional block diagram of the processing unit 23 relating to the illumination condition setting process. The processing unit 23 includes a reference image acquisition unit 31, a virtual image generation unit 32, a composite image generation unit 33, an evaluation value calculation unit 34, and an illumination condition setting unit 35. Each of these units included in the processing unit 23 is a functional module realized by a computer program executed on a processor included in the processing unit 23, for example. Each of these units may be mounted on the control device 14 as a separate circuit and separately from the processing unit 23, or may be processed as one or a plurality of integrated circuits that realize the functions of these units. The controller 23 may be mounted separately from the unit 23.

  The reference image acquisition unit 31 acquires a reference image used for setting illumination conditions for each of the plurality of illumination light sources 11-1 to 11-n.

  In the present embodiment, the reference image acquisition unit 31 executes the same process for each illumination light source, and therefore, the process for the illumination light source 11-1 will be described below.

  The reference image acquisition unit 31 emits illumination light having a predetermined light emission luminance with the sample object 16 placed on the stage 15 and the other illumination light sources are turned off. A control signal is generated. Then, the reference image acquisition unit 31 outputs the control signal to the light source driving unit 12 via the communication unit 21. Note that the predetermined light emission luminance is preferably set such that the luminance value of each pixel in which the object 16 is captured on the image generated by the imaging unit 13 is higher than 0, for example. Then, the reference image acquisition unit 31 causes the imaging unit 13 to photograph the object 16 in a state where only the illumination light source 11-1 emits illumination light having a predetermined emission luminance, and the obtained image is taken from the imaging unit 13. Obtained via the communication unit 21. Then, the reference image acquisition unit 31 determines whether or not the luminance value of the pixel is the upper limit value for each pixel included in the object region where the object 16 is assumed to be reflected in the acquired image. To determine. When the luminance value of the pixel is represented by 0 to 255, the upper limit value is 255. Further, since the position of the object 16 is predetermined on the stage 15, the object region may be set in advance. The object area may include the entire image.

  When the luminance value of each pixel included in the object region is less than the upper limit value and the influence of light sources such as ambient light other than the illumination light source on the surface of the object 16 is small, the reference image acquisition unit 31 displays the image as the illumination light source 11. The reference image for -1. The reference image acquisition unit 31 stores the reference image in the storage unit 22 in association with the identification information for identifying the illumination light source 11-1 and the emission luminance of the illumination light source 11-1 at the time of generating the reference image. On the other hand, when the luminance value of any of the pixels included in the object region has reached the upper limit value, the reference image acquisition unit 31 sends a control signal so as to decrease the emission luminance of the illumination light source 11-1 by a predetermined value. The control signal is generated and output to the light source driving unit 12 via the communication unit 21. The reference image acquisition unit 31 is obtained by the imaging unit 13 imaging the object 16 in a state where the illumination light source 11-1 emits light with the reduced emission luminance and the other illumination light sources are turned off. Get the image. Then, the reference image acquisition unit 31 determines whether or not there is a pixel that has reached the upper limit value of the reference value in the object region for the image, as described above. When the luminance value of each pixel included in the object region is less than the upper limit value, the reference image acquisition unit 31 sets the image as the reference image for the illumination light source 11-1. The reference image acquisition unit 31 stores the reference image in the storage unit 22 in association with the identification information for identifying the illumination light source 11-1 and the emission luminance of the illumination light source 11-1 at the time of generating the reference image. On the other hand, when the luminance value of any of the pixels included in the object region reaches the upper limit value, the reference image acquisition unit 31 controls the control signal so as to further decrease the emission luminance of the illumination light source 11-1 by a predetermined value. And the control signal is output to the light source driving unit 12 via the communication unit 21.

  The reference image acquisition unit 31 repeats the above processing until a reference image is obtained or the light emission luminance of the illumination light source 11-1 reaches the minimum value. Also, for any image obtained when the light emission luminance of the illumination light source 11-1 reaches its minimum value, if any luminance value of the pixels in the object region has reached the upper limit value, the reference image acquisition unit 31 Uses the image as one of the reference images. Furthermore, in this case, the reference image acquisition unit 31 acquires another reference image for the illumination light source 11-1. Specifically, the reference image acquisition unit 31 generates a control signal that increases the emission luminance of the illumination light source 11-1 by a predetermined value from the minimum luminance, and transmits the control signal to the light source driving unit 12 via the communication unit 21. Output. Then, the reference image acquisition unit 31 emits light with the light emission luminance that the illumination light source 11-1 has increased by a predetermined value from the minimum luminance and the other illumination light sources are turned off. To image. The reference image acquisition unit 31 sets the image obtained from the imaging unit 13 as another one of the reference images. Then, the reference image acquisition unit 31 associates the obtained two reference images with the identification information for identifying the illumination light source 11-1 and the emission luminance of the illumination light source 11-1 at the time of generating the reference image in the storage unit 22. save.

  The virtual image generation unit 32 is an image of the object 16 when the light emission luminance is in accordance with the designated illumination condition based on the corresponding reference image for each of the plurality of illumination light sources 11-1 to 11-n. A virtual image is generated.

  In addition, since the virtual image generation part 32 performs the same process with respect to each illumination light source, below, the process with respect to the illumination light source 11-1 is demonstrated.

The reflectance of the object 16 is assumed to be constant. The image sensor of the imaging unit 13 is also assumed to have a linear characteristic with respect to the received light intensity. Therefore, when only one reference image is generated for the illumination light source 11-1, that is, when there is no pixel with a saturated luminance value in the object region of the reference image, the virtual image generation unit 32 performs the operation on the virtual image. The luminance value f (Pm, A) of any pixel A is calculated according to the following equation.
Here, Pn is the light emission luminance of the illumination light source 11-1 at the time of generating the reference image, and Pm is the light emission luminance specified by the illumination condition. F (Pn, A) is the luminance value of the pixel A on the reference image. However, when the luminance value f (Pm, A) calculated by the equation (1) exceeds the upper limit value of the luminance value, the luminance value f (Pm, A) is set to the upper limit value.

On the other hand, when two reference images are generated for the illumination light source 11-1, even if the illumination light source 11-1 emits light with the minimum luminance, there is a pixel with a saturated luminance value in the object region of the reference image. Therefore, in this case, the virtual image generation unit 32 calculates the luminance value f (Pm, A) of an arbitrary pixel A on the virtual image by linear interpolation according to the following equation.
Here, Pa is the minimum luminance of the illumination light source 11-1, and Pb is the emission luminance that is higher by a predetermined value than the minimum luminance at the time of generating the reference image. F (Pa, A) is a luminance value of the pixel A on the reference image when the illumination light source 11-1 emits light with the minimum luminance of the two reference images, and f (Pb, A) Is the luminance value of the pixel A on the other reference image. However, when the luminance value f (Pm, A) calculated by the equation (2) exceeds the upper limit value of the luminance value, the luminance value f (Pm, A) is set to the upper limit value.

  FIG. 4A illustrates an example of a luminance value profile in a predetermined pixel column of the reference image having the light emission luminance of the minimum luminance Pa when two reference images are generated. FIG. 4B shows an example of a luminance value profile in the same pixel column as that in FIG. 4A in the other reference image whose emission luminance is the luminance Pb. 4A and 4B, the horizontal axis represents the position on the reference image, and the vertical axis represents the luminance value (corresponding to the received light intensity of the image sensor of the imaging unit 13). A profile 401 represents a luminance value for each pixel in a predetermined pixel row on the reference image whose emission luminance is the minimum luminance Pa. On the other hand, the profile 402 represents a luminance value for each pixel in the pixel row on the other reference image whose emission luminance is the luminance Pb.

  In the example shown in FIGS. 4A and 4B, the luminance values (b, b ′) are saturated in the vicinity of the position B. On the other hand, at position A, the luminance values (a, a ′) are not saturated for any reference image. Similarly, at position C, the luminance values (c, c ′) are not saturated for any reference image. However, when the illumination light source emits light with the minimum luminance Pa, the luminance value a at the position A and the luminance value c at the position C are very low values. Therefore, the luminance value a and the luminance value c are easily affected by ambient light or the like. Further, as shown in the profile 401 and the profile 402, the other reference image whose emission luminance is the luminance Pb has a wider range in which the luminance value is saturated.

  FIG. 4C is a diagram illustrating a relationship between the luminance value of each reference image and the luminance value of the corresponding pixel of the virtual image when two reference images are generated. In FIG. 4C, the horizontal axis represents the light emission luminance of the illumination light source, and the vertical axis represents the luminance value of the pixel on the reference image or virtual image. A straight line 411 represents the relationship between the light emission luminance Pm and the luminance value on the virtual image calculated according to the equation (2) for the position A. Similarly, the straight line 412 and the straight line 413 represent the relationship between the light emission luminance Pm and the luminance value on the virtual image calculated according to the equation (2) for the position B and the position C, respectively.

  As indicated by each line, the luminance value f (Pm, X) (where X∈A, C) of the virtual image at the emission luminance Pm is a plurality of emission luminances (Pa and Pb in this example). It is calculated based on the luminance value of the corresponding pixel on the reference image. Therefore, even when the luminance a and luminance c at the light emission luminance Pa are low and affected by the ambient light, the virtual image generation unit 32 can estimate the luminance change on the image according to the light emission luminance change relatively accurately. . Therefore, the virtual image generation unit 32 can relatively accurately estimate the luminance value f (Pm, X) on the virtual image for any light emission luminance Pm. Note that for a pixel whose luminance value on the reference image is saturated even at the minimum luminance Pa, such as the position B, the luminance value f (Pm, X) on the virtual image at an arbitrary emission luminance Pm is also saturated. Therefore, it is not estimated accurately. However, in the present embodiment, since the emission luminance is set to the minimum luminance Pa for one reference image, the number of pixels whose luminance values are saturated on the reference image is also minimized. Therefore, the virtual image generation unit 32 can reduce the number of pixels whose luminance values are not accurately estimated in the virtual image.

  The virtual image generation unit 32 has a small influence on the surface of the object 16 due to light sources other than the illumination light source, such as ambient light, and the luminance value of each pixel included in the object region is less than the upper limit. When only one image is generated, the luminance value is calculated according to the expression (1) for each pixel in the object area of the virtual image. On the other hand, when two reference images are generated for the illumination light source of interest, the virtual image generation unit 32 calculates a luminance value for each pixel in the object area of the virtual image according to the equation (2). As a result, the virtual image generation unit 32 does not include a pixel whose luminance value is saturated in the object region, and the virtual image generation unit 32 performs virtual processing when the influence of light sources such as ambient light other than the illumination light source on the surface of the object 16 is small. When the amount of calculation required to create an image can be reduced, and when the object area includes pixels with saturated luminance values, or when the influence of light sources such as ambient light other than the illumination light source on the surface of the object 16 is not small Can create a more accurate virtual image. Then, the virtual image generation unit 32 passes a virtual image of each illumination light source to the composite image generation unit 33.

  The composite image generation unit 33 displays a composite image that virtually represents an image obtained by the imaging unit 13 when each illumination light source is turned on under the designated illumination condition, and corresponds to the illumination condition. Generate based on image.

  For example, the composite image generation unit 33 generates a composite image by calculating the sum of luminance values between corresponding pixels for the object region of the virtual image for each illumination light source generated according to the specified illumination condition. To do. Note that the composite image generation unit 33 does not have to use the virtual image for the illumination light source designated to be turned off under the designated illumination condition for the generation of the composite image. Then, the composite image generation unit 33 passes the generated composite image to the evaluation value calculation unit 34.

  The evaluation value calculation unit 34 calculates an evaluation value representing the appropriateness of the designated illumination condition based on the composite image. In the present embodiment, the evaluation value calculation unit 34 calculates the evaluation value such that the higher the possibility that the object 16 can be accurately recognized from the composite image, the higher the evaluation value.

  FIG. 5 is a diagram illustrating an example of a composite image. As described above, in the present embodiment, the object 16 is a component tray that is an example of a support and a component placed on the component tray. Therefore, the composite image 500 includes a component tray 501 and one or more components 502. The component tray 501 is provided with a plurality of pockets 503 for placing components 502. Each pocket 503 is an example of an article placement position. As shown in the composite image 500, in the target object 16 that is used for setting the illumination condition, the component 502 is actually placed only in the pocket 503 at a predetermined position on the component tray 501, The component 502 is not placed in the pocket 503 other than the predetermined position. That is, a smaller number of components 502 than the number of pockets 503 included in the component tray 501 are placed on the component tray 501. For example, in the example shown in FIG. 5, the component 502 is placed in the pocket 503 located at each corner of the component tray 501 and the central pocket 503, and the component 502 is placed in the other pocket 503. Not.

  Therefore, the evaluation value calculation unit 34 calculates a normalized cross-correlation value for each pocket 503 by performing template matching between the template representing the component 502 and the composite image for each pocket 503 of the component tray 501. The template and the position of the pocket 503 where the component 502 is placed on the component tray 501 and the position of the pocket 503 where the component 502 is not placed are stored in the storage unit 22 in advance. The normalized cross correlation value is an example of the degree of coincidence.

  Since the component tray 501 is placed at a preset position on the stage 15, the range of each pocket 503 on the composite image is also known. Therefore, the evaluation value calculation unit 34 calculates the normalized cross-correlation value for each pocket 503 while changing the relative positional relationship between the region on the composite image corresponding to the pocket and the template. Then, the evaluation value calculation unit 34 may set the maximum value among the calculated normalized cross-correlation values as the normalized cross-correlation value for the pocket.

  The evaluation value calculation unit 34 calculates, for example, a first representative value of the normalized cross-correlation value calculated for each pocket 503 on which the component 502 is not placed. The evaluation value calculation unit 34 also calculates a second representative value of the normalized cross-correlation value calculated for each pocket 503 on which the component 502 is placed. Then, the evaluation value calculation unit 34 sets the ratio of the second representative value to the first representative value as the evaluation value.

  The evaluation value calculation unit 34 may calculate, for example, the maximum value CFmax among the normalized cross-correlation values calculated for each pocket 503 on which the component 502 is not placed as the first representative value. . Further, the evaluation value calculation unit 34 may calculate, for example, the minimum value CTmin among the normalized cross-correlation values calculated for each pocket 503 on which the component 502 is placed as the second representative value. In this case, the evaluation value calculation unit 34 calculates a ratio (CTmin / CFmax) as the evaluation value.

  Alternatively, the evaluation value calculation unit 34 may calculate an average value or a median value of normalized cross-correlation values calculated for each pocket 503 on which the component 502 is not placed as the first representative value. Similarly, the evaluation value calculation unit 34 may calculate an average value or median value of normalized cross-correlation values calculated for each pocket 503 on which the component 502 is placed as the second representative value.

  In order to accurately determine the presence or absence of the component 502 on the composite image, it is preferable that the normalized cross-correlation value for the pocket 503 in which the component 502 is actually placed is as high as possible. On the other hand, the normalized cross-correlation value for the pocket 503 on which the part 502 is not placed is preferably as low as possible. Therefore, it can be seen that the larger the evaluation value, the more accurately the presence / absence of the component 502 can be determined on the composite image.

  According to the modification, the evaluation value calculation unit 34 may use a difference obtained by subtracting the first representative value from the second representative value, for example, (CTmin−CFmax) as the evaluation value.

  Alternatively, the evaluation value calculation unit 34 may use the number of pockets in which the presence or absence of a component is wrong as the evaluation value. In this case, the evaluation value calculation unit 34 counts the number n1 of the pockets in which the normalized cross-correlation value calculated for the pocket is not less than a predetermined threshold among the pockets 503 on which the component 502 is not placed. To do. Similarly, the evaluation value calculation unit 34 counts the number n2 of pockets in which the normalized cross-correlation value calculated for each pocket is less than a predetermined threshold among the pockets 503 on which the component 502 is placed. Then, the evaluation value calculation unit 34 sets the total of those numbers (n1 + n2) as the evaluation value. Note that the predetermined threshold value can be a threshold value used to determine whether or not there is actually a component in the pocket.

  Alternatively, the evaluation value calculation unit 34 may calculate a sum of squared errors as another example of the degree of matching instead of calculating a normalized cross-correlation value for each pocket 503. In this case, the evaluation value calculation unit 34 may calculate, as the first representative value, for example, the minimum value SFmin of the error sum of squares calculated for each pocket 503 on which the component 502 is not placed. Further, the evaluation value calculation unit 34 may calculate, for example, the maximum value STmax of the error square sum calculated for each pocket 503 on which the component 502 is placed as the second representative value. Then, the evaluation value calculation unit 34 may calculate (SFmin-STmax) as the evaluation value.

  The evaluation value calculation unit 34 stores the evaluation value in the storage unit 22 in association with the designated illumination condition.

  The illumination condition setting unit 35 selects, as the illumination condition to be set, the illumination condition that corresponds to the value that provides the best corresponding evaluation value among the plurality of illumination conditions. For example, when the evaluation value is (CTmin / CFmax), (CTmin-CFmax), or (SFmin-STmax), the illumination condition setting unit 35 sets the illumination condition corresponding to the maximum evaluation value. Determine as a condition. Alternatively, when the evaluation value is the number of pockets (n1 + n2) in which the presence / absence of the component has failed to be detected, the illumination condition setting unit 35 sets the illumination condition corresponding to the evaluation value that is minimum. Determine as.

FIG. 6 is an operation flowchart of the illumination condition setting process.
The reference image acquisition unit 31 acquires a reference image from the imaging unit 13 for each of the plurality of illumination light sources 11-1 to 11-n (step S101). At that time, the reference image acquisition unit 31 acquires two reference images having different emission luminances for an illumination light source including a pixel whose luminance value is saturated in the object region on the image even if light is emitted with the minimum luminance. .

  Next, the processing unit 23 designates a lighting condition of interest from among a plurality of lighting conditions prepared in advance (step S102). In the second and subsequent times, the processing unit 23 specifies an illumination condition different from the illumination condition for which the evaluation value has already been calculated.

  The virtual image generation unit 32 generates a virtual image based on the reference image for each of the plurality of illumination light sources 11-1 to 11-n according to the designated illumination condition (step S103). At that time, the virtual image generation unit 32 generates a virtual image from two reference images for an illumination light source having a pixel whose luminance value is saturated even at the minimum luminance, and generates a virtual image from one reference image for the other illumination light sources. Generate an image. Then, the composite image generation unit 33 generates a composite image by calculating the sum of luminance values between corresponding pixels in the object region for the virtual images generated for each of the plurality of illumination light sources 11-1 to 11-n. (Step S104).

  The evaluation value calculation unit 34 calculates an evaluation value for the designated illumination condition based on the composite image (step S105). Then, the evaluation value calculation unit 34 stores the evaluation value in the storage unit 22 together with the designated illumination condition.

  The processing unit 23 determines whether or not evaluation values have been calculated for all illumination conditions (step S106). When the evaluation value is not calculated for any of the lighting conditions (No at Step S106), the processing unit 23 repeats the processes after Step S102.

  On the other hand, when the evaluation values are calculated for all the illumination conditions (step S106—Yes), the illumination condition setting unit 35 identifies the illumination condition that provides the best evaluation value, and sets the identified illumination condition. The illumination condition is determined (step S107). The illumination condition setting unit 35 stores the set illumination condition in the storage unit 22. Thereafter, the processing unit 23 ends the illumination condition setting process.

  The set illumination conditions are used when an assembly process or an inspection process is executed. For example, during the execution of the assembly process, the processing unit 23 turns on or off each illumination light source with light emission luminance corresponding to the set illumination condition. Then, during the execution of the process, the imaging unit 13 captures the object 16 at regular intervals, generates an image representing the object 16, and outputs the image to the control device 14. Then, the processing unit 23 executes template matching for each pocket of the component tray based on the image, and calculates a normalized cross-correlation value for each pocket. Then, the processing unit 23 compares the normalized cross-correlation value with a predetermined threshold value for each pocket, and determines that there is a part in the pocket when the normalized cross-correlation value is equal to or greater than the predetermined threshold value. The processing unit 23 calculates the number of pockets determined to have parts, and outputs the number to other devices connected to the control device 14 via the communication network. Alternatively, when the number becomes equal to or less than a threshold value (for example, 1/10 of the total number of pockets that the component tray has), the processing unit 23 may output the number to another device.

  As described above, this illumination condition setting device generates an image of an object when each illumination light source is turned on with a predetermined light emission luminance as a reference image, and various illuminations are generated based on the reference image. A composite image corresponding to an image when the object is illuminated under conditions is generated. And this illumination condition setting device calculates an evaluation value representing the appropriateness of the illumination condition based on the composite image for each illumination condition, and sets the illumination condition when the evaluation value is the best as the illumination condition to be set decide. For this reason, the illumination condition setting device does not need to re-photograph the object every time the illumination condition is specified, and therefore the man-hour for setting the illumination condition can be reduced. Further, the illumination condition setting device generates two reference images having different emission luminances for an illumination light source including pixels whose luminance values are saturated in the object region on the image even when the emission luminance is minimum. Therefore, since this illumination condition setting device generates a virtual image according to the illumination condition designated based on the two reference images, a more appropriate virtual image can be obtained even when a pixel whose luminance value is saturated is included. Since it can generate | occur | produce, an appropriate lighting condition can be set.

  In some cases, the position or orientation of the illumination light source can be changed for any one of the plurality of illumination light sources. In such a case, the reference image acquisition unit 31 may generate a reference image in the same manner as in the above embodiment for each position or direction of the illumination light source. In this case, each illumination condition includes the position or orientation of the illumination light source. Therefore, the virtual image generation unit 32 may generate a virtual image based on the reference image corresponding to the position or orientation of the illumination light source under the designated illumination condition.

  Note that for an illumination light source that includes pixels whose luminance value is saturated within the object area even when light is emitted at the minimum luminance, the number of pixels whose luminance value is saturated changes even if the emission luminance is higher than the minimum luminance by a predetermined value. There may not be. For such an illumination light source, the reference image acquisition unit 31 may make the emission luminance of the illumination light source higher than the minimum luminance for both of the two reference images. As a result, the luminance value of each pixel in the reference image is higher than the luminance value of each pixel when the emission luminance of the illumination light source is the minimum luminance, thereby reducing the influence of ambient light on the reference image. In addition, in order to reflect the actual fluctuation of the illumination brightness due to the illumination light source itself or the ambient light, the brightness value of each pixel of the above-described reference image is used as the brightness value of each corresponding pixel between images obtained by actually performing imaging a plurality of times. An average value may be used. Alternatively, the luminance value of each pixel of the reference image may be adjusted when the virtual image is generated with reference to the actually observed fluctuation range of the luminance value.

  Further, for an illumination light source that includes a pixel whose luminance value is saturated in the object region even if light is emitted at the minimum luminance, the reference image acquisition unit 31 generates three or more reference images having different emission luminances at the time of imaging. Also good. The virtual image generation unit 32 may calculate the luminance value of each pixel of the virtual image by executing linear interpolation or spline interpolation processing based on the luminance value of the corresponding pixel in each reference image.

  In addition, for each reference image used in the illumination condition setting process, the processing unit 23 determines the number of gradations of the luminance value in the image acquired by the imaging unit 13 during the assembly process or the inspection process. The number of gradations may be increased. Thereby, in the composite image, the illumination state of the object 16 under the designated illumination condition is more faithfully represented.

  All examples and specific terms listed herein are intended for instructional purposes to help the reader understand the concepts contributed by the inventor to the present invention and the promotion of the technology. It should be construed that it is not limited to the construction of any example herein, such specific examples and conditions, with respect to showing the superiority and inferiority of the present invention. Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention.

The following supplementary notes are further disclosed regarding the embodiment described above and its modifications.
(Appendix 1)
When a target is illuminated with a minimum settable emission luminance among a plurality of illumination light sources, the luminance value is within a region corresponding to the target on an image obtained by imaging the target with an imaging unit. A first reference image obtained by illuminating the object with a first light emission luminance and picking up an image by the image pickup unit with respect to a first illumination light source including a saturated pixel, and the first light emission luminance The second reference image obtained by illuminating the object with a second light emission brightness different from that of the second image and capturing an image by the imaging unit, while the first illumination light source among the plurality of illumination light sources is stored. For a second illumination light source other than the illumination light source, a storage unit that illuminates the object with a third emission luminance and stores a third reference image obtained by imaging with the imaging unit;
For each of a plurality of illumination conditions, for each of the plurality of illumination light sources, a virtual image that generates a virtual image of the object when the illumination light source illuminates the object with a light emission luminance corresponding to the illumination condition. An image generator;
For each of the plurality of illumination conditions, the composite image is calculated by calculating a sum of luminance values between corresponding pixels for the virtual image generated for each of the plurality of illumination light sources as a luminance value of the corresponding pixel of the composite image. A composite image generation unit to generate;
An illumination condition setting unit that selects any of the plurality of illumination conditions based on the composite image for each of the plurality of illumination conditions;
Have
The virtual image generation unit generates the virtual image for the first illumination light source based on the first reference image and the second reference image, whereas the virtual image generation unit generates the virtual image for the second illumination light source. Generating a virtual image based on the third reference image;
Lighting condition setting device.
(Appendix 2)
The virtual image generation unit calculates a luminance value of each pixel of the virtual image for the first illumination light source by interpolation of luminance values of corresponding pixels of the first reference image and the second reference image. The virtual image is generated, and the luminance value of each pixel of the virtual image for the second illumination light source is calculated based on the luminance value of the corresponding pixel of the third reference image. The illumination condition setting device according to appendix 1, which generates a virtual image.
(Appendix 3)
For each of the plurality of illumination conditions, further comprising an evaluation value calculation unit that calculates an evaluation value representing the appropriateness of the illumination condition based on the composite image corresponding to the illumination condition,
The illumination condition setting device according to appendix 1 or 2, wherein the illumination condition setting unit selects an illumination condition in which the evaluation value is the best among the plurality of illumination conditions.
(Appendix 4)
The object includes a support having a first number of article arrangement positions, and a second number of articles less than the first number among the first number of article arrangement positions, Each of the second number of articles is disposed at any of the first number of article placement positions,
The storage unit further stores an article arrangement position where the second article among the template representing the article and the first number of article arrangement positions is arranged,
For each of the plurality of illumination conditions, the evaluation value calculation unit includes a degree of matching by template matching using the template for each of the first number of article placement positions in the composite image corresponding to the illumination condition. And a first representative value of the degree of coincidence calculated for the article arrangement position where the article is not arranged, and a second representative of the degree of coincidence calculated for the article arrangement position where the article is arranged The illumination condition setting device according to any one of appendices 1 to 3, wherein a ratio or difference with a value is calculated as the evaluation value.
(Appendix 5)
The object includes a support having a first number of article arrangement positions, and a second number of articles less than the first number among the first number of article arrangement positions, Each of the second number of articles is disposed at any of the first number of article placement positions,
The storage unit further stores an article arrangement position at which the second article among the first number of article arrangement positions is arranged,
The evaluation value calculation unit determines, for each of the plurality of illumination conditions, whether or not the article can be detected for each of the first number of article placement positions in the composite image corresponding to the illumination condition. The number of the article arrangement positions where the article is detected among the article arrangement positions where the article is not arranged, and the article arrangement where the article is not detected among the article arrangement positions where the article is arranged The illumination condition setting device according to any one of appendices 1 to 3, wherein a total with the number of positions is calculated as the evaluation value.
(Appendix 6)
The illumination condition setting device according to any one of appendices 1 to 5, wherein the first emission luminance is a minimum value of the emission luminance of the first illumination light source.
(Appendix 7)
For each of a plurality of illumination conditions, for each of a plurality of illumination light sources, generate a virtual image about the object when the illumination light source illuminates the object with a light emission luminance according to the illumination condition,
For each of the plurality of illumination conditions, the composite image is calculated by calculating a sum of luminance values between corresponding pixels for the virtual image generated for each of the plurality of illumination light sources as a luminance value of the corresponding pixel of the composite image. Generate
Selecting one of the plurality of illumination conditions based on the composite image for each of the plurality of illumination conditions;
Including
Generating the virtual image includes
Among the plurality of illumination light sources, when the object is illuminated with a minimum value of light emission luminance that can be set, the luminance within an area corresponding to the object on an image obtained by imaging the object by an imaging unit The virtual image of the first illumination light source including pixels whose values are saturated is obtained by the first illumination light source illuminating the object with a first emission luminance and captured by the imaging unit. A second reference image obtained by illuminating the object with a second light emission luminance different from the first light emission luminance and imaged by the imaging unit. Based on the reference image,
Of the plurality of illumination light sources, the virtual image of a second illumination light source other than the first illumination light source is illuminated with the second illumination light source illuminating the object with a third emission luminance; and Generating based on a third reference image obtained by imaging by the imaging unit,
Lighting condition setting method.
(Appendix 8)
For each of a plurality of illumination conditions, for each of a plurality of illumination light sources, generate a virtual image about the object when the illumination light source illuminates the object with a light emission luminance according to the illumination condition,
For each of the plurality of illumination conditions, the composite image is calculated by calculating a sum of luminance values between corresponding pixels for the virtual image generated for each of the plurality of illumination light sources as a luminance value of the corresponding pixel of the composite image. Generate
Selecting any of the plurality of illumination conditions based on the composite image for each of the plurality of illumination conditions;
Let the computer do
Generating the virtual image includes
Among the plurality of illumination light sources, when the object is illuminated with a minimum value of light emission luminance that can be set, the luminance within an area corresponding to the object on an image obtained by imaging the object by an imaging unit The virtual image of the first illumination light source including pixels whose values are saturated is obtained by the first illumination light source illuminating the object with a first emission luminance and captured by the imaging unit. A second reference image obtained by illuminating the object with a second light emission luminance different from the first light emission luminance and imaged by the imaging unit. Based on the reference image,
Of the plurality of illumination light sources, the virtual image of a second illumination light source other than the first illumination light source is illuminated with the second illumination light source illuminating the object with a third emission luminance; and Generating based on a third reference image obtained by imaging by the imaging unit,
Computer program for setting lighting conditions.

DESCRIPTION OF SYMBOLS 1 Illumination condition setting apparatus 11-1 to 11-n Illumination light source 12 Light source drive part 13 Imaging part 14 Control apparatus 15 Stage 16 Object 21 Communication part 22 Storage part 23 Processing part 31 Reference image acquisition part 32 Virtual image generation part 33 Synthesis | combination Image generation unit 34 Evaluation value calculation unit 35 Illumination condition setting unit

Claims (6)

When a target is illuminated with a minimum settable emission luminance among a plurality of illumination light sources, the luminance value is within a region corresponding to the target on an image obtained by imaging the target with an imaging unit. A first reference image obtained by illuminating the object with a first light emission luminance and picking up an image by the image pickup unit with respect to a first illumination light source including a saturated pixel, and the first light emission luminance The second reference image obtained by illuminating the object with a second light emission brightness different from that of the second image and capturing an image by the imaging unit, while the first illumination light source among the plurality of illumination light sources is stored. For a second illumination light source other than the illumination light source, a storage unit that illuminates the object with a third emission luminance and stores a third reference image obtained by imaging with the imaging unit;
For each of a plurality of illumination conditions, for each of the plurality of illumination light sources, a virtual image that generates a virtual image of the object when the illumination light source illuminates the object with a light emission luminance corresponding to the illumination condition. An image generator;
For each of the plurality of illumination conditions, the composite image is calculated by calculating a sum of luminance values between corresponding pixels for the virtual image generated for each of the plurality of illumination light sources as a luminance value of the corresponding pixel of the composite image. A composite image generation unit to generate;
An illumination condition setting unit that selects any of the plurality of illumination conditions based on the composite image for each of the plurality of illumination conditions;
Have
The virtual image generation unit generates the virtual image for the first illumination light source based on the first reference image and the second reference image, whereas the virtual image generation unit generates the virtual image for the second illumination light source. Generating a virtual image based on the third reference image;
Lighting condition setting device.   The virtual image generation unit calculates a luminance value of each pixel of the virtual image for the first illumination light source by interpolation of luminance values of corresponding pixels of the first reference image and the second reference image. The virtual image is generated, and the luminance value of each pixel of the virtual image for the second illumination light source is calculated based on the luminance value of the corresponding pixel of the third reference image. The illumination condition setting device according to claim 1, which generates a virtual image. For each of the plurality of illumination conditions, further comprising an evaluation value calculation unit that calculates an evaluation value representing the appropriateness of the illumination condition based on the composite image corresponding to the illumination condition,
The illumination condition setting device according to claim 1, wherein the illumination condition setting unit selects an illumination condition that provides the best evaluation value among the plurality of illumination conditions. The object includes a support having a first number of article arrangement positions, and a second number of articles less than the first number among the first number of article arrangement positions, Each of the second number of articles is disposed at any of the first number of article placement positions,
The storage unit further stores an article arrangement position where the second article among the template representing the article and the first number of article arrangement positions is arranged,
For each of the plurality of illumination conditions, the evaluation value calculation unit includes a degree of matching by template matching using the template for each of the first number of article placement positions in the composite image corresponding to the illumination condition. And a first representative value of the degree of coincidence calculated for the article arrangement position where the article is not arranged, and a second representative of the degree of coincidence calculated for the article arrangement position where the article is arranged The illumination condition setting device according to any one of claims 1 to 3, wherein a ratio or a difference with a value is calculated as the evaluation value. For each of a plurality of illumination conditions, for each of a plurality of illumination light sources, generate a virtual image about the object when the illumination light source illuminates the object with a light emission luminance according to the illumination condition,
For each of the plurality of illumination conditions, the composite image is calculated by calculating a sum of luminance values between corresponding pixels for the virtual image generated for each of the plurality of illumination light sources as a luminance value of the corresponding pixel of the composite image. Generate
Selecting any of the plurality of illumination conditions based on the composite image for each of the plurality of illumination conditions;
Including
Generating the virtual image includes
Among the plurality of illumination light sources, when the object is illuminated with a minimum value of light emission luminance that can be set, the luminance within an area corresponding to the object on an image obtained by imaging the object by an imaging unit The virtual image of the first illumination light source including pixels whose values are saturated is obtained by the first illumination light source illuminating the object with a first emission luminance and captured by the imaging unit. A second reference image obtained by illuminating the object with a second light emission luminance different from the first light emission luminance and imaged by the imaging unit. Based on the reference image,
Of the plurality of illumination light sources, the virtual image of a second illumination light source other than the first illumination light source is illuminated with the second illumination light source illuminating the object with a third emission luminance; and Generating based on a third reference image obtained by imaging by the imaging unit,
Lighting condition setting method. For each of a plurality of illumination conditions, for each of a plurality of illumination light sources, generate a virtual image about the object when the illumination light source illuminates the object with a light emission luminance according to the illumination condition,
For each of the plurality of illumination conditions, the composite image is calculated by calculating a sum of luminance values between corresponding pixels for the virtual image generated for each of the plurality of illumination light sources as a luminance value of the corresponding pixel of the composite image. Generate
Selecting any of the plurality of illumination conditions based on the composite image for each of the plurality of illumination conditions;
Let the computer do
Generating the virtual image includes
Among the plurality of illumination light sources, when the object is illuminated with a minimum value of light emission luminance that can be set, the luminance within an area corresponding to the object on an image obtained by imaging the object by an imaging unit The virtual image of the first illumination light source including pixels whose values are saturated is obtained by the first illumination light source illuminating the object with a first emission luminance and captured by the imaging unit. A second reference image obtained by illuminating the object with a second light emission luminance different from the first light emission luminance and imaged by the imaging unit. Based on the reference image,
Of the plurality of illumination light sources, the virtual image of a second illumination light source other than the first illumination light source is illuminated with the second illumination light source illuminating the object with a third emission luminance; and Generating based on a third reference image obtained by imaging by the imaging unit,
Computer program for setting lighting conditions.