JP5757156B2 - Method, apparatus and program for calculating center position of detection object - Google Patents

Description

  The present invention relates to a method, an apparatus, and a program for calculating a center position of a detection object including at least a part of a cylinder or a sphere having a known radius.

  For example, in the field of FA (Factory Automation) and the like, from the viewpoint of labor saving and automation, various types of images are obtained based on images obtained by imaging a detection target (hereinafter also referred to as “work”) handled at a manufacturing site. Image measurement technology that automates this process has been developed and put into practical use.

  As an example of such an image measurement technique, a method for calculating the center position of a workpiece in order to grip a cylindrical workpiece has been proposed. For example, in Japanese Patent Application Laid-Open No. 2004-294234 (Patent Document 1), a distance image is input by photographing an object including a cylindrical portion from the side surface, and a cylindrical region distance image that is a region of the cylindrical portion is obtained from the distance image. Extract and calculate the normal vector at each point of the cylindrical area distance image, calculate the principal axis from the normal vector, project the cylindrical area distance image on a plane perpendicular to the main axis, and turn the projected cylindrical area distance image into a circle An object detection method is disclosed that approximates and obtains the center of a circle and uses the principal axis passing through the center of the circle as the position and orientation of the object. That is, Patent Document 1 discloses a method of calculating a cylinder center and a diameter by projecting a three-dimensional point group obtained by measuring a cylinder with a three-dimensional measuring device onto a plane in the direction of the cylinder central axis.

  Japanese Patent Laid-Open No. 2000-329520 (Patent Document 2) irradiates laser slit light so that a slit light scanning device has a plurality of brightness-changing portions when the coil is viewed in the radial direction. A coil position detection device is disclosed that calculates the width of the coil and the center position in the width direction for a portion where a plurality of cross-sectional shape data exists when the coil calculation device projects the cross-sectional shape data in the coil radial direction. That is, in Patent Document 2, light cutting is performed while changing the light projecting direction, a glossy cylindrical coil is three-dimensionally measured, and the width and diameter of the cylinder are calculated from a point group obtained from the three-dimensional measurement. A technique is disclosed.

JP 2004-294234 A JP 2000-329520 A

  However, in a state where a plurality of workpieces are placed in a tray or the like (that is, in a state where they are “unstacked”), a process such as gripping each workpiece from the tray using a robot or the like is performed as described above. Even if the technique was applied as it was, the desired result could not be obtained.

  That is, in the method disclosed in Patent Document 1, a group of points having a predetermined area is necessary because the center of the cylinder and the diameter are calculated using circle fitting. Therefore, although it can be applied to a cylindrical workpiece whose surface is diffusely reflected, it is difficult to obtain a sufficient number of measurement points with a cylindrical workpiece having glossy surface characteristics (specular reflection). Therefore, the invention described in Patent Document 1 cannot be applied universally to a workpiece whose surface is not specified as one of the diffuse reflection surface and the specular reflection surface.

  Further, in the method disclosed in Patent Document 2, in order to acquire a sufficient number of measurement points, imaging (light cutting) is required a plurality of times at different angles. Therefore, the measurement time for one workpiece becomes relatively long, and a large number of workpieces cannot be processed in a practical time. In addition, in order to obtain a sufficient number of measurement points for a workpiece, the direction of the workpiece (cylindrical direction) needs to be known to some extent, and can be applied to the above-described bulk workpiece. Can not.

  Therefore, an object of the present invention is to provide a method, apparatus, and program capable of calculating the center position of each detection object even when a plurality of detection objects are arranged in an arbitrary direction. Is to provide.

  According to one aspect of the present invention, there is provided a method for calculating the center position of a detection object including at least a part of a cylinder or a sphere having a known radius. The method includes obtaining an input image obtained by imaging at least one detection object, extracting a region corresponding to the shape of the detection object from the input image, and obtaining height information about the extracted region. Identifying a portion where the brightness is maximized in the extracted region, determining a normal direction of the portion, determining the normal direction, a known radius, and determining that the brightness is a maximum. Determining the center position of the detection target object from the position of the corresponding portion and the corresponding height information.

  Preferably, in the step of determining the normal direction, the method of determining the normal direction is made different depending on the optical reflection characteristic of the detection target.

  More preferably, the input image is an image in which at least one detection object is irradiated with illumination light from the first direction and arranged with a line of sight in a second direction different from the first direction. Acquired by imaging using the apparatus.

  More preferably, in the step of determining the normal direction, when the surface of the detection target is specularly reflected, the three-dimensional coordinate point determined from the position of the portion where the brightness is maximum and the corresponding height information And a vector corresponding to a bisection angle between a first vector extending toward the light source of the illumination light and a second vector passing through the three-dimensional coordinate point and extending in the direction of the imaging device As a normal vector.

  Alternatively, more preferably, in the step of determining the normal direction, when the surface of the detection object is diffusely reflected, the three-dimensional coordinates determined from the position of the portion where the brightness is maximum and the corresponding height information Calculating a vector passing through the point and extending toward the light source of the illumination light as a normal vector.

  Alternatively, more preferably, the step of acquiring the height information irradiates the at least one detection object with a plurality of different shading patterns from the first direction, and each image is captured by the imaging device at the time of each irradiation. Height information is acquired using a plurality of images.

  Alternatively, more preferably, the step of acquiring the input image includes the first image acquired by the imaging device in a state where the illumination light is not irradiated from the first direction to the at least one detection object, and at least one detection. Including a step of generating an input image from a difference from the second image acquired by the imaging device in a state in which illumination light is irradiated from the first direction to the object.

  Preferably, the step of determining the center position includes a step of determining a center axis of the detection object from a plurality of center positions determined for the same detection object.

  Preferably, the method further includes a step of outputting the determined center position of the detection object.

  According to another aspect of the present invention, there is provided an apparatus for calculating a center position of a detection object including at least a part of a cylinder or a sphere having a known radius. The apparatus is connected to an imaging apparatus, acquires an input image obtained by imaging at least one detection object, means for extracting an area corresponding to the shape of the detection object from the input image, and the extracted area Means for obtaining the height information, means for identifying the part where the brightness is maximized in the extracted region, determining the normal direction of the part, the determined normal direction, and the known radius And means for determining the center position of the detection object from the position of the portion where the brightness is maximum and the corresponding height information.

  According to still another aspect of the present invention, there is provided a program for calculating the center position of a detection object including at least a part of a cylinder or a sphere having a known radius. The program includes, in a computer, obtaining an input image obtained by imaging at least one detection target, extracting a region corresponding to the shape of the detection target from the input image, and height of the extracted region. A step of acquiring information, a step of identifying a portion where brightness is maximized in the extracted region, a step of determining a normal direction of the portion, a determined normal direction, a known radius, and a brightness The step of determining the center position of the detection target object is executed from the position of the portion where is maximum and the corresponding height information.

  According to the present invention, even if a plurality of detection objects are arranged in an arbitrary direction, the center position of each detection object can be calculated.

1 is a schematic diagram showing an overall configuration of a gripping system including an image processing apparatus according to an embodiment of the present invention. 1 is a schematic configuration diagram of an image processing apparatus according to an embodiment of the present invention. It is a flowchart which shows the whole process sequence performed in the image processing apparatus which concerns on this Embodiment. It is a figure which shows the process example of the pre-processing which concerns on embodiment of this invention. It is a figure which shows the process example of the shading process which concerns on embodiment of this invention. It is a figure which shows an example of the pattern used for the three-dimensional measurement process which concerns on embodiment of this invention. It is a figure which shows the process example of the three-dimensional measurement process which concerns on embodiment of this invention. It is a figure for demonstrating the principle of the normal calculation process which concerns on this Embodiment. It is a flowchart which shows the more detailed procedure of the normal calculation process performed in the image processing apparatus which concerns on this Embodiment. It is a figure which shows the process example of the process which extracts the part which the brightness which concerns on this Embodiment becomes maximum. It is a figure for demonstrating the calculation process of the light vector which concerns on this Embodiment. It is a figure for demonstrating the normal line calculation process which concerns on this Embodiment. It is a figure for demonstrating the position calculation process which concerns on this Embodiment. It is a figure which shows an example of the recognition result produced | generated by the image processing which concerns on embodiment of this invention.

  Embodiments of the present invention will be described in detail with reference to the drawings. Note that the same or corresponding parts in the drawings are denoted by the same reference numerals and description thereof will not be repeated.

<< A. Overview"
The image processing method according to the present embodiment is directed to a method for calculating the center position of a detection target (work W) including at least a part of a cylinder or a sphere having a known radius. In the image processing method according to the present embodiment, illumination light (white light) is projected onto a work W from a projector that is a kind of surface light source, and reflected light generated by projection of the illumination light using an imaging device. Image. From the input image obtained by this imaging, the cylindrical cross section of the workpiece W is obtained from the position information and height information (that is, the coordinate value in the three-dimensional coordinate system) of the portion where the brightness is maximum (the luminance maximum portion). Considering this, the position of the center of the workpiece W is determined using position information (three-dimensional coordinate values) of a portion (luminance maximum portion) where the radius and brightness of the cylindrical cross section are maximum. Depending on the reflection characteristics of the workpiece W, it is preferable to vary the method for calculating the center position.

  As described above, the image processing method according to the present embodiment uses a result of image processing on a two-dimensional image and height information by three-dimensionally measuring the same workpiece W, and uses a smaller amount of work. The center position of W is calculated sequentially.

<< B. Overall device configuration >>
FIG. 1 is a schematic diagram showing an overall configuration of a gripping system SYS including an image processing apparatus 100 according to an embodiment of the present invention. A gripping system SYS illustrated in FIG. 1 performs a process of gripping detection objects (workpieces W) that are incorporated in a production line or the like and stacked in a tray or the like one by one. Note that FIG. 1 illustrates a case where only two workpieces W exist in the tray for easy understanding, but a large number of workpieces W may be arranged in an overlapping state.

  Here, since the workpiece | work W is conveyed from the upstream manufacturing apparatus etc. which are not shown in figure, the direction in the tray shall be random. Therefore, the gripping system SYS (mainly, the image processing apparatus 100) calculates the center position of the detection target (work W) including at least a part of the cylinder or the sphere, and uses the calculated center position to perform the gripping operation. I do. Here, since the dimension of the workpiece W is known in advance, the radius of the cylindrical portion is assumed to be known.

  In the following description, the description will be given mainly assuming a workpiece W (for example, a bolt) having a columnar shape among cylinders, but it is a cylindrical shape such as a U-shaped pipe or a spherical shape. Even if it exists, it is applicable similarly. That is, the calculation method of the center position according to the present embodiment can be generally applied to an object having a circular cross section.

  More specifically, the gripping system SYS includes an image processing apparatus 100, a robot controller 200, and a robot 300. A projector (illumination device) 7 and an imaging device 8 are connected to the image processing apparatus 100.

  As will be described later, the projector 7 functions as a kind of surface light source, and projects illumination light of an arbitrary shade pattern onto a predetermined position (a position where a tray including the workpiece W is disposed) according to an instruction from the image processing apparatus 100. It shall be possible. This arbitrary shading pattern includes a pattern in which the brightness is uniform in the irradiation surface and a pattern in which the brightness changes periodically along a predetermined direction in the irradiation surface.

  The imaging device 8 images the light reflected by the surface of the workpiece W from the illumination light irradiated by the projector 7. Then, the imaging device 8 outputs an image (hereinafter also referred to as “input image”) indicating the reflected light (image of the workpiece W) from the workpiece W.

  In the present embodiment, it is assumed that the relative position and orientation relationship between the projector 7 and the imaging device 8 is known. That is, the direction in which the projector 7 irradiates illumination light (first direction) and its arrangement position, and the line-of-sight direction (second direction) of the imaging device 8 and its arrangement position are acquired in advance. Note that the relative position and orientation relationship between the projector 7 and the imaging device 8 is reflected in the calibration data obtained by the calibration process. Specific contents of the calibration data include a three-dimensional position and orientation (coordinates (x, y, z) and angles (α, β, γ) on three-dimensional coordinates) between the projector 7 and the imaging device 8. And information such as their viewing angle.

  More specifically, the projector 7 includes, as main components, a light source such as an LED (Light Emitting Diode) or a halogen lamp, and a filter disposed on the irradiation surface side of the projector 7. The light source used for the projector 7 preferably has a relatively wide wavelength range. The filter generates a shading pattern necessary for three-dimensional measurement, which will be described later, and can arbitrarily change the in-plane transmissivity according to a command from the image processing apparatus 100 or the like.

  The imaging device 8 includes, as main components, an optical system such as a lens, and an imaging element such as a CCD (Coupled Charged Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

  The image processing apparatus 100 is typically a computer having a general-purpose architecture, and provides an image processing function according to the present embodiment by executing a preinstalled program (instruction code). To do. Such a program is typically distributed in a state of being stored in various recording media or installed in the image processing apparatus 100 via a network or the like.

  When using such a general-purpose computer, an OS (Operating System) for providing basic functions of the computer is installed in addition to the application for providing the functions according to the present embodiment. It may be. In this case, the program according to the present embodiment may be a program module that is provided as a part of the OS and calls a necessary module at a predetermined timing in a predetermined arrangement to execute processing. Good. That is, the program itself according to the present embodiment does not include the module as described above, and the process may be executed in cooperation with the OS. The program according to the present embodiment may be a form that does not include some of such modules.

  Furthermore, the program according to the present embodiment may be provided by being incorporated in a part of another program. Even in that case, the program itself does not include the modules included in the other programs to be combined as described above, and the processing is executed in cooperation with the other programs. That is, the program according to the present embodiment may be in a form incorporated in such another program. A part or all of the functions provided by executing the program may be implemented as a dedicated hardware circuit.

  FIG. 2 is a schematic configuration diagram of the image processing apparatus 100 according to the embodiment of the present invention. Referring to FIG. 2, an image processing apparatus 100 includes a CPU (Central Processing Unit) 110 that is an arithmetic processing unit, a main memory 112 and a hard disk 114 as a storage unit, a camera interface 116, an input interface 118, and a display. A controller 120, a projector interface 122, a communication interface 124, and a data reader / writer 126 are included. These units are connected to each other via a bus 128 so that data communication is possible.

  The CPU 110 performs various operations by developing programs (codes) installed in the hard disk 114 in the main memory 112 and executing them in a predetermined order. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory), and in addition to a program read from the hard disk 114, an input image acquired by the imaging device 8, calibration, and the like. Data related to measurement data and measurement results. Further, the hard disk 114 may store various setting values. In addition to the hard disk 114 or instead of the hard disk 114, a semiconductor storage device such as a flash memory may be employed.

  The camera interface 116 mediates data transmission between the CPU 110 and the imaging device 8. That is, the camera interface 116 is connected to the imaging device 8 for imaging the workpiece W and generating an input image. More specifically, the camera interface 116 can be connected to one or more imaging devices 8 and includes an image buffer 116 a for temporarily storing an input image from the imaging device 8. Then, when an input image having a predetermined number of frames is accumulated in the image buffer 116 a, the camera interface 116 transfers the accumulated input image to the main memory 112. In addition, the camera interface 116 gives an imaging command to the imaging device 8 in accordance with an internal command generated by the CPU 110.

  The input interface 118 mediates data transmission between the CPU 110 and an input unit such as a mouse 104, a keyboard, or a touch panel. That is, the input interface 118 accepts an operation command given by the user operating the input unit.

  The display controller 120 is connected to the display 102 which is a typical example of a display device, and notifies the user of the processing result in the CPU 110. In other words, the display controller 120 is connected to the display 102 and controls display on the display 102.

  The projector interface 122 mediates data transmission between the CPU 110 and the projector 7. More specifically, the projector interface 122 gives an illumination command to the projector 7 according to an internal command generated by the CPU 110.

  The communication interface 124 mediates data transmission between the CPU 110 and a console (or a personal computer or server device). The communication interface 124 typically includes Ethernet (registered trademark), USB (Universal Serial Bus), or the like.

  The data reader / writer 126 mediates data transmission between the CPU 110 and the memory card 106 that is a recording medium. That is, the memory card 106 circulates in a state where a program executed by the image processing apparatus 100 is stored, and the data reader / writer 126 reads the program from the memory card 106. Further, the data reader / writer 126 writes the input image acquired by the imaging device 8 and / or the processing result in the image processing device 100 to the memory card 106 in response to an internal command of the CPU 110. The memory card 106 is a general-purpose semiconductor storage device such as CF (Compact Flash) or SD (Secure Digital), a magnetic storage medium such as a flexible disk, or a CD-ROM (Compact Disk Read Only Memory). ) And other optical storage media.

  Further, the image processing apparatus 100 may be connected to another output device such as a printer as necessary.

  Referring to FIG. 1 again, robot 300 includes a hand mechanism for gripping workpiece W and a joint mechanism for positioning the hand mechanism in an arbitrary position and direction. The operation of the robot 300 is controlled by the robot controller 200.

  The robot controller 200 recognizes the position and orientation of the workpiece W based on the information indicating the center position (center axis) of the workpiece W calculated by the image processing apparatus 100, and also recognizes the position of each recognized workpiece W, Based on the attitude, a command for positioning the hand mechanism of the robot 300 in an appropriate position and direction is generated and output to the robot 300. In response to this command, the robot 300 starts gripping the workpiece W.

<< C. Overall processing >>
FIG. 3 is a flowchart showing an overall processing procedure executed in the image processing apparatus 100 according to the present embodiment. Each step shown in FIG. 3 is basically executed by the CPU 110 of the image processing apparatus 100.

  Referring to FIG. 3, CPU 110 of image processing apparatus 100 performs preprocessing (step S1). This pre-processing includes processing for acquiring an input image obtained by imaging at least one workpiece W in a bulk state. A typical example of this input image acquisition method is a method of inputting an image captured using the imaging device 8 as shown in FIG. However, the input image obtained by imaging using the imaging device 8 may be stored and input to the image processing device 100 afterwards. Such a processing method is suitable for use in investigating the cause, such as when there is an abnormality in the gripping operation on the workpiece W.

  Furthermore, in the present embodiment, a shading image is generated in order to eliminate the influence of disturbance light from around the work W. This shading image is captured in an image obtained by illuminating illumination light from the projector 7 (environment light + illumination light) and in a state in which illumination light is not illuminated from the projector 7 (only ambient light). It is generated by the difference from the obtained image. In other words, in the preprocessing shown in step S1, the image acquired by the imaging device 8 in a state where the illumination light is not irradiated on at least one workpiece W and the illumination light is irradiated on the at least one workpiece W. The process which produces | generates an input image from the difference with the image acquired by the imaging device 8 in the state is included. Details of this shading image generation processing will be described later.

  After executing the preprocessing, the CPU 110 of the image processing apparatus 100 executes a segmentation process (step S2). This segmentation process is a process for specifying the position of the workpiece W included in the input image. That is, the segmentation process includes a process of extracting an area corresponding to the shape of the workpiece W from the input image. An area corresponding to the shape of the workpiece W is also referred to as a “cylindrical area” for convenience. Details of this segmentation processing will be described later.

  After execution of the segmentation process, the CPU 110 of the image processing apparatus 100 executes a three-dimensional measurement process (step S3). This three-dimensional measurement process is a process for acquiring height information about the cylindrical region extracted in step S2. As an example of such a three-dimensional measurement process, a phase shift method is employed in the present embodiment. By adopting the phase shift method, the height information can be acquired using the projector (illumination device) 7 and the imaging device 8 necessary for the preprocessing in the above-described step S1, so that the apparatus can be simplified. is there.

  Of course, any three-dimensional measurement process may be adopted as long as the height information about the cylindrical region can be acquired. For example, a spatial code method or the like can be employed instead of the phase shift method. Further, a three-dimensional position information of the workpiece W may be acquired using a measuring device different from the image processing apparatus 100, and the acquired three-dimensional position information may be input to the image processing apparatus 100. Details of the three-dimensional measurement process (phase shift method) will be described later.

  After executing the three-dimensional measurement process, the CPU 110 of the image processing apparatus 100 executes a normal line calculation process (step S4). This normal calculation processing is processing for specifying a portion where brightness (typically luminance) is maximized in the cylindrical region extracted in step S2 and determining the normal direction of the portion in the workpiece W. is there. The normal determined in step S4 is calculated as a vector quantity in a three-dimensional space.

  In particular, in the image processing apparatus 100 according to the present embodiment, the normal line is calculated using the reflection characteristics on the surface of the workpiece W. More specifically, the normal line calculation process in step S4 varies the normal direction determination method depending on the optical reflection characteristics (specular reflection / diffuse reflection) of the workpiece W. Details of the method for determining the normal direction will be described later.

  After determining the normal direction, the CPU 110 of the image processing apparatus 100 executes position calculation processing (step S5). This position calculation process is performed based on the normal direction determined in step S4, the known radius of the workpiece W, the position of the portion where the brightness is maximized, and the corresponding height information (obtained in step S3). The process of determining the center position of W is included. Depending on the shape of the workpiece W, this step S5 includes processing for determining the center axis of the workpiece W from a plurality of center positions determined for the same workpiece W. That is, the central axis of the cylindrical workpiece W is calculated using the normal line and the radius information. When the workpiece W is a sphere, basically only one center position (center point) is calculated, so the center axis is not calculated.

  After execution of the position calculation process, the CPU 110 of the image processing apparatus 100 executes an output process (step S6). This output process is a process of outputting information on the center position of the workpiece W determined in step S5 to the robot controller 200 (FIG. 2) or the like. The output center position information includes a coordinate value in a three-dimensional space and a value indicating a vector. In many cases, the coordinate system acquired by the image processing apparatus 100 through three-dimensional measurement and the coordinate system used in the processing of the robot 300 do not coincide with each other. It is preferable to convert and output. Further, in a situation where a plurality of workpieces W are stacked, when information on the center position is acquired for the plurality of workpieces W, information for identifying each workpiece W and the corresponding center position are obtained. Information may be output. Or you may output only the information about the workpiece | work W located in the uppermost part, ie, the calculated center position most likely. In the gripping system SYS as shown in FIG. 1, the robot 300 grips the uppermost workpiece W as the information on the center position of the workpiece W is output, so that the workpiece W is removed. The measurement process may be executed again.

  In the flowchart shown in FIG. 3, the segmentation process in step S2 and the three-dimensional measurement process in step S3 may be executed in parallel.

<< D. Preprocessing"
First, details of the preprocessing (step S1) shown in FIG. 3 will be described. As described above, the preprocessing is a process for removing only the influence of light incident on the imaging device 8 from the ambient light and observing only light (reflected light) generated when the illumination light from the projector 7 is reflected by the workpiece W. It is. In order to remove the influence of such environmental light, a difference is obtained from a difference between an image acquired without illuminating illumination light from the projector 7 and an image acquired with illuminating illumination light from the projector 7. An image (shaded image) is generated.

  FIG. 4 is a diagram showing a processing example of the preprocessing according to the embodiment of the present invention. Referring to FIG. 4, in the image processing method according to the present embodiment, (c) a shading image is generated in order to eliminate the influence of disturbance light (environment light) from around the workpiece W. More specifically, an image ((a) white pattern projection + environment light image shown in FIG. 4) captured in a state where a white pattern is emitted as illumination light from the projector 7 is acquired. The white pattern means a pattern in which the light and shade in the irradiation surface can be regarded as uniform, and the range and length of the spectrum in the irradiation surface are not particularly limited. Therefore, a pattern in which the irradiation surface is uniformed with red or blue light may be used.

  In addition, an image ((b) ambient light image shown in FIG. 4) captured in a state where no illumination light is emitted from the projector 7 is acquired.

  Then, the shading image is generated by calculating the difference between these images. The calculation of the difference between the images includes a process of subtracting color information (for example, R, G, and B gray values) for the corresponding pixels between the two images.

  Such preprocessing is particularly effective in a situation where ambient light that changes with time, such as sunlight, is incident on the field of view of the imaging device 8. In other words, if the ambient light is stable and relatively small with respect to the illumination light emitted by the projector 7, such pre-processing may be omitted. In this case, the subsequent processing may be performed only with (a) white pattern projection + ambient light image shown in FIG.

<< E. Segmentation process >>
Next, the details of the segmentation process (step S2) shown in FIG. 3 will be described. As described above, the segmentation process is a process for extracting an area (cylindrical area) indicating (a part of) the workpiece W included in the input image. As the input image, a shading image obtained by the above-described processing is used.

  As such a segmentation process, a cylindrical region is extracted by dividing an input shading image (input image) according to a predetermined rule. As a method for extracting such a cylindrical region, various known methods can be employed. As a simple method, there may be mentioned a method of specifying an area in which the brightness (luminance or brightness) has almost the same value in the input image.

  In the present embodiment, the “Normalized Cuts” method is adopted as an example of such segmentation processing. For details of this "Normalized Cuts" method, refer to "Jianbo Shi and Jitendra Malik," Normalized Cuts and Image Segmentation ", IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 22, No. 8, August 2000", etc. I want.

  FIG. 5 is a diagram illustrating a processing example of the shading processing according to the embodiment of the present invention. By applying the above-described “Normalized Cuts” method to (a) input image shown in FIG. 5, a result image such as (b) segmentation result is generated. When a region is extracted by applying a condition according to the shape of the workpiece W to such a segmentation result, a result image such as (c) a cylindrical region extraction result is generated. Basically, subsequent processing is performed on the extracted cylindrical region. The range of the extracted cylindrical region is specified using the coordinate value in the two-dimensional coordinate system on the input image.

<< F. Three-dimensional measurement process >>
Next, details of the three-dimensional measurement process (step S3) shown in FIG. 3 will be described. As described above, the three-dimensional measurement process is a process for acquiring height information about the cylindrical region extracted by the segmentation process.

  As a process for acquiring such height information, a three-dimensional coordinate value for each part of the workpiece W in a bulk state is calculated using a known three-dimensional measurement process. As described above, in the present embodiment, the phase shift method is employed as a specific example of the three-dimensional measurement process. This phase shift method is a method using an image (sine wave projection image) captured in a state where illumination light having a pattern in which shading is changed in a sine wave pattern on the irradiation surface is irradiated.

  More specifically, a plurality of patterns having different shade change periods and phases in the irradiation surface are prepared, and images captured when the respective patterns are irradiated are acquired as sinusoidal projection image groups. Then, height information (three-dimensional coordinate values) of each part is calculated based on a change in brightness (luminance and brightness) of the corresponding part between the sine wave projection images.

  FIG. 6 is a diagram showing an example of a pattern used in the three-dimensional measurement process according to the embodiment of the present invention. FIG. 7 is a diagram illustrating a processing example of the three-dimensional measurement processing according to the embodiment of the present invention.

  As shown in FIGS. 6A to 6D, in the phase shift method, a plurality of patterns are used in which the grayscale pattern in the irradiation surface has different change periods and phases. By using a plurality of different shade patterns in this way, it is possible to perform three-dimensional measurement of the measurement object while expanding the measurement range.

  By such a phase shift method, (b) a height image can be generated from the (a) sine wave projection image group of FIG. FIG. 7 illustrates a height image in which height information is represented by color information (shading change) for easy understanding. In the phase shift method, a cubic including the height information of each part is illustrated. The original coordinate value is acquired.

  As described above, in the three-dimensional measurement processing according to the present embodiment, the projector 7 irradiates the workpiece W with a plurality of different shade patterns from a predetermined direction, and each image is captured by the imaging device 8 at each irradiation. Height information is acquired using a plurality of images.

<< G. Normal calculation processing >>
Next, details of the normal calculation processing (step S4) shown in FIG. 3 will be described. As described above, the normal calculation processing is processing for calculating a normal (normal vector) to the surface of the workpiece W.

  In the image processing method according to the present embodiment, higher-speed processing can be realized by calculating the center position of the workpiece W from the relationship between the normal and the radius of the workpiece W. Furthermore, from the viewpoint of increasing the measurement accuracy, the normal direction determination method can be made different depending on the optical reflection characteristics (specular reflection / diffuse reflection) of the workpiece W, as will be described later.

  Hereinafter, in addition to the basic processing procedure for calculating the normal line, a process for changing the method for determining the normal direction depending on the optical reflection characteristics of the workpiece W will be described.

(G1: Normal calculation principle)
First, the principle for calculating the normal in the image processing method according to the present embodiment will be described.

  FIG. 8 is a diagram for explaining the principle of normal calculation processing according to the present embodiment. FIG. 8 illustrates a case where the surface of the workpiece W has a specular reflection characteristic and a case where the surface of the work W has a diffuse reflection characteristic.

(I) When it has specular reflection characteristics First, the case where the surface of the workpiece W has specular reflection characteristics will be described. When the workpiece W has specular reflection characteristics, the light incident on the surface of the workpiece W is reflected at the same angle as the angle (incident angle) defined with respect to the normal of the incident position. Therefore, when viewed from a certain position on the surface of the workpiece W, most of the illumination light from the projector 7 is reflected in a direction having the same angle as the incident angle.

  The brightest position in the input image indicating the light incident on the imaging device 8 corresponds to the surface position of the workpiece W on which the illumination light from the projector 7 is incident. That is, the maximum luminance observation position in the input image shown in FIG. 8 can be acquired based on the luminance information of the input image captured by the imaging device 8. The three-dimensional coordinate value indicating the maximum luminance observation position in the input image shown in FIG. 8 can also be obtained from the result of the above-described three-dimensional measurement process.

  Since the relative position and orientation relationship between the projector 7 and the imaging device 8 is known, a vector (hereinafter referred to as a vector) connecting the projector 7 and the three-dimensional coordinate value indicating the observation position of the maximum luminance in the input image shown in FIG. , Also referred to as “light source vector l”) and a vector (hereinafter also referred to as “line-of-sight vector v”) connecting the imaging device 8 and the three-dimensional coordinate value indicating the maximum luminance observation position in the input image shown in FIG. ) Can be calculated.

  The bisected angle between the light source vector l and the line-of-sight vector v thus calculated is the normal direction (normal vector n).

  That is, in the normal calculation processing (step S4) shown in FIG. 3, when the surface of the workpiece W is specularly reflected, the tertiary determined from the position of the portion where the brightness is maximum and the corresponding height information. A light source vector l (first vector) that passes through the original coordinate point and extends toward the light source (projector 7) of the illumination light, and a line-of-sight vector that passes through the three-dimensional coordinate point and extends in the direction of the imaging device 8 A vector corresponding to a bisecting angle with v is calculated as a normal vector n.

(Ii) Case of having diffuse reflection characteristics Next, a case where the surface of the workpiece W has diffuse reflection characteristics will be described. When the workpiece W has diffuse reflection characteristics, the light incident on the surface of the workpiece W is basically diffusely reflected in all directions. Therefore, when viewed from a certain position on the surface of the workpiece W, the illumination light from the projector 7 is reflected most in the normal direction of the position.

  The brightest position in the input image indicating the light incident on the imaging device 8 corresponds to the surface position of the workpiece W on which the illumination light from the projector 7 is incident. That is, the maximum luminance observation position in the input image shown in FIG. 8 can be acquired based on the luminance information of the input image captured by the imaging device 8. The three-dimensional coordinate value indicating the maximum luminance observation position in the input image shown in FIG. 8 can also be obtained from the result of the above-described three-dimensional measurement process.

  Since the relative position and orientation relationship between the projector 7 and the imaging device 8 is known, a vector (light source) connecting the projector 7 and the three-dimensional coordinate value indicating the observation position of the maximum luminance in the input image shown in FIG. If the vector l) is calculated, the calculated light source vector l matches the normal direction (normal vector n).

  That is, in the normal calculation processing (step S4) shown in FIG. 3, when the surface of the workpiece W is diffusely reflected, the tertiary determined from the position of the portion where the brightness is maximum and the corresponding height information. A light source vector l passing through the original coordinate point and extending toward the light source (projector 7) of the illumination light is calculated as a normal vector n.

  When the workpiece W has a cylindrical shape, the illumination light from the projector 7 is incident on a plurality of positions, so that a plurality of “brightest positions in the input image” are detected. Accordingly, in any of the above cases, when the workpiece W has a cylindrical shape, a plurality of normal vectors n are calculated.

(G2: Normal calculation processing flow)
FIG. 9 is a flowchart showing a more detailed procedure of normal calculation processing executed in the image processing apparatus 100 according to the present embodiment. The processing procedure shown in FIG. 9 shows the processing content of step S3 shown in FIG. 3 in more detail.

  Referring to FIG. 9, the CPU 110 of the image processing apparatus 100 executes an extraction process of a portion where the brightness is maximized in the input image (step S41). The extraction process of the portion where the brightness is maximum is realized by image processing on the input image. The contents of this image processing will be described later. By the processing in step S41, a portion where the luminance is maximum in the input image (hereinafter also referred to as “luminance maximum portion”) is extracted.

  Subsequently, the CPU 110 of the image processing apparatus 100 acquires the reflection characteristics of the target work W (step S42). The reflection characteristics of the workpiece W may be input in advance by the user, or may be input as information about the workpiece W from an external device (such as a process management server) to the image processing apparatus 100.

  Subsequently, the CPU 110 of the image processing apparatus 100 executes a light ray vector (generic name for the line-of-sight vector v and the light source vector 1 shown in FIG. 8) (step S43). This ray vector calculation process is a three-dimensional process executed in a three-dimensional coordinate system. In step S42, the CPU 110 of the image processing apparatus 100 calculates a light vector (line-of-sight vector v and light source vector l) corresponding to the luminance maximum extracted in step S41. Note that the type of the calculated light vector changes depending on the reflection characteristic of the workpiece W input in step S42. Details of the light ray vector calculation process will be described later.

  Finally, the CPU 110 of the image processing apparatus 100 executes normal calculation processing (step S44). In this normal calculation processing, normal calculation processing is executed in accordance with the reflection characteristics.

  As described above, when the normal line (normal vector n) is calculated, the process of step S5 shown in FIG. 3 is subsequently performed.

(G3: Extraction of the part where the brightness is maximized)
Next, details of the process of step S41 shown in FIG. 9 (a process of extracting a portion where the brightness is maximized) will be described. In step S41 shown in FIG. 9, the luminance maximum portion is extracted from the cylindrical region extracted by the segmentation process (step S2 in FIG. 3). In the process of step S41, basically, the brightest pixel in the cylindrical region and its peripheral region are extracted as the maximum luminance portion.

  FIG. 10 is a diagram illustrating a processing example of processing for extracting a portion where the brightness is maximum according to the present embodiment. Referring to FIG. 10, first, a target from which a luminance maximum portion is extracted from a cylindrical region included in the segmentation result is determined. In the case where the maximum luminance portion is extracted from each of the plurality of cylindrical regions, the procedure described below is performed on each cylindrical region.

  More specifically, the following procedures (1) to (6) are repeatedly executed for the number of target cylindrical regions.

(1) Determination of a cylindrical region from which a luminance maximum portion is extracted from cylindrical regions included in the segmentation result (2) Determination of a scanning direction from a ratio of lengths of sides defining the cylindrical region (3) Each input image Acquisition of luminance profile by operating in the scanning direction (4) Extraction and recording of coordinate values of area (maximum value and surrounding area) where luminance is higher than threshold value in luminance profile (5) (3) and (4) Processing is repeatedly performed for all cylindrical regions. (6) The maximum coordinate values in each luminance profile are integrated to extract the luminance maximum portion in the cylindrical region. FIG. 10 includes (a) the segmentation result. The scanning direction is determined for one cylindrical region, and the luminance change of the cylindrical region acquired along the scanning direction is indicated ((b) luminance change of the cylindrical region). In order to facilitate understanding, FIG. 10 illustrates a height image in which the absolute value of the luminance in the cylindrical region is shown by color information (change in shading). Information such as a coordinate value that takes a value and a maximum value is extracted.

(G4: ray vector calculation process)
Next, details of the light vector calculation processing shown in step S43 shown in FIG. 9 will be described. In step S43 shown in FIG. 9, first, a light vector in a three-dimensional space is calculated (line-of-sight vector v and v) with reference to the coordinate values (x, y, z) of the luminance maximum extracted in the process of step S41. A light source vector l) is calculated.

  FIG. 11 is a diagram for explaining the light vector calculation processing according to the present embodiment. With reference to FIG. 11A, the line-of-sight vector v is calculated from the coordinate value (x, y, z) of the luminance maximum portion on the input image and the coordinate value (xc, yc, zc) of the imaging device 8. The This calculation process is executed using calibration data acquired in advance.

  That is, the line-of-sight vector v (v1, v2, v3) is calculated according to the following equation.

Line-of-sight vector v (v1, v2, v3) = (x-xc, y-yc, z-zc)
Subsequently, referring to FIG. 11B, the light source vector l is obtained from the coordinate values (x, y, z) of the luminance maximum portion on the input image and the coordinate values (xp, yp, zp) of the projector 7. Calculated. This calculation process is executed using calibration data acquired in advance.

  That is, the light source vector l (l1, l2, l3) is calculated according to the following equation.

Light source vector l (l1, l2, l3) = (x-xp, y-yp, z-zp)
The line-of-sight vector v and the light source vector l are preferably standardized for convenience in the calculation process of the normal vector n described later.

(G5: normal calculation processing)
Next, details of the normal calculation processing shown in step S44 shown in FIG. 9 will be described. In step S44 shown in FIG. 9, the calculation method of the normal direction (normal vector n) is varied depending on the reflection characteristics on the surface of the workpiece W.

  FIG. 12 is a diagram for explaining the normal calculation processing according to the present embodiment. FIG. 12A shows a processing example when the surface of the workpiece W has specular reflection characteristics, and FIG. 12B shows a processing example when the surface of the workpiece W has diffuse reflection characteristics.

  Referring to FIG. 12A, when the surface of the workpiece W has specular reflection characteristics, specularly reflected light is generated at a portion where the brightness is maximum (the luminance maximum portion). Therefore, as described above, the normal vector n on the surface of the workpiece W is calculated according to the following equation using the line-of-sight vector v and the light source vector l.

Normal vector n = (line-of-sight vector v + light source vector l) / 2
On the other hand, referring to FIG. 12B, when the surface of the workpiece W has a diffuse reflection characteristic, diffuse reflected light is generated in a portion where the brightness is maximum (luminance maximum portion). Therefore, as described above, the normal vector n on the surface of the workpiece W coincides with the light source vector l and is calculated according to the following equation.

Normal vector n = light source vector l
Through the processing as described above, one or more normal vectors n are calculated.

<< H. Position calculation process >>
Next, details of the position calculation process (step S5) shown in FIG. 3 will be described. As described above, the position calculation process is a process for calculating the position and orientation of the workpiece W. In the present embodiment, since the radius information (radius) of the workpiece W is known, from the normal (normal vector) calculated by the above-described processing and the corresponding three-dimensional coordinate point of the maximum luminance, The center position (center axis) of the cylindrical portion of the workpiece W is calculated.

FIG. 13 is a diagram for explaining the position calculation processing according to the present embodiment.
As shown in FIG. 13A, the three-dimensional coordinate point (x, y, z) of the luminance maximum portion and the corresponding normal vector n are calculated, and radius information (radius r) in the cylindrical cross section of the workpiece W is calculated. Is known, the center position of the workpiece W can be uniquely specified from the geometric relationship.

  Further, as shown in FIG. 13B, a plurality of luminance maximum portions and corresponding normal vectors n1, n2, n3,... Are calculated for the same workpiece W. The center position of the workpiece W can be calculated. Furthermore, the center axis of the cylindrical portion of the workpiece W can be determined using a plurality of center positions extracted from the workpiece W. As a method for determining the central axis, a known fitting method can be employed.

<< I. Measurement example >>
FIG. 14 is a diagram showing an example of a recognition result generated by image processing according to the embodiment of the present invention. The recognition result shown in FIG. 14 is obtained from an input image obtained by imaging a state in which a plurality of workpieces W (a large number of bolts as shown in FIG. 4) are stacked. Note that the line shown in FIG. 14 indicates the central axis of the workpiece W calculated by the above-described processing, and the circle drawn on the periphery of the workpiece W is determined when the corresponding normal vector is calculated. Indicates the radial position (outer shape).

  As described above, according to the image processing method according to the present embodiment, the position of the workpiece W arranged in an arbitrary direction and its central axis can be determined in a relatively short time.

<< J. Modifications >>
In the above-described example, the processing example in the case where the image processing method according to the present invention is mainly applied to the cylindrical workpiece W has been described. However, as described above, the processing can be similarly applied to the spherical workpiece W. In this case, in principle, since the center position of the workpiece W is one, only one normal vector needs to be extracted from one workpiece W.

<< K. advantage"
According to the image processing apparatus according to the present embodiment, the result of image processing on a two-dimensional image and the height information obtained by three-dimensionally measuring the same workpiece W can be used with a smaller amount of processing. The center position can be calculated.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

7 Projector 8 Imaging Device 100 Image Processing Device 102 Display 104 Mouse 106 Memory Card 110 CPU
112 main memory 114 hard disk 116 camera interface 116a image buffer 118 input interface 120 display controller 122 projector interface 124 communication interface 126 data reader / writer 128 bus 200 robot controller 300 robot SYS gripping system W work

Claims (9)

A method for calculating a central position on one or more circular sections of a cylinder or sphere, with respect to a detection object that at least partially includes a cylinder or sphere having a known radius and whose surface is specularly reflected, comprising:
A line of sight in a second direction that is in the same plane as the first direction and is different from the first direction in a state in which illumination light is irradiated from at least one of the detection objects from the first direction. Obtaining an input image by imaging using an imaging device arranged with
Extracting a region including at least a part of a cylindrical portion or a spherical portion of the detection target object from the input image based on brightness information in the input image and information on the shape of the detection target;
The at least one detection object is irradiated with a plurality of different shade patterns from the first direction, and the plurality of images respectively captured by the imaging device at the time of each irradiation are used for the extraction. Obtaining height information about the region;
For each of a plurality of cylindrical cross sections or a spherical cross section included in the extracted region , a portion where the brightness on the outer periphery of the cross section is maximized is specified, and a normal direction of the portion where the brightness is maximized is determined. And steps to
The determined normal direction of each of the plurality of cylindrical sections or the normal direction of the spherical section, the known radius, and the position of each portion at which the brightness of the plurality of cylindrical sections is maximized or the Determining the center position of each cylindrical portion of the detection target object or the center position of the detection target object from the position of the portion where the brightness of the sphere cross-section is maximized and the corresponding height information. ,
The step of determining the normal direction includes a first direction extending through the three-dimensional coordinate point determined from the position of the portion where the brightness is maximized and the corresponding height information and extending toward the light source of the illumination light. Calculating a vector corresponding to a bisector between a vector and a second vector passing through the three-dimensional coordinate point and extending in the direction of the imaging device, as a normal vector. A method for calculating a center position on one or more circular sections of a cylinder or a sphere with respect to a detection object whose surface is diffusely reflected, the cylinder or sphere having a known radius at least in part.
In a state where illumination light is irradiated from at least one of the detection objects from the first direction, an image is picked up using an imaging device arranged with a line of sight in a second direction different from the first direction. To obtain an input image,
Extracting a region including at least a part of a cylindrical portion or a spherical portion of the detection target object from the input image based on brightness information in the input image and information on the shape of the detection target;
The at least one detection object is irradiated with a plurality of different shade patterns from the first direction, and the plurality of images respectively captured by the imaging device at the time of each irradiation are used for the extraction. Obtaining height information about the region;
For each of a plurality of cylindrical cross sections or a spherical cross section included in the extracted region , a portion where the brightness on the outer periphery of the cross section is maximized is specified, and a normal direction of the portion where the brightness is maximized is determined. And steps to
The determined normal direction of each of the plurality of cylindrical sections or the normal direction of the spherical section, the known radius, and the position of each portion at which the brightness of the plurality of cylindrical sections is maximized or the Determining the center position of each cylindrical portion of the detection target object or the center position of the detection target object from the position of the portion where the brightness of the sphere cross-section is maximized and the corresponding height information. ,
In the step of determining the normal direction, a vector extending through the three-dimensional coordinate point determined from the position of the portion where the brightness is maximized and the corresponding height information and extending toward the light source of the illumination light is used. A method comprising calculating as a line vector. Acquiring the input image, the first image acquired by the imaging device in a state of not irradiated with illumination light from the first direction to at least one of the detection object, wherein the at least one detection comprising the step of generating the input image from the difference between the second image acquired by the imaging device in a state in which illumination light is irradiated from the first direction to the object, according to claim 1 or 2 the method of. The method according to any one of claims 1 to 3, wherein the step of determining the center position includes a step of determining a center axis of the detection target object from a center position of each cylindrical portion of the detection target object. .   The method according to claim 1, further comprising a step of outputting the determined center position of the detection object to a controller of a robot that holds the detection object. An apparatus for calculating a central position on one or more circular sections of a cylinder or a sphere with respect to a detection target that includes a cylinder or a sphere having a known radius at least in part and whose surface is specularly reflected.
An interface connected to an imaging device and acquiring an input image obtained by imaging at least one of the detection objects, wherein the input image irradiates the at least one detection object with illumination light from a first direction. In such a state, it is acquired by imaging using an imaging device that is in the same plane as the first direction and is arranged with a line of sight in a second direction different from the first direction,
Means for extracting an area including at least a part of a cylindrical portion or a sphere portion of the detection target object from the input image based on brightness information in the input image and information on a shape of the detection target;
The at least one detection object is irradiated with a plurality of different shade patterns from the first direction, and the plurality of images respectively captured by the imaging device at the time of each irradiation are used for the extraction. Means for obtaining height information about the region;
For each of a plurality of cylindrical cross sections or a spherical cross section included in the extracted region , a portion where the brightness on the outer periphery of the cross section is maximized is specified, and a normal direction of the portion where the brightness is maximized is determined. Means to
The determined normal direction of each of the plurality of cylindrical sections or the normal direction of the spherical section, the known radius, and the position of each portion at which the brightness of the plurality of cylindrical sections is maximized or the Means for determining the center position of each cylindrical portion of the detection target object or the center position of the detection target object from the position of the portion where the brightness of the sphere cross-section is maximized and the corresponding height information. ,
The means for determining the normal direction includes a first direction extending through a three-dimensional coordinate point determined from the position of the portion where the brightness is maximized and the corresponding height information and extending toward the light source of the illumination light. An apparatus that calculates, as a normal vector, a vector corresponding to a bisector between a vector and a second vector that passes through the three-dimensional coordinate point and extends in the direction of the imaging apparatus. An apparatus for calculating a central position on one or more circular cross sections of a cylinder or a sphere with respect to a detection object whose surface is diffusely reflected, the cylinder or sphere having a known radius at least in part.
An interface connected to an imaging device and acquiring an input image obtained by imaging at least one of the detection objects, wherein the input image irradiates the at least one detection object with illumination light from a first direction. In such a state, it is acquired by imaging using an imaging device arranged with a line of sight in a second direction different from the first direction,
Means for extracting an area including at least a part of a cylindrical portion or a sphere portion of the detection target object from the input image based on brightness information in the input image and information on a shape of the detection target;
The at least one detection object is irradiated with a plurality of different shade patterns from the first direction, and the plurality of images respectively captured by the imaging device at the time of each irradiation are used for the extraction. Means for obtaining height information about the region;
For each of a plurality of cylindrical cross sections or a spherical cross section included in the extracted region , a portion where the brightness on the outer periphery of the cross section is maximized is specified, and a normal direction of the portion where the brightness is maximized is determined. Means to
The determined normal direction of each of the plurality of cylindrical sections or the normal direction of the spherical section, the known radius, and the position of each portion at which the brightness of the plurality of cylindrical sections is maximized or the Means for determining the center position of each cylindrical portion of the detection target object or the center position of the detection target object from the position of the portion where the brightness of the sphere cross-section is maximized and the corresponding height information. ,
The means for determining the normal direction is a vector that passes through a three-dimensional coordinate point determined from the position of the portion where the brightness is maximum and the corresponding height information and extends toward the light source of the illumination light. Device that calculates as a line vector. A program for calculating a central position on one or more circular sections of a cylinder or a sphere with respect to a detection target that includes a cylinder or a sphere having a known radius at least in part and whose surface is specularly reflected. The program is stored on the computer
A line of sight in a second direction that is in the same plane as the first direction and is different from the first direction in a state in which illumination light is irradiated from at least one of the detection objects from the first direction. Obtaining an input image by imaging using an imaging device arranged with
Extracting a region including at least a part of a cylindrical portion or a spherical portion of the detection target object from the input image based on brightness information in the input image and information on the shape of the detection target;
The at least one detection object is irradiated with a plurality of different shade patterns from the first direction, and the plurality of images respectively captured by the imaging device at the time of each irradiation are used for the extraction. Obtaining height information about the region;
For each of a plurality of cylindrical cross sections or a spherical cross section included in the extracted region , a portion where the brightness on the outer periphery of the cross section is maximized is specified, and a normal direction of the portion where the brightness is maximized is determined. And steps to
The determined normal direction of each of the plurality of cylindrical sections or the normal direction of the spherical section, the known radius, and the position of each portion at which the brightness of the plurality of cylindrical sections is maximized or the The step of determining the center position of each cylindrical portion of the detection target object or the center position of the detection target object from the position of the portion where the brightness of the sphere cross section becomes maximum and the corresponding height information is executed. Let
The step of determining the normal direction includes a first direction extending through the three-dimensional coordinate point determined from the position of the portion where the brightness is maximized and the corresponding height information and extending toward the light source of the illumination light. A program comprising a step of calculating a vector corresponding to a bisector between a vector and a second vector passing through the three-dimensional coordinate point and extending in the direction of the imaging device as a normal vector. A program for calculating a center position on one or more circular cross sections of a cylinder or a sphere with respect to a detection object whose surface is diffusely reflected and including a cylinder or a sphere having a known radius at least in part. The program is stored on the computer
In a state where illumination light is irradiated from at least one of the detection objects from the first direction, an image is picked up using an imaging device arranged with a line of sight in a second direction different from the first direction. To obtain an input image,
Extracting a region including at least a part of a cylindrical portion or a spherical portion of the detection target object from the input image based on brightness information in the input image and information on the shape of the detection target;
The at least one detection object is irradiated with a plurality of different shade patterns from the first direction, and the plurality of images respectively captured by the imaging device at the time of each irradiation are used for the extraction. Obtaining height information about the region;
For each of a plurality of cylindrical cross sections or a spherical cross section included in the extracted region , a portion where the brightness on the outer periphery of the cross section is maximized is specified, and a normal direction of the portion where the brightness is maximized is determined. And steps to
The determined normal direction of each of the plurality of cylindrical sections or the normal direction of the spherical section, the known radius, and the position of each portion at which the brightness of the plurality of cylindrical sections is maximized or the The step of determining the center position of each cylindrical portion of the detection target object or the center position of the detection target object from the position of the portion where the brightness of the sphere cross section becomes maximum and the corresponding height information is executed. Let
In the step of determining the normal direction, a vector extending through the three-dimensional coordinate point determined from the position of the portion where the brightness is maximized and the corresponding height information and extending toward the light source of the illumination light is used. A program comprising the step of calculating as a line vector.