JP6884398B2 - Judgment device for transported objects and transport system using this - Google Patents

Description

The present invention relates to a device for determining a transported object and a transport system using the device.

Conventionally, a device for determining a transported object has been used, which detects a transported object to be transported along a predetermined transport path and determines the presence / absence, posture, quality, etc. of the transported object according to the detection mode. Such a determination device is used for grasping the conveyed state, selecting the processing operation of the conveyed object processing unit for aligning, sorting, and the like based on the result of determining the conveyed object. ..

In the prior art, the most common method for determining a transported object is a detection sensor consisting of an optical detector or the like that detects the arrival of the transported object and outputs a detection signal, or outputs the detection signal or the like. A determination method using a transmission sensor or the like including an optical detector or the like for determining the shape of the conveyed object when the conveyed object reaches a predetermined position by timing is known.

On the other hand, there is also known an image processing type determination method in which image data of a transported object is acquired by using an image pickup device such as a line sensor or a camera, and the image data is processed to determine the transported object. By using a line sensor, the position of the transported object can be grasped, and the shape of the transported object can be reproduced using multiple line data, or the posture and shape of the transported object can be determined by image processing such as template matching from the image data taken by the camera. A method of recognizing is used (see, for example, Patent Document 1 below). Among these methods, there is also a technique for identifying an object by using a neural network for inputting a feature amount extracted in advance from image data (see, for example, Patent Document 2 below).

Japanese Unexamined Patent Publication No. 6-137837 Japanese Unexamined Patent Publication No. 5-128262

However, due to the recent miniaturization of the transported object and the increase in the transport speed, it has become difficult to realize the processing speed corresponding to the increased transport speed while maintaining the determination accuracy of the transported object by the above-mentioned conventional technology. There is a problem that it is.

Therefore, an object of the present invention is to solve the above-mentioned problems, and an object of the present invention is to provide a conveyed object determination device capable of increasing the speed of determination processing while ensuring the determination accuracy of the conveyed object.

In order to solve the above problems, the method for determining a transported object of the present invention is based on the image data of a limited area set in a range in which only a part of the transported object on the transport path can be included, not the whole. A method for determining a transported object for determining an object, which is a data acquisition step of repeatedly acquiring image data in the limited region at an acquisition timing having a time interval at which image data in the limited region can be formed for each transported object. A data classification step of classifying the distribution pattern of the pixel values of the image data in the limited region into one of a plurality of classification categories for each acquisition timing and obtaining the classification result of the image data in the limited region is provided. It is characterized by.

In this case, it is preferable that the acquisition timing is set at a time interval in which the image data of the limited region can be formed a plurality of times for each transported object. Here, the number of image data in the limited region that can be formed for each transported object is preferably in the range of 2 to 10, and particularly preferably in the range of 3 to 7.

In the present invention, it is preferable to further include a determination processing step for obtaining determination results such as the total number of passages, posture, and quality of the transported object based on a plurality of classification results. At this time, it is desirable that the plurality of classification results are the classification results of the image data of the limited region acquired at the plurality of acquisition timings. Further, when a plurality of the limited areas are set, at least a part of the plurality of classification results may be a classification result of image data of the plurality of the limited areas acquired at a single acquisition timing. Good.

Further, the conveyed object determination device of the present invention determines the conveyed object based on the image data of the limited area set in the range where only a part of the conveyed object on the conveyed path can be included. A data acquisition means for repeatedly acquiring image data in the limited region at an acquisition timing having a time interval capable of forming image data in the limited region for each transported object, and a data acquisition means for each acquisition timing. It is characterized by including a data classification means for classifying the distribution pattern of pixel values of the image data in the limited region into any of a plurality of classification categories and outputting the classification result of the image data in the limited region.

In this case, it is preferable that the acquisition timing is set at a time interval in which the image data of the limited region can be formed a plurality of times for each transported object. Here, the number of image data in the limited region that can be formed for each transported object is preferably in the range of 2 to 10, and particularly preferably in the range of 3 to 7.

In the present invention, it is preferable to further provide a determination processing means for obtaining determination results such as the total number of passages, posture, quality, etc. of the transported object based on a plurality of classification results. At this time, it is desirable that the plurality of classification results are the classification results of the image data of the limited region acquired at the plurality of acquisition timings. That is, it is desirable to further provide a determination processing means for outputting determination results such as the total number of passages, posture, and quality of the transported object based on the classification results of the image data of the limited area acquired at the plurality of acquisition timings. .. Further, when a plurality of the limited areas are set, at least a part of the plurality of classification results may be a classification result of image data of the plurality of the limited areas acquired at a single acquisition timing. Good.

According to the present invention, by using the image data in the limited region corresponding to only a part of the transported object by the data acquisition means, it is not necessary to process the image data beyond the range including the entire transported object. Since the amount of data to be processed can be limited, the processing speed can be increased. Further, the distribution pattern of the pixel value of the image data in the limited area is classified into a plurality of classification categories by the data classification means, and the image information regarding a part of the transported object in the limited area is reflected in the classification result, which is necessary for the determination process. The amount of data can be further reduced. Further, by acquiring the image data of the limited area for each transported object at a time interval that can be formed a plurality of times, it becomes possible to more accurately and easily grasp the information on the transported state of the transported object, so that the determination accuracy can be improved. Can be done. Further, by obtaining the determination result of the transported object by the determination processing means using the plurality of classification results, the transported object can be reliably determined in various modes only by simple information processing. In particular, by using the classification results of the image data of the limited area acquired at a plurality of acquisition timings, the determination mode can be further expanded.

In the present invention, the data classification means is preferably a classifier including a neural network. Further, it is more desirable that the data classification means is composed of a trained model. At this time, the data classification means uses the learning data in which the classification category determined based on the positional relationship of the limited region with respect to the transported object at the acquisition timing for each image data is labeled with respect to the distribution pattern. It is preferably learned. The neural network has an input layer, an intermediate layer, and an output layer, the input layer includes a number of inputs (number of units) matching the number of pixels of the image data, and the output layer is the classification category to be classified. It is desirable to have a number of outputs (number of units) that matches the number of. Further, it is desirable that the neural network includes a plurality of the intermediate layers. Further, it is desirable that the intermediate layer (intermediate first layer) directly connected to the input layer includes a number of units (artificial neurons) larger than the number of the inputs. In particular, it is desirable that the neural network is a fully connected type.

In the present invention, the limited region is preferably a linear or strip-shaped region. At this time, it is desirable that the limited region extends in a direction intersecting the transport direction of the transport path. Further, it is particularly desirable that the limited region extends linearly. Further, it is desirable that the limited region extends in a direction orthogonal to the transport direction. In particular, it is desirable that the limited region is a region composed of pixels arranged in a row. The limited area may extend along the transport direction. The linear or strip-shaped limited region preferably overlaps with the passing range of the transported object on the transport path at a ratio of 80% or more. When classifying the presence or absence of a transported object, the linear or strip-shaped limited region may be configured to be limited to a region narrower than the passing range. Further, it is desirable that the linear or strip-shaped limited region is provided with end portions protruding from the passage range on both sides of the passage range. In this case, it is desirable that the limited region is a region of 120% or less of the passing range.

In the present invention, the pixel value is preferably a luminance value (brightness value). However, it is also possible to use other pixel values such as the hue and saturation of the pixels depending on the appearance of the transported object and the content of the determination.

In the present invention, the data classification means is not particularly limited as long as the image data can be classified into the plurality of separation categories. However, the data classification means is preferably a machine learning classifier, and is preferably a classifier having a neural network as described above. Regarding the physical configuration, the data classification means or the classifier can be configured by executing a program by various computers (those using a general-purpose arithmetic processing unit (CPU)) such as a personal computer. In this case, it is preferably executed by a computer equipped with a GPU (Graphics Processing Unit). Further, the data classification means or the classifier uses an electronic circuit (chip) corresponding to specific data processing such as a PLD (programmable logic device) such as an FPGA (field-programmable gate array) or a DSP (digital signal processor). It can also be configured.

In the present invention, it is preferable to provide a display means (display device) for displaying the classification result or the determination result, or a recording means (recording device) for recording the classification result or the determination result. Further, it is possible to provide a transported object control means for removing the transported object from the transport path or changing (reversing, rolling over) the posture of the transported object based on the classification result or the determination result. desirable.

According to the present invention, by classifying the image data in a limited area corresponding to only a part of the transported object into a plurality of classification categories, the determined object can be determined accurately and the determination process can be speeded up. A determination device can be provided. In particular, by performing the determination process based on a plurality of classification results, the determination speed can be further shortened while ensuring the determination accuracy, and the determination contents can be widely set.

It is a schematic block diagram which shows the whole structure of the transport system including the structural example of the embodiment of the transport object determination apparatus which concerns on this invention. It is a top view which shows the transported object on the transport path which is the judgment target of 1st Embodiment. A plan view (a) showing the imaging range by the camera, an enlarged explanatory view (b) showing the limited areas CR1 and CR2 of the first embodiment within the imaging range, and a dimensional relationship between the transported object and the limited area are shown. It is explanatory drawing (c). Explanatory drawing (a) showing the positional relationship of the pixel value distribution patterns Ds1 to Ds7 of the image data D of the limited regions CR1 and CR2 of the first embodiment with respect to the transported object, and the corresponding patterns of the examples Ds1 to Ds7 of each distribution pattern. It is a graph (b) to (h) shown together with the type PT1 to PT7. It is a schematic block diagram (a) which shows the structure of the classifier of 1st Embodiment, and the schematic flowchart (b) which shows an example of the procedure of the learning process of this classifier. It is a schematic flowchart which shows the procedure of the transported goods determination operation including the classification processing process by the classifier (data classification means) of 1st Embodiment, and the determination processing process by a determination processing means. It is a schematic flowchart which shows the procedure of the whole system operation of the transfer system including the determination device of the conveyed object of 1st Embodiment and 2nd Embodiment. An explanatory diagram (a) showing the positional relationship of the pixel value distribution patterns Ds11 to Ds17 of the image data D of the limited region CR3 of the second embodiment with respect to the transported object, and the pattern type PT11 corresponding to the examples Ds11 to Ds17 of each distribution pattern. It is a graph (b) to (h) shown together with ~ PT17. It is sectional drawing (a) which shows the photographing direction of a camera with respect to the conveyed object on a transport path, and is the perspective view (b) which shows an example which looked at the transport posture of the transported object on a transport path from the photographing direction. It is explanatory drawing (a) and (b) which shows the example in the case of determining the total number of transported goods on a transport path, and the number of non-defective products of the transported goods after the sorting process. It is explanatory drawing which shows the example which determines the transport posture of a transported object based on a classification category. It is explanatory drawing which shows another example which determines the transport posture of a transported object based on a classification category. It is explanatory drawing which shows the different example which determines the transport posture of a transported object using different limited regions CR4 and CR5.

Next, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. First, with reference to FIGS. 1 to 3, the overall configuration of the transport system including the embodiment of the transport object determination device according to the present invention will be described.

The transport device 10 includes a parts feeder 11 provided with a bowl-shaped transport body 110 having a spiral transport path 111, and an inlet configured to receive the transported object from the outlet of the transport path 111 of the parts feeder 11. It is a vibration type transfer device including a linear feeder 12 provided with a transfer body 120 having a linear transfer path 121 provided. The transport device 10 is controlled by a transport object determination control system (hereinafter, referred to as “control system”) including the transport object determination device of the present embodiment. The entire combination of the transfer device 10 and the control system is hereinafter referred to as a transfer system. In the control system, the transported object P on the transport path 121 of the transport body 120 of the linear feeder 12 is inspected and determined based on the captured image GPX. In the present invention, the configuration not limited to the vibration type transfer device can be used for various transfer devices in which the conveyed object P is conveyed along the transfer path. Further, the vibration type transfer device is not limited to the combination of the parts feeder 11 and the linear feeder 12, and can be used for other types of transfer devices such as a circulation type parts feeder. Further, even in the above combination, not only the one that inspects the conveyed object P on the conveyed path 121 of the linear feeder 12 but also the one that inspects the conveyed object P on the conveyed path 111 of the parts feeder 11. I do not care.

The parts feeder 11 is driven and controlled by the controller CL11. Further, the linear feeder 12 is driven and controlled by the controller CL12. These controllers CL11 and CL12 AC drive the vibrating means (including the electromagnetic drive body and the piezoelectric drive body) of the parts feeder 11 and the linear feeder 12, and transport the transport bodies 110 and 120 on the transport paths 111 and 121. The object P is vibrated so as to move in a predetermined transport direction F. Further, the controllers CL11 and CL12 control the conveyed objects CP11 and CP12 that constitute the conveyed object control means provided in the conveyed object 110 of the parts feeder 11 and the conveyed object 120 of the linear feeder 12 to control the conveyed object. By using the control mechanisms CP11 and CP12, for example, by blowing an air flow, the transported object is removed from the transport paths 111 and 112, and the posture on the transport paths 111 and 112 is changed (reversed or rolled over). The controllers CL11 and CL12 are connected to the inspection processing unit DTU having an image processing function, which is the main body of the control system, via an input / output circuit (I / O).

Further, the controllers CL11 and CL12 perform a predetermined operation input (debug operation) to the arithmetic processing device MPU described later that executes the following operation program via the operation input devices SP1 and SP2 described later such as a mouse. Then, the drive of the transfer device 10 is stopped according to the above operation program. At this time, according to the above operation program, for example, the image measurement process in the inspection processing unit DTU is also stopped. This debugging operation and the operation of each part corresponding to the operation will be described in detail later.

The inspection processing unit DTU has an arithmetic processing unit MPU (microprocessing unit) such as a personal computer as a core configuration. In the illustrated example, the arithmetic processing unit MPU includes central processing units CPU1, CPU2, cache memory CCM, memory controller MCL, and the like. It is composed of a chipset CHS and the like. Further, the inspection processing unit DTU is provided with image processing circuits GP1 and GP2 for performing image processing connected to the cameras CM1 and CM2, which are imaging means, respectively. These image processing circuits GP1 and GP2 are connected to the image processing memories GM1 and GM2, respectively. The outputs of the image processing circuits GP1 and GP2 are also connected to the arithmetic processing device MPU, process the image data of the captured image GPX captured from the cameras CM1 and CM2, and perform an appropriate processed image (for example, an image in the image area GPY described later). Data) is transferred to the arithmetic processing device MPU. The operation program of the control system is stored in the main storage device MM in advance. When the inspection processing unit DTU is started, the operation program is read out and executed by the arithmetic processing unit MPU. In addition, the main storage device MM stores the image data of the captured image GPX or the image area GPY, which is the target for which the image measurement process described later is executed, by the arithmetic processing unit MPU.

Further, the inspection processing unit DTU is connected to display devices DP1 and DP2 such as a liquid crystal monitor and operation input devices SP1 and SP2 via an input / output circuit (I / O). The display devices DP1 and DP2 include image data of the captured image GPX or image area GPY processed by the arithmetic processing device MPU, image data D and distribution patterns Ds of the limited areas CR1 to CR5 described later, and patterns corresponding to these. The display of the type PT, the display of the classification category CG, and, if necessary, the result of the image measurement processing, that is, the result of the transported object detection process and the transported object determination process, etc. are displayed in a predetermined display mode. To. It should be noted that this display function is not limited to the case where the transported object is actually transported, but also functions when the past data is read and reproduced as described later. Further, by operating the operation input devices SP1 and SP2 while looking at the screens of the display devices DP1 and DP2, it is possible to input various operation commands, set values and other processing conditions.

Next, an example of a basic detection method and a determination method of a transported object in the transport device 10 using the above-mentioned control system including the present embodiment will be described. FIG. 2 is an explanatory diagram showing the shape of the transported object and the transport posture on the transport path 121 in this embodiment. In the illustrated example, the conveyed object P is an electronic component (for example, a chip resistor, a chip inductor, a chip capacitor, etc.) having a substantially cubic shape (for example, a shape in which eight corners of the cube are rounded). The transported object P is transported on a transport path 121 provided with transport surfaces 121a and 121b orthogonal to each other in the shown horizontal posture with the major axis (main axis) mainly directed in the transport direction F. In this lateral posture, metal terminal portions Pa are exposed at both front and rear ends of the transported object P, and a white surface Pb made of an insulating material and a black surface Pc which is a direction identification mark are exposed on the side surface portions between them. The normal transport posture of the transported object P is, for example, a posture in which the front end surface Pt5 is directed toward the transport destination (downstream side, left side in the drawing) and the rear end surface Pt6 is directed toward the transport source (upstream side, right side in the drawing). Of the four side surfaces Ps1 to Ps4, the side surface Ps1 in which the white surface Pb appears on the transport destination side and the black surface Pc appears on the transport source side is in an upward facing posture (see FIG. 9), and the whole is a white surface Pb. The side surface Ps2 is in a posture of facing the open side of the transport path 121. However, the regular transport posture may be a posture other than the above, and depending on the mode of the transported object P, only the front-back orientation may be determined or only the posture around the spindle may be determined.

In FIGS. 2 and 3, the transport surface 121a of the transport path 121 is a relatively steep surface, the transport surface 121b is a relatively gentle surface, and the cameras CM1 and CM2 are on the front side in the lower part of the drawing. (That is, the image when the image is taken obliquely from the front upper side of the transport surface 121b) is shown. An example of this situation is shown in the cross-sectional structure of the carrier shown in FIG. Therefore, in the transported object P, the side surface (side surface arranged on the transport surface 121a side) arranged on the upper side in the drawing on the transport path 121 is a surface facing upward (hereinafter, simply referred to as “upper side surface”). The side surface (the side surface arranged on the transport surface 121b side) arranged on the lower side in the drawing is a surface facing sideways (hereinafter, simply referred to as “side surface”). Regarding the transported object P at the left end in FIG. 2, the upper side surface is the side surface Ps1 and the side side surface is the side surface Ps2.

3A and 3B are explanatory views (a) to (c) for explaining a setting example of a measurement area for determining whether or not the transport posture of the transported object P shown in FIG. 2 is normal. The captured image GPX captured by the cameras CM1 and CM2 is appropriately processed by the image processing circuits GP1 and GP2, and is required in a direction orthogonal to the transport direction F on the transport path 121 as shown in FIG. 3A. Only the image data included in the image width GPW, which is within a range, is captured. Further, as for the range of the captured image GPX along the transport direction F, the image data may be captured in a range limited to the image length GPL as shown in the figure. By limiting the image area GPY that is actually captured from the captured image GPX and transferred to the arithmetic processing unit MPU in this way, the capture speed and the transfer speed can be improved. The image area GPY of the present embodiment is typically a rectangular area long in the transport direction F as shown in FIG. 3A.

In the control system, detection (inspection) and determination of the transported object P are performed by image measurement processing performed according to the inspection processing component incorporated and executed in the operation program. This image measurement process is not performed over the entire image area GPY shown in FIG. 3A, but is performed only on a limited area of the image area GPY. This area can be set differently depending on the processing content. In particular, in the first embodiment, in the counting process of the transported object P described later, the image data of the limited regions CR1 and CR2 that can include not the whole but only a part of the transported object P passing through the transport path is processed.

The limited areas CR1 and CR2 shown in FIG. 3 are set in the conveyed object determination device of the first embodiment according to the present invention. The limited regions CR1 and CR2 are formed in a linear or strip shape extending in the extending direction intersecting the transport direction F (orthogonal in the illustrated example). Further, the limited areas CR1 and CR2 are arranged so as to overlap the passing range Wt (see FIGS. 3A and 3B) of the transported object P on the transport path. Here, in the case of the present embodiment, since the transported object P moves on the transport path 121 due to vibration, the passing position of the transported object P varies to some extent. Therefore, the above-mentioned passing range Wt is set to a range corresponding to the average position through which the transported object P passes. In the illustrated example, as shown in FIGS. 3 (b) and 3 (c), the limited regions CR1 and CR2 are set so as to be within the passing range Wt. More specifically, assuming that the length of the limited regions CR1 and CR2 along the transport direction F is Lcr and the width in the extension direction is Wcr, the entire width Wcr is arranged within the passage range Wt and the width Wcr. Is configured to be smaller than the passing range Wt. At this time, the width Wcr is preferably 80% or more of the passing range Wt. With this setting, even if the passing position of the conveyed object P is slightly different from the passing range Wt, the entire limited regions CR1 and CR2 can be configured to almost always overlap with the passing position of the conveyed object P. In the illustrated example, the passing range Wt when only the laterally oriented conveyed object P passes is shown. However, when the vertically oriented transported object P also passes, the passing range Wt corresponding to the horizontally oriented transported object P is used as it is in order to set the range of the necessary limited area according to the classification category and the judgment content. Alternatively, a wider passage range may be set including the vertically oriented conveyed object P.

The linear or strip-shaped limited regions CR1 and CR2 may be any region of the captured image GPX or the image area GPY in which at least a row of pixels is arranged along the extending direction. However, it may be a region in which two or more rows of pixels are arranged along the extension direction. In particular, as shown in the illustrated example, it is preferable that the limited regions CR1 and CR2 are formed in a straight line, and it is desirable that the extending direction is orthogonal to the transport direction F (width direction). As shown in FIG. 3C, where Lp is the length of the transported object P along the transport direction F in the normal posture and Wp is the width, then Lp × Wp> Lcr × Wcr. Further, in the illustrated example, Wp> Wcr. Further, in the illustrated example, Lcr <Wcr.

In the present embodiment, the cameras CM1 and CM2 repeatedly perform shooting in a preset shooting cycle, and the shot image GPX or the image data in the image area GPY is the image processing device GP1 and GP2 at each shooting timing. It is transferred to the arithmetic processing unit MPU via the above. The arithmetic processing unit MPU acquires the limited image data D in the limited area CR1 from the transferred image data, takes it into the above-mentioned arithmetic processing memory RAM or the like, processes it as described later, and detects or determines it. I do. However, in this control system, a trigger sensor is separately provided, or a predetermined shape pattern of the transported object P is searched for in a predetermined area from the image data of the transported object P, and an internal trigger is detected when the shape pattern is detected. By introducing an external trigger that indicates a predetermined shooting cycle, or by outputting a trigger signal of a fixed cycle from the arithmetic processing unit MPU to the cameras CM1 and CM2, etc. Shooting is repeated at intervals. Therefore, for all the transported objects P transported on the transport path 121, the image data of the limited region CR can be acquired at least one acquisition timing for each transported object P. That is, the acquisition timing of the image data D of the limited areas CR1 and CR2 is set so that each conveyed object P overlaps the limited areas CR1 and CR2 at least once. The above conditions are satisfied by setting the shooting interval to a constant value in advance as described later. However, if the above conditions are satisfied, the shooting interval and the time interval of the acquisition timing may be set to fluctuate. For example, a method such as setting the shooting interval or the time interval of the acquisition timing according to the detection value of the transport speed of the transported object P, the control value or the detected value correlated with the transport speed, or the like may be used.

In the present embodiment, the shooting interval, that is, the time interval of the acquisition timing of the image data of the limited areas CR1 and CR2 is made constant. At this time, when the photographing cycle is Ts [sec], the length of the conveyed object P in the conveyed direction F is Lp [mm], and the conveyed speed of the conveyed object P is Vp [mm / sec], the conveyed object P is In order to ensure that the limited regions CR1 and CR2 overlap at least once at the imaging timing, the imaging cycle Ts is set as in the following equation (1).
Ts <Lp / Vp ... (1)
For example, if the length Lp of the transported object P in the transport direction F is 0.6 [mm] and the transport speed Vp is 50 [mm / sec], the imaging cycle Ts <0.012 [sec]. This means that if the shooting interval, that is, the time interval of the acquisition timing of the image data D of the limited areas CR1 and CR2 is less than 12 [msec], the image data D is described for each transported object P for all the transported objects P. Means that you can get.

However, under the above conditions, although the processing speed can be increased because the amount of data is small, it is the minimum condition that the image data D of the limited areas CR1 and CR2 can be acquired for each transported object P. Depending on the case, sufficient judgment accuracy cannot be obtained. Further, in reality, the transport speed Vp of the transported object P varies from individual to individual, depending on the location, or over time. Therefore, it is more preferable to obtain a plurality of (n) acquisition timings for each transported object P. Here, n is a natural number of 2 or more. As a result, the setting is as shown in the following equation (2).
Ts <Lp / (n × Vp) ... (2)
In the case of this embodiment, n is set to be in the range of 3 to 7. This is because when n is small, there is a high possibility that the image of the transported object P will be missed due to variations in the transport speed, and conversely, when n is large, the load of image processing will increase. For example, the acquisition timing of the image data D shown in FIGS. 3C and 11 is an example of n = 4, and the acquisition timing of the image data D shown in FIG. 12 is an example of n = 6. In general, the natural number n is preferably in the range of 1-10. When n is a plurality, it is desirable that n is in the range of 2 to 10. In FIG. 3C, the position change of the limited regions CR1 and CR2 with respect to the transported object P at each acquisition timing corresponds to the time interval Ts of the acquisition timings of the image data D1 to D6 of the limited regions CR1 and CR2 in the transport direction. A distance of Dt is generated in F. In the conveyed object determination device of the present embodiment, as an example, a case where n = 6 and the photographing cycle Ts is 2 [msec] = 0.002 [sec] will be described below. As a result, the image data of the limited regions CR1 and CR2 can be acquired about 6 times for each conveyed object P, respectively. In the present embodiment, the image data D of the limited areas CR1 and CR2 is acquired from the image data of the captured image GPX or the captured area GPY, but the images of the limited areas CR1 and CR2 are acquired by a line sensor, a small camera, or the like. You may try to get the data directly.

In the above example, the description has been made with reference to the total length Lp of the laterally-oriented conveyed object P whose main axis direction is the conveying direction F. However, when the determination content targets only a part of the transported object P, the length of the part is used instead of the total length Lp. For example, when only the rotational posture around the main axis of the transported object P is to be determined, the determination target portion is only the side surface portions Ps1 to Ps4, so the length of this determination target portion (for example, FIG. 10B). Refer to the length LDR shown in (see). Further, as shown in FIG. 9, not only the horizontally oriented conveyed objects P1 and P2 but also the vertically oriented conveyed objects P4 (however, the vertically oriented conveyed objects P7 that have fallen to the opposite side are excluded) are also conveyed. At the location, if the length Lp in the horizontal posture is larger than the width Wp as shown in the illustrated example, the image data D in the vertical posture can be obtained at least once for the transported object P in the vertical posture. The width Wp, which is the length in the transport direction of, is used as a reference. In the case of the present embodiment, as described above, the trigger signal for detecting that the conveyed object P reaches the position where it overlaps with the limited areas CR1 and CR2 is not used, so that the captured image GPX or the image area GPY It may occur that the transported object P and the determination target portion thereof are not arranged at all in the limited areas CR1 and CR2.

In the present embodiment, as a typical example of the image data D of the limited regions CR1 and CR2 shown in FIG. 3C, as shown in FIG. 4A, the pixel values of the image data D of the limited regions CR1 and CR2 The relative positions of the limited regions CR1 and CR2 from which the distribution patterns Ds1 to Ds5 are obtained with respect to the transported object P are set. That is, the image data D has distribution patterns (one-dimensional profile) Ds1 to Ds7 shown in FIG. 4B along the extending directions (width direction, vertically shown) of the limited regions CR1 and CR2. Here, the pixel value is a luminance value (brightness) in the illustrated example. In each graph of FIG. 4B, the vertical axis shows the luminance value represented by full scale 256, and the horizontal axis shows the pixel number in the pixel string of the image data D from 0 to 40. In the illustrated example, in the limited areas CR1 and CR2, a case where 40 pixel pixels are arranged in a row along the extending direction (width direction) is shown. Therefore, if the image data is D = {d1, d2, ... dm} (pixel value, m is a natural number), then m = 40. The limited regions CR1 and CR2 do not have to be composed of one row of pixel arrays, and may have two or more rows of pixel arrays. Further, in this case, a two-dimensional profile may be used instead of the one-dimensional profile as the distribution pattern of the pixel values. Further, in this case, preprocessing (processing such as extraction of representative values such as median value and average value) for forming a one-dimensional profile (distribution pattern) from a pixel array of two or more columns may be executed. ..

The transported objects P1 to P6 shown in FIG. 4 include the transported objects P1 to P3 in the horizontal posture and the transported objects P4 to P6 in the vertical posture. Here, as shown in FIG. 9, as the vertical posture conveyed objects, the vertically oriented conveyed object P4 having the main shaft along the conveyed surface 121b and the vertical oriented conveyed object P7 having the main shaft along the conveying surface 121a are included. Although it may exist, in the present embodiment, the conveyed object P having the same posture as the conveyed object P7 is supplied via the airflow supply path 121c on the upstream side of the locations where the limited regions CR1 and CR2 are set, and the airflow is supplied. It is either eliminated or overturned by the airflow blown from the mouth 121d. Therefore, FIG. 4 includes the conveyed objects P5 and P6 having the same vertical posture as the conveyed object P4, but does not include the conveyed objects having the same vertical posture as the conveyed object P7. However, unlike the present embodiment, it is necessary to consider such a transported object in a place where the transported object P having the same vertical posture as the transported object P7 may arrive or in an embodiment.

In the present embodiment, the distribution pattern Ds of the pixel values of the image data D of the limited regions CR1 and CR2 is categorized into the pattern types PT1 to PT7 corresponding to the distribution patterns Ds1 to Ds7 in the illustrated example. That is, the acquired image data D is associated with any one of the seven pattern types PT1 to PT7 corresponding to the distribution patterns Ds1 to Ds7 of the pixel values. Here, the pattern type PT1 corresponding to the distribution pattern Ds1 corresponds to the distribution pattern when the limited regions CR1 and CR2 overlap with the terminal portion Pa of the transported object P1 in the horizontal posture. Further, the pattern type PT2 corresponding to the distribution pattern Ds2 corresponds to the distribution pattern when the limited regions CR1 and CR2 overlap over the white surface Pb and the black surface Pc of the transported object P1 in the horizontal posture. Further, the pattern type PT3 corresponding to the distribution pattern Ds3 corresponds to the distribution pattern when the limited regions CR1 and CR2 overlap with the white surface Pb of the transported object P1 in the horizontal posture. The pattern type PT3 also includes a distribution pattern in the case where the vertically oriented conveyed object P6 overlaps from the terminal portion Pa to the white surface Pb. Further, the pattern type PT4 corresponding to the distribution pattern Ds4 corresponds to the distribution pattern when the limited regions CR1 and CR2 overlap over the black surface Pc and the white surface Pb of the transported object P2 in the vertical posture. Further, the pattern type PT5 corresponding to the distribution pattern Ds5 corresponds to a distribution pattern in which the limited regions CR1 and CR2 overlap from the terminal portion Pa of the transported object P4 in the vertical posture to the white surface Pb and the black surface Pc. Further, the pattern type PT6 corresponding to the distribution pattern Ds6 corresponds to a distribution pattern in which the limited regions CR1 and CR2 overlap from the terminal portion Pa of the transported object P5 in the vertical posture over the white surface Pb and the black surface Pc. Further, the pattern type PT7 corresponding to the distribution pattern Ds7 corresponds to the distribution pattern when the limited regions CR1 and CR2 overlap with the gap between the two front and rear transported objects. In this pattern type PT7, not only the gap when the front and rear conveyed objects are conveyed apart, but also the front and rear when the two conveyed objects P2 and P3 are conveyed in a continuous state as shown in the illustrated example. The distribution pattern of the contact portion of the transported object is included.

The above pattern types are not limited to the above combinations PT1 to PT7. For example, the pattern type PT3 has a type corresponding to the distribution pattern when it overlaps the white surface Pb of the transported object P1 in the horizontal posture, and a distribution pattern when it overlaps the terminal portion Pa and the white surface Pb of the transported object P6 in the vertical posture. It may be divided into the corresponding types. Further, the pattern type PT7 may be divided into a distribution pattern when the front and rear conveyed objects P are separated from each other and a distribution pattern of the contact portion when the front and rear conveyed objects P are in contact with each other. Further, the association with the above-mentioned plurality of pattern types is not essential, and as a result, the distribution pattern Ds of the acquired image data D can be clearly classified into any of the classification categories described below. I just need to be there. That is, the classifier (data classification means) may be set so that the correct classification category always exists for the predetermined image data D. Here, the pattern type corresponds to the classification of the positional relationship of the limited region CR with respect to the transported object P.

In the present embodiment, as the classification categories classified by the classifier, the classification category CGA corresponding to the pattern types PT1 to PT6 and the classification category CGB corresponding to the pattern type PT7 are set. As a result, the image data Ds of the limited areas CR1 and CR2 are classified as data classification means configured by the execution of the classification processing unit in the inspection processing component by the arithmetic processing unit MPU in the inspection processing unit DTU, respectively. Depending on the vessel, it is classified into two classification categories, CGA and CGB. That is, in the present embodiment, the image data D of the limited regions CR1 and CR2 is processed as a classification problem for classifying the image data D of the limited regions CR1 and CR2 into the classification category CGA corresponding to the pattern types PT1 to PT6 and the classification category CGB corresponding to the pattern type PT7. Execute the image processing step to be performed. Here, the classification category CGA corresponds to the case where the limited areas CR1 and CR2 overlap with the transported object P, that is, the case where the transported object P exists, and the classification category CGB does not overlap the limited areas CR1 and CR2 with the transported object P. , When it overlaps with the gap, that is, when there is no conveyed object P. The classifier that executes this image processing step is a machine learning classifier, and in the present embodiment, it is a trained model using a neural network. FIG. 5A schematically shows a schematic configuration of the classifier, and FIG. 5B shows a schematic procedure for training the classifier.

As shown in FIG. 5A, the classifier which is the data classification means of the present embodiment has a fully connected neural network. This neural network includes an input layer I, intermediate layers M1, M2, M3, and an output layer O. The input layer I has m units i1 corresponding to m (= 40) of the distribution pattern Ds = {d1, d2, ... dm} (m is a natural number, d1, d2, ... dm is a pixel value). , I2, ... im. Here, it is proved that any continuous nonlinear function can be approximated by a three-layer neural network. Therefore, it is possible to solve the flexible classification problem by providing an intermediate layer. In this case, the intermediate layer M1 includes h units m11, m12, ... m1h (h is a natural number). The number of units h is larger than the number of units m of the input layer I. As a result, the information of the image data D is amplified (mapped from the low-dimensional data to the high-dimensional data) in the intermediate first layer M1, so that the determination accuracy can be improved by learning using the training data set. Be expected. Further, by providing a plurality of intermediate layers M1, M2 and M3, the information amplified by the intermediate layer M1 is aggregated by the intermediate layers M2 and M3 (mapping from high-dimensional data to low-dimensional data). It is expected that the output to the output layer O will be appropriately realized.

In the present embodiment, the limited regions CR1 and CR2 are classified into two categories: the position where they overlap with the transported object P (CGA) and the position where they do not overlap with the transported object P (CGB). Therefore, the output layer O of the neural network includes two units (elements) o1 and o2, which are the number of categories to be classified. In the illustrated example, the output values {o1, o2} of the output layer O are processed by the activation function, and finally the classification result E = {e1, e2} is obtained. Here, for example, o1 and o2 are real numbers, and each presents a value representing the probability of the classification result. Further, e1 and e2 are 0 or 1 (or ± 1). For example, E = {1,0} corresponds to the case where there is a transported object P, and E = {0,1} corresponds to the case where there is no transported object P and there is a gap. The activation function is for deriving the classification result E from the output values {o1, o2} of the output layer O, and is not particularly limited, but for example, the Softmax function or the like can be used.

The classifier (data classification means) including the neural network can be configured by, for example, a method including the procedure shown in FIG. 5 (b). First, the acquisition of the image data D and the labeling process for the image data D, that is, which of the above pattern types PT1 to PT7 the distribution pattern should belong to is examined, and the classification category of the correct answer is examined. Label. By performing this for a large number of image data D, training data (train data), validation data (validation data), evaluation data (test data), etc. having an association between the image data D and its correct answer category can be created. Can be done. The above labeling process for creating each of the data can be performed manually. Further, the labeling process can be automatically performed by a conventional image processing method (template matching or the like) described with reference to FIG. 10 or the like. In the present embodiment, as described above, each image data D is labeled with the classification categories CGA and CGB, but before that, training data in which each image data D is associated with the pattern types PT1 to PT7 is created. By doing so, it is possible to prepare basic data that can be used for classification processing into a classification category different from the present embodiment.

The labeling process is not performed based on the image data D (distribution pattern Ds) of the limited areas CR1 and CR2 itself, but the overall condition of the transported object P included in the captured image GPX and the image area GPY. , It is performed according to the positional relationship of the limited areas CR1 and CR2 with respect to a plurality of conveyed objects P (front and rear conveyed objects P, etc.). For example, the classifier (data classification means) of the present embodiment classifies into one of the classification categories CGA and CGB only by the input distribution pattern Ds. On the other hand, it is preferable that the correct classification categories CGA and CGB in which each distribution pattern when creating the learning data should be classified are labeled by the positional relationship of the limited region CR with respect to the transported object P. That is, the classification category of the correct answer is determined by the positional relationship of the limited regions CR1 and CR2 shown in FIG. 4A, which directly correspond to the purpose of the classification process, with respect to the transported objects P1 to P6. The positional relationship is represented by the pattern types PT1 to PT7. Therefore, regardless of the distribution pattern Ds of the image data D actually acquired, it is preferable that the labeling of each of the training data and the like is based on (or corresponds to) the above pattern type. In particular, it is desirable that the final classification category of the correct answer is determined by ignoring the distribution pattern itself and determining the correct pattern types PT1 to PT7 only by the above positional relationship. This is expected to realize classification processing according to the actual situation, excluding the idea of subjective distribution patterns in human and image processing.

When the training data is created, the training process of the classifier is first executed using the training data. In this learning process, values called hyperparameters such as the number of layers of the neural network, the number of units (number of elements) m, h, j, k of each layer, and the activation function for the output value are set, and the basic design of the classifier is performed. Do. After that, the above training data is applied, and the classifier is trained by using a general error backpropagation method (BP method) or the like. After that, verification is performed using verification data different from the training data, and the basic design is changed (hyperparameter resetting) according to the result. If the verification result is good, further evaluation is performed using the evaluation data. If there is a problem with this evaluation result, the learning process or basic design will be changed again. If the evaluation result is finally good, it is used as a trained model as a classifier (data classification means) of the present embodiment.

In FIG. 6, the total number of passages of the transported object P transported on the transport path is counted based on the classification result obtained by using the classifier (data classification means) of the present embodiment configured as described above. The outline procedure of the case is shown. In a state where the transport device 10 shown in FIG. 1 is operating and the transport object P is being transported on the transport paths 111 and 112, the cameras CM1 and CM2 sequentially perform repeated shooting so as to have the above-mentioned shooting interval Ts. The image data D of the limited areas CR1 and CR2 is acquired from the image data of the captured image GPX or the image area GPY generated thereby. In the classifier, the above-mentioned preprocessing is performed from the image data D as necessary to form a pixel value distribution pattern Ds = {d1, d2, ..., Dm}. Then, when the final classification result E is output from the classifier, whether or not there is a transported object P by the determination processing means shown by the illustrated alternate long and short dash line according to the classification result E = {e1, e2} ( Whether or not the limited areas CR1 and CR2 overlap with the conveyed object P) is determined.

When the classification result E = {1,0} and it is finally classified into the classification category CGA, that is, when there is a transported object P (the limited area CR1 is at a position where the transported object P overlaps), the past It is determined whether or not the continuous detection number Nc of the image data D is larger than 6, for example, and when the continuous detection number Nc is larger than 6, it is assumed that the transported object P has newly arrived, the total number of passages Nd is added by one, and the continuous detection number Nc is continuous. Reset the number of detections Nc to 0. After that, the next image data D is acquired. This is because when the gap between the front and rear conveyed objects P is narrow (or when the front and rear conveyed objects P are in contact with each other), the boundary between the front and rear conveyed objects P may not be detected by the limited regions CR1 and CR2. This is to enable the subsequent counting of the conveyed objects P to be performed even in this case. On the other hand, when the continuous detection number Nc is 6 or less, it is determined that the transported object P is the same as the previously detected one, and one continuous detection number Nc is added without adding the total number of passages Nd. After that, the next image data D is acquired.

When the classification result E = {0,1} and the product is finally classified into the classification category CGB, that is, when there is no transported object P (limited areas CR1 and CR2 do not overlap with the transported object P and are located in a gap). Is determined whether or not the continuous detection number Nc is larger than 0, and when the continuous detection number Nc is larger than 0, it is assumed that the transported object P that has been detected so far has passed, and the total number of passages Nd is added by one. , Resets the number of continuous detections Nc to 0. After that, the next image data D is acquired. On the other hand, when the number of continuous detections Nc is 0, it is assumed that the new conveyed object P has not arrived yet, and the next image data D is processed without doing anything. In the above determination processing step (determination processing means), a determination processing chip incorporating a predetermined logic circuit or the like may be used in addition to the above classifier, and the above operation may be performed in the same manner as the above classifier. The judgment processing unit in the inspection processing component that constitutes the program may be executed. The inspection processing unit DTU that executes the procedure of the determination processing step described above constitutes a determination processor (determination processing means) based on the classification result.

As described above, the signal indicating the total number of passages Nd of the transported objects P counted based on the classification result of the classifier is output from the inspection processing unit DTU as the determination result. The total number of passes Nd is finally displayed on the display devices DP1, DP2 and the like, or is input to the controllers CL11, CL12 and the like. In the present embodiment, as a determination device for the transported object P, a classifier that is a data classification means performs a classification process as to whether or not the limited areas CR1 and CR2 overlap with the transported object P, and the classification result E = {e1. , E2}, the determination processing means described above is provided with determination content such as counting the total number of passages Nd of the conveyed objects P in the limited areas CR1 and CR2 on the conveying path.

As a specific example of this embodiment, a fully connected neural network was used for the classifier. In this neural network, the number of inputs of the input layer I is m = 40, the number of outputs of the output layer O is 2, the number of units of the intermediate first layer M1 is h = 800, and the number of units of the intermediate second layer M2 is j = 500. The number of units of the intermediate third layer M3 is k = 100. A classifier equipped with this neural network was trained using 400 sets of training data in which the correct classification categories were CGA and CGB, respectively. The learning rate was 0.001, the batch size (stochastic gradient descent method) was 100, and the optimizer was Adam. The accuracy of the training data and the verification data was 94.9% at the best. Further, when the ratio of the total number of passages determined by the transported object P based on the evaluation data to the actual total number of passages is calculated as the classification accuracy, it is 97.72%, which is 92.92% of the determination accuracy by the conventional technique (template matching method). It surpassed. Further, the processing time for each image data D is 0.276 [msec], which is also significantly shorter than the processing time of 2.186 [msec] of the prior art.

In this embodiment, as described above, the classifier classifies the two final classification categories CGA and CGB. However, the classifier classifies the classification patterns Ds of each image data D into seven pattern types PT1 to PT7, and based on the classification result E = {e1, e2, e3, e4, e5, e6, e7}, the above The total number of passages Nd may be finally obtained by deriving whether or not the transported object P is present or absent by the determination processing means. In the above-mentioned determination processing step performed based on the classification result in this case, when there is a transported object P from each classification category corresponding to any of the seven pattern types PT1 to PT7 with respect to the present embodiment (pattern). A process is added to determine whether the type PT1 to PT6) or the case where the transported object P is absent (pattern type PT7).

In the procedure of the operation program shown in FIG. 7, the conveyed object determination device of the present embodiment counts and conveys before and after the conveyed object control process based on the result of the predetermined conveyed object inspection determination process. It can be used to perform counting processing after object control. Here, the control process of the transported object is a process performed by the transported object controlling means, for example, sorting the transported object P such as removing the transported object P from the transport path or changing the posture on the transport path. It refers to the process of giving some action to the transported object P for alignment. This control process of the transported object includes, for example, various processes for selecting and aligning the transported object P according to the quality of the transported object P itself and the appropriateness of the transport posture of the transported object P. In the present embodiment, in the "counting process before controlling the transported object" shown in FIG. 7, the limited area CR1 is used to set the total number of passing Nd described above to the total number of the transported objects P on the transport path before performing the control process. Count as the number of transports. On the other hand, in the "counting process after controlling the transported object" shown in FIG. 7, the limited region CR2 is used to reduce the total number of passages Nd described above to the non-defective product P of the transported object P on the transport path remaining after performing the control process. Count as a number.

FIG. 10 is an explanatory diagram showing an example of the inspection determination process and the control process of the transported object shown in FIG. 7. In the image processing related to the inspection determination process shown in FIG. 10, the prior art is adopted without using the conveyed object determination device of the present embodiment. In this technique, a range (search area SAS) that can include the transported object P passing through the transport path, as shown by the dotted line in FIG. 10A, is set in the image area GPY, and is within this range. Image is processed. As shown in FIG. 10B, the search area SAS includes a control area MES for selecting the transported object P. In the case of the illustrated example, the control area MES is an area for sorting the transported object P. Further, the control area MES is an area for selecting the transported object P depending on whether it is passed on the transport path 121 or excluded from the transport path 121, and sends only the desired transported object P to the downstream side. is there. Regarding the sorting process of the transported object P, only the image data in the search area SAS is the target of the image measurement process.

As shown in FIG. 10B, the search area SAS includes a first measurement area ME1 adjacent to the upstream side of the control area MES and a second measurement area ME2 adjacent to the downstream side of the control area MES. And are further included. Here, the control area MES is an area on the transport path 121 in which the transport object P can be excluded by the sorting fumarole OPS formed at the central position CLN. Further, the first measurement area ME1 is inside the search area SAS and is provided on the upstream side of the control area MES, and the transported object P transported from the upstream side is not excluded by the sorting fumarole OPS. This area includes the area on the downstream side. Further, the second measurement area ME2 is inside the search area SAS, is provided on the downstream side of the control area MES, and when the transported object P escapes to the downstream side after passing through the control area MES, the transported object P is It is a region including the upstream range that is not excluded by the sorting fumarole OPS.

In the search area SAS, in the image measurement process, an image having an outer edge shape corresponding to the image of the transported object P registered in advance (hereinafter, simply referred to as “reference image”) (hereinafter, simply referred to as “detection image”). .) Is searched for. When the detected image exists, the position of the area occupied by the detected image is specified as the transported object detection area WDS. This is the transported object detection process. The transported object detection process does not need to detect the posture or defect of the transported object P, but only detects the existence and position of the transported object P. Therefore, the pattern shape such as the outer shape of the conveyed object P and the average inside the outer shape are averaged. The degree of agreement such as brightness is obtained, and this is compared with a predetermined threshold value to determine the presence or absence of detection. Further, at the time of detection, the position of the pattern shape in the search area SAS is calculated, and the conveyed object detection area WDS is specified as described above. In the transported object detection process, the degree of matching of the pattern shape such as the outer shape described above may be used as a determination factor, but as described above, the degree of matching such as the average brightness inside the outer shape may also be used as a judgment factor. , The detection accuracy of the transported object P can be improved. For example, if the brightness of the transported object P becomes dark as a whole due to the relationship between the illumination direction and the component posture, it becomes difficult to distinguish it from the background of the image, so that detection omission is likely to occur, but the threshold value of the average brightness Detection omission can be reduced by setting low.

In the above-mentioned transported object detection process, when the transported object detection area WDS is in the first measurement area ME1, the transported object determination process described below is subsequently performed. Further, when the transported object detection area WDS is in the control area MES and the second measurement area ME2, the measurement is performed as it is, and when the transported object detection area WDS in the control area MES and the second measurement area ME2 disappears. Then, the transported object passage detection process for outputting the transported object passage detection signal is performed. As will be described later, when a plurality of captured images GPX capture a state in which a certain transported object P is arranged in the first measurement area ME1, the transported object detection process is performed each time. Is carried out to derive the transported object detection region WDS, but the following transported object determination process may be performed only once (for example, the first time).

The transported object determination process is carried out as follows. First, as shown in FIG. 10B, the first determination area GWA and the second determination area GWB are positioned with reference to the transported object detection area WDS specified as described above, and the brightness thereof is determined. It is detected whether or not it corresponds to the side surfaces Ps1 to Ps4. For example, the first determination area GWA is arranged on the upper side surface of the conveyed object P, and the second determination area GWB is arranged on the lateral side surface of the conveyed object P. In the present embodiment, when the upper side surface is the side surface Ps1 and the side side surface is the side surface Ps2, it is set that the conveyed object P is being conveyed in a normal posture. At this time, the first determination area GWA has an elongated determination auxiliary area GWA1 extending in the transport direction F, a determination auxiliary area GWA2 arranged on the upstream side, and a determination arranged on the downstream side in order to detect the side surface Ps1. It has an auxiliary area GWA3. The determination auxiliary area GWA1 detects the boundary between the white surface Pb and the black surface Pc of the side surface Ps1 by the edge detection process in the transport direction F, and corrects the positions of the determination auxiliary areas GWA2 and GWA3 using the detected edge as the boundary position. .. After that, by comparing the brightness of the position-corrected determination auxiliary areas GWA2 and GWA3 with a predetermined threshold value, it is determined whether or not each brightness matches the transported object P in the normal posture. In the illustrated example, when the determination auxiliary area GWA2 detects the white surface Pb and the determination auxiliary area GWA3 detects the black surface Pc, it is determined that the conveyed object P is a good product in a normal posture. The mode of determining the transported object P (determination of quality) is not limited to the posture, and may be good or bad in shape, dimensions, or the like.

The second determination area GWB determines whether or not the lateral side surface is the side surface Ps2 (the side surface that is all white surface Pb). Also in this case, the determination can be made based on the fact that the brightness of the second determination area GWB is higher than a predetermined threshold value. By determining both the first determination area GWA and the second determination area GWB, the acquired information obtained from the image data to be determined can be made redundant, so that the brightness of the image and the like can be determined. Judgment accuracy can be improved, such as avoiding erroneous judgment due to variation.

On the other hand, the image area GPY is subjected to an inversion or rollover process for inversion or rollover of the conveyed object P at a position different from the search area SAS (in the illustrated example, a position on the upstream side of the search area SAS). Judgment areas GV1 and GV2 for determining whether or not to perform are provided. The first determination area GV1 and the second determination area GV2 are arranged at positions where the upper side surface of the conveyed object P passes. The first determination area GV1 is when the upper side surface of the transported object P is not the side surface Ps1 including the black surface Pc, that is, the side surfaces Ps2 to Ps4 which are entirely white surfaces Pb (for example, a predetermined threshold value). The determination result NG is output when it is brighter than), and the determination result PASS is output when the terminal portion Pa is included or the side surface Ps1 or the like is included (for example, when it is darker than a predetermined threshold value). Further, the second determination area GV2 is a region narrower in the transport direction F than the first determination area GV1. When an edge is detected by scanning in the transport direction F in the second determination area GV2, it is assumed that the boundaries of the conveyed objects P before and after being conveyed in close contact with each other are arranged, and the determination result is also PASS. .. Then, only when the determination result is NG, the airflow is ejected from the reversing or rollover fumarole OPR, and the airflow is inverted or rolled over so that the upper side surface of the conveyed object P becomes another side surface. By doing so, the posture of the transported object P is changed only when the upper side surface is arranged in the first determination area GV1 and the upper side surface is the side surfaces Ps2 to Ps4 (that is, not Ps1). be able to.

The pre-control counting process and the post-control counting process, which are the determination devices of the present embodiment, performed before and after the control process of the transported object P include the limited area CR1 arranged in the first measurement area ME1 shown in FIG. It is carried out by the limited area CR2 arranged on the downstream side of the measurement area ME2. Here, the limited area CR1 used for the pre-control counting process is set in the first measurement area ME1. This limited area CR1 prevents the image data D of the transported object P from being unable to be acquired due to the conveyed object P being excluded by the control area MES, and the conveyed object P reaching the control area MES is caused by an air flow or the like. In order not to detect the non-carrying posture of the indefinite posture when being moved, it is desirable to set it within the range closer to the upstream in the first measurement area ME1.

Further, the limited region CR2 can basically have exactly the same shape and size as the limited region CR1. The limited area CR2 is arranged at a position adjacent to the downstream side of the second measurement area ME2. This is to prevent accidentally acquiring the image data D of the transported object P before being excluded by the control area MES in the limited area CR2.

By counting the total number of passes Nd as described above based on the image data D of the limited area CR1 described above, the total number of transported objects before the control process of the transported object can be obtained based on the result of the inspection determination process. Further, by counting the total number of passages Nd as described above based on the image data D of the limited area CR2, it is possible to obtain the number of non-defective products after the control processing of the transported object based on the result of the inspection determination processing. Therefore, the non-defective product rate and the number of defective products can be known from the total number of transported products and the number of non-defective products.

In the image display fields appropriately formed in the image display devices DP1 and DP2 shown in FIG. 1, the image data in the image area GPY is displayed, and the limited areas CR1 and CR2 and CR3 to be described later are displayed. Each area such as CR5, the search area SAS, the first measurement area ME1, the second measurement area ME2, and the control area MES can be displayed by a frame line or the like. Here, in addition to or separately from the above, for controlling the transported object detection area WDS by the transported object detection process, the determination areas GWA and GWB used for the transported object determination process, and the fumarole OPR for reversing or rolling over. At least one of the determination areas GV1 and GV2 used for the transported object determination process can be displayed by a frame line or the like. In these cases, the above classification result and determination result may be identified by a display mode capable of distinguishing the display color and line type of each frame line and the like. For example, when it is classified into the classification category CGA by the above classifier, the borders of the limited areas CR1 and CR2 are set as the first display mode (for example, displayed in green), and when it is classified into the classification category CGB, it is limited. The borders of the areas CR1 and CR2 are set as the second display mode (for example, yellow display). Further, when the conveyed object determination process is OK (determination mode is non-defective), the frame line or the like is set to the third display mode (for example, blue display). Further, when the transported object determination process results in an NG determination (determination mode is defective), the frame line or the like is set to the fourth display mode (for example, red display). In addition, each display mode is not limited to the color of the above example, and is not particularly limited as long as it can be distinguished from each other, such as a line type such as a solid line, a dotted line, a broken line, and a dash-dotted line, and a thickness.

In the present embodiment, while the transported object P transported on the vibrating transport path 121 by the vibrating transport device 10 is targeted for inspection, the cameras CM1 and CM2 are located in a place where they do not vibrate (on the base 100). Since it is installed, in the image data of the captured image GPX or the image area GPY, the transport path 121 that vibrates with a predetermined amplitude in a mode of reciprocating back and forth in the transport direction F changes the vibration phase at the time of shooting the image data. It is arranged at the displaced position in the transport direction F according to the above. Therefore, when the appearance of the transported object P is to be detected and determined at a fixed position based on the transport path 121, the limited areas CR1 and CR2 in the image, CR3 to CR5 described later, the search area SAS, and the like, respectively. It is necessary to move the positions of the measurement areas ME1 and ME2 with the same amplitude in synchronization with the vibration of the carrier 120 according to the shooting timing. For example, the carrier 120 is subjected to vibration having an amplitude of 0.1 mm and a vibration frequency of 300 Hz.

Therefore, in the present embodiment, the positions of the limited areas CR1 and CR2, CR3 to CR5 described later, the search area SAS, and the measurement areas ME1 and ME2 are set to the captured image GPX or the image area GPY at the time of photographing. In order to match the vibration position of the image, the position correction mark (not shown) set on the carrier 120 is used as a reference for correction. The position correction mark is not particularly limited as long as the position can be easily and reliably detected. However, by making the position correction mark a single color (same gray scale) mark that can be reliably recognized as a blob in the image and that the position of the center of gravity can be stably detected, the detection accuracy of the position can be improved. Can be done. The position correction mark is not intentionally provided, but is inherently present in the transport device and can be detected by image processing, for example, a ridge line, a corner portion, or a bolt head formed on the transport body 120. , It may be a fumarole or the like. However, it is preferable that the material is not hidden by the transported object P.

In the present embodiment, for the above position correction, the positions of the limited areas CR1 and CR2 and CR3 to 5 described later, or the search area SAS and the measurement areas ME1 and ME2 with respect to the transport path 121 are set to the vibration during shooting. Regardless of the phase timing, it is always in the same position with respect to the transport path 121. Therefore, for example, the position where the exhaust air for eliminating the transported object P in the bad posture is blown from the exhaust fumarole OPS, or the airflow for correcting the posture of the transported object P in the bad posture is reversed or the fumarole for rolling over. The limited areas CR1 and CR2, CR3 to CR5 described later, or the measurement areas ME1 and ME2 are set to always have a constant positional relationship with respect to the position to be sprayed from the OPS. As a result, it is possible to prevent erroneous determination of the conveyed object P (in the above example, erroneous counting), and when an exclusion force or a reversing force is applied to the conveyed object P according to the result of the conveyed object determination process. , The action can always occur at an approximate timing.

In the present embodiment, the type and size of the transported object P, the posture of the non-defective product, the positions of the limited areas CR1 and CR2 and CR3 to CR5 described later (locations in the image), and the engagement required for the pretreatment for forming the distribution pattern Ds. Used for detecting and determining the transported object P, such as numerical values, reference image data, various set values such as the threshold value of the brightness of the transported object detection process, and various set values such as the threshold value of the brightness of the transported object determination process. Various types of data are stored in the main storage device MM or the like, and are appropriately read out and used in each process. In addition, the set values for determining the shooting timing of the cameras CM1 and CM2, the set values of the image capture conditions when capturing the captured image GPX or the image area GPY, and the mode of position correction of each set area due to the vibration of the transport path 121. The same applies to the set values to be set, the set values to set the mode of various setting screens and display screens, the control mode at the inversion or rollover position and the sorting position, for example, the set values such as the airflow blowing timing and the pressure value. Be treated.

In the present embodiment, it is possible to select, read, and display an image file in which the past captured image GPX or image area GPY stored in the main storage device MM is repeatedly stored in chronological order. Then, a means for executing various operation processes on the selected image file is also prepared. Such a display / playback function of the captured image GPX or the image area GPY has a great effect when later verifying a data processing defect that cannot be understood only by the image data D of the limited areas CR1 and CR2 and CR3 to CR5 described later. Demonstrate.

The image file stored in the main storage device MM is a file in which image data of a plurality of captured image GPX or image area GPY captured in the operation mode is automatically recorded by the arithmetic processing unit MPU. This image file can be saved for all image data when the main storage device MM has free space, but even if the main storage device MM does not have free space, the latest default period can be saved. It is preferable to always save the latest default number of image files (for example, one hour) or the latest default number of images (for example, 1000).

With the captured image GPX or the image area GPY recorded in the past displayed as described above, before and after the control process corresponding to the conveyed object determination device of the present embodiment by performing an appropriate operation on this image data. The counting process, or the image measurement process including the transported object detection process and the transported object determination process can be executed again. As one of the display mode control functions, it is possible to switch a plurality of captured image GPX or image area GPY stored in the same file one by one to other image data captured before and after by an appropriate operation. it can. It is also possible to repeatedly display a plurality of captured image GPX or image area GPY in the same image file and execute image measurement processing on the displayed image data in parallel.

Next, with reference to FIG. 7, the flow of the entire operation program of the present embodiment will be described. FIG. 7 is a schematic flowchart of processing executed according to an operation program by the arithmetic processing unit MPU of the inspection processing unit DTU. When this operation program is started, first, the above-mentioned image shooting and the image measurement processing including the above-mentioned pre-control coefficient processing and post-control coefficient processing are started, and the transfer device 10 (parts feeder 11) is started by the controllers CL11 and CL12. And the driving of the linear feeder 12) is started. Then, if the debug setting corresponding to the above-mentioned debug operation is OFF, the image measurement process is executed for the captured image GPX or the image area GPY, and the inspection determination process is executed together with the counting process before and after the control process. If the final determination result is an OK determination, the image measurement process of the next captured image GPX or image area GPY is performed as it is unless the debug operation is performed.

At the sorting position of the transported object P provided on the transport paths 111 and 121, for example, the air flow is always flowing from the exclusion fumarole OPS, but if the determination result is OK (non-defective product), the exclusion fumarole port The airflow of the OPS is stopped, and the airflow is restored after all the non-defective products have passed through the control area MES. As a result, the defective transported object P is excluded from the transport path 121. At this sorting position, the airflow from the exhaust fumarole OPS may be stopped at all times, and the above airflow may be generated when the determination result is NG (defective product). Further, at the reversal or rollover position of the transport posture of the transport object P provided on the transport paths 111 and 121, the airflow from the reversal or rollover fumarole OPR is always stopped. On the other hand, if the determination result is NG (defective product), an air flow is ejected from the reversing or overturning fumarole OPR and reversed on the transport path 121. In this reversal or rollover position, the airflow may be constantly flowed from the reversal or rollover fumarole OPR, and the airflow may be stopped only when a non-defective product is detected.

By controlling the transported object P on the transport path 121 in this way, only non-defective products (parts in the normal posture) are supplied to the downstream side in an aligned state. In this case as well, the determination of the next captured image GPX or image area GPY is performed as it is unless the debugging operation is performed thereafter. When the debug operation is performed in the middle of the above and the debug setting is turned ON, the routine is exited, the drive of the transport device 10 is stopped, and the image measurement process is also stopped. Then, if an appropriate operation is performed in this state, the image file can be selected as described above. At this time, the image file selected and displayed is an image file including a plurality of captured image GPX or image area GPY recorded in the immediately preceding operation mode. If this is selected as it is and an appropriate operation is performed, the mode shifts to the re-execution mode. In this mode, the display, detection, and determination of the image can be re-executed based on the image file in which the control operation already executed as described above is recorded. That is, when a problem occurs in the control process of the transported object P of the transport device 10, in order to solve this problem, first, the image measurement process is re-executed based on the past image data to measure the image. Search for processing problems. If the problematic part is found, the setting contents (setting value) of detection and judgment are changed and adjusted accordingly, and the image measurement process is re-executed for the past image data to adjust and improve the result. Can be confirmed. It should be noted that this point is the same for the above-mentioned counting process of the present embodiment, and is also the same for re-execution of the counting process, change of setting contents, adjustment, etc. when a problem occurs in the counting process. After that, when an appropriate return operation is performed, the debug setting is returned to OFF, the image measurement process is restarted, and the driving of the transport device 10 is restarted. Further, the screen of the display device returns to the display screen of the operation mode.

In the present embodiment described above, the cameras CM1 and CM2 repeatedly shoot at predetermined shooting intervals, and all the transported objects P passing through the transport path 121 due to the relationship between the transport speed Vs and the shooting interval Ts of the transported objects. By presetting each of the transported objects P so as to always overlap with the limited regions CR1 and CR2 and CR3 described later, information on all the transported objects P can be obtained from the image data D in the limited region in any of the captured images. Therefore, it is not necessary to generate a trigger signal for detecting the position of each transported object as in the prior art. Further, the determination accuracy is ensured by performing the determination based on the classification result obtained by classifying the distribution pattern Ds of the pixel values of the image data D into a predetermined classification category by a classifier (data classification means). , It becomes possible to perform processing in a short time. That is, by processing the detection process of the transported object P as a classification problem and reducing the amount of data required for the process, the image measurement process for determining the transported object P can be performed at high speed and with high accuracy.

As described above, by providing the counting processing means for the transported object P corresponding to the conveyed object determining device of the present embodiment, the non-defective product rate and the number of defective products at the sorting position of the transported object P can be obtained. It is possible to estimate the alignment efficiency and the supply efficiency to the supply destination. Therefore, when the above-mentioned non-defective product rate falls below a certain ratio or when the number of defective products exceeds a certain number, a command is automatically issued to the controllers CL11 and CL12 to stop the driving of the transport device 10. You can also do it. As a drive control method for the transport device 10 according to such a determination result, in addition to the drive stop, the driving force (voltage, current, etc.) of the vibration exciting means so as to change the transport speed and other transport modes, It may control the amplitude, frequency, and the like.

As an example of the control processing means for the transported object, for example, an air flow blowing mechanism that blows an air flow such as air to the transported object on the transport path to blow off the transported object or change its posture. Can be mentioned. This airflow blowing mechanism includes an airflow blowing port opened on the transport surface of the transport path, an airflow supply path for allowing the airflow to flow out from the airflow blowing port, and a compressor that supplies the airflow to the airflow supply path. It is equipped with an airflow supply source such as a gas tank. Further, as another example of the conveyed object control means, the conveyed object is brought into contact with the conveyed object on the conveyed object, and the conveyed object is mechanically removed from the conveyed object or the posture is mechanically changed (reversed or rolled over). A mechanical mechanism of action can be mentioned. Further, as another example of the conveyed object control means, the conveyed object is selected by changing or switching the path width of a predetermined portion of the conveyed object, the inclination angle or gradient of the conveyed surface, the shape of the conveyed surface, and the like. An adjustment mechanism of a transport path for adjusting the rate can also be mentioned. Such a transport path adjusting mechanism can change or switch the transport mode of the transported object at the predetermined location, and as a result, can control the selected mode and posture of the transported object.

Next, a case where the above-mentioned inspection and determination processing of the transported object is performed by the device for determining the transported object according to the second embodiment of the present invention will be described. In this case, since it is necessary to determine the transport posture of the transported object, the limited region CR3 having a shape and size different from the limited regions CR1 and CR2 of the first embodiment is used. This is because, in the first embodiment, the distribution patterns Ds of the image data D of the limited regions CR1 and CR2 overlapping the transported objects P having different transport postures cannot be discriminated from each other, and correspond to the same pattern type. 8 is an explanatory diagram (a) showing the relative position of the limited region CR3 with respect to the transported object P, and graphs (b) to (h) showing the distribution pattern Ds of the image data D of the limited region CR3.

As described above, as shown in FIG. 9, the transported object P carried into the first measurement area ME1 has a vertical posture in which the main shaft is along the transport surface 121a among the transported objects P4 and P7 in the vertical posture. The main axis of the conveyed object P7 is eliminated by the airflow blown from the airflow supply port 121d through the airflow supply path 121c, or is rolled over so that the main shaft is only the conveyed object P4 in the vertical posture along the conveying surface 121b. Become. Therefore, in order to surely obtain the information of the transported object P4 in the vertical posture, the limited region CR3 has the transported object P4 in the vertical posture with respect to the limited regions CR1 and CR2 corresponding to the transported objects P1 and P2 in the horizontal posture. The shape is extended to the lower side in the drawing corresponding to P6. When the vertically-positioned conveyed object P7 is also carried in, it is preferable that the limited region CR3 has a shape further extended to the opposite side.

Here, the limited region CR3 is set to be longer in the vertical direction than the limited region CR1 so that at least the horizontally oriented conveyed objects P1 and P2 and the vertically oriented conveyed objects P4 to P6 can be discriminated from each other. Will be done. In the illustrated example, m = 40 in the limited region CR1 while m = 80 in the limited region CR2. Further, it is desirable that the limited region CR3 is provided with end portions protruding from the passing range on both sides of the passing range Wt of the transported object. In this case, it is desirable that the limited region is a region of 120% or less of the passing range. In the illustrated example, the limited region CR3 is provided with end portions protruding from the passing range on both sides in the width direction of both the passing objects P1 and P2 in the horizontal posture and the transported objects P4 to P6 in the vertical posture. There is. The end portion is set to, for example, about 10 to 20% of the total width of the limited region, or about 8 to 16 pixels in the illustrated example (m = 80). As a result, in the distribution pattern Ds of the pixel values of the image data D of the limited area CR3, the end portion is arranged at a position outside the range overlapping with the transported object P, so that the detection range of the transported object P can be easily recognized. Therefore, it becomes easy to improve the discrimination accuracy of each part of the transported object P, so that the classification accuracy of the data classification process by the classifier can be improved.

In the present embodiment, pattern types PT11 to PT17 are assumed corresponding to the distribution patterns Ds11 to Ds17 of the pixel values of the image data D of the limited region CR3. Thereby, the acquired image data D can be associated with the seven pattern types PT11 to PT17 corresponding to the distribution patterns Ds11 to Ds17 of the pixel values. Here, the pattern type PT11 corresponding to the distribution pattern Ds11 corresponds to the distribution pattern when the limited region CR3 overlaps the white portion Pb of the transported object P1 in the horizontal posture. Further, the pattern type PT12 corresponding to the distribution pattern Ds12 corresponds to the distribution pattern when the limited region CR3 overlaps the white surface Pb and the black surface Pc of the transported object P1 in the horizontal posture. Further, the pattern type PT13 corresponding to the distribution pattern Ds13 corresponds to the distribution pattern when the limited region CR3 overlaps the white surface Pb of the transported object P2 in the horizontal posture. Further, the pattern type PT14 corresponding to the distribution pattern Ds14 corresponds to the distribution pattern when the limited region CR3 overlaps the terminal portion Pa of the transported object P4 in the vertical posture and the white surface Pb. Further, the pattern type PT15 corresponding to the distribution pattern Ds15 corresponds to a distribution pattern in which the limited region CR3 overlaps from the terminal portion Pa of the transported object P5 in the vertical posture to the white surface Pb and the black surface Pc. Further, the pattern type PT16 corresponding to the distribution pattern Ds16 corresponds to the distribution pattern when the limited region CR3 overlaps from the terminal portion Pa of the transported object P6 in the vertical posture to the black surface Pc and the white surface Pb. Further, the pattern type PT17 corresponding to the distribution pattern Ds17 corresponds to the distribution pattern when the limited region CR3 overlaps with the gap between the two front and rear objects. This distribution pattern is not limited to the gap when the front and rear transported objects are transported apart, but also the contact portion of the front and rear transported objects when the two transported objects are transported in a continuous state. Good.

In the present embodiment, as the classification categories classified by the classifier, the classification categories CGC corresponding to the pattern types PT11 to PT13, the classification categories CGD corresponding to the pattern types PT14 to PT16, and the pattern type PT17 are supported. Set the classification category CGE. As a result, the image data D of the limited area CR3 is obtained by the classifier, which is a data classification means configured by the execution of the classification unit in the inspection processing component by the arithmetic processing unit MPU in the inspection processing unit DTU, respectively. It is classified into three classification categories, CGC, CGD or CGE. That is, in the present embodiment, the image data D of the limited region CR3 corresponds to the classification category CGC corresponding to the pattern types PT11 to PT13, the classification category CGD corresponding to the pattern types PT14 to PT16, and the pattern type PT17. The image processing step to be processed as a classification problem to be classified into the classification category CGE is executed. Here, the classification category CGC corresponds to the case where the limited area CR3 overlaps with the laterally transported objects P, that is, when there are the laterally oriented objects P1 and P2, and the classification category CGD corresponds to the case where the limited area CR3 is vertically oriented. Corresponding to the case where it overlaps with the object P, that is, when there are vertically oriented objects P4 to P6, in the classification category CGE, when the limited area CR3 does not overlap with the object P but overlaps with the gap, that is, the items P1 to P6 Corresponds when there is no.

In this way, the classifier that classifies the image data D of the limited region CR3 into the classification categories CGC to CGE is basically designed and trained as a trained model with the same configuration as that described with respect to FIG. Therefore, the classification result E'= {e1, e2, e3} can be obtained by the same method as that described with respect to FIG. Then, instead of the determination process described with reference to FIG. 6, for example, in the case of the classification category CGC having the laterally transported objects P1 to P3 and in the case of the classification category CGE when there is a gap, the transported object P is excluded or the posture is changed. Is judged to be unnecessary, and the transported object is passed through as it is without performing the transported object control process. In the case of the classification category CGD in which the vertically oriented transported objects P4 to P6 are present, it is necessary to eliminate the transported object P or change the attitude. By the conveyed object control process, the conveyed object P is removed from the conveyed path, or the conveyed object P is inverted or overturned to change the conveyed posture to the sideways direction. In addition, in the transported object control process, it is also possible to set the transported object control state such as the state in which the air flow is ejected at all times, and set the transported object passing state such as stopping the ejection of the air flow only when it is determined that the product is non-defective. it can. In this case, in the case of the classification category CGC in which the sideways transported objects P1 to P3 are present, it is determined that it is not necessary to remove the transported object P or change the posture, stop the transported object control process, and pass the transported object as it is. In the case of the classification category CGD in which the vertically oriented objects P4 to P6 are present, and in the case of the classification category CGE when there is a gap, it is determined that the transportation object P needs to be eliminated or the attitude is changed, and the transportation object is controlled. By continuing the process as it is, the transported object P is removed from the transport path, or the transported object P is rolled over to change the transport posture to the sideways direction.

As a specific example of this second embodiment, the number of inputs of the input layer I is 80, the number of outputs of the output layer O is 3, the number of units of the intermediate first layer M1 is 100, and the number of units of the intermediate second layer M2 is 100. The number was 120, the number of units of the intermediate third layer M3 was 40, the activation function of the intermediate layer was the Tanh function, and the error function was the mean square error function. The learning rate was 0.00001 and the batch size was 32. The number of training data and verification data was 2000, and the number of evaluation data was 1000. Assuming that the ratio of the number of correct transport postures to the actual number of determined transport postures is the classification accuracy, the classification accuracy of the training data and the verification data was 97%, and the classification accuracy of the evaluation data was 95.4%.

Also in this second embodiment, similarly to the first embodiment, the image data D of the limited region CR3 is classified into the pattern types PT11 to PT17 by the classifier, and then by the determination processing procedure (determination processing means). It may be classified into one of the final classification categories CGC to CGE. In addition, various contents and other modifications described in the first embodiment described above can be appropriately applied as long as they do not interfere with the application to the second embodiment. Further, in both the first embodiment and the second embodiment, the control system or the transport system can be used alone or both can be used.

Further, in both the first embodiment and the second embodiment, they are classified into two classification categories CGA and CGB by a classifier, or classified into three classification categories CGC to CGE. However, the classification category classified by the classifier is not limited to the above example, and can be set in various modes. For example, in the laterally-positioned transported objects P1 to P3, although the spindle is in a posture along the transporting direction, a plurality of transporting postures in which the front-rear direction of the transported object P and the rotational posture around the spindle are different are included. Is done. These plurality of transport postures can be discriminated by the position of the black layer Pc. Therefore, for example, in the classifier, each image data D of the limited areas CR1 and CR2 is classified into a plurality of classification categories having different front-back orientations and rotational postures around the main axis, and the transported goods are subjected to determination processing according to the classification results. The transport posture of P may be determined. In this case, assuming that the overlapping positional relationship between the transported object P and the limited areas CR1 and CR2 is set as shown in FIG. 11, the pattern types of the plurality of image data D before and after the acquisition timing (limited area CR1 with respect to the transported object P). The transport posture of the transported object P can be determined from the arrangement mode (positional relationship of ~ CR5). For example, in the illustrated example, if the temporal sequence of the pattern type is PTx, PTy, PTz, PTx, it may be the transported product Po, PTx, PTz, PTy, and if it is PTx, it may be the transported product Pp, PTx, PTz, PTw, PTx. For example, if the transported object is Pq, PTx, PTz, PTw, or PTx, it is the transported object Pr. Therefore, in this case, due to the difference between the second and third pattern types of the sequence in which the first and fourth pattern types of the four front and rear pattern types are PTx, one pair for each of the above four pattern types. By setting the classification category corresponding to 1, the above-mentioned transport posture can be determined.

Further, in the example shown in FIG. 11, when the pattern type PTx in which the limited regions CR1 and CR2 overlap with the terminal portion Pa and the pattern type PTv overlapping with the gap appears, the conveyed object P passes through. Further, when the pattern type PTx in which the limited areas CR1 and CR2 overlap with the terminal portion Pa and the pattern type PTx in which the limited areas CR1 and CR2 overlap with the terminal part Pa appears again, the next transported object P has arrived. Become. Therefore, in these cases, in any case, one transported object P has passed. Therefore, it is possible to count the transported object P by looking at the front and rear pattern types PTx, or the front and rear pattern types PTx and PTv. In this case, it is possible to obtain a determination result by providing a classification category corresponding to one-to-one for each of the pattern types PTx and PTv.

In the example shown in FIG. 12, the procedure of the data classification step is the same as the above, but by making the photographing interval Ts smaller than that in the case of FIG. 11, the white surface Pb that appears between the pattern types overlapping the terminal portion Pa. Only one of the pattern type PTz and the pattern type PTy or PTw of the combination of the white surface Pb and the black surface Pc appears twice before and after, or between the two pattern types PTz and PTy or PTw. It is set so that an intermediate pattern type PTs or PTt appears. Therefore, when the first and last of the three consecutive pattern types before and after are one or the other of the above PTZ and PTy or PTw, the intermediate pattern types are PTz, PTy, PTw, and PTs (PTz and PTy). If it is either PTt (intermediate pattern between PTz and PTw), the transport posture of the transported object P can be determined. That is, if the middle is either the first or the last, or an intermediate pattern between them, the first is PTy and the last is PTZ, the transported Po, if the first is PTw and the last is PTZ, the transported Pp, the first is If it is PTz and the last is PTy, it is the transported product Pq, and if the first is PTz and the last is PTw, it is the transported product Pr. In this case, it is possible to obtain a determination result by providing a classification category corresponding to one-to-one for each of the four pattern types PTz, PTy, PTw, PTs, and PTt.

In the example shown in FIG. 13, a plurality of limited regions CR4 and CR5 to be acquired at one acquisition timing are provided. As described above, there may be a plurality of limited areas for acquiring the image data D at one acquisition timing. Further, in the illustrated example, both the limited regions CR4 and CR5 extend along the transport direction F. Further, the limited area CR4 is arranged at a position overlapping the upper side surface in the drawing, and the limited area CR5 is arranged at a position overlapping the lower side surface in the drawing. In this case, the image data D of the limited regions CR4 and CR5 is the pattern type PTp of the pixel value distribution pattern including the pattern portion corresponding to the range covering the white surface Pb and the black surface Pc, and the pattern corresponding only to the white surface Pb. It is classified into a pattern type PTq of a pixel value distribution pattern including only a part and a pattern type PTr other than that. As a result, when both pattern types PTp and PTq are simultaneously observed in the limited areas CR4 and CR5 in the image data D at one acquisition timing, the transport postures of the transported objects Po, Pp, Pq, and Pr are determined by the combination thereof. It can be derived by processing. Therefore, the transport posture of the transported object P can also be determined by combining the pattern types in the image data D of the plurality of limited regions CR4 and CR5 having the same acquisition timing. In this case, the determination can be made by providing at least one classification category corresponding to PTp and PTq, respectively.

In the present embodiment, the data classification means (classifier) performs a plurality of image data D of the limited region CR set in a range where only a part of the conveyed object P can overlap, according to the distribution pattern Ds of the pixel values. Classification Category CGA, CGB, or CGC to CGE. As a result, it may be possible to determine the status of the transported object P that overlaps with the limited area CR only from the classification category obtained as the classification result, and various predetermined determination processes are further performed based on the classification result. The judgment result can also be obtained. In the latter case, as described above, a predetermined determination result can be obtained based on the classification result of the image data D of the plurality of limited area CRs acquired at the acquisition timings before and after, and one acquisition timing. It is also possible to obtain a predetermined determination result based on the classification result of the image data D of the plurality of limited area CRs acquired in. Therefore, the required determination accuracy can be ensured while limiting the amount of data to be processed, so that the determination process can be executed at high speed. Therefore, even when a large number of transported objects P must be supplied at high speed as in the above-mentioned transport system, accurate determination processing can be easily realized.

In the present embodiment, when the training data is created, the distribution pattern Ds of the image data D of the limited region CR is associated with a plurality of pattern types PT1 to PT7 or PT11 to PT17, and is associated with these pattern types. They are labeled in the classification categories CGA, CGB, or CGC to CGE. As a result, labeling can be performed accurately and easily regardless of whether the labeling is performed manually or automatically by image processing. Further, by performing this labeling based not only on the image data D of the limited region CR but also on a wider range of image data including the limited region CR, the image data D (or its distribution pattern Ds) is clearly labeled. Can be expected to improve the classification processing capacity of the classifier based on unrecognizable information (improvement of classification accuracy and responsiveness).

In the present embodiment, when the determination process based on the classification result is performed, the determination processing means processes the image data D of the plurality of limited region CRs temporally or spatially by using the classification result. A wider and more flexible judgment result can be obtained. Further, since this determination process is performed based only on the classification result, the process time can be extremely shortened.

In the present embodiment, as described above, the limited regions CR1 to CR5 are set in a linear or strip-shaped region extending in a predetermined direction. As a result, the pattern along the predetermined extending direction of the transported object P can be captured as image data, so that more information about the transported object can be acquired while reducing the amount of data. Further, since it becomes easy to make the information input to the classifier or the like one-dimensional, the classification process can be further facilitated and speeded up. In particular, by forming a linear or band-shaped region extending in a direction intersecting (preferably orthogonal to) the transport direction, such as the limited regions CR1 to CR3, the transported object can be strictly measured and corresponding to the shooting interval and acquisition timing. You can also get location information. Further, by using the plurality of image data Ds in time series, it is possible to accurately grasp the transport posture of the transported object and the transport status of the plurality of transported objects.

It should be noted that the device for determining the transported object of the present invention is not limited to the above-mentioned illustrated examples, and it goes without saying that various modifications can be made without departing from the gist of the present invention. For example, in each of the above embodiments, a trained model using a fully connected neural network is used as a classifier, but CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), or the like is used as the neural network. Is also possible. Further, as the classifier, a machine learning classifier other than the neural network such as SVM (Support Vector Machine) or Random Forest can also be used.

Further, in the system including each of the above embodiments, as a sorting method at the sorting position, the transported object P is excluded from the transport path 121 by blowing an air flow, but the method for sorting the transported object P is included. The individual processing contents and the range of each measurement area are not particularly limited, and various known techniques for detection and determination can be adopted, such as using mechanical exclusion means. Further, as the mode for controlling the transported object, not only exclusion but also various modes such as inversion and distribution can be used. Further, the transfer system to which each embodiment is applied is not limited to the vibration type transfer device as described above, and various other transfer devices may be used, and the control system to which each embodiment is applied. However, it can be used in various situations where some measurement is performed in the process of moving some conveyed object P.

10 ... Conveying device, 11 ... Parts feeder, 110 ... Conveying body, 111 ... Conveying path, 12 ... Linear feeder, 120 ... Conveying body, 121 ... Conveying path, OPS, OPR, OPR1? CA3 ... Transport, CM1, CM2 ... Camera, CL11, CL12 ... Controller, DTU ... Inspection processing unit, DP1, DP2 ... Display device, GP1, GP2 ... Image processing device, GM1, GM2 ... Image processing memory, GPX ... Captured image , GPY ... image area, GWA to GWB ... judgment area, MPU ... arithmetic processing device, MM ... main memory, ME1 ... first measurement area, ME2 ... second measurement area, MES ... control area, SAS ... search area , SP1, SP2 ... Operation input device, RAM ... Memory for arithmetic processing, CR, CR1 to CR5 ... Limited area D, D1 to D6 ... Image data, d1 to dn ... Pixel values, Ds, Ds1 to Ds7, Ds11 to Ds17 ... Distribution pattern, PT1-PT7, PT11-PT17, PTx, PTy, PTz, PTv, PTw ... Pattern type, CGA-CGE ... Classification category, P ... Transport, P1-P3, Po, Pp, Pq, Pr ... Horizontal Transported object, P4 to P7 ... Vertically oriented transported object, Pa ... Terminal part, Pb ... White surface, Pc ... Black surface, Ps1 to Ps4 ... Side surface, Pt5 ... Front surface, Pt6 ... Rear end surface

Claims (10)

A device for determining a transported object that determines the transported object based on image data of a limited area set in a range in which only a part of the transported object on the transport path can be included.
A data acquisition means for repeatedly acquiring the image data of the limited region at an acquisition timing having a time interval capable of forming the image data of the limited region for each transported object.
A data classification means that classifies the distribution pattern of pixel values of the image data in the limited area into one of a plurality of classification categories for each acquisition timing and outputs the classification result of the image data in the limited area.
A determination processing means for outputting the determination result of the transported object based on the plurality of the classification results, and
A device for determining a transported object. The plurality of classification results are classification results of image data of the limited area acquired at a plurality of acquisition timings.
The device for determining a transported object according to claim 1. The acquisition timing is set at a time interval in which image data in the limited region can be formed a plurality of times for each transported object.
The device for determining a transported object according to claim 1 or 2. The data classification means is a classifier including a neural network.
The device for determining a transported object according to any one of claims 1 to 3. In the data classification means, the classification category determined based on the positional relationship of the limited region with respect to the transported object at the acquisition timing for each image data was learned using the learning data labeled with respect to the distribution pattern. A trained model,
The device for determining a transported object according to claim 4. The neural network has an input layer, an intermediate layer, and an output layer.
The input layer includes as many inputs as the number of pixels of the image data.
The output layer comprises a number of outputs that matches the number of the classification categories to be classified.
The device for determining a transported object according to claim 4 or 5. The neural network includes a plurality of the intermediate layers.
Of the plurality of intermediate layers, the first layer directly connected to the input layer includes a number of units larger than the number of inputs of the input layer.
The device for determining a transported object according to claim 6. The limited area is a linear or band-shaped area,
The device for determining a transported object according to any one of claims 1 to 7. The limited region is a region extending in a direction intersecting the transport direction.
The apparatus for determining a transported object according to claim 8. Conveyor and
The device for determining a transported object according to any one of claims 1 to 9, which determines the transported object transported by the transport device.
Conveyance system equipped with.

Images