TW202219894A - Conveyance system - Google Patents

Abstract

To enable discrimination processing of a conveyance object to be executed at a high speed and reduce an error in the discrimination processing due to the fluctuation in the posture of the conveyance object during conveyance in a conveyance system. A conveyance system according to the present invention comprises: a conveyance device which conveys a conveyance object along a conveyance path; image acquisition means which sequentially acquires a plurality of images in such a way that all the conveyance objects passing through the conveyance path are certainly photographed in two or more images of a measurement area; and conveyance object discrimination means which derives a discrimination result on the basis of a processing result for the two or more images by sequentially executing image processing on the two or more images in which the conveyance objects are photographed for each conveyance object.

Description

Conveyor system

The present invention relates to delivery systems.

Conventionally, in various conveying systems, by processing images generated by photographing conveyed objects, the counting of conveyed objects, determination of quality, posture identification, and detection of conveying states (speed, density, interval, etc.) have been performed. As an example of such a conveyance system, the apparatus described in the following patent document 1 and patent document 2 is mentioned.

prior art literature

Patent Literature

Patent Document 1: Japanese Patent Laid-Open No. 2017-121995

Patent Document 2: Japanese Patent Laid-Open No. 2019-169010

However, in the above-mentioned conveying system, since the position or posture of the conveyed object sometimes changes irregularly during the conveying process, there is a possibility that the image processing is affected by the change in the position or posture, resulting in erroneous discrimination. As a result, the first problem arises in this case, that is, the unproblematic conveyed material is excluded from the conveying path, or the problematic conveyed material flows directly downstream, which is caused by an error in the identification result. Poor delivery form.

In addition, when the image processing of the conveyed object is performed, if the area of the measurement area to be the image processing target is too large, the conveyed object needs to be identified after the position of the conveyed object inside the measurement area is determined. Therefore, the data to be processed As the amount increases, the load of image processing increases, and the time required for image processing increases. Therefore, there is a second problem that high-speed discrimination processing cannot be performed, and high-speed transportation cannot be handled. Here, if the speed of image processing is pursued, the accuracy of discrimination against the change in the position or posture of the conveyed object tends to be lowered, and the misjudgment caused by the change in the position or posture of the conveyed object will further increase. predicament.

Therefore, the present invention is an invention to solve the above-mentioned problems, and its object is to reduce errors in the discrimination processing caused by the fluctuation of the position or posture of the conveyed object during the conveying process in the conveying system. In addition, it is possible to perform the identification processing of the conveyed objects at high speed while reducing errors in the identification processing of the conveyed objects.

In order to solve the above-mentioned problems, the conveying system of the present invention includes: a conveying device for conveying a conveyed object along a conveying path; an image acquisition unit for acquiring an image of a measurement area through which the conveyed object passes on the conveying path, and , to sequentially acquire a plurality of the images in such a way that all the conveyed objects passing through the conveying path must be captured respectively in two or more of the images in the measurement area; and a conveyed object identification unit, which is a A unit for identifying the conveyed object by performing image processing on the image portion of the conveyed object in the image, and, for each of the conveyed objects, for the two or more images of the conveyed object The images are sequentially subjected to image processing, whereby discrimination results are derived from the processing results for the two or more images.

According to the present invention, through the image acquisition unit, all the conveyed objects are captured in a manner of being photographed in two or more images, respectively, so that for each conveyed object, the conveyed object identification unit can be used to distinguish between the two or more images. Two or more processing results obtained by performing image processing on the image part of the conveyed object to derive the discrimination result for each conveyed object, thereby reducing the impact on the processing result caused by the change of the position or posture of the conveyed object. The possibility of erroneous identification results.

In the present invention, it is preferable that the conveyed object identification unit derives, for each of the conveyed objects, the identification result corresponding to the most processing result among the respective processing results for the two or more images. take this. The discrimination accuracy can be improved by deriving the discrimination result corresponding to the most processing result among the respective processing results for two or more images. It is particularly preferable that the two or more images are three or more images.

In the present invention, it is preferable that, for each of the conveyed objects, the conveyed object identification unit derives a predetermined processing result corresponding to the predetermined processing result when the predetermined processing result for the two or more images reaches a set number. the discrimination result. Particularly preferably, the two or more images are three or more images, and the set number is two or more.

In the present invention, it is preferable that, when the conveyed object identification unit performs image processing on the two or more images, the image after the second time is determined according to the image processed from the previous time. The image processing is performed within an observation area defined by the position information of the conveyed object obtained from the image and the predicted movement form of the conveyed object. In this way, although two or more images need to be processed per conveyed object, since the second and subsequent images are set to be processed based on the image processing results of the images processed last time Image processing is performed within the observation area limited by the obtained position information of the conveyed object and the predicted movement shape, so that the burden of image processing can be reduced and the speed of image processing can be increased.

In the present invention, it is preferable that the predicted movement form is a predicted movement amount in the conveying direction corresponding to the conveying speed of the conveyed object and the time difference between the imaging of the image processed this time and the image next time.

In the present invention, it is preferable that the observation area for the second and subsequent images is set as an area formed by expanding the reference area so as to include a peripheral area arranged around the reference area. , wherein the reference range is a range formed by arranging the detection range of the conveyed object obtained in the image processing of the image processed last time at the predicted position after the movement. Thereby, the conveyed object can be detected more reliably in the observation area for the next image, and the processing result of the discrimination processing can be derived. Here, the above-mentioned peripheral region may be provided in at least a part of the periphery of the reference range. However, from the viewpoint of improving the reliability of derivation of the processing result, it is preferable to provide the above-mentioned peripheral region over the entire periphery of the reference range. In this case, the expansion degree of the observation area with respect to the reference range is preferably increased or decreased according to the magnitude of the deviation of the past movement form of the conveyed object.

In the present invention, preferably, the predicted movement form is obtained from the past movement form of the conveyed object. In this case, the past moving form refers to the past moving speed, moving direction, change of position or posture of the conveyed object, and the like. In addition, various methods may be employed in accordance with the movement characteristics of the conveyed object as a method of obtaining the predicted movement form. For example, the predicted movement pattern may be derived from a representative value such as an average value and a median value of a plurality of the past movement patterns, or may be derived from the latest past movement pattern.

In the present invention, it is preferable that the conveyed object identification unit is in a predetermined first observation area set so that the conveyed object can be arranged for the first image among the two or more images. Image processing is performed to obtain the conveyed object identification information. Here, in the case where the image acquisition unit captures the image of the passing portion of the conveyed object, it is preferable to determine the initial observation area corresponding to the passing detection position of the conveyed object . In addition, in this case, it is preferable to set the observation area for the next image based on the predetermined predicted movement form for the first time.

In the present invention, it is preferable to further include a conveyed object passing through which detects that the conveyed object has passed through the limited area by performing image processing on a limited area in the image set in the measurement area. A determination unit, and when it is determined by the conveyed object passing determination unit that the conveyed object has passed through the limited area, the first-time said image is compared in the first-time said observation area that is in a predetermined positional relationship with the said limited area. like performing image processing. Here, the limited area is preferably a short area in the conveying direction and a long area in a direction perpendicular to the conveying direction. In addition, it is preferable that the above-mentioned conveyed object passage determination unit can detect a first state in which the conveyed object exists in the limited area and a second state in which the conveyed object does not exist in the limited area, and can detect the first state in which the conveyed object does not exist in the limited area. When a state is switched to the second state, it is determined that the conveyed object has passed.

Preferably in the present invention, the image acquisition unit is independent of the presence or absence of the conveyed object and sequentially acquires the images at a predetermined acquisition time without using a trigger signal, and provides the conveyed object identification unit. This eliminates the need to use a trigger signal for determining the timing to acquire an image from a detection sensor of the conveyed object or the like, thereby preventing an error in the detection of the conveyed object and simplifying the configuration of the image acquisition unit. In this case, according to the relationship between the maximum moving speed of the conveyed object and the time interval at the time of image acquisition, it is preferable to set a number of times in the image that can include all the conveyed objects that have passed through. A measurement area is used, and the conveyed object identification unit processes the primary image by using the entire measurement area as the observation area, thereby obtaining the primary conveyed object detection information. In this case, it is preferable that the observation area for the next image is set based on the predetermined initial predicted movement form.

In the present invention, it is preferable that the identification process for the observation area performed by the conveyed object identification unit is repeatedly performed a plurality of times, and the next The observation area is set for the image, so that the conveyed object identification information is obtained through image processing in the observation area for the two or more images. In this case, it is preferable that the conveyed object identification unit performs the identification processing in the above-mentioned two or more images for each conveyed object in parallel with respect to a plurality of conveyed objects, and derives the identification results from the plurality of processing results.

In this invention, it is preferable to further comprise the conveyance control part which controls the said conveyance apparatus based on the said conveyance object identification information. Here, the conveyance control unit controls a conveyed object processing mechanism provided in the conveying device for sorting the conveyed objects or changing the posture based on the conveyed object identification information. For example, as the conveyed material processing means, there are a conveyed material removal processing unit that removes the conveyed material from the conveying path by air pressure, a posture inversion processing unit that changes the posture by applying air pressure to the conveyed material, and the like. As the control objects of these conveyed object processing units, there are the operation time of the air pressure, the operation pressure, and the like.

According to the present invention, it is possible to reduce errors in the identification process due to changes in the position and posture of the object to be conveyed in the conveying system. In addition, for each conveyed object, the target portion of the image processing for the second and subsequent images is set within a limited observation area, whereby the identification processing of the conveyed object can be performed at high speed.

10: Conveying system

10P: Action Program

11: Feeder

110: Conveyor

111: Conveyor Road

12: Linear feeder

120: Conveyor

121: Conveyor Road

OP: air vent

CA, CA1~CA3: conveyed material

CM1, CM2: Camera

CL11, CL12: Controller

DTU: Check Processing Unit

DP1, DP2: Display device

GP1, GP2: Image processing device

GM1, GM2: Image processing memory

GPX: Capture images

GPY: Image area

MPU: arithmetic processing unit

MM: main storage device

SP1, SP2: Operation input device

RAM: memory for arithmetic processing

θ: Spindle angle (angle pose)

100: conveyed object identification processing department

101: Conveying object detection and processing process

A: Image acquisition unit

Ai:image

B: Image processing unit

Bi: conveyed object detection information

Bia: Detection range of conveyed objects

C: dynamic shape prediction unit

Ci: predict movement patterns

D: Observation area setting unit

Di: observation area

di: benchmark range

Ei: The moving form of the conveyed object

MS: Conveyor movement status

Ji: Handling result of conveyed material

K: Distinguishing result of conveyed object

FIG. 1 is a schematic flowchart showing a schematic procedure of a conveyed object discrimination processing unit in an embodiment of the conveying system of the present invention.

FIG. 2 is a schematic configuration diagram schematically showing an example of the overall configuration of the embodiment.

(a) to (g) of FIG. 3 are image explanatory diagrams schematically showing an example of the process of the conveyed object identification processing of this embodiment.

FIG. 4(a) is a schematic configuration diagram schematically showing the overall structure of the function realization unit corresponding to the conveyed object detection process used in the conveyed object identification process of this embodiment, and (b) is used to explain the implementation of the structure. An explanatory diagram of the principle of the processing content.

(a) to (e) of FIG. 5 are explanatory diagrams for explaining the process of deriving the conveyed object identification information by the image processing performed by the image processing unit in the conveyed object identification processing of this embodiment.

(a)-(e) of FIG. 6 is a schematic diagram of each example of the implementation result of the conveyed object discrimination process of this embodiment.

FIG. 7( a ) is a schematic diagram showing an example of a problem in the prior art example in the conveyed object discrimination process, and (b) is a schematic diagram of another example.

FIG. 8 is a schematic flowchart showing an example of the processing procedure of the operation program 10P in this embodiment.

Next, an embodiment of the conveying system of the present invention will be described in detail with reference to the drawings. First, with reference to FIGS. 1-5, the outline|summary of embodiment of the conveyance system of this invention is demonstrated. FIG. 1 shows a part of the operation program 10P executed by the computer of the conveying system 10 of the present invention, that is, the conveyed object identification process A schematic flowchart of the processing procedure of the unit 100 is shown. FIG. 2 is a schematic configuration diagram schematically showing the overall configuration of the embodiment of the transport system 10 . FIGS. 3( a ) to ( g ) are explanatory diagrams illustrating examples of the conveying form of the conveyed objects on the conveying path 121 of the conveying system 10 . Fig. 4(a) is a schematic configuration diagram showing the tracking process of the conveyed object used in the conveyed object identification processing unit 100 by the function realization units A to D, and (b) is for explaining the concept of the tracking process. explanatory diagram. (a) to (e) of FIG. 5 are explanatory diagrams schematically showing various kinds of image processing performed in the process.

First, the overall configuration of the transport system 10 will be described with reference to FIG. 2 . As shown in FIG. 2, this conveyance system 10 is a conveyance system which conveys the conveyance CA along a predetermined conveyance path. The conveying system 10 constitutes a vibrating conveying device including a feeder 11 including a bowl-shaped conveying body 110 having a spiral conveying path 111 and a linear feeder 12 having a linear feeder 12. The conveying body 120 of the conveying path 121 of the linear conveying path 121 is provided with an inlet configured to receive the conveyed material from the exit of the conveying path 111 of the feeder 11 . In addition, this conveying device has a conveying management function that inspects and determines the conveyed object CA on the conveying path 121 of the conveying body 120 of the linear feeder 12 based on the captured image GPX. Moreover, in this invention, the structure which is not limited to a vibrating conveying apparatus can be used for various conveying apparatuses which convey the conveyance CA along a conveyance path. In addition, even if it is a vibration type conveying apparatus, it is not limited to the combination of the said feeder 11 and the linear feeder 12, It can be used for conveying apparatuses of other types, such as a circulating feeder. Furthermore, even in the above-mentioned combination, the inspection is not limited to the inspection of the conveyed object CA on the conveying path 121 of the linear feeder 12, and the inspection of the conveyed object CA on the conveying path 111 of the feeder 11 is also possible.

The feeder 11 is driven and controlled by the controller CL11. In addition, the linear feeder 12 is driven and controlled by the controller CL12. The controllers CL11, CL12 AC drive the vibration excitation mechanism (including the electromagnetic drive body or the piezoelectric drive body, etc.) of the feeder 11 or the linear feeder 12, so that the conveying bodies 110 and 120 are connected to the conveying path. The conveyed objects (the conveyed objects CA) on 111 and 121 are vibrated so as to move in the predetermined conveying direction F. In addition, the controllers CL11 and CL12 are connected via an input/output circuit (I/O) to an inspection processing unit DTU having an image processing function as the main body of the conveyance management system.

In addition, when a predetermined operation input (debugging operation) is performed to the arithmetic processing device MPU executing the above-mentioned conveyed object discrimination processing unit 100 through the operation input devices SP1 and SP2 described later, such as a mouse, the controllers CL11 and CL12 follow the above-mentioned operation program. 10P stops the driving of the conveying device of the conveying system 10 . At this time, according to the above-described operation program 10P, for example, the image measurement processing in the inspection processing unit DTU is also stopped. The debug operation and the operations of the respective parts corresponding to the operation will be described in detail later.

The inspection processing unit DTU is constituted by an arithmetic processing unit MPU (Micro Processing Unit) such as a personal computer as a core. In the illustrated example, the above-mentioned arithmetic processing device MPU is composed of central processing units CPU1, CPU2, a cache memory CCM, a memory controller MCL, a chip set CHS, and the like. In addition, image processing circuits GP1 and GP2 are provided in the inspection processing unit DTU, and the image processing circuits GP1 and GP2 are respectively connected to cameras CM1 and CM2 as imaging units and used to perform image processing. The image processing circuits GP1 and GP2 are connected to the image processing memories GM1 and GM2, respectively. The outputs of the image processing circuits GP1 and GP2 are also connected to the above-mentioned arithmetic processing unit MPU to process the image data of the captured images GPX acquired from the cameras CM1 and CM2, and appropriately process the images (for example, images described later). image data in the area GPY) is transferred to the arithmetic processing unit MPU. The operation program 10P of the conveyance management system is previously stored in the main memory device MM. When the checking processing unit DTU is activated, the above-mentioned action program 10P is read and executed through the arithmetic processing unit MPU. In addition, in the main memory device MM, image data of the captured image GPX or the image area GPY, which is the subject of image measurement processing to be described later, to be executed by the arithmetic processing device MPU is stored.

In addition, the inspection processing unit DTU is connected to display devices DP1 and DP2 such as a liquid crystal monitor or operation input devices SP1 and SP2 via an input/output circuit (I/O). The display devices DP1 and DP2 display, in a predetermined display form, the image data of the captured image GPX or the image area GPY processed by the above-mentioned arithmetic processing device MPU, and the result of the image measurement processing, that is, the identification processing of the conveyed object or the conveying behavior. The result of detection processing, etc. In addition, this display function is not limited to functioning when the conveyed object CA is actually conveyed, but also functions when reading past data and reproducing it, as will be described later. In addition, by operating the operation input devices SP1 and SP2 while viewing the screens of the display devices DP1 and DP2, processing conditions such as various operation commands and setting values can be input to the arithmetic processing device MPU.

In the present embodiment, the cameras CM1 and CM2 continuously capture images at predetermined capturing intervals, and perform image measurement processing on image data in the measurement zone having the conveyance speed Vs and the capturing interval according to the conveyed object. The relationship of Ts always includes the range of the conveyance direction F which is preset so that all the conveyances (the conveyances CA) passing through the conveyance path are included. Thereby, all the conveyed objects CA must be detected in the captured image of any measurement area, so that there is no need to generate a trigger signal for detecting the position of each conveyed object as in the prior art. In addition, by processing the image data of the conveyed object CA included in the image, it is possible to reliably extract information necessary for the conveyed object identification process, the conveyed object detection process, the conveyance behavior detection process, and the like. In addition, in the conveying system of the present invention, without employing the triggerless imaging method as described above, an image can be acquired at the imaging time corresponding to the detection time of the conveyed object determined by a normal sensor or the like.

In the present embodiment, the image measurement process described above is further configured so that the following conditions are satisfied. That is, all the conveyed objects CA passing through the measurement area are captured in two or more images Ai within the predetermined range of the measurement area. Here, the setting is within a predetermined range, and it is assumed that discrimination processing is performed on the upstream side of the conveyed object processing unit (position facing the air injection port OP) in the embodiment to be described later, and this input is used. The result of the identification of the conveyed material controls the state of the above-mentioned conveyed material processing unit. If it is not necessary to set the above-mentioned predetermined range in the above-mentioned measurement area in this way, all the conveyed objects CA may be captured in two or more images Ai. By doing so, since the conveyed object identification process can be performed on two or more images Ai for each conveyed object CA, two or more conveyed object processing results Ji can be obtained as described later. Then, the conveyed object discrimination result K can be finally derived from the two or more conveyed object processing results Ji.

In the conveyance system 10, the conveyed object discrimination during conveyance is performed by the execution of the conveyed object discrimination processing unit 100 (refer to FIG. 1 ) included in the below-described operation program 10P (refer to FIG. 8 ) executed by the arithmetic processing unit MPU. processing, and the above-mentioned conveying device is controlled according to the discrimination result of the processing. In this conveyed object identification processing part 100, the conveyed object detection process is performed using the conveyed object detection process procedure 101 shown to Fig.4 (a).

As shown in FIG. 4( a ), the above-mentioned conveyed object detection processing process 101 includes: an image acquisition unit A that sequentially acquires a plurality of images Ai including an area (area) where the conveyed object CA may exist; an image processing unit B, is to perform image processing in the observation area Di of the images Ai; the movement shape prediction unit C is to obtain the prediction of the conveyed object CA related to the object detection information Bi obtained by the image processing unit B The movement form Ci; and the observation area setting unit D, set the observation area Di+1 related to the next image Ai+1 based on the predicted movement form Ci. Here, i is a natural number and represents a plurality of numbers from 1 to n (n is a natural number of 2 or more).

The conveyed object detection information Bi includes the conveyed object detection range Bia indicating the configuration of the conveyed object CA, and may also include information about the type, appearance, defect, or type of the conveyed object, the quality of the conveyed object, and the position or posture of the conveyed object. information, etc. In addition, the above-mentioned predicted movement form Ci may include the amount and direction of movement of the conveyed object CA in the next image Ai+1 when the conveyed object CA in the current image Ai is used as a reference, for example For example, it can be represented in the form of a motion vector. In addition, the predicted movement form Ci may include a change in the posture of the conveyed object CA in the next image Ai+1 when the conveyed object CA in the current image Ai is used as a reference.

(b) of FIG. 4 shows a situation in which, when the conveyed object detection range Bia in which the conveyed object CA is arranged in the observation area Di is obtained in a certain image Ai, based on the conveyed object detection range Bia and the prediction The movement form Ci sets the observation area Di+1 in the next image Ai+1. The next observation area Di+1 is set by predicting the area including the next conveyed object detection range Bia+1, and the setting is performed based on the previous conveyed object detection information Bi. For example, in the example shown in the figure, by using the reference range di+1 formed by moving the previous conveyed object detection range Bia to the predicted position in the next image Ai+1 according to the predicted movement form Ci as a reference, and according to the predetermined The method expands the reference range di+1, thereby setting the next observation area Di+1.

Here, if (x _i , y _i ) are the center coordinates of the conveyed object detection range Bia, (v _i ^x , v _i ^y ) are values representing the predicted movement vector of the predicted movement pattern Ci, (x _i+1 , y _i+1 ) is the position coordinate of the center of the reference range di+1, Δt is the time interval (time difference) between the previous image and the next image, Δw is the width Wi in the observation area Di+1 The amount of increase from the width Wi of the reference range di+1 (the amount of expansion in the x direction), Δh is the amount of increase in the height Hi in the observation region Di+1 from the height Hi of the reference range di+1 (the amount of expansion in the y direction). expansion amount), the following equations (1) to (4) hold. Here, the functions σ(v _i ^x ) and σ(v _i ^y ) represent the standard deviation of the x component or the y component of the set of moving speeds up to now. c is a proportional constant, and can be set to an arbitrary number such as 0.01 and 0.1. For example, Δw and Δh can be set to values corresponding to the expansion amount of the surrounding area on the left and right sides of the reference range di+1 as shown in the figure and on the upper and lower sides of the figure, respectively, as shown in the example in the figure, or can be set to the same value as the figure. Shows the value corresponding to the total expansion amount of the surrounding area on the left and right sides and the upper and lower sides of the illustration. In the latter case, the ratio of the left-to-right ratio of the left and right sides of the graph and the enlargement of the upper and lower sides of the graph of the reference range di+1, or the ratio of the top to the bottom of the graph can be set to 1:1, It can also be set to a ratio corresponding to other information of the predicted movement pattern Ci, such as the ratio of the movement probability between the left and right of the icon and the movement probability of the upper and lower sides of the icon.

x _i+1 = x _i +v _i ^x . △t…(1)

y _i+1 =y _i +v _i ^y . △t…(2)

_Δwi = c. σ(v _i ^x )…(3)

Δh _i = c. σ(v _i ^y )…(4)

In addition, in the illustrated example, as shown in the above equations (1) to (4), only the amount of movement of the predicted motion vector between the center point of the conveyed object detection range Bia and the center point of the reference range di+1 is used and direction to express the predicted mobile morphology Ci. However, the predicted movement form Ci is not only the amount and direction of the movement of the above-mentioned center position, but also generally includes, for example, the position and range after the movement of the conveyed object CA corresponding to the conveyed object detection range Bia (width Wi, height Hi) , that is, information from which the reference range di+1 can be derived, and further, information from which the expansion degrees Δw and Δh can be derived. In addition, the predicted movement form Ci may further include information related to changes in the posture of the conveyed object CA.

The predicted movement form Ci can be obtained by various prediction methods based on the movement characteristics of the conveyed object CA. At this time, it is preferable that the predicted movement form C1 which is a prediction of the movement form of the conveyed object CA between the first image A1 and the next image A2 is set to a predetermined initial value in advance. For example, when the general movement speed, movement direction, and posture change of the conveyed object CA are known, the first predicted movement form C1 is set to the movement speed, movement direction, and posture change. After the predicted movement pattern C2 and beyond, better The calculation is carried out in consideration of the movement form of the conveyed object CA derived previously. For example, it is preferable that the predicted movement form C2 is actually determined in consideration of the first actual movement form E1 of the conveyed object CA, wherein the first actual movement form E1 of the conveyed object CA is determined by using the first observation area D1 by using the The conveyed object detection information B1 obtained by image processing and the conveyed object detection information B2 obtained by image processing of the next observation area D2 are derived. At this time, the next predicted movement form C2 may be determined in consideration of both the above-mentioned first predicted movement form C1 and the actual movement form E1. In addition, the next predicted movement form C2 may be determined only from the actual movement form E1. This includes setting the next predicted movement pattern Ci+1 (eg C2) to be the same as the previous actual movement pattern Ei (eg E1). Furthermore, when there are a plurality of actual movement patterns Ei in the past, the predicted movement patterns Ci+1 may be obtained by an appropriate method based on the plurality of movement patterns Ei. As an appropriate method, for example, a case where a representative value such as an average value and a median value is set, or a method in which a representative value is obtained by giving a larger weight to a movement pattern that is adjacent to each other, or the like, can be considered.

Since it is necessary to give the first observation area D1 at a stage where the predicted movement pattern is not obtained, the first observation area D1 is set in advance as a predetermined area according to the disposition characteristics of the above-mentioned area of the conveyed object CA, or according to The detection position of the conveyed object CA is set in advance as an area determined according to a predetermined rule. For example, in the first image A1, the first observation area D1 may be set as the entire image A1, or may be set as an area in the image A1 where the possibility of arranging the conveyed object CA is high. If it is difficult to predict in advance the position where the conveyed object CA can be arranged in the image A1, the initial observation area D1 is set within a relatively wide range. On the other hand, when the first image A1 is acquired at the same time as or immediately after the position of the conveyed object CA is detected, the observation area D1 may be set as the first image using the position information of the detected conveyed object CA. Like the area in A1 that reliably contains the above-mentioned conveyance CA.

The expansion amounts Δw and Δh of the reference range di from the observation region Di can be obtained, for example, from the above-mentioned equations (3) and (4). The degree of expansion or expansion ratios Δw and Δh at this time are preferably determined according to the movement characteristics of the conveyed object CA. For example, when the deviation of the movement form Ei of the conveyed object CA is large, the expansion degree or expansion ratio needs to be increased, and when the deviation of the movement form Ei is small, the expansion degree or expansion ratio can be decreased. When obtaining the movement patterns Ei of a plurality of actual conveyed objects CA in the past, it is preferable to increase or decrease the expansion degree or the expansion rate according to the deviation (eg, standard deviation) of the aggregate of the movement patterns Ei. By doing so, the conveyed object detection information Bi related to the conveyed object CA can be obtained more reliably through the image processing in the observation area Di.

In the present embodiment, when the detection information of the conveyed object CA is analyzed in more detail, as indicated by the dotted line in FIG. 4( a ), additional information processing other than the above may be performed. For example, a process of obtaining a plurality of movement patterns Ei of the conveyed object CA, and calculating the movement state MS of the conveyed object CA based on these movement patterns Ei is exemplified. The conveyed object movement state MS may include, for example, the state of the movement trajectory of the conveyed object CA, the state of change in the moving direction, the state of change in the conveyance position or the conveyance posture, and the like. The conveyed object movement status MS can be used as a detection amount for various control and determination of the conveyed object CA.

Next, the above-mentioned conveyed object identification processing unit 100 which can be used as an example of the above-mentioned conveyed object detection process will be described with reference to FIG. 1 . The conveyed object identification processing unit 100 is configured to set conditions for the start of the entire conveyed object identification processing unit 100 , the first observation area D1 or the first predicted movement form C1 in various processes for each conveyed object CA. Initial conditions such as setting values for the observation area Di, processing routines for image processing of the observation area Di, image processing conditions such as various thresholds, and predictions such as processing routines and calculation constants used for predicting the movement pattern of the conveyed object CA In a state where processing conditions are stored in a predetermined memory or the like, a program is executed according to the above-mentioned conditions that are appropriately read. Furthermore, the imaging device (camera) connected via the input/output circuit of the computer can be operated under predetermined imaging conditions, whereby a plurality of images can be sequentially acquired for the range including the measurement area where the conveyed object CA can be arranged. However, conveyed object identification The processing unit 100 may be configured as a device for processing data formed by storing a plurality of images sequentially captured in advance. In this case, the data of the plurality of images may be sequentially stored in the computer according to predetermined conditions, or the data of the plurality of images already stored may be sequentially read out for each image.

The conveyed object identification processing unit 100 starts to operate when a trigger signal is given, which is a signal when a predetermined start operation is completed, or when a certain conveyed object CA to be detected is detected by a device such as a sensor. , and it is a signal when an imaging start signal of the imaging device has been input. At this time, when the initial value of the initial condition is read, for example, the first image A1 is acquired for the above-mentioned one of the conveyed objects CA, image processing is performed in the first observation area D1 set to a predetermined range. In general, in this step, as shown in FIG. 5( a ), the conveyed object detection process is performed by performing image processing on the observation area Di of the i-th image Ai. In this conveyed object detection process, first, as shown in FIG. 5( b ), the image data of the observation area Di is processed by a predetermined filtering process, so that the conveyed object identification process is converted into an easy form. For example, when converting into an image that enhances the outline of the conveyed object CA, edge extraction processing or edge enhancement processing is performed. On the other hand, when it is desired to reduce the influence outside the contour, other parts are smoothed. Examples of filtering (processing methods) satisfying these requirements at the same time include a guided filter, a bilateral filter, and the like. In addition, as the filtering process, a binarization filter, a moving average filter, a Gaussian filter, a median filter, a Sobel filter, a Prewitt filter, or the like can also be used.

Next, image processing for detecting the position, range, posture, pattern, color, and other forms of the conveyed object CA is performed on the image after the filtering process described above. As one example, the range of the conveyed object CA is determined as shown in FIG. 5( c ) or ( d ), for example. In general, contour tracking processing, pattern detection processing, pattern fitting processing, and the like are performed in this process. Through these processes, the range in which the conveyed object CA is detected in the observation area Di can be determined, for example, the conveyed object detection range Bia can be obtained. Inspection of the conveyed material The measurement range Bia can be expressed by, for example, a boundary graph such as a Bounding Box shown in FIG. 5( c ), or a fitting graph such as an ellipse fitting shown in FIG. 5( d ).

A boundary pattern such as the above-mentioned bounding box forms an area such as a rectangle surrounding the image of the conveyed object CA, and the area (boundary pattern) of the predetermined condition surrounding the conveyed object CA is made as small as possible (for example, it becomes the area of the circumscribing area). method) to find out. In addition, in the fitting figure such as the above-mentioned ellipse fitting, the image of the conveyed object CA is fitted to various figures such as circle, ellipse, rectangle, polygon, etc., and the figure which is the closest is formed. In addition, in the conveyed object detection process, shape detection technology, pattern detection technology, area extraction technology, etc. of image processing techniques can be used, and thus various information related to the conveyed object CA can be recognized, classified, distinguished, and the like. The specific methods of the above technologies are not limited, and template matching, blob detection methods, various edge detections, various conversion processes, and the like can be used. In this embodiment, in order to form a bounding box, the following method can be used. The rotation calipers algorithm is used to obtain the angle of each included angle through calipers rotated with respect to the point group of edge pixels. The rectangle with the smallest area among the resulting enclosing rectangles. In addition, as the ellipse fitting, for example, a method of obtaining an ellipse in which the squared error is minimized for a point group of edge pixels can be used.

From the conveyed object detection range Bia that can be derived as described above, the position and range of the conveyed object CA in the observation area Di can be known. In addition, the posture of the conveyed object CA can also be detected from the conveyed object detection range Bia. For example, as shown in FIG. 5( e ), the angle posture of the conveyed object CA can be obtained by obtaining the direction of each side of the boundary graph, the fitting graph, and the direction of the main axis. In the illustrated example, the posture of the conveyed object CA can be easily obtained by calculating the angle (main axis angle) θ of the main axis of the fitted ellipse. In the present embodiment, an example of recognizing the two-dimensional posture is given, but the three-dimensional posture of the conveyed object CA can also be obtained by analyzing the appearance of the conveyed object CA. In addition, in general, when the conveyed object CA is far from the camera or the lighting environment is uneven, as will be described later, it may be difficult to obtain a boundary pattern (eg, a rectangle) reflecting the precise outer shape of the conveyed object CA with high accuracy. , preferably a method to obtain a compact fitting figure such as an ellipse or a circle.

In the present embodiment, when at least a part of the image portion of the conveyed object CA is not included in the observation area Di, the observation area Di may be further included in the observation area Di so that the entire conveyed object CA is included in the observation area Di. Observation area correction unit for correction. In this way, the observation area correction means corrects the observation area so as to include the entire image portion of the conveyed object CA, so that even when the prediction of the set observation area is insufficient, the object can be detected reliably. Here, as a method of detecting that at least a part of the image portion of the conveyed object CA is not included in the observation area Di, for example, a method of detecting whether the conveyed object detection range Bia obtained by the above-mentioned method is not A method for detecting that the conveyed object detection range Bia is in contact with the boundary of the observation area Di, and a method for detecting the conveyance A method in which the shape of the object detection range Bia is greatly deformed from a predetermined shape, or a combination of the above-mentioned methods, or the like. When it is determined by these methods that the conveyed object CA is not contained in the observation area Di, the observation area Di is further expanded or moved according to the detection state to be reset, and then the reset observation area Di is further implemented. like processing.

Next, the case of the conveyed object CA on the conveyance path 121 processed by the conveyed object identification processing unit 100 will be described with reference to (a) to (g) of FIG. 3 . Here, an example of the process of identifying the conveyed object CA by the conveyed object identification processing unit 100 based on the images Ai on the conveying route captured by the above-mentioned cameras CM1 and CM2 of the conveying device of the conveying system 10 is shown. In addition, the plurality of images A01 , A02 , A1 to A5 shown in FIG. 3 are constituted by at least a part of the image data of the above-mentioned captured image GPX or image area GPY.

First, as shown in (a) and (b) of FIG. 3 , in the images A01 and A02 in the image area GPY obtained by the above-mentioned images of the cameras CM1 and CM2 , the images A01 and A02 in the conveying direction F will be short and The limited area T of the shape long in the direction intersecting with the conveyance direction F is set in advance within the range where the conveyed objects on the conveyance path, that is, the conveyed objects CA pass through. Then, the pixel group limited in the limited area T changes from the "presence" state that indicates the existence state of the conveyed object CA (the distribution of pixel values such as luminance along the extension direction of the pixel group) as in the image A01. When the state is switched to the "none" state indicating the absence of the conveyed object CA (gap) as in the image A02, it is determined that the conveyed object CA has passed through the limited area T. When making this determination of the passage of the conveyed object, as shown in the figure, this image A02 is set as the first image A1, or the next image of the image A02 is set as the first image A1, and the conveyed object CA is set as the first image A1. The first observation area D1 is set at the passing detection position of , that is, the downstream side of the limited area T. When the image A02 is shown as the first image A1, the first observation area D1 includes the conveyed object CA in the downstream area adjacent to the limited area T as shown in FIG. 3( c ). The size is set at a fixed position. On the other hand, when the image A02 is the first image A1, the difference between the time (time difference) between the image A02 and the image A1 and the conveyance direction F of the conveyed object CA is considered. The movement amount of the conveyed object CA obtained by the movement speed is set as a range deviated from the limited area T to the downstream side by a distance corresponding to the movement amount in the first observation area D1. In these cases, the initial observation area D1 can be limited and set within a narrow area close to the amount of one conveyed object CA, so that it becomes easier to increase the speed and height of image processing.

The above-mentioned conveyed object passing determination is based on the distribution of pixel values in the limited area T to determine whether the conveyed object CA is passing through, and this identification process can be realized by various processing procedures. As an example, the discrimination process can also be performed by using a neural network that inputs the pixel values of the pixel columns defining the region T to an input layer with the same number of nodes, via an appropriate intermediate layer (hidden) layer), and discern whether the conveyance CA is passing through the defined area T, or whether the defined area T is in the gap between the conveyed articles CA, through the output of, for example, an output layer having one or more nodes. In the present embodiment, for example, a fully coupled neural network is configured, and an example of discrimination processing of a conveyed object CA that is learned by supplying 29 frames (images) or more of learning data to the fully coupled neural network is used. The fully coupled neural network has: an input layer with nodes corresponding to 53 pixels, a two-layer middle layer, and An output layer with one node. In this process, for example, it is possible to distinguish between multiple nodes of the output layer. The direction of the conveyed object CA, the position of the appearance pattern (the marking part described later), and the like. In addition, the conveyed object passage determination means that performs this conveyed object passage determination process is not limited to the above-mentioned configuration as long as it can detect that the conveyed object CA has passed through the predetermined limited area T. In addition, by setting the initial observation area D1 in the primary image A1 when it is detected that the conveyed object CA has passed through, and the above-mentioned limited area T in a predetermined positional relationship, the position of the conveyed object CA that has passed through the limited area T can be set. scope.

In addition, in the illustrated example, in any case, the first observation area D1 is set in a range that faces the air outlet OP for ejecting the airflow opened on the conveyance path with respect to the conveyance CA (the conveyance processing part, the same below) on the more upstream side. In addition, in the conveyed object processing section provided with the air outlet OP, a solenoid valve, not shown, etc. is controlled based on the determination result of whether or not the conveying object CA should be sprayed on the conveying path 121 through the conveyed object discrimination process. , so as to determine whether the airflow is ejected from the air outlet OP.

In the conveyed object identification processing, image processing is performed in the above-mentioned observation area D1 in the same manner as described above, and the conveyed object detection information Bi is obtained by the above-mentioned conveyed object detection processing. In the illustrated example, the conveyed object detection range B1a is depicted in (c) of FIG. 3 . In addition, processing required for identification of the conveyed object other than the conveyed object detection process is performed as necessary. As described above, the observation area D1 is set on the upstream side of the range in which the conveyed object CA faces the air ejection port OP. Here, the first predicted movement form C1 is obtained for the conveyed object CA related to the conveyed object detection information B1a obtained in the first observation area D1. In this case, the first predicted movement form C1 can be obtained from the moving speed of the conveyed object CA in the conveying direction F (for example, the average conveying speed of objects on the conveying path) and the advancing direction of the conveying direction F as the moving direction.

Here, in the first predicted movement pattern C1, only the movement component in the x direction along the conveyance direction F may be set, and the movement component in the y direction (vertical direction) may be set to 0 without consideration. However, the conveyed object CA moves up and down on the conveyance path 121 due to vibration or the like, and it is known in advance that the conveyed object CA is in the y direction When the moving speed (including the direction, the same below) is high, the possibility of continuing to move at the same moving speed due to inertia is high, so the moving component in the y direction can also be considered. In addition, when the vertical movement during the conveyance of the conveyed object CA is large, the expansion degree Δh may be previously set to a certain value or more in accordance with the vertical movement. Furthermore, in the case where the speed of the conveyance direction F increases or decreases sharply during the conveyance of the conveyed object CA, the expansion degree Δw may be previously set to a certain value or more. Here, if certain degrees of expansion Δw and Δh are set in the observation area Di, the predicted movement form Ci can also be set as a fixed predicted movement amount where the movement direction is The advancing direction of the conveyance direction F is obtained by assuming that the conveyance speed of the conveyed object CA is a constant speed. In this way, since the observation area Di can be easily set, the conveyed object detection processing procedure 101 can be executed simply and at high speed.

Then, as shown in (d) of FIG. 3 , a reference range (d2), not shown, is obtained from the conveyed object detection range B1a and the predicted movement pattern (C1), not shown, and based on this, the next step is set. Observation area D2 in image A2. Here, since the conveyed object CA moves in the conveyance direction F on the conveyance path 121, the observation area D2 is shifted in the conveyance direction F by a predetermined movement amount with respect to the above-mentioned observation area D1. Here, the moving amount corresponds to the product of the conveyance speed of the conveyed object CA (the actual speed may be measured, or the average conveyance speed may be preset and used) and the photographing time difference Δt between the images A1 and A2 . When the conveyed object detection processing is performed in the observation area D2, the conveyed object detection information Bi is obtained, and the conveyed object detection range B2a is also determined. Here, the past movement pattern E1 of the conveyed object CA, that is, the moving amount (moving speed and moving direction) between the actual images A1 and A2 of the conveyed object CA can be obtained from the conveyed object detection ranges B1a and B2a. In general, from these past movement patterns Ei, the predicted movement patterns Ci after the setting of the observation area Di+1 can be obtained.

Then, as shown in (e) of FIG. 3 , the observation area D3 in the next image A3 is set in the same manner as described above. After that, as shown in (f) in FIG. 3 , the observation area D4 in the next image A4 is set as in FIG. 3 As shown in (g), the observation area D5 in the next image A5 is set. In this way, it is possible to grasp the conveyance CA in the conveyance path 121 based on the conveyed object detection information Bi obtained in the above-mentioned process, in particular, the movement conditions of the conveyed object detection ranges B1a to B5a, that is, the above-mentioned movement patterns Ei (E1 to E3). condition of the transport.

In the example shown in FIG. 3 described above, before the conveyed object CA reaches the conveyed object processing unit in which the air ejection port OP is provided, the plurality of images A1 to A4 can be transmitted through the image processing in the observation areas D1 to D4, respectively. Then, the identification process of the conveyed object CA is performed. The identification processing of the conveyed object CA acquires the conveyed object processing information G1 to G4 for the purpose of identifying the conveyed object processing section located on the downstream side of the arrangement position of the conveyed object CA in the plurality of images A1 to A4 described above. Information necessary to control the handling of conveyed objects such as screening and posture change in the The conveyed object processing information Gi may include the above-mentioned conveyed object detection information Bi. In addition, in the case where the above-mentioned necessary information is sufficient for the conveyed object detection information Bi obtained in the above-mentioned conveyed object detection processing, the identification of the conveyed object CA is performed directly based on the conveyed object detection information Bi. In each image Ai, the conveyed object processing result Ji is derived from the conveyed object processing information Gi. The conveyed object processing result Ji is a result that directly indicates the above-mentioned necessary information.

When the conveyed object processing result Ji is obtained as described above, as long as the predetermined measurement process termination condition is not satisfied, the prediction setting of the next observation area Di+1 is completed, and the process proceeds to the next conveyed object identification process. On the other hand, when the above-mentioned measurement process termination condition is satisfied, a predetermined conveyed object discrimination result K is calculated based on the plurality of conveyed object processing results Ji obtained as described above. Here, the measurement processing termination condition is whether the predetermined maximum processing count m has been reached, or whether the transported object discrimination result K can be derived from the combination of a plurality of transported object processing results Ji even though the maximum processing count m has not been reached. The above-mentioned maximum number of processing times m is determined by the imaging time difference Δt of the image Ai, the conveyance speed ^vx of the conveyed object CA, the position of the first observation area D1, and the like. In the case of the illustrated example, the maximum number of times m of processing is 4 according to the above-mentioned various conditions. In addition, the relationship between the conveyed object processing result Ji and the conveyed object discrimination result K will be described later.

The method of the conveyed object detection processing procedure 101 used in the conveyed object identification processing unit 100 is not limited to the methods represented by the above-mentioned equations (1) to (4). For example, the conveyed object CA basically moves on the conveying path 121 along the conveying direction F. Therefore, a method of shifting the observation area Di along the conveying direction F by a fixed moving amount corresponding to the time difference Δt between images may be employed.

As an example of the conveyed object processing unit including the above-mentioned air outlet OP, in order to control the conveyed object CA from which the predetermined conveyed object discrimination result K is derived, a conveyed object removal unit for excluding a part of the conveyed object CA from the conveying path 121 can be exemplified. and a posture changing unit for changing the posture by inverting a part of the conveyed object CA. In addition, a part of the conveyance CA may be considered to distribute a part of the conveyance CA to the conveyance distribution part on two or more downstream branch paths.

FIGS. 6( a ) to ( e ) are explanatory diagrams showing the identification form of the conveyed object CA by the conveyed object identification processing unit 100 of the above-described embodiment of the conveying system 10 . In addition, in these explanatory diagrams, the conveyance form of the conveyance CA on the conveyance path 121 shows the state observed from a different direction from that of FIG. 3 showing the images Ai captured by the cameras CM1 and CM2 , and is drawn with a cross-sectional structure. Transport path 121. The conveyed-object processing part provided with the air outlet OP is shown on the right side of the figure on the downstream side. Here, the plurality of conveyed objects CA shown in the figure are all the same specific conveyed object CA, and the above-mentioned specific conveyed objects CA captured in the plurality of images Ai are repeatedly displayed.

In the example shown in FIG. 6( a ), the conveyed object CA is sequentially subjected to the conveyed object discrimination processing in the observation areas D1 to D4 using a plurality of (four) images A1 to A4 . Since the above-mentioned conveyed object processing result J1 If it is OK, J2 is OK, and J3 is OK, the same processing result occurs more than half times three times, so the fourth processing is aborted (unnecessary), and the final conveyed object discrimination result K is set as OK. Therefore, the airflow will not flow from the air port OP flow, and the conveyed material CA directly passes through the conveyed material processing unit. In this example, if processing is performed in the observation area D4 for the fourth time, an erroneous processing result J4=NG may occur due to a change in the position or posture of the conveyed object CA, however, since the processing result for the third time is already OK, The conveyed object discrimination result K=OK is set based on these results, whereby erroneous discrimination results can be avoided. In this example, as the measurement end condition, among the two processing results OK and NG, the maximum number of processing times is four, and if the set number of the same conveyed object processing result Ji is three or more times at the same time as the maximum number, the conveyed object will be removed. The discrimination result K is set to this result, and the subsequent processing is terminated. In addition, in the case where the processing is carried out to the end, if the same number of processing results OK and NG appear twice each, if the conveying capacity (conveying efficiency) is given priority, the OK of the conveyed object processing result Ji is set as the conveyed object discrimination priority. result K. On the other hand, if the screening accuracy (screening effect) is given priority, the NG of the conveyed object processing result Ji is set as the conveyed object discrimination result K with priority.

In the example shown in FIG. 6( b ), the conveyed object processing result Ji and the conveyed object discrimination result K are the same as those shown in FIG. 6( a ), but the time difference Δ when a plurality of images A1 to A4 are acquired Since t is shorter than the above, although the conveyance speed of the conveyed object CA is the same, the setting interval of the observation areas D1 to D4 is smaller than the above. In this way, since the conveyed objects CA are frequently subjected to the discrimination processing at positions close to each other so as to overlap each other, the conveyed objects CA can be processed close to the conveyed object processing unit using the conveyed object discrimination result K. Therefore, the maximum number of times m of processing can be easily increased, and the discrimination accuracy can be improved on the conveyance path where characteristics such as the posture of the conveyed object CA may frequently change during conveyance (such as when a plurality of conveyed object processing units are close to each other).

In the example shown in FIG. 6( c ), it is assumed that since the posture of the conveyed object CA is bad, the first to third conveyed object processing results are all NG, but the fourth The next processing result is OK due to the change in the posture of the conveyed object CA. However, even in this case, since the conveyed object processing result Ji=NG appears in G1 to G3, the discrimination process is not performed, and the conveyed object discrimination result K=NG is set. Thereby, the air flow is ejected from the air outlet OP, and the conveyance CA is removed from the conveyance path 121 or perform a posture reversal. In this example, it is assumed that the posture of the conveyed object CA in the fourth observation area D4 is close to the good product posture, and an OK judgment is given. However, if it does not finally reach the good product posture and passes directly downstream in the bad product posture, the produce a bad situation. Therefore, the conveyed object CA is controlled by the air flow from the air injection port OP in the state of NG determination, and the inconvenience caused by an erroneous determination result can be avoided.

In the example shown in FIG. 6( d ), the case where the conveyed object processing result Ji is unstable due to the change in the posture of the conveyed object CA is shown. In this case, the maximum number of processing times is that NG occurs continuously for the first two times (the set number) among the four times. Therefore, the subsequent processing is suspended, and the conveyed object discrimination result K=NG is set. In this example, since the screening accuracy (screening effect) is prioritized, a conclusion can be made when NG occurs twice or more in the four processings, so no further processing is required, and the conveyed object discrimination result K=NG is set .

In the example shown in FIG.6(e), the example in which the conveyed object processing result Ji differs by the positional change by the up-and-down movement of the conveyed object CA is shown. In this example, if the vertical movement is too large, the accuracy of image processing is lowered, and the conveyed object processing result Ji=NG, but as long as the vertical movement is normal, Ji=OK. In this example, the conveying capacity (conveying efficiency) is set first, and no further processing is required when the conveyed object processing result Ji=OK appears twice (the set number), and the conveyed object discrimination result K=OK is set.

However, the conveyed object processing result Ji is generally not limited to the case where there are two processing results (OK and NG) as described above, and there may be three or more processing results. In these cases, in general, when a predetermined number (set number) is generated for each processing result, the result can be set as the conveyed object discrimination result K. In this case, the above-mentioned set numbers may be different from each other among the plurality of processing results. This is because, for example, as described above, it may be considered from the viewpoint of emphasizing the conveyance capability, and may be considered from the viewpoint of emphasizing the conveyance accuracy. It is usually preferable to obtain a certain treatment result that exceeds half (the maximum number) of the maximum treatment number m. If it is, it is set as the conveyed object discrimination result K corresponding to the processing result Ji. However, it is not necessary to exceed half, and when the same processing result Ji is obtained at least twice, the conveyed object discrimination result K corresponding to the processing result Ji is derived, which can greatly improve the accuracy of the discrimination result compared with the prior art.

<Configuration of Action Program 10P>

Next, the flow of the overall operation program 10P of each embodiment of the present invention will be described with reference to FIG. 8 . FIG. 8 is a schematic flowchart showing various processing procedures for conveyance management executed by the arithmetic processing unit MPU of the inspection processing unit DTU according to the operation program 10P. When the operation program 10P is activated, the above-described image capturing and image measurement processing is first started, and at the same time, the driving of the conveying devices (the feeder 11 and the linear feeder 12 ) is started through the controllers CL11 and CL12 . Then, if the debug setting corresponding to the debug operation described above is OFF, the image measurement process is performed on the captured image GPX or the image area GPY, and if the final determination result of the conveyed object discrimination process is an "OK" determination, only The image measurement processing of the next captured image GPX or image area GPY is directly performed without performing the debugging operation. For example, in the conveyed object removal processing section, the air flow of the air injection port OP is always stopped, and when the determination result is "NG" (defective product), the air flow flows out from the air injection port OP. Thereby, the defective conveyed material CA is excluded from the conveyance path. In addition, in the posture inversion processing unit, the air flow from the air outlet OP is normally stopped, and when the determination result is "NG" (defective product), the air flow is ejected from the air outlet OP to turn the conveyed object CA on the conveyance path. In addition, contrary to the above, the airflow may flow normally, and when the determination result is "OK" (good product), the airflow may be stopped.

In this way, by discriminating the conveyed object CA as the conveyed object on the conveyance path, and processing according to the result of the discrimination, only the good products are supplied to the downstream side in a state of being aligned. In this case, as long as no debugging operation is performed thereafter, the image measurement process and the conveyed object identification process are directly performed in the measurement area of the next captured image GPX or the image area GPY. Here, the conveyed object performed by the conveyed object identification processing unit 100 In the discrimination processing, the processing for a certain conveyed object CA is performed as described above, but for a plurality of conveyed objects CA that are continuously conveyed into the measurement area, the same conveyed object discrimination processing is usually performed in parallel for each conveyed object CA, respectively. . In addition, in parallel with the above-described image measurement processing or conveyed object identification process, for example, a conveyance behavior of tracking the conveyed object CA on the conveying path in order to detect the conveying behavior of the conveyed object CA when conveyed in the conveying direction F on the conveying path may be performed. detection processing. Furthermore, it is also possible to adjust the conveyance form by controlling the drive of the conveyance device based on the detection result of the conveyance behavior detection process. The control of this conveyance drive, for example, controls the drive conditions of the excitation element of the excitation mechanism of a conveyance apparatus, for example, controls the frequency or voltage of a piezoelectric driver, and adjusts to an appropriate conveyance form. In addition, the above-mentioned conveyance behavior detection process may be executed in parallel with the conveyed object detection processing procedure 101 used in the above-mentioned conveyed object identification processing unit 100, or may be completely performed by separate image processing independently of the above-mentioned conveyed object detection process. implement. For example, the frequency and amplitude of the vibration can be controlled according to the magnitude of the positional variation in the direction perpendicular to the conveying direction F of the conveyed object CA when moving in the conveying direction F on the conveying path 121, thereby preventing the conveyed object CA from being trapped on the conveying path. Unnecessary turmoil on the 121.

When the debugging operation is performed in the middle of the above, and the debugging is set to "ON", the above routine mode (operation mode) is exited, the driving of the conveying device is stopped, and the image measurement processing, conveyed object identification process, conveyed object detection process, Conveying movement detection processing. Then, if an appropriate operation is performed in this state, a state in which a past image file can be selected is obtained. At this time, the image file selected to be displayed is an image file including a plurality of shot images GPX or image areas GPY recorded in the operation mode just now. When the image file is directly selected and an appropriate operation is performed, the re-execution mode is entered. In this mode, based on the image file that records the processing results of the image measurement process, the conveyed object identification process, the conveyed object detection process, the above-mentioned conveyance behavior detection process, etc. that have been executed, the image display, various processing or control, etc. That is, when a defect occurs in the control (rejection, inversion, conveyance detection) of the conveyed object of the conveying device, that is, the conveyed object CA, in order to eliminate the defect, first, based on the past image data, the Image processing is performed each time to find out the problem area of each processing or control. If the problem location is identified, the setting contents (setting values) of each process or control can be changed or adjusted according to the problem location, and the image measurement process or the like can be performed again on the past image data, thereby making it possible to adjust and improve the work. results are confirmed. Then, if an appropriate reset operation is performed, the debug setting returns to "OFF", thereby restarting the image measurement process and restarting the drive of the conveyor.

In addition, in each of the above-described embodiments, for each conveyed object CA on the conveying path, the two or more images Ai in which the conveyed object CA was captured are sequentially processed, whereby the images Ai from each image are processed in sequence. A plurality of processing results Ji of the conveyed object CA obtained by Ai are obtained, and the conveyed object identification result K is derived. Thereby, it is possible to reduce errors in the discrimination results due to changes in the position and posture of the conveyed object CA.

On the other hand, in the prior art example, as shown in FIG. 7( a ), a photo sensor SA is provided on the conveyance path 131 to detect the passage of the conveyed object CA, and an image is captured at the detection timing. SB, the conveyed object CA is determined based on the image SB. However, in this case, there is a situation where image processing is affected by changes in the position or posture of the conveyed object CA, and an erroneous determination that the conveyed object CA is a defective product NG occurs. In this case, even if the conveyed object CA is originally a good product, it is OK , is also excluded from the conveying path 131 by the airflow from the air outlet OP, or the posture is reversed.

Further, as shown in (b) of FIG. 7 , even in the case of performing image measurement processing based on triggerless shooting as in the above-described present embodiment, since the position of the conveyed object CA, which is the object of the conveyed object identification process, is determined by Since it is limited to the predetermined range SD in the measurement area SC, similarly to the above, it is easy to erroneously determine that the conveyed object CA is a defective product NG due to a change in the position or posture of the conveyed object CA.

Different from the above-mentioned prior art, in this embodiment, two or more conveyed object processing results Ji are obtained based on two or more images Ai for each conveyed object CA, and conveyance is derived from these conveyed object processing results Ji. In the case of the object discrimination result K, the observation area Di for performing the image processing in the two or more images Ai is further provided for each conveyed object CA, so that the load of image processing for each conveyed object CA can be reduced. In addition, even if image processing for a plurality of conveyed objects CA is performed in parallel, high-speed processing can be performed.

In the present embodiment, the observation area Di is set for each conveyed object CA based on the position information of the conveyed object CA and the predicted movement pattern Ci of the conveyed object CA, and the image is performed while tracking the conveyed object CA in the observation area Di. Therefore, even when a plurality of conveyed objects CA continuously pass through the measurement area, the conveyed object identification process and the conveyed object detection process can be executed in parallel on the plurality of conveyed objects CA without being confused with each other.

In the present embodiment, the next observation area Di+1 corresponding to the conveyed object detection range Bia and the predicted movement form Ci is set, and the conveyed object detection processing based on the image processing in the observation area Di+1 is performed. Since the conveyed object CA is detected and the conveyed object detection information Bi+1 is derived, it is possible to reliably detect the conveyed object CA while tracking the conveyed object CA while limiting the image data to be processed. Therefore, it is possible to increase the processing speed, and it is possible to perform processing more advanced than that of the prior art within a limited time. In addition, by repeating the prediction of the observation area and the image processing described above, it is possible to easily grasp the complicated moving form and operation of the conveyed object CA (the conveyed object moving state MS).

In particular, through the above-mentioned conveyed object detection processing procedure 101, the moving form and operation of the conveyed object can be grasped at high speed and with certainty. Thereby, not only the speed-up of the conveyance system but also the high density of the conveyed objects CA on the conveying path and the high precision of the arrangement of the conveyed objects CA can be achieved.

In this embodiment, when setting the next observation area Di+1, the surrounding area is further added to the reference area di+1 obtained from the conveyed object detection information Bi and the predicted movement pattern Ci obtained from the observation area Di. Therefore, even if the conveyed object CA shows a different movement pattern from the predicted movement pattern Ci, the conveyed object detection information Bi can be more reliably derived from the next image Ai+1 +1.

In addition, the conveying system of the present invention is not limited to the above-described examples, and it is needless to say that various modifications can be added without departing from the gist of the present invention. For example, the present invention only needs to be a conveying system that obtains a conveyed object discrimination result K based on a plurality of conveyed object processing results Ji for a plurality of images Ai, and the method of obtaining K from this Ji depends on the purpose of the conveying system Or use the method to set it.

100: conveyed object identification processing department

Claims (12)

A conveying system, characterized in that it has: The conveying device is used to convey the conveyed objects along the conveying road; An image acquisition unit is a unit that acquires an image of the measurement area where the conveyed object passes through the conveying path, and ensures that all the conveyed objects passing through the conveying path must be photographed in the measurement area respectively acquiring a plurality of said images in sequence by means of two or more of said images; and The conveyed object identification unit is a unit that performs image processing on the image portion of the conveyed object in the image to identify the conveyed object, and, for each of the conveyed objects, records the image of the conveyed object. The image processing is sequentially performed on the two or more images of , whereby a discrimination result is derived from the processing results for the two or more images. The conveying system of claim 1, wherein, The conveyed object identification unit derives, for each of the conveyed objects, the identification result corresponding to the most processing result among the respective processing results for the two or more images. The conveying system of claim 1, wherein, The conveyed object identification means derives the identification result corresponding to the predetermined processing result when the predetermined processing result for the two or more images reaches a set number for each of the conveyed objects. The conveying system of claim 1, wherein, The two or more images are three or more images. The delivery system of any one of claims 1 to 4, wherein, The conveyed object identification means determines the position information of the conveyed object and the conveyed object based on the position information of the conveyed object obtained from the previously processed image for the second and subsequent images of the two or more images. The image processing is performed within the viewing area defined by the predicted movement morphology. The conveying system of claim 5, wherein, The predicted movement form is a predicted movement amount in the conveying direction corresponding to the conveying speed of the conveyed object and the time difference between the imaging of the image processed this time and the image next time. The conveying system of claim 5, wherein, The observation area for the images after the second time is set as an area expanded and formed so as to include a peripheral area arranged around the reference area on the basis of the reference area, wherein the reference area is The range is a range formed by arranging the detection range of the conveyed object obtained by the image processing of the previous image at the predicted position after the movement. The delivery system of claim 7, wherein, The degree of expansion of the observation area with respect to the reference range increases or decreases according to the magnitude of the deviation of the past movement form of the conveyed object. The conveying system of claim 5, wherein, The predicted movement form is calculated based on the past movement form of the conveyed object. The conveying system of claim 5, wherein, The conveyed object identification unit executes image processing in a predetermined first observation area set so that the conveyed object can be arranged for the first image among the two or more images, and performs image processing according to the predetermined image. The first time the predicted movement shape is set for the next time the viewing area of the image. The delivery system of any one of claims 1 to 4, wherein, It is further provided with a conveying control unit for controlling the conveying device; The conveyance control unit controls the conveyance device based on the conveyed object identification information. The delivery system of claim 11, wherein, The said conveyance control part controls the conveyed object processing means provided in the said conveyance apparatus for sorting and changing a posture of the said conveyed object based on the said discrimination|determination result.

Images