JP4765942B2 - Monitoring device and monitoring method, and control device and control method - Google Patents

Description

  The present invention relates to an apparatus and method for performing monitoring and control by performing image processing on a captured image.

  With the recent development of video cameras, image recognition, and image processing technologies, technologies for monitoring or controlling using image recognition have been developed in the field of monitoring or controlling the operation of movable members such as automatic assembly equipment. Yes. For example, in an assembly process, image recognition is performed on a captured image and the process is monitored. In this case, the timing for executing image recognition on the captured image is given by a programmable logic controller (PLC) that detects the position of the movable member by various sensors and receives the detection signal.

  In facilities where automatic assembly and inspection are performed, the movable unit is often moved at high speed, or a plurality of movable units are operated simultaneously. In a facility in which a plurality of movable units move at high speed, a movable unit that is a monitoring target, that is, an imaging target, may be prevented from being captured by a camera due to the operation of another movable unit. For example, if the movable unit that hinders imaging is a work robot equipped with a vacuum chuck, air piping or electrical wiring is bundled, and the imaging screen is greatly affected during high-speed movement. In such a case, in order to guarantee the accuracy of the image processing, the PLC determines that the movable unit that may interfere with imaging has stopped at the moving end that does not interfere with imaging, and performs image processing of the captured image. The command to execute was issued. As a result, there is a waiting time before the start of image processing, and it is difficult to shorten the cycle time of the equipment.

  Conventional problems will be described with reference to FIGS. 13 (a) and 13 (b). FIG. 13A is a diagram conceptually showing image processing for monitoring or control in a control device for a facility having a conventional movable member, and FIG. 13B illustrates the timing of the image processing. FIG.

  The facility 70 includes a first movable member 71 and a second movable member 72, and the video camera 73 monitors the first movable member 71. An image captured by the video camera 73 is input to the image processing unit 75, and the image processing unit 75 performs image processing in synchronization with a command from the PLC 76. The PLC 76 receives signals regarding the positions of the first and second movable members 71 and 72 from the sensors 74-1 and 74-2, and determines the timing at which the image processing unit 75 starts image processing. The result of the image processing is sent to the PLC, and if there is an abnormality, the facility is stopped.

  The first movable member 71 moves between the positions p1 and p2, and the second movable member 72 moves between the positions p3 and p4. After the first movable member 71 moves to the position p1, the second movable member 72 moves to the position p4 that prevents the camera 73 from capturing the first movable member 71, and then returns to the position p3.

  In order to monitor the first movable member 71, it is necessary to perform image processing on an image captured when the first movable member 71 is at the position p1 and the second movable member 72 is at the position p3. Therefore, conventionally, when the sensor 74-2 confirms that the first movable member 71 is stopped at the position p1, and the sensor 74-1 confirms that the second member 72 is at the position p3, the PLC 76 Instructs the image processing unit 75 to start image processing.

  The timing of image processing by the image processing unit 75 is shown in FIG. After the first movable member 71 reaches a predetermined position (t1), the second movable member 72 moves upward. Thereafter, after confirming that the second movable member 72 has retracted (t2) from above the first movable member, image processing is performed on the captured image. Conventionally, in order to ensure the accuracy of image processing, a waiting time WT is required for confirming that the second movable member 72 has been retracted to a position that does not hinder the imaging of the camera 73, that is, the position p3. Since the waiting time WT is related to the accuracy of image processing, it is difficult to shorten the waiting time WT. Therefore, it is difficult to shorten the cycle time of the equipment. Even if the waiting time until the start of such image processing is 0.5 seconds, for example, if the cycle time is 3 seconds, the ratio of the waiting time is large and the cycle time cannot be shortened. There was also a problem that had to be increased.

Japanese Utility Model Publication No. 5-40908 JP-A-9-76185

  In view of the above problems, the present invention provides a monitoring device, a monitoring method, and a control device that can autonomously determine the timing of image processing by the image processing unit in the image processing unit and, as a result, reduce the operating time of the equipment. An object is to provide a control method.

In order to achieve the above object, the invention of the monitoring device according to claim 1 includes a first member (3, 10) and a second member (5) that moves relative to the first member. , 41, 43), an imaging unit (6, 51) for imaging the first member, and an image processing unit (7, 53) for processing an image captured by the imaging unit, The members (3, 10) may be prevented from being imaged by the imaging unit due to the positional relationship between the first members (3, 10) and the second members (5, 41, 43). Two feature portions (4, A1, A2, 251, 252), and the first member (3, 10) has a movable portion (3, 11-22), and the at least one feature portion ( 4, A1, A2) further indicates that the movable part of the first member (3, 10) is stopped. And than, the image processing section (7,53), said characteristic portion of the image captured by the imaging unit (6,51) (4, A1, A2,251,252) first image the presence or absence of The second image processing is executed on the image according to the presence / absence of the characteristic portion (4, A1, A2, 251, 252).

In the invention of the monitoring device of claim 1, since the image processing unit autonomously determines the timing for executing the second image processing, the first member or the second member can be determined without waiting for confirmation of the position of the second member. Two image processes can be executed. Moreover, the sensor which detects the position of a 1st member or a 2nd member can also be abbreviate | omitted. Furthermore, reliable monitoring can be performed even if the first member has a movable part.

According to a second aspect of the present invention, the second member (41, 43) is a movable member, and is a robot that operates on the first member. Thereby, it is possible to reliably monitor before and after the robot work.

Furthermore, the invention of the control device according to claim 3 includes a first member (3, 10), a second member (5, 41, 43) that moves relative to the first member, The operation control unit (8, 55) for controlling the operation of at least one of the first member and the second member, the imaging unit (6, 51) for imaging the first member, and the imaging unit An image processing unit (7, 53) for processing a captured image is provided, and the first member (3, 10) includes the first member (3, 10) and the second member (5, 41). 43) at least one characteristic part (4, A1, A2, 251, 252) that may be prevented from being captured by the imaging unit due to the positional relationship with the image processing unit (7, 53). ) Are the characteristic portions (4, A1, A2, 2) in the image captured by the imaging unit (6, 51). 51, 252) is determined by the first image processing, and the second image processing is executed on the image according to the presence / absence of the characteristic portion (4, A1, A2, 251, 252), The control of the operation control unit (8, 55) is changed according to the result of the second image processing.

In the invention of the control device according to the third aspect , since the timing for executing the second image processing is autonomously determined by the image processing unit, the first member or the second member can be determined without waiting for confirmation of the position of the second member. 2 image processing can be executed, and the control cycle time can be shortened. Further, the sensor for detecting the position of the first member or the second member can be omitted, and the apparatus itself can be simplified.

Furthermore, in the control device according to claim 4 , the first member (3, 10) has a movable part, and the at least one characteristic part is further the first member (3, 10). It is characterized by showing that the movable part is stopped. Thereby, even if the 1st member has a movable part, reliable control can be performed.

According to a fifth aspect of the present invention, the second movable member (41, 43) is a robot that performs an operation on the first member. As a result, it is possible to monitor the robot before and after work, and to perform effective control.

The invention of the control method according to claim 6 is a control method including a step of moving the second member (5, 41, 43) relative to the first member (3, 10). The imaging is hindered by the positional relationship between the first member (3, 10) and the second member (5, 41, 43) provided on the first member (3, 10). The first member is imaged including at least one characteristic part (4, A1, A2, 251 and 252), and the first member (3, 10) is imaged with respect to the first image. Image processing is performed to determine the presence or absence of the characteristic portion (4, A1, A2, 251, 252) in the image obtained by imaging the first member, and the characteristic portion (4, A1, A2, 251, 252). ), The second image processing is executed on the image and the result of the second image processing is determined. By, and it performs changing of control.

In the invention of a control method of claim 6, since the timing for executing the second image processing is determined by the first image processing, without waiting for the determination of the position of the first member or the second member, the second Image processing can be executed, and the control cycle time can be shortened. Moreover, the sensor which detects the position of a 1st member or a 2nd member can also be abbreviate | omitted.

  In the following, embodiments of the present invention will be described with reference to the drawings, but before that, in order to facilitate understanding of the present invention, a diagram that can be compared with a diagram for explaining the problems of the conventional example of FIG. Thus, the outline of the present invention will be described.

  FIG. 1 is a diagram for explaining the outline of the present invention. The facility 1 includes a first movable member 3 and a second movable member 5, and a camera 6 that images the first movable member 3 is disposed to monitor the first movable member 3. In FIG. 1, the camera 6 is positioned directly above the first movable member 3, but may be arranged with an appropriate angle. An image captured by the camera 6 is input to the image processing unit 7. On the first movable member 3, the characteristic portion 4 is designated as a portion in which imaging may be hindered by the second movable member 5. The feature portion 4 may be affixed with a marker or may be a location different from the other of the first movable member.

  The image processing unit 7 inputs an image captured by the camera 6 every predetermined time, and determines whether or not the feature portion 4 exists at a predetermined position on the image by the first image processing. The timing for starting the second image processing for the purpose of control is autonomously determined. That is, the first movable member 3 moves between the positions p1 and p2, and the second movable member 5 moves to the position p4 when the first movable member 3 is at the position p1, and then the position p3. , When the feature point 4 is found at a predetermined position on the image, the first movable member 3 is at the position p1, and the second movable member 5 is at a position that does not interfere with imaging. I understand. If the feature point 4 can be found in the image until it moves to the position p3 or when it moves, the target second image processing can be performed with the image. Therefore, it is possible to determine the timing of image processing without putting a waiting time as in the prior art. Then, the result of the image processing is sent to the PLC, and if there is an abnormality, the equipment control can be changed, such as stopping the equipment.

  Thus, according to the present invention, the image processing unit 7 determines the presence of a feature point of an image captured by the camera 6 and finds a timing at which the image processing unit 7 autonomously performs image processing. Therefore, compared with the determination of the image processing timing by the conventional PLC, the waiting time for confirming that the second movable member has returned to the moving end can be omitted, and the cycle time can be shortened accordingly. it can. Further, a sensor for confirming the positions of the first and second members is not necessary.

  FIG. 1 is for explaining the outline of the present invention in comparison with the conventional example of FIG. 13, and the present invention shows the movement of the first movable member and the second movable member in FIG. It is not limited to movement. The movement of the second movable member may be any movement as long as it shows a movement that hinders imaging of the first movable member. Furthermore, it is only necessary that the first member and the second member move relative to each other so that the second member is positioned so as to prevent the first member from capturing an image by the camera. That is, either the first member or the second member may be movable, and both need not be movable.

  Hereinafter, with reference to FIGS. 2-12, monitoring or control of the components washing equipment which is one Embodiment of this invention is demonstrated.

  FIG. 2 is a diagram illustrating an outline of a component cleaning facility according to an embodiment of the present invention, and FIGS. 3 and 4 are explanatory diagrams illustrating a component cleaning machine of the component cleaning facility.

  The parts washing facility 100 includes a parts washing machine 10 for washing parts. The parts washer 10 includes a cleaning tank 30 filled with a cleaning liquid 33 and an index holding frame 11 that rotates intermittently about a central axis O. On the peripheral edge of the index holding frame 11, indexes 12 to 23 (see FIGS. 3 and 4) for holding components are held so as to be swingable around a horizontal axis. The index holding frame 11 rotates intermittently about the central axis O, and the index moving to the lower part of the index holding frame 11 passes through the cleaning liquid 33, thereby cleaning the parts held by the index.

  In order to take out the cleaned parts from the index, a parts take-out loader 41 as a work robot is arranged. The component take-out loader 41 takes out the cleaned component when the index on which the cleaned component is placed arrives at a predetermined position (see FIGS. 3 and 4: the component take-out station ST1) and stops. The picked-up parts are conveyed to an assembling station (not shown) by the parts pick-up loader 41 and assembled at a predetermined position.

  In addition, a component loading loader 43 is arranged to load a component to be cleaned into the index. The component loading loader 43 loads a component to be cleaned when the index from which the cleaned component has been taken out has arrived and stopped at a predetermined position (FIGS. 3 and 4: component loading station ST3). The parts to be input are carried from a parts feeder (not shown) for supplying the parts by the part input loader.

  In the present embodiment, the component pick-up loader 41 and the component loading loader 43 use vacuum chucks. The vacuum chuck can insert or assemble parts by sucking and taking out the parts by using the difference in atmospheric pressure and stopping the suction of the parts.

  A camera 51 such as a video camera using a CCD as an image sensor is provided to check whether the cleaned parts have been taken out or whether the parts to be cleaned have been input. Images captured by the camera 51 are continuously input to the image processing unit 53 at a predetermined timing (for example, every 0.1 second). The image processing unit 53 autonomously synchronizes the captured image as will be described later, and performs image processing at an optimal timing for checking the removal of components and the insertion of components. As a result, when an error is found, an error signal is sent to the PLC 55 that controls the facility 100, and the PLC controls to stop the operation of the facility 100.

  FIG. 3 is an explanatory diagram for explaining the index holding frame 11 of FIG. 2, and FIG. 4 is a top view of the index holding frame 11. The index holding frame 11 is a circular support frame arranged in a vertical plane, and supports indexes 12 to 23 which are component holding members that hold components on the periphery thereof. The index 12 has component holding parts 121 to 123 (see FIG. 4). In the present embodiment, the component holding unit 121 holds a spring, the component holding unit 122 holds a washer, and the component holding unit 123 holds an E-ring. The other indexes 13 to 23 have the same configuration.

  The indexes 12 to 23 are arranged at every central angle of 30 ° and move 30 ° intermittently counterclockwise. Since the indexes 12 to 23 move in a circular orbit, they are supported so that the component support portion is always on the top so as to be swingable so that the component does not fall. However, as shown in FIG. 4, at the component take-out station ST1 and the component take-up station ST3, the index is locked by the lock members 27 and 28 for taking out or putting in the components.

  FIG. 3 shows a state in which the indexes 16 to 22 are immersed in the cleaning liquid 33 in the cleaning tank 30. Since the index holding frame 11 rotates 30 ° intermittently, the index is sequentially washed by the rotation of the index holding frame 11. In FIG. 3, the indexes 12 to 14 are on the upper part of the index holding frame 11, and the camera 51 monitors the indexes 12 to 14. In order to prevent the parts from falling into the cleaning tank when the parts are taken out or put in, a flat plate-like fall prevention member 25 is provided. The fall prevention member 25 is a member independent of the index holding frame 11 and is fixed to the cleaning tank 30.

  In FIGS. 3 and 4, the index 12 is stopped at the component take-out station ST1. The cleaned parts held by the index 12 are taken out by the part take-out loader. The index 13 is stopped at the intermediate station ST2. Since the index at which the parts are taken out at the parts pick-up station ST1 arrives at the intermediate station, the parts pick-up at the parts pick-up station ST1 can be confirmed by monitoring the parts at the intermediate station. The index 14 is stopped at the parts input station ST3. In the component loading station ST3, the component to be cleaned is loaded into the index component holding unit by the component loading loader.

  As described above, the component take-out loader 41 (see FIG. 3) takes out the cleaned spring, washer, and E-ring from the component holding unit of the index at the component take-out station ST1. The component take-out loader 41 reciprocates between the component take-out station ST1 and an assembly station for assembling the taken-out cleaned parts. Further, the component loading loader 43 (see FIG. 3) loads a spring, a washer and an E-ring to be cleaned into the component holding portion of the index at the component loading station ST3. The parts loading loader 43 reciprocates between the parts picking station ST3 and the parts feeder that supplies the parts to be cleaned.

  FIG. 4 is also an example of a screen imaged by the camera 50. In the present embodiment, in order to synchronize image processing for checking whether a part is taken out or put in, an index stationary detection area A1 or A2, a part picking marker 251, and a part putting marker 252 are provided. The index still detection area A1 or A2 indicates an image processing area for detecting whether the index is stationary or stopped. In the present embodiment, only one of the index still detection areas A1 and A2 is used according to the monitoring target screen. In the index still detection area A1 or A2, for example, when the index is recognized by the shape pattern matching technique, it is determined that the index is stationary or stopped. The area for index stillness detection is not limited to areas A1 and A2, but may be set anywhere as long as it can be determined that the index is stopped by image processing.

  Furthermore, in order to guarantee the stillness of the index, it may be added to determine whether or not the lock member 27 or 28 is in the lock position based on the position of the lock member on the image. For example, by using at least one image of the lock members 27 and 28 together with the image of the index still area A2, confirming that the lock is applied, and determining whether or not the index is stationary, the index can be stopped more reliably. I can judge.

  The reason why the index stationary detection area is provided is that if a marker is provided in the index, the marker may be immersed in the cleaning liquid together with the index, which may adversely affect the cleaning liquid. If the cleaning liquid is not adversely affected, a marker may be provided on the index.

  In the present embodiment, the component pick-up marker 251 and the component insertion marker 252 are, for example, those in which a black circle is arranged on a white ground color, and whether or not the center of gravity of the black circle calculated from the captured image is at a predetermined position. It is possible to recognize the presence of each marker.

  The component removal marker 251 is provided at a location on the component drop prevention member 25 that is a fixing member. The part pick-up loader 41 is disturbed in imaging until the part pick-up loader 41 moves to the part pick-up station ST1, picks up the part, holds the picked-up part, and moves from the part assembling station. The However, the component take-out marker 251 can be imaged when the component take-out loader 41 moves from the component take-out station ST1. In other words, the image of the part extraction marker 251 in the captured image is an image that is inappropriate for image processing for part removal monitoring, or an image suitable for image processing for part removal monitoring. This is an index for determining on the image whether or not the image is captured.

  The component insertion marker 252 is also provided at a location on the component drop prevention member 25 which is a fixing member. The component insertion loader 41 is disturbed in imaging until the component insertion loader 43 moves to the component extraction station ST3 to input the component and moves from the component assembly station. However, the component insertion marker 252 can be imaged when the component insertion loader 43 moves from the component extraction station ST3. That is, the image of the component insertion marker 252 in the captured image is an image that is inappropriate for image processing for monitoring component insertion, or an image suitable for image processing for component monitoring. This is an index for determining on the image whether or not the image is captured.

  The part removal marker 251 and the part insertion marker 252 are not specially arranged, but the part drop prevention member 25 can be identified as surroundings in a place where it can be determined whether or not the image is suitable for image processing. If there is a member or feature such as, it can be used as a marker.

  Next, with reference to FIG. 5, a visual device composed of a camera and an image processing unit used in this embodiment will be described.

  A monitoring camera 51 using a CCD is a USB-connected USB camera, and is connected to the image processing unit 53 by USB2. The image processing unit 53 is connected to a liquid crystal monitor 56 as an output unit, and is connected via a USB hub 59 to a keyboard 57 or a mouse (not shown) as an input unit. A dongle key 58 is connected to the USB hub, and unauthorized use of the imager folding software can be prevented. With this configuration, the operator can check the image input to the image processing unit on the liquid crystal monitor. The image processing software can be operated with the keyboard 57 or the mouse.

  Hereinafter, a specific part take-out or part input monitoring or confirmation operation will be described. First, an example of monitoring the removal or insertion of a component for the image of the component loading station ST3 will be described. FIGS. 6A, 6B, 7C, and 7A and 7B are examples in which some of the images of the component loading station ST3 captured by the camera 51 are extracted for explaining the present invention. It is. Specifically, these images are extracted from still images taken into the image processing apparatus at intervals of 0.1 seconds.

  Here, the images in FIGS. 6 and 7 are obtained by rotating FIG. 4 by 90 degrees because the images in FIGS. 6 and 7 are created based on the actual monitoring image. Is not essential to the present invention.

  FIG. 6A shows an image captured when the index 14 arrives at the loading station after the cleaned parts are taken out by the take-out loader at the parts take-out station ST1. From the captured image, the index 14 is captured in the index stillness detection area A1, and the component loading loader marker is captured. As a result, it can be seen that the index 14 is stationary and that the component loading loader does not interfere with imaging. Therefore, using this image, the image processing unit analyzes the images of the component holding units 141, 142, and 143. As the analysis method, a known one can be used as appropriate, but in this embodiment, shape pattern matching is mainly used. However, since the shape pattern matching is for detecting an edge, a trace of a component may be erroneously determined even if there is no actual component. In such a case, a method may be adopted in which the images are binarized and the areas are obtained and compared. In any case, known image processing techniques can be used as appropriate.

  By such image processing, it is determined whether or not a component remains in the component holding units 141 to 143. If no part remains, the process proceeds to the next part input process. If a part remains, if the process is continued as it is, the part will be thrown in twice, so error information is issued and the equipment is stopped.

  Next, FIG. 6B shows an image when the component loading loader 43 arrives at the loading station ST3 and is loading components. In this image, the index still detection area A1 and the component loading loader marker are both disturbed by the component loading loader 43 and are not shown in the image. The component loading loader 43 includes a vacuum chuck pipe or electrical wiring (not shown), and an image of the component loading loader 43 that operates at high speed flows. Therefore, the influence of the component loading loader on the captured image is wide. It covers a range.

  FIG. 6C is an image when the component loading loader 43 is retracted after the components to be cleaned are inserted into the component holding units 141, 142, and 143 with the index 14. Here, the index stationary detection area A1 is detected, the component loading loader marker 252 is detected, and the index 14 is stationary, indicating that the component loading loader does not interfere with imaging. Therefore, using this image, the image of the component holding units 141 to 143 is analyzed by the image processing unit. Through such image processing, it can be confirmed whether springs, washers, and E-rings are inserted in the component holding portions 141 to 13. If a part has been inserted, proceed to the next. If no parts are put in, error information is issued and the equipment is stopped.

  FIG. 7A shows an image in the middle of the index being rotated by 30 °. In the image of FIG. 7, the index 14 has left the screen, and the index 13 has appeared on the screen. Here, no index is detected in the index still detection area A1. Therefore, since it is determined that this image cannot be used to determine the presence / absence of a component, image processing for that purpose is not performed. In an actual image, the index is more blurred or flowing during index rotation.

  FIG. 7B is an image when the index 13 arrives at the input station. This image is the same image as in FIG. 6A only after one cycle has been advanced. Therefore, the same processing as that described with reference to FIG. 6A is performed.

  6 and 7, the image of the lock mechanism 28 of the loading station ST3 is not shown. However, as described above, the index detection area A1 and the loading loader marker are used in order to increase the accuracy of confirmation of the stillness of the index. In addition to the confirmation of 252, it may be confirmed on the image that the index locking mechanism 28 in the input station ST <b> 3 is operating.

  Further, if there is a portion where the index can be stationary and the loading loader can be determined at one location, the image processing may be started only by confirming the portion on the image. In the present embodiment, the shape or pedestal of the component holding portion for holding the index spring, for example, in the index 13, if the shape or pedestal of the component holding portion 131 is detected, the shape of the component holding portion or Image processing for confirming the presence / absence of a component can be started in synchronization with only detection of the pedestal. In addition, if there is a part that can detect the stillness of the index or the retraction of the input loader, the start of the image processing can be synchronized by combining these parts.

  In the above example, since an image obtained by imaging only the index that has arrived at the component input station ST3 is used, one index that is an imaging target that can be imaged on one screen can be imaged greatly, and the detection accuracy of the presence or absence of components can be improved. it can.

  Next, with reference to FIG. 8, a flow of processing operations for component monitoring or facility control according to the present embodiment will be described.

  First, in step S1, an image of the index still detection area A1 in the component loading station ST3 is confirmed to determine whether or not the index is still. If the index is rotating and not stopped, step S1 is repeated. If it is determined that the index is stationary, the process proceeds to step S2.

  In step S2, the image of the input loader marker 252 is confirmed, and it is determined whether the input loader 43 has been retracted from the input station ST3. If the input loader marker 252 cannot be confirmed, it is determined that the input loader has not been retracted, so step S2 is repeated. When the input loader marker 252 is confirmed, it is determined that the input loader 43 has been retracted. If step S1 and step S2 are determined affirmatively, it means that synchronization has been achieved autonomously and the process proceeds to image analysis for component detection.

  Steps S <b> 3 to S <b> 5 are steps for detecting whether or not there are components in all the component holding units. In the check before putting parts, it is confirmed that no parts are placed in all the parts holding parts. On the other hand, the check after putting in the parts confirms that the parts are placed in all the part holding parts. In this way, double assembly or forgetting assembly can be prevented.

  In step S3, the image of the E-ring component holder is processed to check the presence or absence of the E-ring. In step S4, the image of the component holder for the washer is processed to check the presence or absence of the washer. In step S5, the image of the spring component holding part is processed to confirm the presence or absence of the spring. Note that the order of steps S3 to S5 is not limited to this.

  Steps S3 to S5 are completed, and if there is, for example, an image process that has not been executed successfully, it is determined whether or not to request a retry of the image process. If no retry is requested, the process proceeds to step S7, and in steps S3 to S5, it is determined whether or not a satisfactory result has been obtained with respect to the presence or absence of a component in the component holding unit. When a desired result is not obtained, that is, when an image of at least one component holding unit cannot be detected despite the image after the component is input, or a component is detected in the image before the component is input In the case, the process proceeds to step S8, error information is sent to the PLC, and the operation of the equipment is stopped.

  In step S7, when all the results of steps S3 to S5 are satisfactory, this flow is ended and the process returns to step S1.

  Next, FIGS. 9 to 12 illustrate an example in which the component take-out station and the component take-up station are captured in one image, and an example in which the component take-out station and the component take-up station are simultaneously monitored using this image will be described.

  As shown in FIG. 9, in this example, an extraction loader marker 251, an input loader marker 252, and an index stillness detection area A2 are used in order to synchronize for starting image processing. Of course, as in the previous example, the index stillness detection area A1 in the input station may be used. In order to confirm the stillness of the index, it may be added to determine whether or not the lock is applied based on the position of the members of the lock mechanisms 27 and 28.

  In the image shown in FIG. 9, at the index 12 in the take-out station ST1, the part take-out process by the take-out loader 41 is completed, and the take-out loader 41 moves to the assembly station side, and at the index 14 in the load station ST3. This is an image of the moment when the parts are not yet put in by the loading loader 43.

  From the captured image of FIG. 9, the index 13 is recognized in the index still detection area A2, and the component take-out loader marker 251 and the component input loader marker 252 are recognized. Therefore, it can be seen from this image that the index is stationary, the component take-out loader 41 is withdrawn from the take-out station ST1, and the component input loader 43 is withdrawn from the component input station.

  Therefore, using this image, the images of the component holding portions of the indexes 12 to 14 in the take-out station ST1, the intermediate station ST2, and the input station ST3 are processed, and the presence / absence of components is determined.

  Next, the component loading loader 41 moves to the loading station ST3, and E-rings, washers, and springs are loaded into the component holding portions 141, 142, and 143 of the index 14. Thereafter, the component loading loader 41 moves to the parts feeder side.

  FIG. 10 shows an image when the component loading loader moves from the loading station after component loading. Also at this time, the index 13 is recognized in the index stillness detection area A2, and the component take-out loader marker 251 and the component loading loader marker 252 are recognized. Therefore, using this image, the images of the component holding portions of the indexes 12 to 14 in the take-out station ST1, the intermediate station ST2, and the input station ST3 are processed, and the presence / absence of components is determined.

  Here, when it is confirmed that there is a predetermined part in the index 14 of the loading station ST3 and that there is no part in the indexes 12 and 13, the locking of the index by the lock mechanisms 27 and 28 is released, and the index is counterclockwise. The components held by the index 14 are turned to the washing tank by rotating around 30 °. This state is shown in FIG. In this state, since the index is not correctly recognized in the index detection area A2, image processing is not performed on this image.

  In FIG. 12, after the index has been rotated by 30 °, the index 13 arrives stationary at the input station ST3, and the intermediate station ST2 remains stationary when the index 12 arrives. In the take-out station ST1, the index 22 on which the cleaned parts are placed arrives, and the image is taken when the parts take-out loader 41 is performing the work of taking out the parts from the parts holding part of the index 22. In this case, as shown in FIG. 12, the component take-out loader marker 251 is not captured behind the image of the take-out loader 41, and is naturally not recognized by image processing. Therefore, it is impossible to detect the state of the index 22 in the take-out station ST1.

  However, since the index 13 is correctly recognized in the index stationary detection area A2 and the component loading loader marker 252 can be recognized, the intermediate station ST2 and the loading station ST3 can be checked. Therefore, in this image, it is determined whether or not there is a component in the component holding unit of the index 14 in the intermediate station ST2 and the index 13 in the loading station ST3.

  In the take-out station ST1, when the part take-out operation by the part take-out loader 41 is finished and the part take-out loader 41 moves to the part assembly station in order to assemble the taken-out parts, the same state as the image shown in FIG. 9 is obtained. Return. However, since the cycle is advanced, the index 13 stops at the input station ST3, the index 12 stops at the intermediate station ST2, and the index 22 stops at the take-out station ST1. From this state, the processes shown in FIGS. 9 to 12 are repeated.

  In the example shown in FIGS. 9 to 12, since it is possible to check simultaneously including the input station ST3, the take-out station ST1, and further the intermediate station ST2, compared to monitoring only the input station ST3, more Judgment in real time is possible.

  The image processing method in this example is the same as the method described above, and it is shape pattern matching that recognizes the index 14 in the index still detection area A2, and is for the component take-out loader marker 251 and the component input loader. It is the center of gravity detection that recognizes the marker 252. In addition, it is shape pattern matching that recognizes the images of the component holding portions of the indexes 12 to 14, and if necessary, binarization and area comparison processing are used together. However, the image processing method is not limited to these. A known image processing technique can be used as appropriate.

It is a figure for demonstrating the concept of this invention. It is a figure which shows the outline of the components washing equipment used for 1 embodiment of this invention. It is a figure explaining the washing machine used for parts washing equipment. It is a top view of the washing machine of FIG. It is a figure which shows the visual apparatus used for one Embodiment of this invention. (A)-(c) is a figure (the 1) for explaining operation of an example of one embodiment of the present invention. (A), (b) is a figure (the 2) for demonstrating operation | movement of an example of 1 embodiment of this invention. It is a figure which shows the flow of operation | movement of one Embodiment of this invention. It is FIG. (1) for demonstrating operation | movement of the other example of 1 embodiment of this invention. It is FIG. (2) for demonstrating operation | movement of the other example of 1 embodiment of this invention. It is FIG. (3) for demonstrating operation | movement of the other example of 1 embodiment of this invention. It is FIG. (4) for demonstrating operation | movement of the other example of 1 embodiment of this invention. (A), (b) is a figure for demonstrating the problem of the conventional drive control apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Equipment 3 1st member 4 Characteristic part 5 2nd member 6 Camera 7 Image processing apparatus 8 PLC
DESCRIPTION OF SYMBOLS 100 Member battlefield apparatus 10 Member washing machine 11 Index holding member 12-22 Index 121, 122, 123 Parts holding part 131, 132, 133 Parts holding part 141, 142, 143 Parts holding part 25 Parts fall prevention member 251 Takeout loader marker 252 Input loader marker 27, 28 Lock member ST1 Component extraction station ST2 Intermediate station ST3 Component input station 30 Cleaning tank 33 Cleaning liquid 41 Component extraction loader 43 Component input loader 51 Camera 53 Image processing unit 55 PLC

Claims (6)

A first member (3, 10);
A second member (5, 41, 43) that moves relative to the first member;
An imaging unit (6, 51) for imaging the first member;
Image processing unit (7, 53) for processing an image picked up by the image pickup unit
With
The first member (3, 10) is prevented from being imaged by the imaging unit due to the positional relationship between the first member (3, 10) and the second member (5, 41, 43). And at least one feature (4, A1, A2, 251, 252), and the first member (3, 10) has a movable part (3, 11-22), The two characteristic parts (4, A1, A2) further indicate that the movable part of the first member (3, 10) is stopped,
The image processing unit (7, 53) determines whether or not the characteristic portion (4, A1, A2, 251, 252) is present in the image captured by the imaging unit (6, 51) by the first image processing. And
Second image processing is performed on the image according to the presence or absence of the characteristic portion (4, A1, A2, 251, 252).
A monitoring device characterized by that.   The monitoring apparatus according to claim 1, wherein the second member (41, 43) is a movable member, and is a robot that performs an operation on the first member. A first member (3, 10);
A second member (5, 41, 43) that moves relative to the first member;
An operation control section (8, 55) for controlling the operation of at least one of the first member and the second member;
An imaging unit (6, 51) for imaging the first member;
Image processing unit (7, 53) for processing an image picked up by the image pickup unit
With
The first member (3, 10) prevents imaging by the imaging unit due to a positional relationship between the first member (3, 10) and the second member (5, 41, 43). At least one feature (4, A1, A2, 251, 252) that may be
The image processing unit (7, 53) determines whether or not the characteristic portion (4, A1, A2, 251, 252) is present in the image captured by the imaging unit (6, 51) by the first image processing. And
Depending on the presence or absence of the feature (4, A1, A2, 251, 252), the second image processing is performed on the image,
The control apparatus characterized by changing the control of the operation control unit (8, 55) according to the result of the second image processing.   The first member (3, 10) has a movable part, and the at least one characteristic part further indicates that the movable part of the first member (3, 10) is stopped. The control device according to claim 3.   The control device according to claim 3 or 4, wherein the second movable member (41, 43) is a robot that performs an operation on the first member. A control method including a step of moving the second member (5, 41, 43) relative to the first member (3, 10),
Imaging may be hindered by the positional relationship between the first member (3, 10) and the second member (5, 41, 43) provided on the first member (3, 10). Imaging the first member including at least one feature (4, A1, A2, 251, 252);
A first image processing is performed on an image obtained by imaging the first member (3, 10), and the feature portions (4, A1, A2, 251, 252) in the image obtained by imaging the first member. )
Depending on the presence or absence of the feature (4, A1, A2, 251, 252), the second image processing is performed on the image,
A control method, wherein control is changed according to a result of the second image processing.

Images