JP3052493B2 - Visual recognition system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual recognition system for determining the appearance and type of a product in a manufacturing or inspection line of a factory or the like.

[0002]

2. Description of the Related Art In general, such a kind of visual recognition system is shown in FIG.
As shown in FIG. 5, a belt conveyor 2 for guiding the objects 1A and 1B to an observation position is provided, and a visual recognition device 3 is provided near the observation position.

The visual recognition device 3 has an observation system 4, an image processing device 5, and a video monitor 6. In the observation system 4, a television camera 8 is illuminated by a lighting device 7 so that the television camera 8 can detect the object 1A,
1B is imaged. The image is captured by the image processing device 5 and binarized, and features such as the area and the center of gravity of the binary image are measured, and the appearance, type, and the like are recognized. In the image processing device 5, a window for designating a measurement region is set for a target portion of a binary image, and a feature amount is usually measured for an image portion in the window.

In such a visual recognition system, when different types of objects 1A and 1B are inspected one after another, every time the type of the inspection object changes, the intensity of the illumination device 7 is changed.
It is necessary to change observation conditions such as the zoom magnification of the television camera 8 and to change measurement conditions such as a binarization threshold value for image processing, a setting position of the window, and a feature amount to be measured.

Conventionally, when the distance L between the last object 1A of a certain type and the first object 1B of another type is sufficiently large, the attendant manually changes observation conditions and measurement conditions. When the distance L is short, a signal is output from the sequencer 9 to the image processing device 5 to instruct the observation condition and the measurement condition to be changed from a certain timing.

[0006]

However, in the case of manual operation, it is necessary for the attendant to constantly monitor the inspection line, and it is not only difficult to completely automate the inspection, but also to perform the change operation in a short time. Therefore, there is a possibility that an operation error of the attendant may be caused. In the case of using the sequencer 9, not only the sequencer 9 is required and the cost is increased, but also the work of connecting the sequencer 9 and the image processing apparatus 5 is complicated, and the following types of objects are required. It is difficult to predict when will arrive at the observation position.

The present invention has been made in view of the above problems, and reduces the work load on the staff and prevents the occurrence of operation errors. Also, the present invention automatically adjusts observation conditions and measurement conditions without using a sequencer or the like. It is an object to provide a visual recognition system that can be changed.

[0008]

According to a first aspect of the present invention, there is provided a visual recognition system comprising: an information display body on which visible information for defining observation conditions is displayed; a plurality of objects and the information display body at a predetermined observation position; And a visual recognition device arranged at the observation position. The visual recognition device has an observation system and an image processing device, and the observation system captures an illumination device that illuminates an observation position, and an image of the information display object and the target object that are conveyed to the observation position. And an imaging device for the same. Further, the image processing apparatus, reading means for reading the visible information by inputting an image of the information display from the imaging device, setting means for setting the observation conditions of the observation system based on the reading result by the reading means,
An adjusting unit that automatically adjusts an observation system according to the observation condition set by the setting unit; and an image processing unit that processes an image of the object captured by the imaging device after adjustment by the adjusting unit. .

According to a second aspect of the present invention, there is provided a visual recognition system comprising: an information display on which visible information for defining measurement conditions is displayed; a transport mechanism for transporting a plurality of objects and the information display to a predetermined observation position; And a visual recognition device arranged at the observation position. The visual recognition device has an observation system and an image processing device, and the observation system includes an imaging device for imaging the information display body and the object transported to an observation position. Further, the image processing apparatus includes:
Image processing means for processing an image of the object imaged by the imaging device according to predetermined measurement conditions; reading means for inputting an image of an information display from the imaging device to read the visible information; Setting means for setting the measurement condition based on the read result and outputting the measurement condition to the image processing means.

According to a third aspect of the present invention, there is provided a visual recognition system, comprising: a work robot for performing a predetermined work; an information display body on which visible information for defining the work content of the work robot is displayed; It comprises a transport mechanism for transporting the information display to a predetermined observation position, and a visual recognition device arranged at the observation position. The visual recognition device has an observation system and an image processing device, and the observation system includes an imaging device for imaging the information display body and an object transported to an observation position. The image processing apparatus may further include: an image processing unit configured to process an image of the target object captured by the imaging device; a reading unit configured to input an image of an information display from the imaging device and read the visible information; Setting means for setting the work content of the work robot based on the read result by the means and outputting the work content to the work robot, wherein the work robot is set by the setting means in accordance with the processing result of the image processing means Perform the work performed.

According to a fourth aspect of the present invention, in the visual recognition system, the information display body is moved between a group of a plurality of objects conveyed by the conveyance mechanism and a group of objects of a type different from the objects. Position.

In a visual recognition system according to a fifth aspect of the present invention, the information display body is positioned and arranged in correspondence with each object transported by the transport mechanism.

[0013]

In the visual recognition system according to the first aspect, when the information display body first reaches the observation position, the imaging device takes an image of the information display body, and the image processing device reads the visible information from the image to determine the observation conditions. Set. The observation system is automatically adjusted in accordance with the observation conditions. Thereafter, the imaging device takes an image of the target object, and the image processing device processes the image to execute recognition.

[0014] In the visual recognition system according to claim 2,
First, when the information display arrives at the observation position, the imaging device captures an image of the information display, and the image processing device reads visible information from the image and sets measurement conditions. Thereafter, the imaging device captures an image of the object, and the image processing device processes the image according to the set measurement conditions to execute recognition.

In the visual recognition system according to claim 3,
First, when the information display arrives at the observation position, the imaging device captures an image of the information display, and the image processing device reads visible information from the image and sets the work content of the work robot. After that, the imaging device captures an image of the target object, the image processing device processes the image and executes recognition, and the work robot performs work of the set work content according to the recognition result.

[0016] In the visual recognition system according to claim 4,
Since the information display is positioned and arranged in the middle of each group of different types of objects, observation conditions, measurement conditions, and work contents of the work robot are set for each group of the objects.

In the visual recognition system according to claim 5,
Since the information display body is positioned and arranged corresponding to each object, observation conditions, measurement conditions, and work contents of the work robot are set for each object.

[0018]

FIG. 1 shows a visual recognition system according to an embodiment of the present invention. A belt conveyor 2 for conveying objects 1A and 1B and an observation set in the middle of the conveying path of the belt conveyor 2 are shown. And a visual recognition device 3 arranged at the position.

The belt conveyor 2 guides a group of a certain type of object 1A to an observation position and sets the group of the object 1A.
Is subjected to an inspection, and subsequently, a group of other types of objects 1B is guided to an observation position, and the group of objects 1B is subjected to the inspection. Each object 1A, 1B is placed on the belt conveyor 2. Are arranged at a constant interval d for each group,
A relatively long interval L is provided between a group of one object 1A and a group of the other object 1B.

At a position on the belt conveyor 2 corresponding to the distance L, an information display 10 on which visible information that defines observation conditions for each type of object is displayed is positioned. In this embodiment, a card-shaped information display body 10 is used and is simply placed on the belt conveyor 2, but the present invention is not limited to this. It may be attached to the surface of the belt conveyor 2. Further, in this embodiment, the information defining the observation conditions is represented by a barcode on the surface of the information display body 10, but may be represented by another identification pattern.

The visual recognition device 3 comprises an observation system 4, an image processing device 5, and a video monitor 6. The observation system 4
It has a lighting device 7 that illuminates toward the observation position, and a television camera 8 that captures images of the objects 1A and 1B under illumination by the lighting device 7, and the lighting device 7 depends on the type of the object. Thus, the illumination intensity and the zoom magnification of the objective lens 11 of the television camera 8 can be adjusted according to the type of the object.

The image processing apparatus 5 has a basic configuration as shown in FIG. 2, and includes a code reading unit 12, an observation condition setting unit 13, an observation system adjustment unit 14, and an image processing unit 15. I have. The code reading unit 12 reads an image of the information display 10 from the television camera 8 and reads the barcode. The observation condition setting unit 13 sets the observation conditions of the observation system 4 based on the reading result of the code reading unit 12, specifically, the illumination intensity of the illumination device 7 and the objective lens 1 of the TV camera 8.
Set a zoom magnification of 1. The observation system adjustment unit 14 adjusts the illumination intensity of the illumination device 7 and the objective lens 1 of the television camera 8 in accordance with the observation conditions set by the observation condition setting unit 13.
Automatically adjusts the zoom magnification of 1. The image processing unit 15 adjusts the television camera 8 after adjustment by the observation system adjustment unit 14.
The image of the target object 1A, 1B captured by is captured and binarized, and the feature amount is measured for the binary image.

FIG. 3 shows a specific example of the image processing apparatus 5, which includes a sync separation circuit 16, an A / D converter 17, and a sync signal generation circuit 18. Synchronous separation circuit 1
Reference numeral 6 fetches a video signal from the television camera 8, separates the synchronization signal, and outputs the synchronization signal to the generation circuit 18. The synchronizing signal generation circuit 18 includes a vertical synchronizing signal VD and a horizontal synchronizing signal H.
D, the pixel clock CK is generated, the vertical synchronization signal VD and the horizontal synchronization signal HD are sent to the image processing unit 15, and the pixel clock CK is sent to the A / D converter 17, the specific pattern detection unit 19, and the image processing unit 15, respectively. At the same time, the horizontal synchronizing signal HD is output to the AND circuit 21 via the inverting circuit 20. The A / D converter 17 inputs an analog video signal which has passed through the sync separation circuit 16 and converts it into a digital video signal. The digital video signal is output to the specific pattern detecting section 19 and the image processing section 15. Is done.

The bar code 22 displayed on the information display body 10 includes a partial pattern 22a for instructing the visual recognition device 3 to change the observation conditions, and a code in which the contents of the observation conditions to be set are recorded in codes. The specific pattern detection unit 19 detects whether or not the partial pattern 22a is a specific pattern.

FIG. 4 is a block diagram showing the function of the specific pattern detecting section 19. In the figure, block 23
Then, a video signal obtained by imaging the information display body 10 is input, a brightness histogram is generated for one horizontal scanning line image, and a known method such as Otsu's threshold method is used from the histogram. The binarization threshold value TH is extracted using the threshold value TH, and the video signal is binarized at the block 24 by the binarization threshold value TH. In this process, an appropriate binarization threshold TH is not obtained from an image of one horizontal scanning line 29 (see FIG. 3) for the white background portion 28 of the information display body 10, and a binary image is not generated. From the image of the first horizontal scan line 31 for the code display portion 30, an appropriate binarization threshold TH is obtained, and
A value image will be generated.

In block 25, the start position st and the end position ed of the partial pattern 22a are detected from the binary image on the next horizontal scanning line. In block 26, the partial pattern 22a between the start position st and end position ed for the binary image of the barcode 22 is expanded and contracted to normalize it to a pattern of a predetermined length, and in block 27, the normalized partial pattern and Pattern matching is performed with the stored reference pattern, and it is determined whether or not both match.

The specific pattern detecting section 19 outputs binary pattern data of the bar code 22 and a flag f for instructing a change of the observation condition. If it is performed, f = 1, and if a match is determined, f = 0. The flag f is provided as one input of an AND circuit 21 shown in FIG. 3. When the signal obtained by inverting the horizontal synchronizing signal HD is turned on and the content of the flag f is "1", the AND circuit 21 outputs the signal as an AND output. Interrupt signal INT of C
Give to PU32.

Returning to FIG. 3, the image processing unit 15 inputs video signals for the objects 1A and 1B from the A / D converter 17 and performs preprocessing such as binarization, and then performs the binary image processing. After measuring the characteristic amounts such as the area and the center of gravity of the object portion, the measurement result is output to the CPU 32 via the bus 33.

The CPU 32 is a main component of control and calculation in the microcomputer. The CPU 32 fetches measurement results from the image processing section 15 to determine the appearance and type of the object, and also performs binary processing from the specific pattern detection section 19. The pattern data is input, the code is read from the code recording unit 22b, and the observation conditions are set according to the contents of the code.

The CPU 32 includes a ROM 34 in which a program is stored and an RA 34 for reading and writing various data.
A microcomputer is constituted by the M35 and the input / output interface 36. The microcomputer decodes and executes a program in the ROM 34, reads and writes data from and in the RAM 35, and controls the operation of each component related to setting of observation conditions.

The ROM 34 stores a table for setting different observation conditions according to the contents of the code, and FIG. 5 shows a specific example of the table TB. The table TB shown in the figure is a table in which the illumination intensity and the zoom magnification correspond to a plurality of codes.
2, when the code C1 is read from the binary pattern data, P1 is used as the illumination intensity, M1 is used as the zoom magnification,
Each is read and output to the input / output interface 36.

An illumination intensity control unit 37 and a zoom magnification control unit 38 are connected to the input / output interface 36. The illumination intensity control unit 37 drives the illumination device 7 by adjusting the illumination intensity set by the CPU 32. The zoom magnification controller 38 adjusts the zoom magnification set by the CPU 32 to adjust the objective lens 1 of the television camera 8.
1 is driven. In the drawing, reference numeral 39 denotes a light-emitting diode that lights up when an interrupt occurs.

FIG. 6 shows an interrupt processing routine performed by the CPU 32 when the interrupt signal INT is received. In step 1 (indicated by "ST1" in the figure) of FIG.
The PU 32 reads the code recording section in the binary pattern data, and generates a code in the next step 2. Next, in step 3, the CPU 32 sets the table TB in the ROM 34.
After reading the illumination intensity and zoom magnification corresponding to the code, the zoom magnification is output to the zoom magnification control unit 38 in step 4 and the illumination intensity is output to the illumination intensity control unit 37 in step 5. Set.

In the above embodiment, the observation condition is represented by a code on the information display body 10. However, when the information amount of the code is large, the set value of the observation condition may be directly represented.

FIGS. 7 to 9 show a visual recognition system according to a second embodiment of the present invention. The first shown in FIGS.
In the second embodiment, observation conditions such as the zoom magnification of the television camera 8 and the illumination intensity of the illumination device 7 are set. In the second embodiment, measurement conditions for image processing, for example, a binarization threshold, The window size and the setting position, the feature quantity to be measured, the criterion for quality judgment, and the like are set. The external appearance of the entire system is the same as that in FIG. 1 and is not illustrated here.

FIG. 7 shows the basic configuration of the image processing apparatus 5, in which the code reading unit 12 and the measurement condition setting unit 40 are used.
And an image processing unit 15. Code reading unit 1
2 reads an image of the information display 10 from the television camera 8 and reads a barcode. The measurement condition setting unit 40 sets the measurement conditions for image processing based on the result of reading by the code reading unit 12 and outputs the measurement condition to the image processing unit 15. The image processing unit 15 captures the images of the objects 1A and 1B captured by the television camera 8 after setting the measurement conditions, and executes various processes such as binarization and measurement of the feature amount according to the set measurement conditions.

FIG. 8 shows a specific example of the image processing apparatus 5. In the illustrated example, the outline is substantially the same as that in the first embodiment shown in FIG. In the case of the second embodiment, the code recording unit 2 of the information display 10
2b stores measurement conditions to be set as codes. When the binary pattern data is input from the specific pattern detection unit 19, the CPU 32 reads the codes from the code recording unit 22b and performs measurement in accordance with the codes. Set conditions. Therefore, a table for setting measurement conditions in accordance with the codes is stored in the ROM 34. The CPU 32 reads the codes from the binary pattern data, reads the measurement conditions corresponding to the codes from the ROM 34,
Output to the image processing unit 15 via the bus 33.

FIG. 9 shows an interrupt processing routine by the CPU 32 when the interrupt signal INT is received. In step 1 of FIG. 9, the CPU 32 reads the code recording section in the binary pattern data, and Step 2 generates a code. Next, in step 3, the CPU 32 sets R
After accessing the table of the OM 34 and reading out the measurement conditions corresponding to the codes, the measurement conditions are output to the image processing unit 15 via the bus 33 and set in step 4.

FIGS. 10 and 11 show a visual recognition system according to a third embodiment of the present invention. In the first embodiment, the observation conditions of the observation system 4 are set, and in the second embodiment, the measurement conditions for image processing are set.
In the embodiment, as shown in FIG. 10, the work contents of the work robot 41 arranged along the transport path of the belt conveyor 2 are set.

In FIG. 10, reference numeral 10 denotes an information display body, in which a code recording section records the work contents of the work robot 41 in codes, and the work robot 41 responds to the recognition result by the visual recognition device 3. Then, the operation content corresponding to the code content read by the visual recognition device 3 is executed. In the drawing, reference numeral 8 denotes a television camera constituting the visual recognition device 3, reference numeral 5 denotes an image processing device, reference numeral 6 denotes a video monitor, and reference numeral 2 denotes a belt conveyor for transporting a plurality of objects 1A and 1B.

FIG. 11 shows a basic configuration of the image processing apparatus 5, in which a code reading section 12 and a work content setting section 42 are provided.
And an image processing unit 15. Code reading unit 1
2 reads an image of the information display 10 from the television camera 8 and reads a barcode. The work content setting unit 42 sets the work contents based on the result of reading by the code reading unit 12 and outputs the work contents to the work robot 41. The image processing unit 15 captures images of the objects 1A and 1B captured by the television camera 8, performs measurement and recognition processing together with a CPU (not shown), and outputs the recognition results to the work robot 41.

In each of the above embodiments, the belt conveyor 2
A group of the above-mentioned one kind of object 1A and another kind of object 1
The information display body 10 is arranged between the first group B and the group B. As shown in FIG. 12, the information display body 10 is arranged for each object 1 to set observation conditions, measurement conditions, work contents, and the like. Can also be performed.

In FIG. 12, each information display body 10 is directly mounted on the belt conveyor 2, but, for example, as shown in FIG.
Are arranged at regular intervals, the object 1 is placed on each plate 43, and the information display 10 is placed at a predetermined position on each plate 43.
May be placed or affixed. in this case,
The observation field of view of the television camera 8 is set to a size including the object 1 and the information display body 10, and an image 44 as shown in FIG. On the other hand, the window 46 is set, and the code is read from the image in the window 46.

[0044]

As described above, according to the present invention, the information display body displays visual information that defines observation conditions, measurement conditions, and work contents of the work robot. The information is read, and the observation conditions, measurement conditions, and the work contents of the work robot are automatically set based on the read result. It is not necessary to monitor the inspection line, the inspection can be completely automated, the work load on the staff can be reduced, and the occurrence of operation errors can be prevented. Further, there is no need to use a special device such as a sequencer, so that the cost can be reduced and the connection work between the devices can be simplified. Furthermore, by arranging information display bodies in groups of objects or by arranging each object, it is possible to perform various inspections and operations according to the purpose, thereby improving the quality of the inspection and the like. Has a number of notable effects.

[Brief description of the drawings]

FIG. 1 is a perspective view showing a configuration of a visual recognition system according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a basic configuration of the image processing apparatus.

FIG. 3 is a block diagram illustrating a specific example of an image processing apparatus.

FIG. 4 is a functional block diagram of a specific pattern detection unit.

FIG. 5 is an explanatory diagram showing the contents of a table stored in a ROM.

FIG. 6 is a flowchart illustrating a procedure of an interrupt processing routine of a CPU.

FIG. 7 is a block diagram illustrating a basic configuration of an image processing apparatus according to a second embodiment of the present invention.

FIG. 8 is a block diagram showing a specific example of an image processing device according to a second embodiment of the present invention.

FIG. 9 is a flowchart showing a procedure of an interrupt processing routine of a CPU according to a second embodiment of the present invention.

FIG. 10 is an explanatory diagram showing a configuration of a third embodiment of the present invention.

FIG. 11 is a block diagram illustrating a basic configuration of an image processing apparatus according to a third embodiment of the present invention.

FIG. 12 is an explanatory diagram showing another arrangement method of the information display body.

FIG. 13 is an explanatory view showing still another arrangement method of the information display body.

FIG. 14 is an explanatory diagram showing a window setting method in the embodiment of FIG.

FIG. 15 is an explanatory diagram showing a configuration of a conventional visual recognition system.

[Explanation of symbols]

 1, 1A, 1B Object 2 Belt conveyor 3 Visual recognition device 4 Observation system 5 Image processing device 7 Illumination device 8 Television camera 10 Information display 11 Objective lens 12 Code reading unit 13 Observation condition setting unit 14 Observation system adjustment unit 15 Image Processing unit 40 Measurement condition setting unit 42 Work content setting unit

──────────────────────────────────────────────────続 き Continued on the front page (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 1/00 B25J 13/00 B25J 19/04 G06T 7/00

Claims (5)

(57) [Claims] 1. An information display body on which visible information defining observation conditions is displayed, a transport mechanism for transporting a plurality of objects and the information display body to a predetermined observation position, and a visual recognition device arranged at the observation position. The visual recognition device has an observation system and an image processing device, the observation system is a lighting device for illuminating an observation position, and the information display body and the object conveyed to the observation position. An image pickup device for picking up an image of the information display device, wherein the image processing device receives an image of an information display from the image pickup device and reads the visible information, and the observing system based on a reading result by the reading unit. Setting means for setting the observation conditions, adjustment means for automatically adjusting the observation system in accordance with the observation conditions set by the setting means, and an adjustment of the object imaged by the imaging device after adjustment by the adjustment means A visual recognition system comprising: image processing means for processing an image. 2. An information display body on which visible information defining measurement conditions is displayed, a transport mechanism for transporting a plurality of objects and the information display body to a predetermined observation position, and visual recognition arranged at the observation position. The visual recognition device has an observation system and an image processing device, the observation system includes an imaging device for imaging the information display body and a target object conveyed to an observation position, The image processing apparatus includes: an image processing unit configured to process an image of an object captured by the imaging device according to predetermined measurement conditions; and a reading unit configured to input an image of an information display from the imaging device and read the visible information. A visual recognition system comprising: a setting unit configured to set the measurement condition based on a reading result of the reading unit and output the measurement condition to the image processing unit. 3. A work robot for performing a predetermined work,
An information display body on which visible information that defines the work content of the work robot is displayed, a transport mechanism for transporting a plurality of objects and the information display body to a predetermined observation position, and a visual recognition device arranged at the observation position The visual recognition device has an observation system and an image processing device, and the observation system includes an imaging device for imaging the information display body and a target object transported to an observation position, An image processing device configured to process an image of the object captured by the imaging device; a reading device that inputs an image of an information display from the imaging device and reads the visible information; Setting means for setting the work content of the work robot based on the read result and outputting the work content to the work robot, wherein the work robot performs the setting in accordance with a processing result of the image processing means. A visual recognition system configured to execute the work content set by the means. 4. The information display body is positioned and arranged between a group of a plurality of objects conveyed by the conveyance mechanism and a group of objects of a different type from the objects. The visual recognition system according to any one of the above. 5. The visual recognition system according to claim 1, wherein the information display body is positioned and arranged in correspondence with each object transported by the transport mechanism.