JP3506161B2 - Agricultural and fishery product appearance inspection device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual inspection apparatus for agricultural and marine products, which visually inspects and grades the shape, size, color and the like based on images of agricultural and marine products.

[0002]

2. Description of the Related Art As a conventional appearance inspection device for agricultural and marine products,
Kazuhiko Fukuda, Masao Nito, Ryo Watanabe, "Processing and sorting system based on color and shape of agricultural and marine products", Fuji Jikho, VOL.67, No.9,19
There is 94. It processes the images obtained by automatically capturing the agricultural and marine products to be inspected by the camera,
Shape recognition, color recognition, and surface inspection are performed. In these apparatuses, a conveyor is generally used to convey the inspection object, and the inspection object is imaged while being conveyed on the conveyor. The surface of the conveyor, which is the background when imaging the inspection object, is usually black or a color close to black. Therefore, for general agricultural and marine products whose surface color has high brightness, the brightness difference from the background becomes large, and the processing of the captured image is easy.

[0003]

However, in the case where the surface of the conveyor is black or a color close to black in this way, the inspection object having a low brightness of the surface color, for example, apples such as eggplant, fuji and starking is inspected. In this case, since the difference in brightness between the surface color of the inspection object and the surface color of the background conveyor is small, the distinction between the background and the inspection area becomes unclear, and the shape characteristics of the inspection object from the captured image. In some cases, the amount of measurement may be inaccurate, or the amount of surface state such as color, scratch, or pattern may not be sufficient. Therefore, the accuracy of the inspection result is lowered, which may hinder the sorting and grading in the subsequent process. The present invention has been made in order to solve the above problems, and its object is to produce an agricultural and marine product that can always obtain an inspection result with a constant accuracy, even for an inspection target having a low surface color lightness. It is to provide a visual inspection device.

[0004]

In order to achieve the above object, the invention of claim 1 is a visual inspection apparatus for agricultural and marine products which processes an image obtained by picking up an image of agricultural and marine products with a camera to inspect the appearance. In, a half mirror that splits the reflected light from the inspection object into two optical paths, a near-infrared filter that is installed on one optical path and passes only the near-infrared light region, and another optical path on the other optical path The visible light filter that is installed and passes only the visible light region, the cameras installed at the ends of both optical paths, and the shape feature amount of the inspection target from the shape of the near infrared image captured by one camera From a means for measuring, a means for subjecting a visible light image picked up by the other camera to a masking process using a binarized near-infrared light image to extract only an image of an inspection target area, and only an inspection target area. Is it a visible light image Characterized by a visual inspection of the round test object and a means for measuring the surface features of the inspection object.

According to a second aspect of the present invention, in the first aspect of the invention, a mirror is disposed on either one of the optical paths separated by the half mirror so that the phases of the images picked up by both cameras coincide with each other. The feature is that the appearance inspection of a non-circular inspection object is enabled.

According to a third aspect of the present invention, in the first or second aspect of the invention, one of the images is contracted or expanded before the masking process in accordance with a difference in magnification of both picked-up images measured in advance. It is characterized in that it is provided with a means for applying the same magnification.

According to a fourth aspect of the present invention, in the first, second or third aspect of the present invention, one of the images is moved before the mask processing in accordance with the displacement of the two picked-up images measured in advance. It is characterized in that a means for matching both images is provided.

[0008]

According to the first aspect of the invention, the invention is applied to an inspection object having a circular outer shape, and the reflected light from the inspection object is split into two optical paths of transmitted light and reflected light by the half mirror. . One of the separated lights passes through the near-infrared light filter and then is imaged by the camera, and the other light passes through the visible light filter and is then imaged by the camera. The shape feature of the inspection target is measured from the shape of the captured near-infrared light image. A mask process using a binarized near-infrared light image is applied to the captured visible light image to extract only the image of the inspection target region. The surface feature amount of the inspection object is measured from the visible light image that includes only the extracted inspection object region.

According to the second aspect of the present invention, the optical paths of the two systems separated by the half mirror in the first invention are mutually inverted in phase, but the mirrors are arranged in either one of the optical paths and are further inverted. As a result, the phases of the images captured by both cameras match each other. As a result, it becomes possible to perform mask processing on a non-circular inspection object.

According to the third aspect of the present invention, before the mask processing, one image is subjected to contraction processing or expansion processing in accordance with a difference in magnification of both captured images which is measured in advance, so that the magnification of both images is increased. Match.

According to the fourth aspect of the invention, before the mask processing, one of the images is moved according to the pre-measured positional deviation between the two captured images so that the two images coincide with each other.

[0012]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is an explanatory view showing the configuration of the first embodiment according to the invention of claim 1. This embodiment is intended for inspection of agricultural and marine products such as apples whose outer shape is circular. In the figure, reference numeral 1 denotes a conveyor, which continuously conveys the inspection object 2. Normally, the conveyor 1 serves as the background of the inspection object 2, and therefore a black color having low lightness is selected. A plurality of illuminators 3 are concentrically arranged above the conveyor 2 to irradiate the inspection object 2 uniformly and without halation. In this embodiment, an incandescent lamp having a visible light region and a near infrared light region is used as the illuminator 3.

An image pickup unit 4 is installed further above. The image pickup unit 4 is entirely composed of a dark box 41.
Reflected light from the inspection object 2 is incident on the opening formed in the lower portion of the inspection object 1. A half mirror 42 is installed in the dark box 41 at an angle of 45 degrees with respect to the optical axis of the incident light to split a part of the incident light in the horizontal direction. In this embodiment, as the half mirror 42, a glass plate on which metal is vapor-deposited is used.
Although visible light to near-infrared light is set to 50% for transmission and 50% for reflection, it is also possible to use a prism to disperse light.

CCD cameras 43 and 44 are provided on both optical paths separated by the half mirror 42 in two directions. Here, when installing the CCD cameras 43 and 44,
The magnification and the optical axis position are finely adjusted so that the images of the inspection object 2 obtained by the both cameras have the same size and the same position. A filter 47 for cutting near infrared light is provided in front of the lens 45 of the CCD camera 43.
Is attached, the CCD camera 43 images the inspection object 2 with visible light, and sends the image signal a to the image processing unit 5. The CCD camera 43 may be either a color camera or a monochrome camera.

A filter 48 for cutting visible light is attached to the front part of the lens 46 of the CCD camera 44, and the CCD camera 44 images the inspection object 2 with near-infrared light and images the image signal b thereof. It is sent to the processing unit 5. This C
A monochrome camera is used as the CD camera 44. Further, in the dark box 41, a light shielding plate 49 that cuts ambient light is installed on a surface orthogonal to both optical paths. The camera 4
It is also possible to dispose the filters 47 and 48 attached to 3, 44 at other positions on the optical path, or to form them on the half mirror 42.

Further, a sensor 6 is installed on the conveyor 1 to detect that the inspection object 2 has reached an image pickup position directly below the image pickup section 4, and the signal c for detecting the inspection object 2 is sent to the image processing section. Send to 5. The image processing unit 5 takes in the image signals a and b at the timing when the signal c is input, performs image processing described later, and generates and outputs a sorting signal d for the inspection object 2. The output sorting signal d is sent to a not-shown sorting device or the like.

FIG. 2 shows a visible light image picked up by the camera 43, and below that, the level distribution of the image signal on the central scanning line is shown. As can be seen from the level distribution, the visible light image has a small level difference between the background image and the inspection object image. Particularly, when the surface color of the inspection object 2 is an apple such as eggplant or fuji having a low lightness, and the background conveyor 1 is black, the tendency is remarkable. In addition,
In the conventional apparatus, the shape feature amount and the surface feature amount of the inspection object are measured only by this image.

FIG. 3 shows a near-infrared light image picked up by the camera 44, and below that, the level distribution of the image signal on the central scanning line is shown. As can be seen from the level distribution, the near-infrared light image is easy to binarize because the level difference between the background image and the inspection object image is large. this is,
This is because the reflectance of near-infrared light from the surface is particularly large as a characteristic of the plant body itself. That is, even if the inspection object has low brightness with visible light that can be perceived by human eyes, a sufficient level of reflected light can be obtained in the near infrared light region. Of course, with near-infrared light, it is impossible to detect the feature amount from the color of the surface, but it can be sufficiently used to extract the shape of the inspection object from the background image.

That is, it is possible to measure the shape feature amount of the inspection object 2 after binarizing the near-infrared light image shown in FIG. 3 with the threshold value L to extract the shape of the inspection object. At the same time, by using the binarized image as a mask pattern, the visible light image of FIG. 2 is masked to remove the background image portion, and only the inspection target area can be extracted.
In the mask processing, it is necessary to match the positions and magnifications of both images.

FIG. 4 is a block diagram showing the configuration of the image processing section 5. In the figure, 501A and 501B are AD converters for digitizing the image signals a and b sent from the cameras 43 and 44, respectively, both of which are quantized into pixels of a measurement unit size, and image memories 502A and 502B.
Send to. In the image memories 502A and 502B, writing and reading of digitized image signals are performed in accordance with instructions from the computer 509 described later. Switch 503
The image memory 50 according to the instruction of the computer 509.
Either one of the image signals read from 2A and 502B is selected and sent to the cutout unit 504 in the subsequent stage. The cutout unit 504 binarizes the input image signal and sends it to the contraction / expansion processing unit 505.

In accordance with instructions from the computer 509, the contraction / expansion processing unit 505 executes a predetermined contraction or expansion process on the inspection object area having a pixel value of 1 on the near-infrared light image to obtain the shape. It is sent to the feature extraction unit 506 and the mask processing unit 507. That is, it is difficult for the cameras 43 and 44 installed in the imaging unit 4 to match each other even in pixel units, even if mechanically finely adjusting the magnification of each other. Therefore, by contracting or expanding one image,
Match the sizes of both images. The number of pixels of this contraction / expansion processing is calculated in advance using a calibration pattern described later. Further, by this contraction or expansion processing, the peripheral portion of the image can be deleted or smoothed to stabilize the measurement.

The shape feature extraction unit 506 extracts the shape of the inspection object region having a pixel value of 1 from the input infrared light image signal, and measures the shape feature amount such as area, length, width, and position. Then, the data is input to the computer 509. Since the specific procedure for measuring the shape feature amount is already known, the description thereof will be omitted. The mask processing unit 507 masks the visible light image by using the inspection target area in which the pixel value of the input near-infrared light image signal is 1 as a mask pattern to mask only the inspection target area in the visible light image. It is taken out and sent to the surface feature extraction unit 508. The surface feature extraction unit 508 measures the surface feature amount such as the color, scratch, and pattern of the inspection object from the input visible light image and inputs it to the computer 509. Note that the specific procedure for measuring the surface feature amount has already been known, and thus the description thereof will be omitted.

The computer 509 operates on the basis of the timing when the signal c from the sensor 6 is input,
02A, 502B image signal writing and reading, switching device 503 switching, contraction / expansion processing unit 505
Is instructed, and based on the measurement values input from the shape feature extraction unit 506 and the surface feature extraction unit 508,
The grade of the inspection object is discriminated and sent to the sorting signal. In addition,
Since a specific inspection procedure for classifying the inspection object executed here is already known, the description thereof will be omitted. An HCI section 510 is connected to the computer 509, and the HCI section 510 provides an interface with the operator.

FIG. 5 shows the camera 4 installed in the image pickup section 4.
Reference numerals 3 and 44 are calibration patterns used to adjust the magnification and the position of the captured image. This calibration pattern is formed by combining a white circular pattern that reflects the region from visible light to near infrared light uniformly and at a high rate. This calibration pattern is
The images are picked up by the cameras 43 and 44 and input to the computer 509. The computer 509 calculates the center of gravity G _{1 to} G ₄ of each circle for each image, and then calculates the total center of gravity G ₀ . Next, the positional coordinates of the overall center of gravity G ₀ of both images are compared to obtain the positional deviation amount S of both images.

The amount of positional deviation S thus obtained is stored in the computer 509, and is stored in the image memory 5 during operation.
The offset value of the address of 02A or 502B is used to correct the positional deviation of the image. Further, when obtaining the difference in magnification, first, the calibration pattern is imaged by the visible light image camera 43, and the distances La _{1 to} La ₄ between the centroids G _{1 to} G ₄ of the respective circles are calculated. Similarly, the calibration pattern is imaged by the camera 44 for near-infrared light images, and the distances Lb _{1 to} Lb ₄ between the centers of gravity G _{1 to} G ₄ of each circle are calculated. Next, the magnification difference K between the two images is calculated by the following equation.

[0026]

## EQU1 ## K = {(La ₁ -Lb ₁ ) / Lb ₁ + (La ₂ -L
b ₂ ) / Lb ₂ + (La ₃ −Lb ₃ ) / Lb ₃ + (La ₄ −L
b ₄ ) / Lb ₄ } / 4

Next, the number of pixels of the contraction / expansion processing in the contraction / expansion processing unit 505 is obtained using the obtained magnification difference K between both images. Specifically, assuming that the imaged measurement target area is a circle, the area of the binarized measurement target area is calculated, and the radius R of the equivalent circle is calculated. This radius R is multiplied by the magnification difference K to obtain the contraction / expansion processing amount Z. When Z is negative, the contraction process is performed, and when Z is positive, the expansion process is performed. If the area to be measured is not circular, the coefficient R that is experimentally determined according to the shape is multiplied by the radius R of the equivalent circle to correct it, and then the contraction / expansion processing amount Z is determined.

According to this embodiment, there is little difference between the blackness and the brightness of the conveyor, which is the background when the inspection object is imaged,
Even for apples with a deep red tint, at the same time as imaging with visible light, the inspection target is imaged with near-infrared light, and the shape feature amount is measured from the near-infrared light image whose distinction from the background is clear. The visible light image is masked by using the external light image as a mask pattern to extract only the inspection object region, and the surface feature amount can be measured without being affected by the background image.

Further, with respect to the inspection object having a high surface color lightness, both characteristic quantities can be measured by imaging with visible light and near infrared light in the same manner. As a result, the inspection device of this embodiment, regardless of the difference in brightness between the color of the conveyor and the color of the inspection object, always inspects various agricultural and marine products with stable accuracy to determine the grade and sort. You can

FIG. 6 is an explanatory view showing the configuration of the second embodiment according to the invention of claim 2. In FIG. In the first embodiment, the inspection target is agricultural and marine products such as apples having a circular external shape, whereas in the present embodiment, the inspection target is agricultural and marine products such as eggplants having a non-circular external shape. Is. Since the configuration of each part has a part in common with the embodiment of FIG. 1, common parts are given the same reference numerals as those in FIG. 1, and description thereof is omitted, and only different parts will be described.

In this embodiment, as shown in the figure, a mirror 40 is provided parallel to the half mirror 42 on the optical path in the horizontal direction reflected / split by the half mirror 42 in the image pickup section 4.
Is arranged, and the camera 44 is arranged above it in parallel with the camera 43. By arranging this mirror 40,
Since the light reflected by the half mirror 42 is further inverted, the image captured by the camera 44 is
The phase matches that of the image picked up in.

FIG. 7 shows a visible light image taken by the camera 43, and FIG. 8 shows a near infrared light image taken by the camera 44. In addition, the figure shown by the broken line in FIG. 8 is shown as a reference for an image when the mirror 40 is not installed. As shown in both these figures, the second
In the embodiment described above, even if the non-circular inspection target object is imaged, both images will match. That is, in the case of the first embodiment shown in FIG. 1, the inspection object is limited to a circular shape.
Inspection is possible regardless of non-circular shape.

In particular, in the case of a circular inspection target object, in the first embodiment, it is necessary to match the center of gravity of the inspection target object 2 with the optical axis at the time of imaging, so that the control of the alignment of the inspection target object 2 is troublesome. However, in this embodiment, even if the center of gravity is slightly displaced from the optical axis, both images are displaced in the same direction.
It does not interfere with the inspection. In the second embodiment, the image on one optical path is inverted by the mirror 40, but it is also possible to invert one image by software processing after image capturing. Further, even when the rotation phases around the optical axis are deviated from each other when the cameras 43 and 44 are attached, the deviation can be corrected by calculating the rotation phase deviation using the calibration pattern.

[0034]

As described above, according to the present invention, the reflected light from the inspection object is separated by the half mirror, transmitted through the filter, and then captured as a visible light image and a near infrared image. This near-infrared image is sufficiently reflected from the inspection object with a low surface color, so when inspecting an eggplant or a reddish apple, the background image and the inspection image should be clearly separated. It becomes possible to measure the shape feature amount with sufficient accuracy.

Similarly, the visible light image is subjected to mask processing using a binarized near-infrared light image and only the image of the inspection object region is extracted, so that the inspection object is not affected by the background image. It becomes possible to measure the surface feature amount of. That is, even with respect to an inspection object in which the lightness of reflected light from the surface is low, by using the present invention, it is possible to accurately extract only the image of the inspection object and measure the feature amount. It is possible to always obtain an inspection result with a constant accuracy regardless of the brightness of the surface color of an object.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing a configuration of a first embodiment according to the invention of claim 1.

FIG. 2 is a diagram showing a visible light image captured in FIG.

FIG. 3 is a diagram showing a near-infrared light image taken in FIG.

FIG. 4 is a block diagram showing a configuration of an image processing unit in FIG.

5 is a calibration pattern for adjusting the magnification and position of both cameras in FIG. 1. FIG.

FIG. 6 is an explanatory diagram showing a configuration of a second embodiment according to the invention of claim 2;

7 is a diagram showing a visible light image taken in FIG. 6;

FIG. 8 is a diagram showing a near-infrared light image taken in FIG.

[Explanation of symbols]

1 conveyor 2 Inspection object 3 illuminator 4 Imaging unit 5 Image processing unit 6 sensors 40 mirror 41 Dark Box 42 half mirror 43,44 CCD camera 45,46 lens 47,48 filters 49 light shield 501A, 501B AD converter 502A, 502B image memory 503 Switch 504 Cutout part 505 Contraction / expansion processing unit 506 Shape feature extraction unit 507 Mask processing unit 508 Surface feature extraction unit 509 computer 510 HCI department

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Yuko Enko 1-1 Tanabe Nitta, Kawasaki-ku, Kawasaki-shi, Kanagawa Prefecture Fuji Electric Co., Ltd. (56) Reference Japanese Patent Laid-Open No. 1-124073 (JP, A) Kaihei 1-292238 (JP, A) JP 5-26636 (JP, A) JP 5-332739 (JP, A) JP 6-160050 (JP, A) JP 6-258225 ( JP, A) JP 7-91931 (JP, A) JP 56-108579 (JP, A) JP 63-200279 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 1/00 300 G01N 21/84

Claims (4)

(57) [Claims] 1. A half-mirror for splitting reflected light from an inspection object into two optical paths in an appearance inspection apparatus for an agricultural and marine product, which processes an image obtained by imaging an agricultural and marine product with a camera to inspect the appearance. A near-infrared light filter installed on one optical path to pass only the near-infrared light region, a visible light filter installed on the other optical path to pass only the visible light region, and the ends of both optical paths. Cameras installed in each part, means for measuring the shape feature amount of the inspection object from the shape of the near-infrared light image captured by one camera, and binarization into the visible light image captured by the other camera A means for performing mask processing using the near-infrared light image to extract only the image of the inspection target area, and a means for measuring the surface feature amount of the inspection target from the visible light image consisting of the inspection target area only. Prepared circular Agricultural products of the visual inspection apparatus, characterized by a visual inspection of the test object. 2. The agricultural and marine product appearance inspection apparatus according to claim 1, wherein a mirror is disposed on either one of the optical paths separated by the half mirror so that the phases of the images captured by both cameras coincide with each other. The agricultural and marine products visual inspection device is characterized in that it enables the visual inspection of non-circular inspection objects. 3. The apparatus for inspecting the appearance of agricultural or marine products according to claim 1 or 2, wherein one of the images is contracted or expanded before mask processing, depending on the difference in magnification of both captured images measured in advance. An appearance inspection apparatus for agricultural and marine products, characterized in that it is provided with means for applying the above-mentioned factors to match the magnification. 4. Claim 1 or claim 2 or claim 3.
The appearance inspection apparatus for agricultural and marine products described above, wherein before the mask processing, means for moving one image according to a positional deviation between the two captured images measured in advance to match the two images is provided. Visual inspection equipment for marine products.