KR100400861B1 - Machine vision system for on-line extraction and qualification of appearance quality factors of fruit and method thereof - Google Patents

Abstract

The present invention relates to a system and method for online detection and quantification of fruit appearance quality factors, comprising: a conveyor unit (10) provided in a frame unit (20); An optical sensor (37) provided to detect the fruit carried by the conveyor unit (10); The first image acquisition unit 31 and the left and right side images of the fruit are respectively acquired to acquire an upper image of each provided fruit so as to acquire an image related to the overall color, defect and shape of the selected fruit according to the detection signal of the optical sensor 37. Image acquisition means (30) having second and third image acquisition units (33, 35), respectively; The conveyor unit 10, the optical sensor 37, and the image acquisition means 30 are electrically connected to each other, and a rating program of fruits is built therein to read the transmitted multiple images to determine the grade of the fruits, Including the computer system (COM) having a monitor (90) displaying the transmitted multiple images and the fruit grading determination value, the selection process according to the color, defect and shape among the external quality factors of the fruit determining the merchandise By performing grading in real time in the stopped state or in the transport state, it is possible to improve the quality of fruit through the strict screening of fruit compared to the conventional sorting by the selector's supervision. There are features that can be.

Description

System and method for on-line detection and quantification of fruit appearance quality factors {MACHINE VISION SYSTEM FOR ON-LINE EXTRACTION AND QUALIFICATION OF APPEARANCE QUALITY FACTORS OF FRUIT AND METHOD THEREOF}

The present invention relates to a system and method for on-line detection and quantification of fruit appearance quality factors, and more particularly, to optically quantify the color, defect (점, 打傷, 刺傷) and shape of the main appearance quality factors of apples. A system and method for evaluating characteristics and measuring quality factors online.

Apple's scientific name is Malus pumila Miller, which is representative of the deciduous fruit, and its fruit is excellent in shape and taste. In Korea, the natural climatic conditions are suitable for fruit cultivation, and despite the decline in agricultural production, the share of apples in agriculture is increasing, and the production also increased by 17% from 556,160 tons in '85 to 651,000 tons in '97. It was. However, domestically grown cultivation area is showing oversupply, and UR negotiations are internationally inevitable competition with major apple producers such as the United States at home and abroad.

Lee Gye-im (1998), who analyzed the preferences for fruit quality factors, divides consumer buying behavior into household consumption and gifts, and analyzes the preferences of quality factors in each case. Reported differences in quality factors. This means that the consumer's demand for fruit is very subjective and the preferred quality factor varies according to the type of consumption. In order to satisfy the demand for quality, above all, various sorting by quality factors need to be performed.

On the other hand, when exporting fruits, the size and color should be uniform. Currently, in Korea, subjective grading by producers is inevitable in foreign fruits and vegetables. Therefore, in order to increase the added value of the fruits produced, other sorting operations must be added in addition to the currently used weight sorting. Currently, the fruit sorting system is mostly made of weight screening and color sorting by some mechanical visual devices, but most of the other quality factors such as shapes and defects are performed by manpower at the primary sorting table.

In addition, if a simple repetitive task such as sorting is performed by a human being, the quality control of agricultural products that are shipped is virtually impossible because the sorter's subjectivity is very likely to be involved and the sorting result is also less reproducible. This is in stark contrast to the effort to instill confidence in the product by improving the quality of the product through strict quality control. In addition, farmers cannot obtain numerical data on the quality of the produced fruit when selecting a manpower, so the characteristics of the produced fruit cannot be grasped, and further improvement of the cultivation technology for shipping more improved agricultural products is expected. Difficult to do As such, the screening work is not only a task of picking fruit, but also a very important work system in that it provides added value and gives confidence to consumers.

The sorting work is the core work to improve the commerciality of fruits. The sorting process is based on sorting factors for the external quality such as size, weight, shape, color, and the presence of scars, and for the internal quality such as sugar, acidity, taste, and aroma. Can be divided. In Korea, only the screening by the electronic sensor is used for screening in general, and recently, a starter for color screening has been produced and used in exhibitions and at some sites. In advanced countries, various quality factors are screened, but it is not suitable for the fruits produced in Korea because the shape, size, color, grade, etc. of the fruits are different for importing and using them in Korea.

The sorting machine using the mechanical visual device can sort the size and color of the fruit, and the price is falling due to the development of the electronic industry. In Korea, fruit sorting system using a foreign machine visual system has been introduced and used, and they are conveyed by rolling the fruit by a conveying roller, by conveying the plate while the fruit is placed on the plate, and by the free conveying plate. Can be divided in a way. The method of rolling the fruit by the transfer roller has the advantage of observing the whole surface of the fruit, but there is a possibility that the fruit may be shocked during the transfer, and in particular, the impact may be somewhat impacted by the drop motion during the discharge. The conveying method by the dish has no impact when transferring the fruit, but it is similarly impacted upon discharging. The conveying method by the free conveying plate has the advantage that the fruit is hardly impacted from conveying to discharging. This becomes complicated and the disadvantage of not being able to observe the whole surface of the fruit by a feed plate. Fruits produced in Korea, for example, are very sensitive to impact due to their thin skin and soft tissue, so if the object is rolled, it is more likely to be bruised by physical impact during the selection process. There is a risk of falling.

Fruit is a highly commercial crop, and its value depends on its appearance. In other words, before evaluating the taste of fruit, healthy fruit without damage is seen high price.

Accordingly, the present invention was created to overcome the above-mentioned problems, and an object of the present invention is to quantify the sorting process according to color, defect and shape among the external quality factors of fruit to determine the merchandise. The present invention provides a system and method for online detection and quantification of fruit appearance quality factors capable of performing grading in real time.

1 is a perspective view showing a system for on-line detection and quantification of fruit appearance quality factors according to the present invention.

2 is a schematic representation of the FIG. 1 system.

FIG. 3 is a schematic diagram illustrating a first image acquisition unit of FIG. 1. FIG.

FIG. 4 is a schematic diagram illustrating second and third image acquisition units of FIG. 1; FIG.

Figure 5 is a photograph showing a seven-channel image of the fruit obtained by the system of the present invention.

FIG. 6 is a graph showing Mahalanobis distances between normal and defect areas of Fuji fruit in each wavelength band. FIG.

7 is a flowchart showing a grading method according to the system of the present invention.

8 is a flowchart showing a color grade determination step by the system of the present invention.

9 is a flow chart showing the defect determination step by the system of the present invention.

<< Description of main part of drawing >>

10: conveyor portion 20: frame portion

30: image acquisition means 31, 33, 35: image acquisition unit

37: light sensor

312,313,314,333,334,353,354 Camera 90 Monitor

COM: Computer System

The system of the present invention having such an object includes a conveyor unit installed in the frame unit for conveying fruit, an optical sensor for detecting whether the fruit conveyed by the conveyor unit is located at an image acquisition position, and a detection signal of the optical sensor. The first image acquisition unit captures the upper part of the fruit located at the image acquisition position with one color camera and two monochrome cameras to acquire one color image and two near infrared monochrome images, and the second and third image acquisition units Image acquisition means for acquiring two near-infrared monochrome images by photographing the left side and the right side of the camera with two black and white cameras, and an object electrically connected to the conveyor unit, the optical sensor, and the image acquiring means, respectively, and obtained by the image acquiring means. The images of the fruits are read according to the stored grading program, and the grade of the fruits is determined. It can be achieved by a system for on-line detection and quantification of the fruit appearance quality factor configured, including a computer system to display on the monitor.

In addition, in the method of the present invention, when the fruit conveyed by the conveyor unit is located at the image acquisition position, the image acquisition means photographs the upper part of the fruit to acquire one color image and two near infrared monochrome images, and the left part of the fruit. A first step of acquiring two near-infrared monochrome images by respectively photographing the right and the second parts with two black-and-white cameras; and a second step of determining the color grade of the fruit by using the color image of the upper part of the fruit obtained in the first step; A third step of detecting pathologic defects, contusion defects and autologous defects and determining a shape grade using two near infrared black and white images of the upper part of the fruit obtained in the first step, and each of the left part and the right part obtained in the first step; A fourth step of detecting pathologic defects, bruising defects, and autologous defects using a near-infrared monochrome image of each dog; and the color grade and the disease detected in the second to fourth steps. It can be achieved by on-line detection and quantification determination method of fruit appearance quality factor consisting of the fifth step of displaying the half defect, the other defect and the auto defect and the shape grade on the monitor.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The system for on-line detection and quantification of fruit appearance quality factor according to the present invention is to take a picture of the fruit in real time from the upper and left and right sides to reach the measurement position in the fruit continuously transported by the conveyor unit 10 to take a multi-image Install the image acquisition means 30 at any position of the frame portion 20 to acquire, display each acquired image data on the monitor 90, and read the acquired image to determine the grade of the fruit. It consists of an embedded computer system (COM).

Frame portion 20 is a strut frame 21 is formed by a plurality of steel bars in consideration of the overall weight of the system of the present invention, the strut so that the above-described image acquisition means 30 and the conveyor unit 10 can be installed The base frame 23 is provided on the upper surface of the frame 21.

In addition, the lighting means 50 is installed in the support frame 25 installed on the upper surface of the base frame 23 so as to realize the best illuminance when the real-time shooting of the fruit is made by the image acquiring means 30. . In addition, the image acquisition means 30 is installed on the upper surface of the support plate 27 installed on the upper surface of the upper surface of the fruit, for example, the upper surface of the support frame 25, for example, the first image acquisition unit It is. In addition, an opening is formed in the central portion of the support plate 27 so that the camera lens of the first image acquisition unit 31 can photograph the top of the fruit.

The luminaire 50 is provided with a plurality of tungsten-halogen lamps (51; 200W) at a predetermined angle to the support frame 25 in order to obtain uniform illumination, which is about 970nm for the detection of bruises and streaks of fruit. Since the response characteristic of the camera is very low, very strong illumination is required to compensate for this, and a detailed description thereof will be described later.

Conveyor unit 10 is installed on the upper surface of the base frame 23 for the continuous transport of the fruit, the mechanical connection between the motor and the conveyor as a driving means is not clearly shown in this figure. In addition, when the system of the present invention is installed in a shipping place for determining the actual fruit grade, the conveyor unit 10 has a corresponding length. Here, the conveying speed of the conveyor unit 10 is adjusted up to 50 cm / sec. It is desirable to be.

The image acquisition means 30 is installed on the upper surface of the base frame 23 so as to capture the transferred fruit in real time to obtain multiple images, the first image acquisition unit 31 for capturing the upper portion of the fruit ) And second and third image acquisition units 33 and 35 which are provided on both left and right sides of the conveyor unit 10 and photograph the left and right sides of the fruit, respectively. Here, the fruit conveyed by the conveyor unit 10 is detected by the optical sensor 37 controlled by the computer system COM, and the image when the sensor 37 detects that the fruit is located at the image acquisition position Image acquisition by the acquisition units 31, 33, 35 is started.

First, referring to FIG. 3, the structure of the first image acquisition unit 31 will be described. The first image acquisition unit 31 divides the light incident through the light splitter 311 into three directions, one in each direction. Color camera 312 and two black and white cameras 313 and 314 for near-infrared image acquisition are respectively installed in the stand 317.

In this case, when the camera is mounted on one surface of the light splitter 311, two light splitting pieces 311a and 311b are continuously connected because the area of the fruit measured differently appears due to the space occupied by each camera. It is built so that three images can be acquired by three cameras on one side. In more detail, the light splitter 311 has a structure in which a predetermined portion of the upper and lower surfaces of the housing 311c is opened, and when light is incident through the lower opening, the light splitter 311 divides the incident light into a corresponding camera 314. The optical splitter plates 311a and 311b are inclined vertically spaced apart so as to acquire an image of the fruit by 313 and 312. At this time, two light splitting plates 311a and 311b having a transmittance and a reflectance of 7: 3 and 6: 4, respectively, are built in to divide the incident light into a constant ratio. 30%, 28%, and 42% of light is incident on the lens of the lens.

As shown in FIG. 3, the light divided by each of the light splitting plates 311a and 311b is incident on the corresponding cameras 313 and 314 at a constant ratio as described above, respectively. 720 nm filter 311 d and 970 nm filter 311 e are respectively provided in the entrance surface of the housing 311 c to which light is incident, and the reason for using the above-mentioned wavelength band filter 311 d and 311 e will be described later. .

Here, the grading is obtained by receiving an image when a signal indicating that the fruit in the conveying state has reached the image acquisition position, which is the lower end of the color camera 312, by the optical sensor 37. The color grade is determined in the image, the second monochrome camera 314 detects a defect, and the first monochrome camera 313 is configured to determine the shape grade. In particular, the shape determination is performed in the 970nm first monochrome camera 313 because the distance between the fruit and the camera is relatively close and the spatial resolution is larger than other images.

The detailed measurement contents in the image acquired by the color camera 312 are the number of fruit pixels and the number of colored pixels. In the first black and white camera 313, the length and length of short and short axis, the distance between the center and the nipple, and the area are measured.

The color camera 312 applied to this system is the LG-305G (LG Honeywell), and the monochrome cameras 313 and 314 used the LG-145E (LG Honeywell), and a fixed focal length lens having a focal length of 16 mm (f). 1.4, Avenir), and the color image processing board for image acquisition was using FlashBus MV-Pro (Integral tech. USA), and the computer system (COM) equipped with Pentium MMX 233MHz CPU was used as the main computer for grading. Used.

Referring to FIG. 4, the second and third image acquisition units 33 and 35 divide the light divided by the light splitters 331a and 331b and 351a and 351b built into the light splitters 331 and 351. The first and second black and white cameras 333 and 334 and 353 and 354 respectively correspond to the left and right sides of the apple to acquire an image. Here, the cameras 333, 334, 353, and 354 of the second and third image acquisition units 33 and 35 may include the first and second cameras 313 of the first image acquisition unit 31 described above. A camera having the same function and performance as 314 and filters 331d, 351d; 720nm and 331e, 351e; 970nm are also the same wavelength band filters.

The images acquired by the first to third image acquisition units 31, 33, 35 described above are transmitted to the computer system COM by the corresponding multiplexers 50, 60, respectively, to the monitor 90. Is displayed. In this case, the first multiplexer 50 draws one color image and two black and white images acquired by the first image acquisition unit 31, and the second multiplexer 60 receives the second and third image acquisition units 33. Each of the two black and white images obtained at 35 is transmitted to the computer system COM.

5 is a photograph in which images obtained from seven cameras are displayed on the monitor 90 so as to determine the color, defect and shape of the actual fruit by the system of the present invention. Color and near infrared black and white images are shown, and quadrant 4 shows left and right lateral near infrared images. As shown in the photograph, when using the cameras of the second and third image acquisition units 33 and 35, the defects of the side surfaces not visible from the upper surface could be obtained as images.

In the present invention, in order to determine the near-infrared wavelength, pixel values in 15 near-infrared bands were obtained from sample samples and the discriminatibility was investigated using the Mahalanobis distance. Figure 6 shows the Mahalanobis distance between the normal region and the defect region in each wavelength band, and analyzed the defect detection possibility for each wavelength. First, the Mahalanobis distance to the bruise and the magnetic field was weak but increased as the wavelength was increased, showing the largest distance difference at the water absorption wavelength of 970nm. On the other hand, Mahalanobis distance to the lesion showed the maximum distance difference at the wavelength of 720nm, and the distance decreased as the wavelength was increased.

On the other hand, the sum of the inverses of the Mahalanobis distances for the three defects was summed to calculate the discriminant to distinguish between normal and defects. In this case, since the inverse value is taken as the inverse of the distance, the smaller the value, the higher the discrimination rate. The analysis result shows that 920nm having the constant distance difference for the three defects has the lowest discrimination value.

Taken together, the wavelength for detecting each defect is 720nm for lesions and 970nm for bruises and cuts, and the lesions are easier to detect than bruises and scratches. As a single wavelength for 920nm was found to be the most suitable.

Referring to FIG. 7, the grading step according to the system of the present invention is described. First, the fruit transferred to the conveyor unit 10 is the color camera 312 of the first image acquisition unit 31 by the optical sensor 37. If it is detected that the bottom of the (S10) to obtain an image (S11). Then, by reading the first channel image obtained by the color camera 312 to determine the color grade (S12).

The near-infrared defect of the first image acquisition unit 31 is detected by reading the channel 2 image acquired by the near-infrared second monochrome camera 314 of the first image acquisition unit 31 (S13), and the near-infrared ray of the first image acquisition unit 31 is detected. The channel number 3 acquired by the 1 monochrome camera 313 is read to detect faults and shape grades of the fruit (S14). Subsequently, the left and right side defects of the fruit are detected from the channel images 4-7 obtained by the cameras 333, 334 (353, 354) of the second and third image capturing units 33, 35 (S15). The determination result is displayed on the monitor 90 (S16).

Referring to FIG. 8, the color grading step of the grading step is described in detail. First, the fruit transferred to the conveyor unit 10 is the color camera of the first image acquisition unit 31 by the optical sensor 37. When it is detected that the lower portion of the 312 (S20) to detect the start position of the contour of the fruit and performs a chain coding at the starting point to separate the background and the fruit (S21).

Then, in order to rule out the effects of direct reflection and illumination unevenness, 10 pixel widths are analyzed and excluded from the pixel, and the nipples of the fruit should be 20% of the maximum radius from the center. In this case, the color pixel judgment is not performed on this area (S22). The color pixel determination 24 of the fruit refers to a look-up table (LUT) value with respect to the RGB value of the pixel (S23) and calculates the slope for the b ^* / a ^* value. If it is within 71 degrees, it is regarded as a colored pixel, and if it is larger than that, it is regarded as a color pixel and the coloring ratio is determined (S28). Here, since the colored pixel determination 24 is made in pixel units as shown in FIG. 8, the determination is performed on the corresponding pixel, and the determination is subsequently performed on the other pixels S26 by the same method. That is, when it is confirmed that there are no color pixels (S27), the coloring ratio is determined as described above, and the color grade of the fruit is determined (S28).

The LUT described above is a value of L ^* a ^* b ^{* and} is a value resident in the main memory of the computer system COM when the system according to the present invention is operated. In the case of Fuji apple, the final grade was decided as 1 grade, 40-60% grade 2, and less than 40% grade 3, depending on the color requirement.

Referring to FIG. 9, the defect determination step will be described in detail with reference to FIG. 9. First, this step starts from the second channel acquired by the near-infrared second monochrome camera 314 of the first image acquisition unit 31. (S31). Then, the area of interest (analysis area) of the channel area is set (S32), the start position of the fruit is searched, and the NIR image is chaincoded (S33).

Verify that the top image (channel 2 or 3) of the fruit faucet is visible (S34). If the top image is separated from the fruit and background, remove the fruit faucet (S37). After dividing the fruit area (S35) and dividing the fruit vertically into three (S36), the average gray value of the fruit is obtained and the defect pixels are counted (S38). This process is repeatedly performed to channel 7 by increasing the number of channels by one channel in channel 2 (S40), and when the channel 7 is completed, the defect threshold value (defective pixel) is measured by measuring the total number of defective pixels. 70 or more), it is determined that there is a fault in the fruit and is determined to be an abnormal fruit (S43). If it is less than the fault threshold, it is determined that there is no fault in the fruit and is determined to be a normal fruit (S42).

As described above, according to the structure of the present invention, by quantifying the sorting process according to the color, defect and shape of the external quality factors of the fruit to determine the merchandise by performing the grading in real time in the stationary state or transport state of the fruit Compared with the prior art screened by the selector's supervision, there is an advantage to improve the quality of the fruit through strict screening of the fruit to enable consumers to improve the reliability of the fruit.

Claims (9)

A conveyor unit 10 installed at the frame unit 20 to transfer fruits; An optical sensor 37 for detecting whether the fruit conveyed by the conveyor unit 10 is located at an image acquisition position; According to the detection signal of the optical sensor 37, the first image acquisition unit 31 photographs the upper part of the fruit located at the image acquisition position with one color camera 312 and two monochrome cameras 313 and 314. Color images and two near-infrared monochrome images are acquired, and the second and third image acquisition units 32 and 33 photograph the left and right sides of the fruit with two monochrome cameras 333, 334 (353 and 354), respectively. Image acquisition means (30) for acquiring black and white images; The images of the fruits are electrically connected to the conveyor unit 10, the optical sensor 37, and the image acquisition means 30, respectively, and are read out according to the stored grading program. A system for on-line detection and quantification of fruit appearance quality factors comprising a computer system (COM) for determining the quality of the fruit and displaying an image of the fruit and a fruit class judgment value on the monitor (90). The light splitter of claim 1, wherein the first image acquisition unit 31 includes a light splitter 311 having a light splitter 311a and 311b for splitting incident light in three directions, and a light splitter corresponding to the split incident light. The color camera 312 which detects the color of the fruit attached to the 311, and the near-infrared first and second monochrome cameras which were provided with the filters 311e and 311d, respectively, in the adjacent part of each lens to detect the fault and shape of the fruit. 313 and 314, The second and third image acquisition units 33 and 35 may include light splitters 331 and 351 each having light splitter plates 331a and 331b and 351a and 351b for splitting incident light. Near-infrared first and second monochrome cameras 333, 334 (353, 354) provided with filters 331e, 331d, 351e, 351d, respectively, in the vicinity of each lens so as to correspond to each other and detect faults of the fruits. System for on-line detection and quantification of fruit appearance quality factors. The wavelength band of the filters 311d and 311e, 331d and 331e, and 351d and 351e is 720 nm or contusion for detecting lesions of fruit for each of the image acquisition units 31 and 33. A system for on-line detection and quantification of fruit appearance quality factors, characterized in that it has an optional wavelength band of 970nm for detecting the magnetic field. The lighting means according to claim 1 or 2, further comprising a plurality of tungsten-halogen lamps (51) provided on the frame portion (20) at the same angle to obtain a uniform image by the image acquisition means (30). And (50), wherein the blinking is done manually for on-line detection and quantification of fruit appearance quality factors. When the fruit transported by the conveyor is located at the image acquisition position, the image acquisition means captures the upper part of the fruit to acquire one color image and two near infrared black and white images, and two black and white cameras on the left and right sides of the fruit, respectively. A first step of acquiring two near-infrared monochrome images by capturing each other; A second step of determining a color grade by dividing each pixel of the color image of the upper part of the fruit obtained in the first step into a color pixel and a color pixel, and determining a color ratio by the color pixel and color pixel that are separated; The near-infrared monochrome image of the upper part of the fruit obtained in the first step was counted and the number of defect pixels of the fruit was counted and the pathologic defects, contusion crystals and autologous defects of the fruit were detected. Determining a third step; A fourth step of counting the fault pixels of the fruit using the near-infrared monochrome images obtained in the first step and the number of the defect pixels counting the number of defect pixels, bruising defects and autologous defects; And a fifth step of displaying color grades, pathologic defects, bruising defects, autologous defects, and shape defects and shape grades detected in the second to fourth stages on a monitor. The method of claim 5, wherein the second step of determining the color grade of the fruit; Searching the start position of the fruit contour and chain coding to distinguish the fruit and the background and to remove the crust, reading the apple's RGB value and calculating the b ^* / a ^* slope and converting the calculated value to the color threshold On-line detection and quantification determination method of fruit appearance quality factor, characterized in that it comprises the step of determining the color grade of the fruit in comparison. The method of claim 5 or 6, wherein the color threshold value is set to 71 in the case of Fuji apple and is considered to be a colored pixel when the b ^* / a ^* slope is less than 71 degrees. On-line detection and quantification method of fruit appearance quality factor, characterized in that the coloration ratio is determined as 60% or more, 1st grade, 40-60% 2nd grade, and less than 40% 3rd grade. The method of claim 5, wherein the fourth step of detecting the defect of the fruit; A forty-first step of searching for a start position of the fruit and chain coding a near infrared (NIR) image; Step 42, which separates the fruit and the background from the upper upper surface where the fruit faucet is visible, removes the fruit faucet, and separates the fruit area from the background, and divides the fruit area from the background, and divides it into three parts to count the defect pixel of the fruit. ; An online detection and quantification method of fruit appearance quality factor, comprising a forty-third step of determining a fault of a fruit by comparing the counted fault pixel with a fault threshold. 9. The online appearance of fruit appearance quality factor according to claim 5 or 8, wherein the defect threshold value is set to 70, and if the number of defect pixels is 70 or more, it is determined to be abnormal. Detection and Quantification Determination Method.