EP4133405A1

EP4133405A1 - Method of sorting angiosperms of forest trees and equipment for making the same

Info

Publication number: EP4133405A1
Application number: EP21727780.5A
Authority: EP
Inventors: Milan Adámek; Petr Chalupa; Jakub Novák; Vladimír VASEK; Václav Lecián; Josef Samek; Martin ROZMANEK; Ale BÁRTA
Original assignee: Lescus Cetkovice SRO; Tomas Bata University in Zlín
Current assignee: Lescus Cetkovice SRO; Tomas Bata University in Zlín
Priority date: 2020-04-06
Filing date: 2021-04-01
Publication date: 2023-02-15
Also published as: WO2021204308A1; CZ308955B6; CZ2020197A3

Abstract

The sorting method consists in first acquiring two of the evaluated seedling images - an image of the whole seedlings and a detail of its root collar - by each of the pair of camera systems oriented at an angle of 60° to 120°. Images of the entire seedlings are converted into binary images, from which the height of the seedlings is directly evaluated, and these are also converted into a vector model using Mathematical Morphology with binary images and skeletonization to evaluate the construction of planting material. Root collar images are converted into binary imagery using thresholding, and root collar diameters are determined by measuring the number of pixels in the area above the soil. In this way, each seedling is individually evaluated by two views of the whole seedling and two views of the root collar and, based on the parameters found, classified into the appropriate class. Furthermore, the invention involves devices to carry out this method.

Description

METHOD OF SORTING ANGIOSPERMS OF FOREST TREES AND EQUIPMENT FOR MAKING THE SAME

Field of Technology

This invention relates to a containerized forest tree seedling sorting method based on the acquisition and processing an image from a camera system. The invention further relates to a device for performing this method.

Current State of Technology

In the last few years, new progressive forest seedlings in container/planter methods have begun to be used. The main advantages of containerized technology include higher growth from forest tree seeds, shortened cultivation time from sowing to planting in forest nurseries; but mainly - the almost lossless rooting of containerised forest seedlings. In traditional containerized seedlings sorting, individual seedlings are measured manually and evaluated. The appearance and dimensions of each seedling are visually inspected by nursery staff.

The speed and accuracy of this manual sorting process do not meet modern forest nursery requirements. Planting material requirements are clearly defined in the CSN 48 2115 Standard „Forest reproductive material . The basic parameters that affect the future growth and rooting of seedlings include the characteristics of the above-ground part: (seedling height, root collar diameter, morphological structure), and the characteristics of the underground part: (root system quality). Based on these criteria, seedlings can be classified into classes - e.g. A, B and C. Classes A and B differ only in the requirements for minimum seedling size and root collar diameter; where Class C represents seedlings with defects: (small seedlings, deformed seedlings, multiple seedlings in a container cell).

In technical practice, there is a general trend in the Industrial Technology field to replace manual labour with mechanisation - mainly due to savings and the declining number of employees. Mechanisation using machine vision in sorting processes is also increasing.

From Czech Republic Patent No. 308015, for example, an autonomous device for image recognition is known, comprised of a housing in which a camera block with an image sensor and a process block with at least one processing unit is housed.

Both blocks are separated from each other by a thermal insulation partition, the camera block and process block are interconnected by an electrically powered connecting element. This device will find its place in Industrial Automation applications - especially on production or control lines requiring Continuous Operation with Continuous Industrial Process Quality Control, mainly consisting of the product recognition - like parts, shapes, colours or defects. However, this solution was not applied for the specific containerized forest tree seedlings classification issue.

The most likely reason is the fact that an efficient and sufficiently high-quality automated containerized seedling sorting method; respectively the concept of the device for its implementation, requires a specific scanning and image processing method which is not known in the above solution - but neither in other existing solutions.

This fact is not changed by existing approaches to seedling sorting - (e.g., Patent JP20010248333); based on taking pictures of seedlings from only one point-of-view - (ground plan or side-on), seedlings are then sorted by height or visible leaf area. There is also a complete lack of the acquired camera view evaluation - for example, the morphological structure of the seedlings, for example with the aid of the created vector model.

The Essence of the Invention

The method of sorting containerized seedlings according to the invention - similarly as known Machine Vision methods based on the acquisition and processing of an image from a camera system - contributes to the solution of the above-mentioned problems. The essence of the invention lies in a specific method - in which two images of the evaluated seedling are taken by each of the paired angle-oriented camera systems - an image of the whole seedling - and a detail of its root collar. The camera systems angle-of-view means the angle between spatial plan projections of the optical axes of the cameras used to capture views. The optimal viewing-angle is 80 ° to 100 ° - (preferably 90 °); satisfactory results are achieved for viewing angles of 60 ° to 120 °.

The images of the whole seedling are converted to binary images which are used to directly evaluate height of the seedling. These images are also used to obtain vector model using mathematical morphology and skeletonization. Each vector of the vector model represents a branch segment of the seedling and similarly to genealogical trees linkages to the other segments are defined in the form of parents and children. The structure of the seedling is evaluated by identifying faults in the vector tree structure. For root collar images, binary image conversion is performed by thresholding; Morphological Operations are used to clean the binary image from other objects by removing smaller objects while optimising the structuring element and maintaining the dimensions of large objects. Measuring the number of pixels in the above ground area is used to determine root collar diameter.

Thus, each seedling is individually evaluated by two points -of- views of the whole seedling and two more of root collar. Based on the evaluation of the images of the whole seedling, errors in the morphological structure are detected, i.e., two competing terminal shoots, and the height of the seedling is also determined from the images of the whole plant.

Based on the images of the root collar, the two seedling root collar diameters are determined, and the information is obtained whether there is more than one seedling in the container cell - or, conversely, the container cell is empty.

Subsequently - the seedling is classified into its appropriate class based on these identified parameters.

The way to implement the equipment method according to the invention is comprised of a system built on two camera systems, each of which consists of one camera with a wide- angle lens with a short focal length for imaging the whole seedling and a second camera with a longer focal length objective for root collar images. These camera systems are housed in a chamber with front and back lighting - based on FED panels to provide contrast between the seedling and the background.

A link conveyor with seedling carriers passes through the chamber. In the conveyor movement direction is a workstation for filling carriers with seedlings from containers, located at the front of the chamber. Behind the chamber, the line conveyor has two switch systems for sorting seedlings into individual quality classes A, B, C. The controls of these switch systems are connected to a control system with an implemented sorting algorithm.

Explanation of Figures

The accompanying drawings serve to illustrate and clarify the essence of the invention:

Fig. 1 - Seedling Parameter Evaluation Camera System Schematic

Fig. 2 - Ground plan of a seedling sorting line

Fig. 3 - Seedling Input Image and Vector Model

Fig. 4 - Root Collar Diameter Evaluation Procedure

Fig. 5 - Detection of Multiple Seedlings in Container Cells

Fig. 6 - Seedling Evaluation Algorithm Example

During the sorting process, two images of the evaluated seedling are first acquired in each of a pair of camera systems la, lb mutually perpendicularly oriented on the exemplary embodiment - an image of the whole seedling and a detail of its root collar.

The camera image is first cropped to the area of interest for images of the whole seedling, and subsequently converted into a binary image, from which seedling height is directly evaluated. Binary image is converted into a vector model using Mathematical Morphology with binary images and Skeletonization. (See Fig. 3).

Each vector here represents a segment of a seedling branch, and similarly to a genealogical tree, a link to subsequent segments in the form of parents and children is defined. Seedling structure evaluation is then performed on the basis of a vector model of the seedling - (Fig. 3), because it is possible to go through this structure very efficiently and look for defects in the seedling structure, such as - for instance, two terminal shoots.

For root-collar image thresholding using morphological operations, the image is cleaned of other objects. By means of a suitable choice of the structuring element, it is possible to remove smaller objects while maintaining the dimensions of large objects. The diameter of the root collar is then determined by measuring the number of pixels in the area above the ground in the carrier (Fig. 4).

Although special automatic lines can be used for sowing, more seeds can germinate in the container cell, or it can also germinate from seeds from the surrounding trees. Cells with two seedlings must also be detected and classified as Class C (Fig. 5).

Each seedling is individually evaluated from two perpendicular views of the seedling root collar and two perpendicular views of the whole seedling. The results of the evaluation of the root collar images are 2 differing diameters of the root collar of the seedling, and eventually - information on whether there is more than one seedling in the container cell or conversely, that that cell is empty. Based on the evaluation of images of the whole seedling, errors in the morphological structure can be detected, i.e., two competing shoots/sprouts. The height of the seedling is also determined from the image of the whole plant.

The combination of information from two perspectives of the whole seedling even makes it possible to correctly process branches that would be in alignment from one view, and to correctly evaluate the possible inclination of the seedling. If all images meet the required parameters for the planting material then information about resulting seedling class is transmitted to the higher-level control system which places the seedling to the correct line by means of switch- systems (3a, 3b). Fig. 6. Depicts the Evaluation Algorithm for one seedling The equipment for performing the containerized forest tree seedling sorting method is comprised with a system of two camera systems la, lb - (see Fig. 1), mutually oriented in an exemplary embodiment in the perpendicular direction. Each consists of one camera with a wide- angle lens with a short focal length for the image of the whole seedling 10 and a second camera with a lens with a longer focal length for root collar images.

These camera systems la, lb are housed in a chamber 2 (Fig. 1, Fig.2) with front 2a and back 2b lighting based on LED panels to provide contrast between the seedling 10 and the background. Link conveyor 3 passes through Chamber 2, and in a specific example chain conveyor with carriers 4 of seedlings 10. In the link conveyor 3 movement direction 11 is Station 5 for filling carriers 4 with seedlings 10 located in front of Chamber 2, and behind this chamber conveyor link 3 has two switching systems - 3a, 3b for sorting to pick-up places A, B, C of individual quality classes. The control elements of these switches are connected to the control system with the implemented algorithm (see Fig. 6) of the sorting method.

Industrial Applications

This invention is based on the application of industrial automation - especially in forest nurseries. It replaces many unskilled seasonal manual workers with an automated system, which is handled by an order of magnitude fewer but highly qualified workers. Compared to manual sorting, the advantage is also to be found in the reproducibility of the results, outputs independent of worker fatigue and the possibility of the easy archiving of results and feedback control.

Claims

P A T E N T C L A I M S

1. A method for sorting containerized forest tree seedlings based on the acquisition and processing of camera system images; the two camera systems are mutually oriented so that the angle between the planned angles of the cameras optical axis is 60° to 120°; better, 80° to 100°; each camera system is used to acquire two images of the evaluated seedlings - the image of the whole seedling and the detail of its root collar, after that: a) images of the whole seedling are converted into binary images; from which, on the one hand, the seedlings height is directly evaluated and, at the same time, using Mathematical Morphology with Binary Images and Skeletonization, is converted into a vector model where each vector represents a segment of the seedling branch and, as with any genealogical tree, the link to the connected segments - in the form of parents and children is defined. This vector tree if used to identify disorders in the structure of the seedling; b) in the case of root collar images, conversion to a binary image is carried out by means of thresholding, morphological operations clean the binary image of other objects by removing smaller objects by means of the optimal selection of the structure element while maintaining large object dimensions; and by measuring the number of pixels in the area above the soil the diameter of the root collar is determined; hereby, each seedling is classified by two individually evaluated views of the whole seedling and two views of the root collar; the angle of these two views; given the angle between the planned projections of the cameras optical axis with which the views are captured, is 60° to 120° - preferably 80° to 100°; based on the evaluation of the images of the entire seedlings, errors in the morphological structure, i.e. two competing terminal shoots are detected, the height of the seedlings is determined from the images of the whole seedling, and - based on the evaluation of the root collar images, the two seedlings root collar diameters are determined and information is obtained as to whether there is more than one seedling in the container cell - or, on the contrary, that the cell is empty; after which it is subsequently classified in the appropriate class based on these seedling detection parameters.

2. The device for carrying out the method according to claim 1 ; is characterised by the fact that it contains a set of two camera systems - (la, lb); each consisting of one camera with a short focal length wide-angle lens for whole seedlings (10), and a second camera with a longer focal length lens for root collar images; that these camera systems - (la, lb) are located in a chamber (2), with front (2a) and rear (2b) LED panel-based lighting to ensure contrast between the seedlings (10) and background, where a liner conveyor (3) with carriers (4) of seedlings (10) passes through the chamber in the direction (11) of the link conveyor movement (3), is a station - (5) in the front of the chamber (5) that fill the carriers (4) with seedlings (10); behind the chamber, the liner conveyor (3) has two switching systems - (3a, 3b) in place to sort the seedlings into (A, B, C) of each quality class pick-up places, in which the controls of these switches are connected to the control system with the algorithm (6) of the implemented sorting method.