WO2023085992A1

WO2023085992A1 - Image analysis of cut flowers

Info

Publication number: WO2023085992A1
Application number: PCT/SE2022/050968
Authority: WO
Inventors: Eda DEMIR WESTMAN; John Schmidt; Katarzyna Maria DYMEK KRAKOWIAK; Hanshenric Carenborn
Original assignee: Opticept Technologies Ab
Priority date: 2021-11-15
Filing date: 2022-10-24
Publication date: 2023-05-19

Abstract

The present invention describes method for analyzing one or more cut flowers with reference to their life length, said method comprising: - taking images of said one or more cut flowers; and - analyzing the images with a software tool, involving labelling said one or more cut flowers into at least two quality categories.

Description

IMAGE ANALYSIS OF CUT FLOWERS

Field of the invention

The present invention relates to an image analysis of cut flowers. Summary of the invention

The present invention is directed to a method for analyzing one or more cut flowers with reference to their life length, said method comprising:

- taking images of said one or more cut flowers; and

- analyzing the images with a software tool, involving labelling said one or more cut flowers into at least two quality categories.

The present is directed to image analysis of cut flowers to enable labelling of the cut flowers into at least two categories. In the method according to the present invention, a software tool is involved for the labelling. The software tool suitably includes an algorithm. The software tool may involve different forms of computer units and may be linked to different computer programs and the like used to perform the method according to the present invention.

It should be noted that there are known methods for analyzing flowers, also where some form of image analysis is involved. The present invention, however, provides an improved method where the image analysis involves labelling of the cut flowers into at least two categories. This is of great relevance to enable to predict the life length of a certain cut flower. This direction is not intended, provided or hinted in the known methods used today.

Specific embodiments of the invention

Below some specific embodiments of the present invention are disclosed and discussed further.

As mentioned above, at least two quality categories are used in the step of image analysis. It should be noted that several categories may be used, such as for instance good, bad and fungus, which is further discussed below. The categories good and bad may e.g. also be “alive” and “dead”. Furthermore, the step of taking images involves taking images at certain set time points. This time point intervals may vary according to the present invention.

According to one embodiment of the present invention, the software tool operates based on at least flower opening level. This parameter may be seen as a key parameter for several embodiments according to the present invention. The flower opening level is very much linked to the life cycle status of a flower, and thus is very relevant to use according to the present invention for the labelling.

Furthermore, according to yet another embodiment, the software tool operates based on at least size of the flower opening or center position of the flower opening, preferably based both of size of the flower opening and center position of the flower opening, more preferably the software tool operates based on evaluating a change in least size of the flower opening and/or change in center position of the flower opening. Also the center position is a very relevant parameter to involve when performing the method according to the present invention. The center position of the flower opening of a certain cut flower moves over time when the cut flower changes its position, such as when the stem bends when the condition/quality of the cut flower decreases. Moreover, to combine flower opening and center position of flower opening may be relevant according to the present invention to ensure an increased efficiency in the labelling, at least for certain embodiments. Moreover, the method according to the present invention may involve that the software tool operates based on a change in least size of the flower opening and/or change in center position of the flower opening. This change detection may be a way for labelling when a certain limit is set for the parameters, e.g. a limit percentage change.

The software tool according to the present invention may perform the method in different ways, both in relation to estimate the size of the flower opening and the center position of the flower opening. As an example, the method may involve using a certain geometry, such as a triangular or quadratic geometry, and using that to determine the center position. When the pixels have moved in that geometry, this information also triggers that the method can detect a change of the center position. Furthermore, such geometrical direction may also be used for deciding a change in the size of the flower opening.

Furthermore, according to yet another embodiment, the software tool operates based on at least flower opening level and the flower opening level is measured by image analysis of images taken continuously, wherein flower opening level is plotted towards time to provide for a relationship or curve of flower opening level vs time, preferably wherein flower opening level is plotted towards time to provide for a curve of flower opening level vs time and wherein a slope of that curve is calculated at certain given times. According to one embodiment, when the slope reaches 0, which may be seen as an interference point, then the method may provide an output of an expectation that the cut flower is about to die. Moreover, certain slope limits may be used for further labelling of the quality of the cut flower being analyzed. Furthermore, the method according to the present invention may also provide data on the actual life length of the cut flower being assessed.

Also other parameters may be involved according to the present invention. For example, according to one embodiment, the software tool operates based on evaluating color, preferably change in color. This may also be used for quality assessment.

Moreover, also machine learning may be used according to the present invention. In line with this, according to one embodiment of the present invention, the method involves machine learning for labelling, preferably for discovering if one or more cut flowers fall into said at least two quality categories based on labelled datasets and dataset training. Machine learning may be used to increase knowledge for certain cut flower types to increase the data in a database. This implies that that data may be used to provide better limit values for different categories of a certain cut flower type, such as e.g. “prima quality”, “good quality”, “OK quality”, “bad quality” and/or “dead”. Therefore, such machine learning may be used during a training phase for a certain cut flower type according to the present invention.

According to the present invention multiple algorithms may be used. According to one embodiment of the present invention, the step of analyzing is performed with multiple algorithms of which at least one is based on classification and at least one is based on detection. According to one specific embodiment, the output for the detection algorithm is a rectangle over the flower’s openings, then used for calculating the area corresponding to a standard opening.

According to another embodiment of the present invention, the algorithm operates based on at least one of the evaluation parameters falling petal level, bending level, fungus amount and standard flower opening level.

The method according to the present invention is suitably based on looking at opening of a flower under the camera first and then follow its life from small to large openings, bending and falling petals. During the time the flower is straight, fungus might appear, and this may then be categorized as such according to the present invention.

According to yet another embodiment of the present invention, the method comprises saving data on said one or more evaluation parameters in relation to said at least two quality categories.

The present invention may also involve a step to train a neural network to “learn” what a good or bad flower implies. This may be performed by using different types of existing software, e.g. a common process within Google Cloud Platform (GCP).

Three sets of annotated pictures may be set and split into: training, test and validation. High accuracy, precision and recall are required for getting a fair answer of a good/bad candidate, and the result of the training will be a set of weights used for a certain application. During a learning phase different flowers with different parameters may be placed under a camera’s field of view.

According to one implementation of the training set-up, the following may be performed. A set of minimum 500 flowers from categories: good, bad and fungus may be needed. For standard “opening” labels the way of getting the training, test and validation set be based on a time-lapse. In the setup, one may place a number of good flowers with smallest opening under one camera for two weeks. Pictures may be taken every 30 minutes. A software algorithm may cut pictures for each cut flower. This way it is possible to get about 500 pictures with different species of cut flowers and different openings. For the intention of getting a larger dataset, it is possible to use augmentation according to the present invention. Moreover, for the same flower dataset, different picture manipulation artifacts can be used, such as rotation, flip, blur etc.

The present invention may involve comparing a certain cut flower and saved data from an implementation such as in accordance with the one above. Based on this, according to one embodiment, the method also comprises calculating the life length of said one or more cut flowers by comparing one or more evaluation parameters with saved data on said one or more evaluation parameters. For the same flower dataset, the present invention may also involve using detection algorithms, where the detection may be provided as e.g. a rectangle surrounding the flower’s opening. The algorithm may e.g. detect roses from a certain picture and may then add a rectangle over each flower. Given the coordinates of the rectangle, the area may be calculated. This can then be translated into a value of the flower opening and be normalized to picture area.

According to yet another embodiment of the present invention, and in line with the above, then the step of taking images are performed with a camera set-up with at least one camera, where the following steps are performed:

- placing said one or more cut flowers in a field of view of said at least one camera;

- taking images of said one or more cut flowers and saving the images to a database or in the cloud;

- labelling all images into said at least two quality categories, preferably into two categories being “good” or “bad” quality.

In relation to the above, also the category “fungus” may be handled as “bad” according to the present invention.

According to one preferred embodiment, the camera set-up comprises at least one side camera and one top camera, and wherein the method comprises labelling images into said at least two quality categories, preferably into two categories being “good” or “bad” quality, individually from said at least one side camera and one top camera. As hinted above, “bad” quality may also imply “dead” or be complemented with such a category.

According to one embodiment of the present invention, the step of taking images are performed with a camera set-up with at least one top camera being placed above said one or more cut flowers. This direction is a foundation for the system set-up according to the present invention.

Moreover, to use both a top camera and a camera from the side may be of interest for detection of all of fungus, openings, falling petals, and bending of a cut flower.

Furthermore, based on the connection to the software and the analysis of the present invention, the camera(s) should be connected to the Internet via Wi-Fi or PoE (power over ethernet).

The image analysis method according to the present invention may also involve both labelling and scoring. According to one embodiment of the present invention, the top camera is used for labelling of good or bad quality and standard opening in a scale of 1 - 5. According to another embodiment, the side camera is used for labelling of good or bad quality. It should be noted that using a side camera is optional according to the present invention. Moreover, according to yet another embodiment, the labelling is also performed into a category group called fungus, such as hinted above. Example

Below there is provided a simple overview implementation example according to the present invention.

1 . Set camera’s field of view

2. Training phase (with reference to different quality status)

3. Image and data collection

4. Labelling (again focused on different quality status)

5. Train neural network. This may be performed by training two neural networks, one for classification (“good”, “bad”) and one for detection, e.g. by use of a single short detector that will output a rectangle over the flower’s opening as mentioned above).

6. Performance assessment (accuracy, precision, recall) In figs. 1 -6 there are shown different types of typical graphs for the measurement of size of flower opening against time according to the present invention. First, in fig. 1 there is provided a normal graph behavior according to the present invention for a cut flower, such as e.g. a rose. Moreover, in fig. 2 there is shown different points calculated along the typical graph. Below the figures, there is provided an equation which is an example how to calculate the slope a_x of the graph. As seen, at the inflection point, then the curve has a slope = 0. Afterwards, the slope is negative, which implies that the cut flower is on its way to die. In figs. 3 and 4 there are shown slope vs time and how to calculate a different in the angle between different slopes. This also provides yet further information on the change of the slope over time, which can be used as an indicator of the quality of the cut flower in different categories. In fig 5 there is provided a real example where slopes a_x are calculated.

Moreover, in fig. 6 there is provided real data from examples where a certain rose was tested by use of the method according to the present invention. A number of roses were placed in an organized way in a bouquet. Images were taken at certain time points. The positions of the different roses were noted, and curves were established for each rose to evaluate when a specific rose went from “good” to “bad” quality. The slopes for each rose curve were calculated to provide for the analysis. As can be seen, the average for all roses (curves) indicated that the roses went from “alive” to “dead” after an average of 277 tics, which corresponded to 3 days in average.

Claims

8 Claims

1 . A method for analyzing one or more cut flowers with reference to their life length, said method comprising:

- taking images of said one or more cut flowers; and

2. The method according to claim 1 , wherein the software tool operates based on at least one of the evaluation parameters falling petal level, bending level, fungus amount and standard flower opening level.

3. The method according to claim 1 or 2, wherein the software tool operates based on at least flower opening level.

4. The method according to claim 3, wherein the software tool operates based on at least size of the flower opening or center position of the flower opening, preferably based both of size of the flower opening and center position of the flower opening, more preferably the software tool operates based on evaluating a change in least size of the flower opening and/or change in center position of the flower opening.

5. The method according to any of claims 1-4, wherein the software tool operates based on at least flower opening level and the flower opening level is measured by image analysis of images taken continuously, wherein flower opening level is plotted towards time to provide for a relationship or curve of flower opening level vs time, preferably wherein flower opening level is plotted towards time to provide for a curve of flower opening level vs time and wherein a slope of that curve is calculated at certain given times.

6. The method according to any of the preceding claims, wherein the software tool operates based on evaluating color, preferably change in color. 9

7. The method according to any of the preceding claims, wherein the method involves machine learning for labelling, preferably for discovering if one or more cut flowers fall into said at least two quality categories based on labelled datasets and dataset training.

8. The method according to any of claims 1-7, wherein the step of analyzing is performed with multiple algorithms of which at least one is based on classification and at least one is based on detection.

9. The method according to claim 8, wherein the method comprises saving data on said one or more evaluation parameters in relation to said at least two quality categories.

10. The method according to claim 9, wherein the method also comprises calculating the life length of said one or more cut flowers by comparing one or more evaluation parameters with saved data on said one or more evaluation parameters.

11 . The method according to any of the preceding claims, wherein the step of taking images are performed with a camera set-up with at least one camera, where the following steps are performed:

12. The method according to any of the preceding claims, wherein the step of taking images are performed with a camera set-up with at least one top camera being placed above said one or more cut flowers.

13. The method according to claim 11 or 12, wherein the camera set-up comprises at least one side camera and one top camera, and wherein the 10 method comprises labelling images into said at least two quality categories, preferably into two categories being “good” or “bad” quality, individually from said at least one side camera and one top camera.

14. The method according to claim 12 or 13, wherein the top camera is used for labelling of good or bad quality and standard opening in a scale of 1 - 5.

15. The method according to claim 13, wherein the side camera is used for labelling of good or bad quality.

16. The method according to any of claims 11 -15, wherein the labelling is also performed into a category group called fungus.