NZ779697A - Method for recovering a posteriori information about the operation of a plant for automatic classification and sorting of fruit - Google Patents

Abstract

A method for recovering a posteriori information about the operation of a plant for automatic classification and sorting of fruit, wherein digital images of a reference piece of fruit previously processed by means of the plant are processed so as to identify its first group of individualizing features, which are then compared with each of a series of second groups of individualizing features each corresponding to a different piece of fruit processed by the plant, in order to establish a match; then, there is extraction from an electronic database of operating information which corresponds to information used or generated by the plant for automatic classification and sorting for classification of the relative processed piece of fruit and/or to information about sorting of the relative piece of fruit processed by the plant for automatic classification and sorting, where in the electronic database the operating information is associated with the second group of individualizing features for which during the comparison step the match with the first group of individualizing features has been established. es, which are then compared with each of a series of second groups of individualizing features each corresponding to a different piece of fruit processed by the plant, in order to establish a match; then, there is extraction from an electronic database of operating information which corresponds to information used or generated by the plant for automatic classification and sorting for classification of the relative processed piece of fruit and/or to information about sorting of the relative piece of fruit processed by the plant for automatic classification and sorting, where in the electronic database the operating information is associated with the second group of individualizing features for which during the comparison step the match with the first group of individualizing features has been established.

Claims (28)

1. A method for recovering a posteriori information about the ion of a plant for automatic classification and sorting which automatically divides pieces of fruit into a plurality of different qualitative classes, the method 5 comprising the following operating steps: an acquisition step wherein one or more digital images of a reference piece of fruit are acquired, where the reference piece of fruit is a piece of fruit previously processed by means of the plant for automatic classification and sorting; 10 a processing step wherein the one or more digital images are sed by an onic device, so as to identify a first group of individualizing features of the reference piece of fruit; a comparison step wherein an electronic device is used to compare the first group of individualizing es with each of a series of second groups 15 of individualizing features each corresponding to a ent piece of fruit processed by the plant for automatic classification and sorting, in order to establish a match between the individualizing features of the first group and the individualizing features of one of the second groups; an extraction step wherein an electronic device is used to extract from an 20 electronic database operating information which corresponds to ation used or generated by the plant for automatic classification and sorting for fication of the relative sed piece of fruit and/or to information about sorting of the relative piece of fruit processed by the plant for automatic classification and sorting, where in the electronic se the operating 25 information is associated with the second group of individualizing features for which during the comparison step the match with the first group of individualizing features has been established.

2. The method according to claim 1 wherein the individualizing features of each second group have been determined by a computer using electronic 30 data which have been usly generated by the plant for automatic classification and sorting following an analysis of the processed piece of fruit, and where the electronic data and/or the second groups of individualizing features are saved in the electronic se together with the operating information. 5

3. The method ing to claim 1 or 2 wherein the comparison step comprises first a selection step wherein the series of second groups of individualizing features to be compared with the first group of individualizing features is ed, from a plurality of second groups of individualizing features. 10

4. The method according to claim 3 n the selection step comprises an estimation step, wherein a classification date and/or a classification time for the reference piece of fruit when the reference piece of fruit has been classified and sorted by means of the plant for automatic classification and sorting are estimated, and wherein during the selection step second groups 15 of individualizing es relative to pieces of fruit processed by the classification plant respectively on that classification date and/or at that classification time are selected.

5. The method according to claim 4 also comprising a step of picking the reference piece of fruit from a plurality of pieces of fruit processed by the plant 20 for automatic classification and sorting, wherein the picking step is med alternatively by picking up the reference piece of fruit when it comes out of the plant for automatic classification and sorting, and/or by selecting the reference piece of fruit inside a container comprising a plurality of processed pieces of fruit whose fication date and classification time are known. 25

6. The method according to claim 3 wherein the selection step comprises a step of measuring one or more dimensions of the reference piece of fruit and/or of a colouring of the nce piece of fruit, and n during the selection step second groups of individualizing features which relate to processed pieces of fruit which have dimensions and/or average colour 30 similar to those measured are selected.

7. The method according to claim 3 wherein the selection step comprises a calculation step wherein, based on features of the reference piece of fruit deducible from the l images, a descriptor of the reference piece of fruit is calculated, and wherein during the selection step second groups of 5 dualizing features are ed which relate to processed pieces of fruit which have a corresponding descriptor which deviates from that of the reference piece of fruit by less than a predetermined value, where the descriptors associated with the processed pieces of fruit to which the second groups of individualizing features refer have been previously calculated by 10 the plant for automatic classification and sorting.

8. The method according to claim 7 wherein each descriptor is calculated using the result of a penultimate layer of a deep neural network trained to identify each piece of fruit of a set of pieces of fruit.

9. The method according to any one of claims 1 to 8 wherein the electronic 15 data relative to the features of each of the processed pieces of fruit, comprise a virtual model of the processed piece of fruit which comprises a threedimensional ric model with which images of the surface of the piece of fruit itself acquired in one or more bands of wavelengths are combined, and wherein the second individualizing features are determined by 20 processing virtual images of each processed piece of fruit which are generated starting with the relative virtual model.

10. The method according to any one of claims 1 to 9 wherein during the acquisition step one or more digital images of the reference piece of fruit are ed according to one or more predetermined directions of examination. 25

11. The method according to claim 10 wherein the predetermined directions of ation form and angle to the axis of the piece of fruit equal to 0°±10° or to 90°±10°.

12. The method according to claim 10 or 11 wherein the comparison step is performed by comparing the digital images with ponding l images 30 of processed pieces of fruit which were generated by the electronic device starting with a three-dimensional reconstruction of the texture of the processed piece of fruit, as if they had been acquired by a camera positioned according to a same direction of examination.

13. The method according to any one of claims 1 to 8 wherein the electronic 5 data relative to the features of each of the processed pieces of fruit, comprise a map projection of a virtual model of the processed piece of fruit, which comprises a three-dimensional ric model with which images of the surface of the piece of fruit itself acquired in one or more bands of wavelengths are combined, and wherein the second dualizing features 10 are determined by processing that map projection.

14. The method according to any one of claims 1 to 9 wherein during the acquisition step two or more l images of the reference piece of fruit are ed from different viewpoints, and wherein during the sing step the digital images are combined to create a dimensional image of the 15 reference piece of fruit.

15. The method according to claim 13 wherein during the processing step a map projection is created from the three-dimensional image of the reference piece of fruit and wherein the first group of individualizing features is identified in that map projection. 20

16. The method according to any one of claims 1 to 15 wherein during the comparison step two or more second groups of individualizing features are identified which may correspond to the first group of individualizing features, wherein during the extraction step the operating information associated with each of those two or more second groups of individualizing features is 25 extracted from the electronic database.

17. The method according to any one of claims 1 to 16 also comprising, before the acquisition step, and for each piece of fruit processed by the plant for automatic classification and sorting, a generation step, a classification step, a sorting step and a saving step, and wherein: 30 during the tion step, the plant for automatic classification and sorting acquires onic data about the processed piece of fruit, which can be used to identify the individualizing features of that processed piece of fruit; during the classification step, the plant for automatic classification and sorting uses the electronic data to classify the processed piece of fruit; 5 during the sorting step, the plant for automatic classification and sorting sorts the processed piece of fruit based on the classification performed during the classification step; and during the saving step the plant for automatic fication and sorting saves in the electronic database the operating information relative to the 10 processed piece of fruit.

18. The method ing to any one of claims 1 to 17 wherein the acquisition step is performed by means of a mobile device or in a fixed scanning station.

19. The method according to any one of claims 1 to 18 wherein the 15 acquisition step is performed by positioning the reference piece of fruit on a known background and acquiring each digital image in such a way as to include both the nce piece of fruit and part of the known background.

20. The method according to any one of claims 1 to 19 also comprising, after the extraction step, a making available step, wherein, by means of an 20 electronic device, the operating information extracted from the electronic se is made available to an operator.

21. The method according to claim 20 when it is ent on claim 16, wherein during the making ble step all of the operating information extracted from the electronic database which is associated with each of the 25 two or more second groups of individualizing features is made available to an operator.

22. The method according to any one of claims 1 to 21 wherein the plant for automatic classification and sorting comprises: a conveying device which defines a movement path for the sed 30 pieces of fruit which extends n an infeed zone and a plurality of outfeed zones, where each outfeed zone is associated with a different classification of the processed pieces of fruit; one or more analysis stations for analysing the features of the sed pieces of fruit, which are mounted along the movement path upstream of the 5 outfeed zones, where each analysis station generates onic data about the features of each processed piece of fruit and where the features of the processed pieces of fruit comprise information about the three-dimensional shape of the processed pieces of fruit and/or information about the appearance of the surface of the processed pieces of fruit in one or more 10 bands of wavelengths and/or information about the internal part of the sed pieces of fruit; a computer operatively connected to one or more analysis station so as to receive from them the electronic data about the features of the processed pieces of fruit, and to the conveying device so as to control its operation; 15 a tracking system for tracking the on of the processed pieces of fruit along the nt path, active at least between the one or more analysis stations and the outfeed zones, the tracking system being operatively connected to the computer for communicating to the computer the position of each processed piece of fruit; and 20 an electronic database connected to the computer, to which, in use, the er saves the operating information relative to each processed piece of fruit; and wherein the er is programmed to: assign a qualitative class to each processed piece of fruit based on the 25 electronic data received from the one or more analysis stations about each processed piece of fruit; monitor the position of each sed piece of fruit along the movement path; issue commands to the conveying device for feeding each processed 30 piece of fruit to a specific d zone associated with the qualitative class assigned to the processed piece of fruit; perform the processing, comparison and extraction steps.

23. A plant for the classification and the sorting of pieces of fruit into a plurality of different qualitative classes, comprising at least one er 5 programmed to implement the processing step, the comparison step and the extraction step of the method according to any one of claims 1 to 21.

24. Plant for the classification and the sorting of pieces of fruit according to claim 23 when it depends on claim 17, wherein the computer is programmed to further implement the classification step, the sorting step and the saving 10 step.

25. A process for inspecting operating anomalies of a plant for automatic classification and sorting, which automatically divides pieces of fruit into a plurality of ent qualitative s, the process comprising execution of a method for ring a posteriori information about the operation of the 15 plant for tic classification and sorting according to any one of claims 1 to 22, adopting as the reference piece of fruit a piece of fruit which has been incorrectly fied and/or sorted, n during the extraction step there is extraction from the database of virtual information about, respectively, an expected classification and/or sorting assigned to the reference piece of fruit, 20 the process for inspecting anomalies also comprising: a step of obtaining real information which comprises real information about a reference piece of fruit fication and/or sorting error, the step of obtaining real information being performed by means of an electronic device; a verification step during which the virtual information is compared with 25 the real information, med by means of an electronic device.

26. The process according to claim 25, wherein, a result of the verification step is communicated to an or and/or saved.

27. The process according to claim 25, wherein, a result of the verification step is used to correct the operation of the plant. 30

28. The process ing to claim 27 when it is dependent on claim 22, wherein the computer is programmed to assign the ative class to each processed piece of fruit by means of artificial vision algorithms, in particular neural networks, and wherein, when the verification step has identified a classification error, the information used by the plant for automatic 5 classification and sorting for classification of the relative processed piece of fruit which was extracted during the extraction step is used together with information about a correct classification of the nce piece of fruit, for a further ng step for the artificial vision algorithms.