EP3912737B1

EP3912737B1 - System and method for sorting packaged products

Info

Publication number: EP3912737B1
Application number: EP20382436.2A
Authority: EP
Inventors: Emilio DE LA RED BELLVIS; Iván CALLEN BARCELONA; Daniel ARTAL GONZÁLEZ
Original assignee: Inndeo Proyectos Industriales SL
Current assignee: Inndeo Proyectos Industriales SL
Priority date: 2020-05-22
Filing date: 2020-05-22
Publication date: 2023-12-27
Anticipated expiration: 2040-05-22
Also published as: EP3912737A1; EP3912737C0

Description

FIELD OF THE INVENTION

The present invention is useful in the field of manufacturing of packaged products and it belongs, more particularly, to the technical field of sorting packaging, preferably although limited to, food and pharmaceutical product packages.
The main object of the present invention is, therefore, sorting packaged products after the inspection thereof with machine vision means, preferably heat-sealed packages. Said sorting can be useful in, for example and limited to, quality control tasks.
The sorting system and method according to the present invention are able to analyse images of the packaged products to be sorted, both in the visible light spectrum and in the infrared light spectrum, and base said sorting on the inspection of the seals provided in said packages, and detecting any type of surface contamination on the product contained in the package, or printing defect on the films of the packages.

BACKGROUND OF THE INVENTION AND TECHNICAL PROBLEM TO BE SOLVED

In the field of manufacturing packaged products, for example heat-sealed packaged products, the need to control the quality of the packages arises. To do so, it is necessary to check the integrity of the seals and detect the presence of possible defects such as the presence of bubbles or the presence of foreign elements or bodies which prevent the airtightness of the sealing of the films, enabling the entry of air, which produces a premature expiration of the product and represents a serious quality failure.
For this purpose, it is known, for example, from patent applications WO 2010/052431 and WO 2015/052445 , how to measure the diffusion of the light in the heat-sealing area by using a linear light beam. Said patent applications describe methods which enable the presence of intrusions in the seal area to be detected which leads to an alteration, on the one hand, of the quality of the seal from the point of view of airtightness and, on the other hand, of the aesthetic appearance related to the product and the package.
Patent application WO 2013/007951 describes a method for controlling, in particular, the structures of heat-sealed packages with the aid of a linear image sensor, and US patent US5515159 describes a vision system for inspecting package seals.
Patents EP 2350621 and EP 3055681 also describe a method and device for inspecting package seals by optical control by means of digital photography.
In practice, the prior art devices and methods described above are satisfactory for inspecting packaged products (including the seals of said packages) and sorting them based on the results of said inspection provided that said packaged products have homogeneous optical features. However, the prior art devices and methods cease to be effective in cases wherein the seal area is invaded with transparent products, such as the interleaver (or separator) used in slicing, or melted fat, which is colourless. Furthermore, the techniques for inspecting, detecting defects and sorting that are already known use methods for comparing images within the visible light spectrum, which are ineffective on packages with printed films, where defects are commonly confused with printing inks. In other words, the techniques are insufficient to guarantee the quality of the seals and are ineffective if films with different printing inks are used.
On the other hand, hyperspectral machine vision equipment is known in the state of the art, which is mainly used in laboratories, satellites and drones, there being different manufacturers of the same. Such equipment is capable of obtaining the so-called hyperspectral images, i.e., images containing information belonging to several different spectral bands usually comprised between 400 and 2,000 nanometres in frequency and which belong to both the visible region (from 400 to 750 nm) and the infrared region of the electromagnetic spectrum (wavelength greater than 750 nm). Said hyperspectral images are usually processed by means of spectrography and can be used to determine the chemical composition by comparing the spectral signature.
Thus, for example, in patent application EP1055115 , a method is disclosed for determining the spectral signature of the different pixels of hyperspectral images, or in other words, a method for analysing the intensity of the light from each pixel of hyperspectral images in each of the different predefined bands of the infrared light spectrum had by said hyperspectral images.
United States Patents 5512752 , 4719351 and 5206510 , German Patent Publications DE 19601923 , 19543134 and 4340795 , and Japanese Patent Applications JP-A-9138194 , 6288913 and 6210632 disclose several examples of hyperspectral machine vision equipment.
To capture hyperspectral images, it is necessary to obtain sufficient light intensity in said spectral bands which include both the visible and infrared light spectra. The only light sources in the state of the art capable of irradiating a light spectrum that is uniform (linear) and with an intensity which enables the images to be captured at high speed are halogen light sources.
However, said halogen light sources dissipate a high thermal power, such that if the packaged products are exposed to said light sources, the heat generated could degrade them. This fact makes it difficult for halogen light sources to be encapsulated within a compact machine and notably complicates, furthermore, the fact that the hyperspectral machine vision devices can be used in the field of inspecting and sorting packaged products.

DESCRIPTION OF THE INVENTION

An object of the present invention is to address all the difficulties and drawbacks of the prior art indicated above, in addition to enabling packaged products to be sorted depending on different quality parameters related to the surface chemical composition thereof and detecting surface contamination thereon.
More particularly, a first object of the invention relates to a system for sorting packaged products, said system being provided with:

at least one conveyor belt configured to transport the packaged products from an entry area to an exit area;
at least one linear image capture means, configured to capture images of the light reflected or transmitted in the visible spectrum by the packaged products along a predefined image capture line;
at least one linear illumination means configured to emit light in the visible light spectrum and illuminate the image capture line;

at least one linear hyperspectral image capture means, configured to capture hyperspectral images of the light reflected or transmitted in different bands of the infrared spectrum by the packaged products along the predefined image capture line;
at least one linear hyperspectral illumination means configured to emit light in the infrared light spectrum and only illuminate the image capture line;
first image processing means, configured to analyse the intensity of the light from each pixel of hyperspectral images in each of the different predefined bands of the infrared light spectrum (which is also known as determining the spectral signature of each pixel), in order to compare said spectral signature to a reference pattern and colour the pixels of hyperspectral images in a pre-established tone depending on the result of the comparison of the spectral signature thereof with the reference pattern, giving rise to a coloured hyperspectral image (also known as a chemical image);
second image processing means, configured to analyse at least one predetermined control area of each image acquired from the linear image capture means and of each coloured hyperspectral image obtained from the first image processing means; to compare at least one control area of each of said images to at least one reference pattern; to sort said images depending on said comparison with the reference pattern and to send sorting instructions to actuation means; and
actuation means, configured to receive the sorting instructions and actuate sorting means for the packaged products based on said sorting instructions, the sorting means being arranged in the exit area.

The packaged products sorted by the present invention are preferably provided with a package comprising at least two heat-sealed films (i.e., they are sealed together by means of heat). The two films of this type of package are preferably provided with a base made of plastic film, which comprises at least one cell wherein the products to be packaged are deposited and whereon a second film is adhered, sealed by means of heat melting with the aim of guaranteeing the airtightness of the cell. When the products are medicines, said films are usually made of laminated aluminium, or a hybrid of aluminium and plastic.
The linear hyperspectral image capture means contemplated by the present invention enable the light reflected and transmitted by the products packaged in the packages within the infrared spectrum to be automatically captured, breaking it down into different frequency bands (preferably from 1 to 250 frequency bands). This enables a set of images to be obtained for each of the frequency bands, known as a hyperspectral image, the number of image layers of which coincides with the number of frequency bands which are analysed.
Hyperspectral images are processed by the first image processing means which analyse the distribution of the intensity of the light in each of the frequency bands pixel by pixel, which is known as the spectral signature of the pixel, assigning a colour tone to it, after comparing to other spectral signatures (or reference patterns) and applying different rules for identifying pre-established patterns of similar signatures. By assigning different tones to different spectral signature ranges, the hyperspectral images of the packaged products are converted into the so-called chemical images. This technique enables chemical compounds with apparently identical colours in visible light to be distinguished, in addition to enabling the contents of the packages to be viewed through the films, even if they are printed with opaque inks, since in the infrared light spectrum the vast majority of the inks opaque to light are transparent.
Likewise, said chemical images enable, in turn, the surface chemical composition of the packaged product to be analysed by comparing said chemical images to different predefined comparison patterns.
The chemical images obtained from colouring the hyperspectral images are subsequently processed by the second image processing means, configured to determine the conformity or non-conformity of the images based on pre-established comparison parameters. This enables the packaged products to be sorted based on the quality thereof, contamination in the products to be detected, and other parameters of the packages to be verified such as the correct printing of codes, labels or the correct positioning of the film. To do so, the second image processing means preferably comprise programmable logic means provided with machine vision algorithms.
The reference patterns used by the first image processing means to compare them to the spectral signature of the hyperspectral images preferably include at least one of the following colour shades:

a colour shade associated with contamination in the heat sealing,
a colour shade associated with surface contamination,
a colour shade associated with the product contained in the package,
a colour shade associated with the background of the image.

In this manner, thanks to the colour shades listed in the previous paragraph, the sorting systems and methods according to the present invention enable the many defects in a packaged product to be detected (which will be described later in the present description).
The second image processing means capture the light reflected and transmitted by the packaged products within the visible light spectrum, and enable the automatic and high-speed analysis of parameters visible to the human eye such as: the printing of barcodes, codes and printed dates, the correct positioning of the films, the presence and position of adhesive labels and the presence of contaminants which are differentiated by colour from the packaged products.
The capture and processing of these images in the visible light spectrum enables the quality of the packaged products to be guaranteed, since it makes it possible to verify the printed codes and suitably trace the quality control of the products, verify the correct printing of critical data such as expiry or best before dates, prevent aesthetic defects, such as the displacement of the film in the packaging machine, or correctly install and print an additional label required on several occasions by customers.
These images in the visible spectrum, like the chemical images, are also analysed by the second image processing means, determining the conformity or non-conformity of the packaged products with a series of pre-established comparison parameters.
In accordance with the foregoing and according to the present invention, the linear hyperspectral image capture means take images exclusively from a predefined line of sight (or image capture line). The width of the image capture line can vary depending on the specific distance at which said image capture means are located, and may be, for example and without limitation, on the order of tenths of millimetres.
Each hyperspectral image is composed by capturing successive lines in different spectral bands. Subsequently, the light intensity values in each spectral band for each pixel of the hyperspectral image, i.e., the spectral signature of the pixel, are compared to a pattern signature, and as a result a colour tone is assigned, composing what is called a chemical image. Likewise, within each chemical image, one or several regions of interest are established, which enable the presence of specific colour tones to be detected, enabling the images to be sorted depending on different comparison methods related to the desired quality inspection.
Likewise, the image capture means in the visible light spectrum also take images exclusively from one line of sight, composing the images by means of capturing successive lines. Likewise, within each image, the second image processing means establish one or several regions of interest, which by means of different comparison methods enable the images to be sorted depending on additional comparison methods also related to the desired quality inspection.
In this manner, the present invention enables deciding whether a packaged product meets the quality requirements and therefore continues to the shipping line, or on the contrary, if the presence of a defect is detected, ordering the rejection thereof by sending a corresponding instruction to the actuation means of the sorting means.
Said sorting means preferably comprise a pneumatic blowing device, a folding belt, a pneumatic pusher or a pneumatic solenoid valve connected to a rejector.
Likewise, the actuation means preferably comprise a programmable logic controller connected to the second image processing means. The first and/or the second image processing means are preferably provided with programmable logic means, for example a computer.
The present invention contemplates that the linear image capture means and the linear hyperspectral image capture means can take images of both the upper face and the lower face of the packaged products. Furthermore, the present invention contemplates that the linear illumination means and the linear hyperspectral illumination means can illuminate the packaged products, both directly and backlit.
The sorting system according to the present invention, like the sorting method of the invention (which will be described in detail later) enable the following defects of a packaged product to be detected:

defects in the heat sealing of the cells of the packages due to air or product contamination, even if they are due to bubbles, unsealed regions or colourless greasy or wet portions, regardless of the colour of the printing ink used on the sealing film or the colour of the packaged product;
presence or absence of the product inside the cell, being especially useful in low-weight products, the absence of which cannot be detected by a scale, which occurs in pharmaceutical products;
suitable positioning of the product within the cell, verifying that it is within established control regions;
deviations in the chemical composition thereof with respect to a reference pattern in the spectral signature of the packaged products (e.g., moisture, protein, fat, sugars, starch, acidity, salt, etc.);
detection of surface contamination in the product contained in the package, even in the event that it is visually confused with the product;
correct positioning of the films of the packages, detecting whether the printing of the sealing film is displaced with respect to the expected position in the film containing the cells; and
reading printed codes or adhesive labels, enabling the correct printing of expiry dates, batches and labels on any of the films to be verified.

In summary, the present invention enables the correct heat sealing of the packages to be analysed and also enables the packaged products to be sorted according to the quality thereof, and possible contaminating agents and any other printing defect in the films to be detected. This is a significant improvement in the detection of possible defects, with respect to the previous techniques.
Likewise, the present invention captures the light reflected by the packaged products both in the infrared light spectrum and in the visible light spectrum. This enables images of the packaged product to be obtained through the films, even if said films are printed with inks opaque to visible light, which is an improvement with respect to currently existing vision equipment with a similar purpose.
The linear image capture means preferably comprise a linear digital camera. Likewise, the linear hyperspectral image capture means preferably comprise a linear hyperspectral digital camera configured to capture images in the spectrum encompassing from 400 to 2000 nanometres in wavelength.
The sorting system according to the present invention preferably further comprises a display monitor configured to show images obtained by the image capture means (both in the visible spectrum and in the infrared spectrum), the monitor being further provided with actuation buttons and illuminated lights which enable different operating parameters to be configured and navigating through different screens.
In a preferred embodiment of the invention, presence sensors are provided in the entry area and in the exit area. In this manner, it is possible to monitor the entry and exit of the packaged products, which enables possible jams to be detected as well as reducing the energy consumption of the sorting system by timing when the illumination means are turned on and off.
Likewise, the sorting system of the invention preferably comprises two feeder belts arranged in series with each other, there being a gap between both feeder belts and the image capture line being located in said gap. Even more preferably, the sorting system according to the present invention is further provided with an upper enclosure and a lower enclosure wherein the linear image capture means, the linear hyperspectral image capture means, the linear hyperspectral illumination means and/or the linear illumination means are housed.
The upper and lower enclosures preferably have an elastomeric seal which guarantees the airtightness thereof during cleaning operations and at least one removable carriage to which the linear image capture means, the linear hyperspectral image capture means, the linear hyperspectral illumination means and/or the linear illumination means are fastened. Said removable carriage facilitates maintenance operations.
Furthermore, the linear image capture means and/or the linear hyperspectral image capture means may be provided with a communication cable connected to the image processing means by which the corresponding sequence of digital images is transmitted.
The linear hyperspectral illumination means for use in the present invention concentrate the hyperspectral light beam on a line with a predefined width (the image capture line), reducing the electrical power of the halogen lamp, and thereby also reducing the thermal irradiation received by the packaged products. Said hyperspectral illumination means demand less heat dissipation (thereby preventing possible damage to the packaged products) and enable a high light power to be obtained. The level of lighting that they provide enables the necessary exposure time for the linear hyperspectral image capture means to be reduced, and enables the high-speed capture of the lines composing hyperspectral images, while they enable the capture with a higher sharpness of the lines.
This represents a novelty in the state of the art in lighting systems which emit within the infrared spectrum and it makes the use of linear hyperspectral lighting means viable in inline industrial applications.
The linear hyperspectral illumination means for use in the present invention preferably comprise a thermally conductive block (for example, a metal block) coupled to heat-dissipating fins, said block being provided with a parabola-shaped inner cavity with a reflective surface finish, a linear halogen lamp being provided in the focus of said parabola in such a manner that the light emitted by the lamp is reflected by the parabola and is concentrated in a light strip with a fixed width, said light strip being focused on a linear Fresnel lens configured to concentrate the received light into a line with a predefined width, the image capture line, which is precisely the same line along which the infrared light is captured by linear hyperspectral image capture means.
The linear hyperspectral illumination means may also preferably be provided with a displacement mechanism for the linear Fresnel lens. Said mechanism enables the focus of the Fresnel lens to be varied and therefore enables the width of the image capture line to be modified.
The linear Fresnel lens can be separated from the heat source (the linear halogen lamp), which enables it to be manufactured with machined plastic materials that are cheap to manufacture.
A second aspect of the invention relates to a method for sorting packaged products by using a sorting system according to the first aspect of the invention, said sorting method being characterised in that it comprises the following steps:

actuating at least one conveyor belt to transport the packaged products from an entry area to an exit area;
actuating the linear illumination means and the linear hyperspectral illumination means to illuminate the image capture line;
actuating the linear image capture means to capture images of the light reflected or transmitted in the visible spectrum by the packaged products along the image capture line and actuating the linear hyperspectral image capture means to capture hyperspectral images of the light reflected or transmitted in the infrared spectrum by the packaged products along the image capture line;
with the first processing means, analysing the intensity of the light from each pixel of the hyperspectral images in different predefined bands of the infrared light spectrum, in order to establish the spectral signature of the hyperspectral images based on said analysis of the intensity, comparing the spectral signature of the hyperspectral images to a reference pattern and colouring the hyperspectral images in a pre-established tone depending on the result of the comparison of the spectral signature thereof with the reference pattern, in order to give rise to coloured hyperspectral images;
with the second image processing means, analysing at least one predetermined control area of each image acquired from the linear image capture means and of each coloured hyperspectral image obtained from the first image processing means; comparing at least one control area of each of said images to at least one reference pattern; sorting said images depending on said comparison with the reference pattern and sending sorting instructions to actuation means; and
receiving the sorting instructions and with the actuation means, actuating the sorting means for the packaged products based on said sorting instructions.

The control area analysed by the second image processing means preferably comprises a heat-sealing band of the packaged product, a cell wherein the products that have been packaged are deposited or an area wherein printed codes of the packaged product are provided.
Preferably, the second processing means, when analysing at least one predetermined control area, quantify the pixels of at least one predetermined shade which are present in said control area, analysing the outline and surface of the portions of said predetermined shade.

BRIEF DESCRIPTION OF THE FIGURES

What follows is a very description of a series of drawings that aid in better understanding the invention, which are expressly related to an embodiment of said invention and are presented by way of non-limiting examples of the same.

Fig. 1 is a schematic side view of a first embodiment of a system for sorting packaged products according to the present invention;
Fig. 2 is a schematic side view of a detail of the embodiment shown in the Figure, wherein the linear hyperspectral illumination means can be better seen;
Fig. 3A is a schematic cross-sectional view of the linear hyperspectral image capture means and Figs. 3B to 3D are schematic views of the method for forming the chemical images;
Fig. 4A is a top view of a detail of a system for sorting packaged products according to the present invention and Fig. 4B schematically shows a possible example of the regions of interest of the images that are captured;
Figures 5A-C show by way of example a sequence of images obtained by the linear hyperspectral image capture means converted into chemical images and how they are processed by means of the system and method of the invention in order to detect contamination in the heat-sealing area;
Figs. 6 and 7 show a preferred embodiment of the device for sorting packages claimed herein; and
Fig. 8 shows a preferred embodiment of the linear hyperspectral illumination means according to the present invention.

DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

A detailed description of several preferred exemplary embodiments of the present invention is provided below with the aid of the attached Figures 1 to 8.
Throughout the present description, as well as in the attached figures, the elements with the same or similar functions will be designated with the same numerical references.
Figure 1 shows a schematic representation of a system for sorting packaged products according to the present invention.
In the embodiment shown therein, the sorting system is used to inspect heat-sealed packages of food packaged products (1) such as, for example, sausage, ham or cheese or medicines, and enables those products (10) that do not meet predefined quality parameters to be rejected, thanks to the actuation of sorting means (9) consisting, in this embodiment of the invention, of a tilting belt.
The sorting system is further provided with two conveyor belts (2), arranged one after the other, which have two presence sensors (8) which detect the entrance and exit of the packaged products (1), such that both belts (2) have an opening in the transition between them, wherein, in this embodiment of the invention, the capture of the images of the lower surface of the package of the packaged products (1) takes place. The system has two linear image capture means (4) in the visible spectrum, which in this embodiment are linear digital cameras and two linear hyperspectral image capture means (5) in the infrared spectrum, two of the capture means being located in the upper enclosure (41) and the other two in the lower enclosure (42). The means (4) and (5) capture the images along an image capture line (3).
In this embodiment, the linear hyperspectral illumination means (6) are hyperspectral digital cameras and are located in the upper enclosure (41) and enable images to be captured in the visible spectrum and the infrared spectrum of the upper surface of the package by means of direct lighting, capturing the lower surface of the package backlit in the infrared spectrum and directly in the visible light spectrum.
Linear illumination means (7) in the visible light spectrum are also provided, which, in this embodiment, are provided on the lower portion in order to enable the capture of images in the visible light spectrum of the lower surface of the package.
The system also has first processing means (12) for processing the hyperspectral images and second image processing means (13) configured to analyse at least one predetermined control area of each image of the sequence of images; acquired from the linear digital cameras (4) and from the first processing means (12). Both the first processing means (12) and the second processing means (13) are housed in a protective enclosure (11).
Actuation means (14) are also provided which comprise a programmable logic controller (14) and are configured to actuate on the package sorting means (9) by sending them sorting instructions. The actuation means (14) in turn receive instructions from the image processing means (13).
The embodiment of the packaging sorting system shown in Fig. 1 is further provided with a display monitor (15), actuation buttons and operation lights (16) and software with a touch interface. Said software enables the different operating parameters of the equipment to be configured and navigating through the screens of the application, which is programmed to show images from the sequence of images captured by the image capture means (4) and (5), showing a control panel with the operation status of the device, showing the sorted images and the history of sorting that has been performed.
Figure 2 shows a transverse cross-sectional representation of the linear hyperspectral illumination means (6), which are focused on a packaged product (1). The package comprises a cell (31), wherein the product (39) to be packaged is deposited which is sealed by heat sealing of an upper film (32) with a bottom film which makes up the cell.
As seen in said figure, the linear hyperspectral illumination means (6) consist, in this embodiment of the invention, of a metal block (17) coupled to heat-dissipating fins (18), and they have a parabola-shaped cavity (19) along the block, in the focus of which a linear halogen lamp (20) is installed, such that the light it emits is reflected by the parabola, which has a mirrored surface finish, such that the reflection of the light is concentrated in a light strip (22) with a fixed width, which is focused on a linear Fresnel lens (21), which in turn concentrates the light in the image capture line (3), the width of which can be regulated by means of a mechanism (23), which by means of a displacement of the lens (21) regulates the focus thereof.
In this embodiment of the invention, the line of light produced by the linear hyperspectral illumination means (6) focuses a strip on the upper face of the packaged product (1), which enables the image capture means (4) and (5) to capture images along the image capture line (3), both directly by means of the upper hyperspectral digital camera (5), as well as backlit through the opening existing between the two conveyor belts (2), by means of the lower hyperspectral digital camera (5).
Figure 3A shows a representation of a longitudinal cross section of linear hyperspectral illumination means (6) and it further schematically shows how the linear hyperspectral image capture means (5) send the hyperspectral image lines of the light reflected or transmitted in different bands of the infrared spectrum (24) to the first means image processing (12), which, as shown schematically in Figures 3B to 3D, compose a hyperspectral image (29), which is formed by different pixels (25), in each of which the intensity of the light captured in the different frequency bands of the infrared light spectrum is recorded, which is known as the spectral signature (26), and subsequently composes a sequence of images called chemical images (30), wherein the pixels of said images are coloured, the colour tone (28) of the different pixels being determined in this embodiment of the invention by a software implemented in the image processing means (12) based on the comparison of the spectral signature (26) with predetermined reference patterns (27). Subsequently, the chemical images (30) are sent to the second image processing means (13).
Figure 4A is a top view wherein it schematically shows the passage of a heat-sealed packaged product (1) over the gap area of the two conveyor belts (2), as well as the image capture line (3). In this embodiment of the invention, the packaged product is provided with two cells (31), wherein the products (39) are deposited and whereon an upper film is adhered by means of a heat-sealing process, along a heat-sealing band (33) arranged around the cells (31).
Figure 4B shows a diagram of the control area (36) of the images to be processed, said control area (36) comprising different regions of interest to be analysed, the control area (36) includes, in this specific exemplary embodiment of the invention, the cell (38) inside of which the packaged product (39) to be analysed is located, which may have surface contamination (35). The control area (36) also includes the heat sealing (33), which is delimited by the band comprising the heat sealing around the cell (37), said band being able to have contamination from the packaged product itself or from a contaminant (34).
Figures 5A-5C show chemical images (30) of some packages of Serrano ham, illustrating, among other aspects of the invention, how the control area (36) of the package, the region of interest of the heat sealing (37) and the presence of contamination (34) are detected therein, by means of the processing of the different regions.
More particularly, Fig. 5A shows two chemical images (30) generated by the first image processing means (12), from two different models for assigning colour tone (28) based on the comparison with different spectral signature models (26). From left to right, the first image is composed to differentiate the control areas of the package, the second to detect the contamination of the heat sealing.
Fig. 5B shows, by way of example, sequences of the processing of the chemical images of some packages of Serrano ham, which illustrate, among other aspects of the invention, how the control area (36) of the package and the region of interest of the heat sealing (37) are detected. From left to right, the first image is processed to detect the outline of the package, the second image to determine the control area (36) of the package, the third image to determine the regions of interest of the heat sealing (37).
More particularly, Fig. 5C shows, by way of example, how the presence of contamination (34) is detected in the heat sealing (37), by comparing the colour pattern of the contamination in the region of interest of the heat sealing (37), processing the different regions of interest.
Figures 6 and 7 show, in front and isometric views, a possible embodiment of the system for sorting packaged products.
Fig. 6 shows, in a front view, the different elements of the system for sorting packaged products according to the invention, previously described in Figures 1 and 2, with the aim of facilitating the understanding of the diagrams. In this case, the system is provided with two linear hyperspectral image capture means (5), one located in the upper enclosure (41) with direct lighting and the other located in the lower enclosure (42) with backlit lighting. In this embodiment, two different linear hyperspectral illumination means (6) are provided, arranged symmetrically to each other with respect to the image capture line.
In this manner, if the halogen lamp of the first means (6) burns out or said means (6) stop working for some other reason, there are some second backup or replacement means (6) located symmetrically. Linear image capture means (4) in the visible light spectrum are also shown, directly illuminated by linear illumination means (7) also located in the upper enclosure (41).
More particularly, Fig. 7 represents the same as Fig. 6 in an isometric view, only in this case the removable carriages (43) of the upper (41) and lower (42) enclosures are deployed in order to better display the different image capture means and the illumination means of the device for sorting packaged products.
Figure 8 shows an isometric view of a possible embodiment of the linear hyperspectral illumination means (6). In this particular embodiment of the invention, the metal block (17) coupled to the dissipating fins (18) is arranged perpendicular to the passage direction of the packaged products, with a certain inclination on the conveyor belt (2), focusing the beam towards the linear Fresnel lens (21), which is attached to a mechanism (23) that enables the position and orientation of the lens to be regulated, and therefore the width of the lighting line focused on the capture area of the hyperspectral camera (3) which hits the package.

Claims

A system for sorting packaged products (1), said system being provided with:
- at least one conveyor belt (2) configured to transport the packaged products (1) from an entry area to an exit area;

- at least one linear image capture means (4), configured to capture images of the light reflected or transmitted in the visible spectrum by the packaged products (1) along a predefined image capture line (3);

- at least one linear illumination means (7) configured to emit light in the visible light spectrum and illuminate the image capture line (3);
said system being characterised in that it further comprises:
- at least one linear hyperspectral image capture means (5), configured to capture hyperspectral images of the light reflected or transmitted in different bands of the infrared spectrum (24) by the packaged products (1) along the predefined image capture line (3);

- at least one linear hyperspectral illumination means (6) configured to emit light in the infrared light spectrum and only illuminate the image capture line (3);

- first image processing means (12), configured to determine the spectral signature of each pixel (25) of the hyperspectral images (29) in different predefined bands of the infrared light spectrum, in order to compare the spectral signature (26) of each pixel of the hyperspectral images with a reference pattern (27) and in order to colour the hyperspectral images (29) in a pre-established tone (28) depending on the result of the comparison of the spectral signature (26) thereof with the reference pattern (27), giving rise to coloured hyperspectral images (30);

- second image processing means (13), configured to analyse at least one predetermined control area region (36) of each image acquired from the linear image capture means (4) and of each coloured hyperspectral image (30) obtained from the first image processing means (12); to compare at least said control area (36) of each of said images to at least one reference pattern; to sort said images depending on said comparison with the reference pattern and send sorting instructions to actuation means (14); and

- actuation means (14), configured to receive the sorting instructions and actuate sorting means (9) for the packaged products (1) based on said sorting instructions, the sorting means (9) being arranged in the exit area.
The system according to claim 1, wherein the linear image capture means (4) comprise a linear digital camera.
The system according to any of the preceding claims, wherein the linear hyperspectral image capture means (5) comprise a linear hyperspectral digital camera configured to capture images in the spectrum of 400 to 2000 nanometres in wavelength.
The system according to any of the preceding claims, wherein the sorting means (9) comprise a pneumatic blowing device, a folding belt, a pneumatic pusher or a pneumatic solenoid valve connected to a rejector.
The system according to any of the preceding claims, wherein the actuation means (14) comprise a programmable logic controller connected to the second image processing means (13).
The system according to any of the preceding claims, which further comprises a display monitor (15) configured to show images obtained by the image capture means (4) and (5), the monitor (15) being further provided with actuation buttons and illuminated lights (16) which enable different operating parameters to be configured and navigating through different screens.
The system according to any of the preceding claims, wherein presence sensors (8) are further provided in the entry area and in the exit area.
The system according to any of the preceding claims, wherein two feeder belts (2) arranged in series with each other are provided, there being a gap between both feeder belts (2) and the image capture line (3) being located in said gap.
The system according to claim 8, wherein an upper enclosure (41) and a lower enclosure (42) are provided wherein the linear image capture means (4), the linear hyperspectral image capture means (5), the linear hyperspectral illumination means (6) and/or the linear illumination means (7) are housed.
The system according to any of the preceding claims, wherein the linear hyperspectral illumination means (6) comprise a thermally conductive block (17) coupled to heat-dissipating fins (18); said block (17) being provided with a parabola-shaped inner cavity (19) with a reflective surface finish; a linear halogen lamp (20) being provided in the focus of said parabola such that the light emitted by the lamp (20) is reflected by the parabola and is concentrated in a light strip (22) with a fixed width; a linear Fresnel lens (21) being focused on said light strip (22), lens which is configured to concentrate the received light into a line (3) with a predefined width.
The system according to claim 10, wherein the linear hyperspectral illumination means (6) are provided with a displacement mechanism (23) for the linear Fresnel lens (21).
The system according to any of the preceding claims, wherein two different linear hyperspectral illumination means (6) are provided, arranged symmetrically to each other with respect to the image capture line (3).
A method for sorting packaged products (1) using a sorting system according to any of claims 1 to 12, said sorting method being characterised in that it comprises the following steps:
- actuating at least one conveyor belt (2) to transport the packaged products (1) from an entry area to an exit area;

- actuating the linear illumination means (7) and the linear hyperspectral illumination means (6) to illuminate the image capture line (3);

- actuating the linear image capture means (4) to capture images of the light reflected or transmitted in the visible spectrum by the packaged products (1) along the image capture line (3) and actuating the linear hyperspectral image capture means (5) to capture hyperspectral images of the light reflected or transmitted in the infrared spectrum (24) by the packaged products (1) along the image capture line (3);

- with the first processing means (12), analysing the intensity of the light from each pixel (25) of the hyperspectral images (29) in different predefined bands of the infrared light spectrum, in order to establish the spectral signature (26) of the hyperspectral images (29) based on said analysis of the intensity, comparing the spectral signature (26) of the hyperspectral images (29) to a reference pattern (27) and colouring the hyperspectral images in a pre-established tone depending on the result of the comparison of the spectral signature (26) thereof with the reference pattern (27), in order to give rise to coloured hyperspectral images (30);

- with the second image processing means (13), analysing at least one predetermined control area (36) of each image acquired from the linear image capture means (4) and of each coloured hyperspectral image (30) obtained from the first image processing means (12); comparing at least one control area (36) of each of said images to at least one reference pattern; sorting said images depending on said comparison with the reference pattern and sending sorting instructions to actuation means (14); and

- receiving the sorting instructions and with the actuation means (14), actuating the sorting means (9) for the packaged products (1) based on said sorting instructions.
The sorting method according to claim 13, wherein the control area (36) analysed by the second image processing means (13) comprises a heat-sealing band (33) of the product package (1), a cell (31) wherein the products are deposited (39) or an area wherein printed codes of the packaged product (1) are provided.