EP3977222A1 - Method for controlling operation of a packaging machine and related control unit - Google Patents
Method for controlling operation of a packaging machine and related control unitInfo
- Publication number
- EP3977222A1 EP3977222A1 EP20742875.6A EP20742875A EP3977222A1 EP 3977222 A1 EP3977222 A1 EP 3977222A1 EP 20742875 A EP20742875 A EP 20742875A EP 3977222 A1 EP3977222 A1 EP 3977222A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- data flow
- packaging machine
- discontinuity
- acquired
- frames
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004806 packaging method and process Methods 0.000 title claims abstract description 131
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 51
- 238000001514 detection method Methods 0.000 claims abstract description 26
- 239000000463 material Substances 0.000 claims abstract description 24
- 230000007787 long-term memory Effects 0.000 claims abstract description 12
- 238000013178 mathematical model Methods 0.000 claims description 13
- 238000004458 analytical method Methods 0.000 claims description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 11
- 230000015654 memory Effects 0.000 claims description 11
- 230000003287 optical effect Effects 0.000 claims description 7
- 230000000737 periodic effect Effects 0.000 claims description 5
- 238000012937 correction Methods 0.000 claims description 4
- 238000007619 statistical method Methods 0.000 claims description 4
- 230000003213 activating effect Effects 0.000 claims description 3
- 230000001419 dependent effect Effects 0.000 claims description 2
- 238000004519 manufacturing process Methods 0.000 description 14
- 235000019504 cigarettes Nutrition 0.000 description 9
- 235000013305 food Nutrition 0.000 description 8
- 238000002360 preparation method Methods 0.000 description 6
- 238000003860 storage Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 239000007788 liquid Substances 0.000 description 4
- 239000000123 paper Substances 0.000 description 4
- 230000000391 smoking effect Effects 0.000 description 4
- 238000012706 support-vector machine Methods 0.000 description 4
- 241000208125 Nicotiana Species 0.000 description 3
- 235000002637 Nicotiana tabacum Nutrition 0.000 description 3
- 230000004931 aggregating effect Effects 0.000 description 3
- 230000002547 anomalous effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000036512 infertility Effects 0.000 description 3
- 238000007789 sealing Methods 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000001954 sterilising effect Effects 0.000 description 2
- 229920000298 Cellophane Polymers 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000003571 electronic cigarette Substances 0.000 description 1
- 230000007257 malfunction Effects 0.000 description 1
- 239000011087 paperboard Substances 0.000 description 1
- 229920003023 plastic Polymers 0.000 description 1
- 239000002985 plastic film Substances 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000000284 resting effect Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G05B19/4184—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by fault tolerance, reliability of production system
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B23/00—Testing or monitoring of control systems or parts thereof
- G05B23/02—Electric testing or monitoring
- G05B23/0205—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults
- G05B23/0218—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterised by the fault detection method dealing with either existing or incipient faults
- G05B23/0224—Process history based detection method, e.g. whereby history implies the availability of large amounts of data
- G05B23/0227—Qualitative history assessment, whereby the type of data acted upon, e.g. waveforms, images or patterns, is not relevant, e.g. rule based assessment; if-then decisions
- G05B23/0235—Qualitative history assessment, whereby the type of data acted upon, e.g. waveforms, images or patterns, is not relevant, e.g. rule based assessment; if-then decisions based on a comparison with predetermined threshold or range, e.g. "classical methods", carried out during normal operation; threshold adaptation or choice; when or how to compare with the threshold
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B23/00—Testing or monitoring of control systems or parts thereof
- G05B23/02—Electric testing or monitoring
- G05B23/0205—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults
- G05B23/0259—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterized by the response to fault detection
- G05B23/0286—Modifications to the monitored process, e.g. stopping operation or adapting control
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/24—Pc safety
- G05B2219/24097—Camera monitors controlled machine
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Definitions
- This invention relates to a method for controlling operation of a packaging machine for consumer products and a related control unit.
- this invention relates to a method for controlling operation and the related control unit of an automatic packaging machine for consumer products such as, for example, products for pourable food preparations which are liquid, semi-liquid, powdery, semi-solid, or smoking articles.
- An automatic packaging machine for consumer products comprises a plurality of operating units connected to each other and a control apparatus configured for controlling those operating units.
- the operating units may be at least one unwinding unit for unwinding a multi-layer web from a reel; a sterilising unit for sterilising the web unwound; a longitudinal sealing unit for the sterilised web, which forms a continuous tube; a filling unit thanks to which the food product is poured into the continuous tube formed; a forming and separating unit configured for sealing the filled tube along transversal seals which are opposite each other relative to the longitudinal axis of the tube so as to obtain a product and separating the product from the tube by means of a transversal cut; a folding unit, configured for receiving the products and for folding parts of the opposite transversal seals to obtain folded triangular tabs in the product.
- the operating units are configured for carrying out successive processing operations on articles containing tobacco, starting from a paper web intended to be wrapped around tobacco deposited on the paper web itself, or on disposable cartridges usable for electronic cigarettes.
- an automatic packaging machine for smoking articles may be: a machine for making cigarette filters; a machine for making cigarettes, starting from tobacco deposited in a continuous paper web, connected to the filter making machine for receiving the filters made from the latter; a storage machine, for storing the cigarettes received from the cigarette making machine; a cigarette packet making machine, connected to the storage machine or to the cigarette making machine, for receiving the cigarettes made; an overwrapping machine, connected to the cigarette packet packaging machine in order to wrap them in a transparent plastic film, also called cellophane; a cartoning machine, connected to the overwrapper in order to form cartons of cigarette packets.
- the control apparatus runs a control program for such operating units, in order to carry out automated processes intended for making the consumer products described above and/or for managing the wrapping materials, setting new production formats, supervising production or scheduled maintenance of the operating units themselves.
- control apparatus runs the control program by reading the input data supplied by the field devices and by the sensors connected to each operating unit, analysing the packaging machine status resulting from those input data read and generating output data, based on these input data and on a control program logic, so as to make the operating units of the packaging machine operate.
- the aim of this invention is to provide a method for controlling operation of a packaging machine for consumer products and a related control unit which is free of the disadvantages described above and, at the same time, is easy and inexpensive to implement.
- this invention supplies a method for controlling operation of a packaging machine for consumer products and a related control unit as set out in the appended independent claims.
- FIG. 1 is a schematic view of a packaging machine for consumer products comprising a plurality of operating units for processing the products being processed and the materials used to make the products, a control apparatus for the operating units of the packaging machine, a control unit for controlling packaging machine operation, in accordance with this invention, and an acquisition device configured for acquiring a data flow in the form of video or audio video streaming;
- FIG. 2 is a schematic view of the acquisition device of Figure 1 and of a processing device configured for comparing the acquired data flow with a reference data flow, for detecting any discontinuity in the acquired data flow.
- the numeral 1 denotes in its entirety an automatic packaging machine for consumer products P, for example a packaging machine 1 for products P for pourable preparations which may be liquid, or semi-liquid, powdery, granular or semi-solid.
- the packaging machine 1 comprises a plurality of operating units configured for processing the products P and the material M used to make those products P.
- the packaging machine 1 comprises an infeed part 2 comprising a plurality of operating units, made as drum conveyors 201 and placed in sequence, configured for feeding and processing a web of material M, used to make the products P.
- the web of material M is, for example, fed from a reel (not illustrated).
- the material M used to make the products has been shown as a web, for example of a paper material, it should be noticed that the material used to make the products may be, for example, a paperboard blank intended to be folded.
- the packaging machine 1 also comprises, by way of example, an outfeed part 3 configured for supplying the packaged products P, which comprises a respective outfeed operating unit, for example made as a linear conveyor 301 configured for feeding the products P, which comprises a planar conveying branch 302 on which the products P are placed resting on it and are conveyed towards a supplying outfeed (not illustrated).
- a respective outfeed operating unit for example made as a linear conveyor 301 configured for feeding the products P, which comprises a planar conveying branch 302 on which the products P are placed resting on it and are conveyed towards a supplying outfeed (not illustrated).
- the packaging machine 1 comprises a packaging part for packaging the products P starting from the material M, not illustrated.
- the packaging machine 1 comprises a control apparatus 4 configured for controlling the packaging machine 1 , in particular the operating units 201 , 301 of the packaging machine 1 which are indicated above.
- the control apparatus 4 is for example, a PLC, which runs a software program installed in the control apparatus 4 itself. That software program is configured for cyclically reading the input data supplied by field devices and by the sensors connected to each operating unit 201 , 301 , for analysing the packaging machine 1 status resulting from those input data read and for generating output data, based on these input data, so as to make the operating units 201 , 301 of the packaging machine operate based on the control program.
- the packaging machine 1 also comprises a control unit 5 for controlling operation of the packaging machine which comprises an acquisition device 501 , configured for acquiring a data flow in the form of video, or audio video, streaming, that is to say, multi-media streaming, comprising a sequence of frames, that is to say, of images, acquired in sequence relative to the products P being processed in the packaging machine 1 and/or to the materials M used to make the products P.
- a control unit 5 for controlling operation of the packaging machine which comprises an acquisition device 501 , configured for acquiring a data flow in the form of video, or audio video, streaming, that is to say, multi-media streaming, comprising a sequence of frames, that is to say, of images, acquired in sequence relative to the products P being processed in the packaging machine 1 and/or to the materials M used to make the products P.
- the acquisition device 501 is, preferably, an image acquisition device, and optionally acquires sounds, at high speed and high resolution and the video, or audio video, flow acquired comprises a set of images, and optionally also of sounds, acquired in sequence which form the sequence of frames.
- the image acquisition device 501 shall be understood to be able to acquire either a video, or audio video, streaming data flow from an initial event E1 to a final event E2, as shown in Figure 2.
- the initial event E1 that is to say, the start of the acquisition, like the final event E2, that is to say, the end of the acquisition, may be commanded by the control apparatus 4 of the packaging machine 1 and correspond to successive moments.
- the image acquisition device may comprise a body on which an electronic sensor is positioned, for example an array or arrangement of linear or two- dimensional matrix light-sensitive elements of the CCD or CMOS type, and special optical receiver devices fixed to the body, for example, a lens through which the sensor can receive the diffused light to be acquired.
- an image with a resolution of [nxm] pixels may be acquired by means of a single shot using a two-dimensional sensor with two-dimensional matrix of light-sensitive elements [nxm] or by means of n consecutive shots using a linear sensor with m light-sensitive elements.
- the acquisition device 501 is configured for acquiring images in a wavelength range of from 100 nm to 15 pm.
- the acquisition device 501 is configured for acquiring images which are in the following ranges: ultraviolet UV (100 - 400 nm), and/or visible (380 - 750 nm), and/or NIR - Near Infrared (750 nm - 1 .4 pm), and/or SWIR - Short-wavelength infrared (1 .4 pm - 3 pm) and/or MWIR - Medium Wavelength Infrared (3 - 8 pm), and/or LWIR - Long Wavelength Infrared (8 -15 pm) including image thermography.
- the acquisition device 501 may, optionally, be configured for acquiring hyperspectral images included in a spectral range of between 400 nm and 4000 nm, preferably between 950 nm and 1700 nm, more preferable still included between 1250 nm and 1600 nm.
- control unit 5 comprises: a processing device 502, which is configured for comparing the acquired data flow with a reference data flow, for detecting any discontinuity in the acquired data flow.
- discontinuity refers to an anomalous variation or an unexpected irregularity compared with the reference data flow.
- the unexpected irregularity may relate to a reference multi- media signal, which may be uniform or cyclically repetitive (video, or audio video) and may relate to correct operation of the packaging machine 1 .
- the reference data flow may also be determined during a learning step, as described in detail below, as a whole and may not be linked to a specific correct operation of the packaging machine.
- the discontinuity in the streaming data flow may highlight an anomalous operation of the packaging machine 1 relative to the products P being processed in the packaging machine 1 and/or to the materials M used to make the products P. For example, there is a discontinuity if a frame of the streaming data flow is not assessed to be acceptable relative to the reference data flow.
- the control unit 5 also comprises a long-term memory 503. Indeed, the processing device 502 is configured for undertaking corrective actions based on the discontinuity detected and/or saving the acquired data flow in the long-term memory 503 following detection of the discontinuity for a subsequent assessment.
- the control unit also comprises a temporary memory 504, for example a RAM memory, configured for saving the data flow acquired by the acquisition device 501 .
- a temporary memory 504 for example a RAM memory, configured for saving the data flow acquired by the acquisition device 501 .
- an acquisition viewpoint is defined which is no longer dependent on the input data supplied by field devices and by the sensors connected to the operating units 201 , 301 , but instead is processed based on an external acquisition viewpoint, as if an expert machine operator were present, constantly observing the packaging machine itself, paying particular attention to the products P being processed in the packaging machine 1 and/or to the materials M used to make the products P.
- the acquisition device 501 may be positioned facing a portion of packaging machine 1 comprising a plurality of operating units 201 , 301 , for example facing a specific operating unit 201 or 301 , in such a way as to frame a field of view having a“context” which is known beforehand.
- the term“context” refers to the fact that, given a portion of the packaging machine 1 framed, the scenes acquired and the content of the videos acquired are already known beforehand and do not change over time.
- the processing device 502 can detect the discontinuity in the acquired data flow, for example in the acquired video or audio video streaming, without adding filters relating to the significance of the acquired data flow itself.
- The“context” being known beforehand, the whole data flow is entirely analysable and therefore the processing is simplified.
- the acquisition device 501 may be positioned facing the portion of the packaging machine 1 from the front in an observation point in which a machine operator could be positioned for viewing or monitoring operation of the packaging machine 1 .
- the acquisition device 501 may be positioned facing the portion of the packaging machine 1 , that is to say, facing the operating unit 201 or 301 , but from the side rather than from the front, if it is appropriate to acquire the data flow from an observation point different from an observation point of an operator.
- the packaging machine 1 may also comprise multiple acquisition devices 501 , which are facing respective portions of packaging machine at respective observation points. It shall be understood that multiple acquisition devices 501 may be connected to the same processing device 502 or each acquisition device 501 may have a respective processing device 502 associated with it.
- control unit 5 is connectable to the control apparatus 4 of the packaging machine 1 for receiving and/or supplying data and/or commands to the packaging machine 1 itself.
- the acquisition device 501 is a video streaming image acquisition device, or an audio and video multi-media acquisition device, configured for acquiring the sequence of N frames, that is to say, of images, in sequence.
- the sequence may be that of a continuous streaming or, alternatively, since it is connectable to the control apparatus 4 of the packaging machine 1 , the sequence may be that which the acquisition device 501 can select amongst those of the continuous streaming.
- the acquisition device 501 may acquire images, and optionally sounds:
- packaging machine 1 synchronously with the packaging machine 1 , that is to say, at packaging machine 1 significant events (for example, at one or more specific machine encoder degrees in each machine cycle, that is to say, “in phase” with machine cyclical operation).
- the periodic acquisition command has a low acquisition frequency
- a stroboscopic video, or audio video, acquisition is carried out, if it is at a high acquisition frequency an acquisition with slow motion effect is carried out (for example with a high number of fps“frames per second” - 100 fps rather than the usual 25 fps/30fps).
- the number of frames per second acquired may range, for example, from a minimum of 15 fps with low quality video streaming, to a maximum of 300fps for extremely high quality video streaming with slow motion effect.
- nxm the number of pixels acquired per frame, which considering a pixel matrix [nxm] may range from 320x240 pixels/frame, for low quality video streaming, to 640x480 pixels/frame or 854x480 pixels per frame, for a medium quality video, up to 1280x720 (720p Standard HD) or 1920x1080 (1080p Full HD, also called 2K), in compliance with the most common proportions 4:3 and 16:9 (aspect ratio) of video formats.
- the acquired data flow may therefore be produced using images, and optionally sounds, periodically acquired over time in sequence or using images, and optionally sounds, which are acquired in sequence and at packaging machine 1 significant events, as previously illustrated.
- the data flow may be acquired starting from one of the significant events, if this is considered as the initial event E1 from which the streaming data flow is acquired, and is a continuous streaming.
- the video or audio video streaming data flow may be acquired at a start of a processing cycle for processing the products P, or processing of the materials M used to make those products P.
- the acquisition device 501 may be positioned facing from the front a filling unit for the pourable preparation and, for example, the streaming data flow may be activated immediately before the filling starts and may be deactivated when filling has ended.
- the initial event E1 of the start of the acquisition may be commanded by the control apparatus 4 of the packaging machine 1 , which also controls a movement of the product P in front of the acquisition device 501 itself, relative to the movement itself.
- the control unit 5, when connected to the control apparatus 4, can modify a packaging machine 1 operation with respect to the discontinuity detected, for example to modify packaging machine 1 operating parameters associated with the operating units 201 , 301 .
- control unit 5 can activate a dedicated screen page of an operator interface (HMI - Fluman Machine Interface) of the control apparatus 4 of the packaging machine 1 to show, for example to a machine operator, the video, or audio video, streaming saved in the long term memory 503 after the discontinuity and highlighting the discontinuity detected, without modifying the operating parameters of the self-same packaging machine 1 .
- HMI Hemetic Management Interface
- the subsequent assessment indicated above may be carried out by an expert operator who has the opportunity to perform an off-line analysis of the video, or audio video, flow showing the discontinuity detected.
- the corrective actions which the processing device 502 may undertake, based on the discontinuity detected may also relate to the detection logic itself. Indeed, the processing device 502 may be configured for correcting a discontinuity detection logic, for example correcting a mathematical model for said detection of the discontinuity.
- That correction may take place by adapting the mathematical model used and/or selecting a different mathematical model, for example selected from a library of mathematical models available, based on metrics detected from the streaming data flow.
- the processing device 502 comprises a software component configured for analysing the acquired data flow which is installed in the processing device 502 itself.
- the times for processing the data flow must be limited in order to modify packaging machine 1 operation and/or to correct the logic for detection of the discontinuity in the shortest possible time, so as to increase packaging machine 1 production efficiency.
- the data flow acquisition device 501 is separate from the processing device 502 in order to increase processing efficiency.
- the acquisition device 501 and the processing device 502 may be integrated with each other but in any case their functions remain separate.
- the processing device 502 may comprise a first component 505 for carrying out, during a first analysing step, an analysis of the data flow, for detecting at least one metric of the data flow, and a second component 506 for carrying out, during a second detecting step, a detection of the discontinuity, analysing the metric detected by the first component, during the first analysing step, by means of the mathematical model for detection of the discontinuity.
- the first analysis of the analogue or digital video, or audio video, streaming may be carried out on the streaming data flow saved in the temporary memory 504.
- the analysis of the video, or audio video streaming data flow can be rendered more efficient and guarantee high performance.
- the first component 505 comprises a converting module 505’, configured for converting the video, or audio video, streaming data flow which comprises the sequence of N frames acquired in sequence from the initial event E1 to the final event E2, and for obtaining a vector field 505” indicating a direction of a movement and an intensity of the movement of a plurality of uniform zones of pixels in each frame relative to a base frame.
- a converting module 505 configured for converting the video, or audio video, streaming data flow which comprises the sequence of N frames acquired in sequence from the initial event E1 to the final event E2, and for obtaining a vector field 505” indicating a direction of a movement and an intensity of the movement of a plurality of uniform zones of pixels in each frame relative to a base frame.
- the base frame may correspond to the frame initially acquired at the initial event E1 . Therefore, in each frame after the initial one, for each uniform zone of pixels, the intensity and the direction of the movement relative to the base frame will be assessed.
- the first component 505 also comprises a statistics module 505”’ for analysing, by means of a statistical analysis algorithm, during a statistical analysis step following the converting step, the vector field 505” and for identifying some significant parameters of each frame N, that is to say, the so-called metrics, obtaining a set of data having reduced dimensions, that is to say, a data matrix having the size B x N indicated with the number 507, wherein B indicates the number of metrics identified.
- a statistics module 505 for analysing, by means of a statistical analysis algorithm, during a statistical analysis step following the converting step, the vector field 505” and for identifying some significant parameters of each frame N, that is to say, the so-called metrics, obtaining a set of data having reduced dimensions, that is to say, a data matrix having the size B x N indicated with the number 507, wherein B indicates the number of metrics identified.
- significant metrics may be the kurtosis of the angle, or the average intensity of the vector field (that is to say, the average intensity of the modulus of a movement vector).
- the converting module 505’ may, for example, be produced by means of an Optical Flow type algorithm, for example Dense Optical Flow, which is configured for considering the motion of an object in consecutive frames of a frequency of frames acquired in sequence and therefore to assess a movement over time between the object and the acquisition device 501 , assessing the movement of the position of a same pixel over time relative to the position of the self-same pixel in the base frame.
- an Optical Flow type algorithm for example Dense Optical Flow
- the converting module 505’ can identify in each frame the plurality of uniform zones of pixels, that is to say, closed zones inside which there are pixels present which meet the uniformity criteria. These closed zones are also called voxels.
- the Optical Flow type algorithm can consider the movement vector, defined by a pair of values given by the direction of the movement and by the intensity of the movement relative to the same zone of the reference frame for converting the frames, each defined by [nxm] pixels, and obtain the vector field 505” previously described.
- the vector field 505 has a number of values equal to V (number of voxels) x 2 (the direction and intensity of the movement of each voxel) x N (number of frames).
- the statistical processing carried out by the statistics module 505’” on the vector field 505” allows a further reduction in the data associated with each frame in order to identify at least the significant metric of each frame N.
- the first component 505 following the processing carried out by the first component 505 during the first analysing step, it is possible to convert the video streaming data flow, comprising N frames each of which comprises a total number of pixels [nxm], into a matrix of metrics having a reduced size BxN, with more simple processing and analysis which can advantageously be analysed in shorter times by the second component 506, dedicated to detection of the discontinuity.
- the content of each frame is converted and is simplified, detecting at least one, or a plurality of metrics which summarise the information content of the frame itself.
- the second component 506 comprises a classifying module 506’ for classifying, during a classifying step, the metrics of each of the frames N of the video and/or audio video flow and an assessing module 506” for assessing, during an assessing step, whether each frame of the sequence should be considered acceptable, that is to say, “OK”, or unacceptable, that is to say,“NO”.
- the classifying module 506’ may comprise an algorithm of the Support Vector Machine type, for example, with which it is possible to associate with each frame a point in a Cartesian plane, if there are two metrics and they are for example the kurtosis of the angle and the average intensity of the field, or in a space having K dimensions if there are more than 2 metrics and they are equal to K.
- each frame of the sequence of frames N of the streaming data flow may therefore be represented in the Cartesian plane gradually as it is acquired, in accordance with the Support Vector Machine algorithm.
- the classifying module 506 it is possible to consider multiple metrics simultaneously for each video frame, and to classify the video frame itself by means of complex criteria, which consider a combination of multiple metrics.
- the result of the classifying module 506’ is then processed by the assessing module 506”, for the assessment of the acceptability of each frame, which considers the comparison with the reference data flow.
- the assessing module 506 may take into consideration classifying maps, which identify sets of “OK” acceptable frames or sets of “NO” unacceptable frames, and assess whether the acquired frame is included in one of the classifying maps having acceptable frames, or in another of the maps having unacceptable frames.
- the discontinuity in the acquired data flow is detected if one of the frames of the sequence belongs to an unacceptable frames classifying map.
- the classifying maps are defined during a learning step which precedes the operating step of packaging machine operation.
- the second component 506 in order for the second component 506 to be able to effectively detect the discontinuity in the video, or audio video, data flow acquired during an operating step of packaging machine operation, it is necessary to ensure that said operating step is preceded by the learning step.
- the acquisition device 501 is configured for acquiring video, or audio video, streaming data flows in a controlled way, so that the processing device 502 can process them by means of the first component 505 and a configuring module (non illustrated) of the second component 506 prepared for configuring the classifying maps a posteriori.
- the classifying module can request the decision of an operator for each acquired streaming data flow during the learning step.
- the configuring module can submit each acquired data flow to an operator and receive indications about the acceptability, or lack of acceptability, of the data flow and therefore of the N frames acquired.
- the operator can assess the presence or absence of a discontinuity capable of introducing a defectiveness in the products P being processed in the packaging machine 1 and/or in the use of the materials M used to make the products P.
- the learning step it may be the operator who assesses each streaming data flow and classifies it as“OK” or“NO”.
- automatic methods may exist for configuring without a supervisor, which, during the learning step, are supplied with data flows that refer exclusively to correct operation of the packaging machine 1 and in which the deviation from them is assessed in statistical terms. Such automatic methods are faster, but less reliable.
- the configuring module can configure classifying criteria, identifying and aggregating sets of “OK” acceptable frames or sets of“NO” unacceptable frames, taking into consideration all of the frames of all of the streaming data flows, as acquired, and defining a reference data flow by aggregating the a posteriori assessment of all of the streaming data flows of the learning step, by the operator or by the automatic configuration methods.
- the reference data flow is constructed a posteriori and may relate to packaging machine 1 operation which is correct, or incorrect if the decision of the operator is requested, or only correct if automatic configuration methods without a supervisor are used.
- the classifying criteria are classifying maps which define boundaries between acceptance and non-acceptance spaces and which may be acceptance and non-acceptance maps.
- the assessing module 506 is capable of assessing each frame, automatically, verifying which classifying map the frame belongs to.
- each frame of the streaming data flow with the reference data flow, defined by means of the classifying maps, it is possible to identify the discontinuity in the acquired data flow. For example, there is discontinuity when a frame of the N frames of the streaming data flow belongs to a non- acceptance map.
- classifying module 506’ of the second component 506 may even alternatively be produced using Neural Network or Random Forest type algorithms which do not necessitate classifying maps.
- the operating step of packaging machine operation must be preceded by the learning step, in which the operator assesses each streaming data flow and classifies it as“OK” or “NO”, so that the assessing module 506” can define the classifying criteria necessary so that subsequently, during the operating step of operation, the processing device 502 can classify each frame as acceptable, or not, in automatic mode. That is to say, even with different types of algorithms, it is necessary to define the reference data flow by means of the learning step in order to allow identification of the discontinuity in the acquired data flow.
- the processing device 502 is configured for correcting the discontinuity detection logic by correcting the mathematical model, for example, for that purpose the classifying module 506’ and/or the assessing module 506” may be modified in order to carry out an even more precise assessment of the streaming data flow.
- the acquisition device 501 , the processing device 502, the long-term memory 503 and the temporary memory 504 of the control unit 5 are connected to each other and are also connected to the control apparatus 4 by means of a communication network 6, for example the Internet or the factory LAN (Local Area Network).
- a communication network 6 for example the Internet or the factory LAN (Local Area Network).
- a component may be, without limitation, a process being executed on a processor, a processor, a hard disk unit, multiple storage units (optical or magnetic storage media) including a solid state storage unit; an information technology data item; a software program which can be run by a computer and/or a computer and/or a hardware calculation unit as a programmable hardware component or an application in the internet cloud.
- both an application being run on an information technology server and the information technology server itself may be considered a component.
- One or more components may reside in an execution process and a component may be located on a computer and/or distributed between two or more computers.
- the components as described herein may be executed by various computer-readable storage media having various data structures saved on them.
- the long-term memory 503 usually made using a set of hard disks, even SSD, could be saved in an Internet cloud and not physically reside in the same information technology component in which the processing device 502 resides.
- first component 505 and the second component 506 of the processing device 502 may reside on different hardware specially designed in order to obtain distributed processing for an efficient analysis of the video, or audio video, streaming or in the same hardware, but in processes which are separate from each other.
- the packaging machine comprises the sterile processing unit in order to guarantee that the food product P is packaged in a sterile way in a protected atmosphere.
- the discontinuity of interest, for a machine of that type may not be linked to a possible jamming of the materials with consequent packaging machine stop, as could occur in a packaging machine for smoking articles, but may be linked to events which could cause a loss of sterility in the packaging of the consumer food products P, for example a poor quality of the longitudinal and/or transversal seals.
- the packaging machine comprises the filling unit for filling the continuous tube.
- the discontinuity of interest detected may be linked to an incorrect filling due to excessive pressure used when pouring the pourable food product into the continuous tube.
- the processing by the first component 505 and the second component 506 of the processing device 502 there may be detection of a discontinuity in the video flow of the sterile processing unit, and/or of the filling unit, and/or of the longitudinal sealing unit, and/or of the forming and separating unit and therefore it may be possible to avoid a subsequent rejection of products packaged in an atmosphere which is no longer protected, with a significant increase in production efficiency.
- - acquiring a data flow, in particular in the form of video, or audio video, streaming relating to the products P being processed in the packaging machine 1 and/or to the materials M used to make those products;
- the method also comprises the further step of saving the acquired data flow in a temporary memory 504, for example a RAM memory.
- a temporary memory 504 for example a RAM memory.
- control method may comprise the further step of defining the reference data flow by means of a data flow previously saved in the temporary memory 504, for example relating to correct operation of the packaging machine 1 , and/or receiving a reference start setting instruction from an operator and defining the reference data flow by means of the data flow acquired following that start setting instruction.
- the reference data flow may be that which, during a predetermined packaging machine 1 operating time period, corresponds to a maximum production efficiency of the packaging machine 1 . If the packaging machine 1 is kept under observation for a time period which is longer, for example, than the predetermined period, amongst all of the acquired data flows it may be possible to select as the reference data flow the acquired data flow which corresponds to the best performance of the packaging machine 1 itself.
- the processing device 502 may comprise the second component 506 dedicated to executing the mathematical model for detecting the discontinuity which necessitates the learning step
- the reference start setting instruction may even correspond to the start of the learning step, since the data flows acquired during the learning step all contribute to defining the reference data flow to be considered during the packaging machine operating step.
- each acquired data flow may be saved in the temporary memory 504.
- the step of comparing the acquired data flow with a reference data flow comprises a first step of analysing the data flow, for detecting at least one metric of the data flow, preferably a plurality of metrics of the data flow, and a second step of detecting the discontinuity for analysing the metric detected during the first step and assessing the discontinuity in the acquired data flow relative to the reference data flow defined during the learning step, preceding the operating step of packaging machine operation.
- the first analysing step comprises converting the video streaming data flow, comprising the sequence of N frames acquired in sequence, and obtaining a vector field 505” indicating, for each frame, a direction of a movement and an intensity of the movement of a plurality of uniform zones of pixels relative to a base frame. That converting step may be carried out by means of the Optical Flow algorithm.
- the first analysing step additionally comprises a statistical analysis step, following the converting step, for analysing the vector field 505” and for identifying at least one significant parameter of each frame N, preferably a plurality B of significant parameters of each frame, that is to say, the so- called metrics.
- the content of each frame is converted and simplified, by detecting a plurality of metrics which summarise the information content of the frame itself.
- the second step of detecting the discontinuity comprises a classifying step, for classifying the metric, or the plurality B of metrics of each of the frames N of the streaming data flow and an assessing step, following the classifying step, for assessing whether each frame of the sequence should be considered acceptable, that is to say,“OK”, or unacceptable, that is to say, “NO” based on the classification of each metric.
- the assessing step comprises comparing each frame of the acquired streaming data flow with a reference data flow defined considering classifying maps, which identify sets of “OK” acceptable frames or sets of “NO” unacceptable frames, and assessing whether the frame is included in one of the classifying maps having acceptable frames, or in another of the maps having unacceptable frames.
- the assessing step also comprises assessing the discontinuity in the acquired data flow if one of the frames of the sequence belongs to an unacceptable frames classifying map.
- the method also comprises the step of defining the classifying maps during the learning step which precedes the operating step of packaging machine operation.
- the method comprises the step of acquiring video, or audio video, streaming data flows, in a controlled way for allowing processing by means of the first analysing step and for obtaining the plurality of metrics and a configuring step for configuring the classifying maps a posteriori, after having obtained for all of the streaming data flows acquired during the learning step the decision from an operator, or from automatic configuration methods.
- the method according to this invention may comprise classifying the metrics of each of the frames N of the streaming data flow and, subsequently, submitting the acquired streaming data flow to an operator for receiving indications about the acceptability, or lack of acceptability, of the data flow and therefore of the N frames acquired.
- the method according to this invention may comprise acquiring streaming data flows which all relate to correct operation of the packaging machine 1 , classifying the metrics of each of the frames N of the streaming data flow and, subsequently, statistically analysing all of the acquired streaming data flows.
- the learning step comprises the further step of configuring the classifying maps, that is to say, the reference data flow, a posteriori, after the decision from the operator or the automatic configuration methods, identifying and aggregating sets of acceptable frames, or sets of unacceptable frames, taking into consideration all of the frames of all of the streaming data flows acquired during the learning step itself.
- the step of undertaking corrective actions may comprise the step of modifying the operation of the packaging machine, for example modifying operating parameters of the packaging machine, or activating a correction of a mathematical model for detection of the discontinuity.
- the acquisition of the video streaming relates to the filling unit for filling the continuous tube (or individual containers in other packaging machines)
- the discontinuity detected is an incorrect filling due to excessive pressure used when pouring the pourable food product into the continuous tube
- detection of the discontinuity in the streaming data flow that is to say, with respect to detection of a non-optimal start of filling (splashes or diversion of the filling flow)
- the correction may take place by adapting the mathematical model used and/or selecting a different mathematical model, for example selected from a library of mathematical models available, based on the metrics detected.
- the classifying module 506’ of the second component 506 of the processing device 502 may be modified so as to deactivate classifying by means of Support Vector Machine, activating that using Neural Networks, if the discontinuity in the streaming data flow is more easily identifiable using the latter.
- a control apparatus 4 for controlling the operating units 201 , 301 of the packaging machine 1 and a control unit 5 for controlling packaging machine 1 operation
- a control apparatus 5 for controlling packaging machine 1 operation
- the control apparatus 4 receives any warnings about video, or audio video, streaming data flow discontinuity from the control unit 5 and the control unit 5 receives warnings about events indicative of packaging machine 1 operation from the control apparatus 4, for example the initial event E1 of acquisition of the streaming data flow and the final event E2 of stopping the acquisition.
- the step of acquiring a data flow comprises the step of acquiring the sequence of images from the initial event E1 to the final event E2, asynchronously relative to the packaging machine 1 , at a periodic acquisition command, or synchronously or in phase with events indicative of that operation, for example cyclical operating events.
- control method also comprises the further step of identifying at least one portion of packaging machine 1 comprising a plurality of operating units for packaging the products P and/or processing the materials M, and positioning an acquisition device, 501 , configured for acquiring the video, or audio video, streaming data flow, facing the portion of packaging machine 1 identified, for example one of the operating units 201 , 301 , for associating the discontinuity detected with said portion.
- the streaming data flow may be only video or audio video, that is to say, multi-media.
- control method may comprise the step of positioning the acquisition device 501 in the same observation point in which an expert machine operator could be positioned for viewing packaging machine 1 operation at the portion of machine identified.
- the control method of this invention amounts to a smart supervising or monitoring of a complex operation such as that of a modern packaging machine with high production speed.
- the control method of this invention allows easy control of the operation of a packaging machine in a non-traditional way, by considering a viewpoint outside the packaging machine by means of acquisition of a video, or audio video, flow in which to detect anomalous discontinuities or irregularities. It should be noticed that, advantageously, the discontinuity detected may give rise to corrective actions on the packaging machine itself, but not only that, since the data flow may also advantageously be saved for an a posteriori control by an expert operator.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- General Engineering & Computer Science (AREA)
- Manufacturing & Machinery (AREA)
- Quality & Reliability (AREA)
- Containers And Plastic Fillers For Packaging (AREA)
- Basic Packing Technique (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
IT102019000007205A IT201900007205A1 (en) | 2019-05-24 | 2019-05-24 | Method of controlling an operation of a packaging machine and relative control unit |
PCT/IB2020/054889 WO2020240379A1 (en) | 2019-05-24 | 2020-05-22 | Method for controlling operation of a packaging machine and related control unit |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3977222A1 true EP3977222A1 (en) | 2022-04-06 |
Family
ID=67998587
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP20742875.6A Pending EP3977222A1 (en) | 2019-05-24 | 2020-05-22 | Method for controlling operation of a packaging machine and related control unit |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP3977222A1 (en) |
IT (1) | IT201900007205A1 (en) |
WO (1) | WO2020240379A1 (en) |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102013207139A1 (en) * | 2013-04-19 | 2014-10-23 | Krones Ag | Method for monitoring and controlling a filling installation and device for carrying out the method |
US9805460B2 (en) * | 2015-03-27 | 2017-10-31 | Rockwell Automation Technologies, Inc. | Systems and methods for assessing a facility based on audio/visual delta analysis |
JP6476228B2 (en) * | 2017-04-14 | 2019-02-27 | オムロン株式会社 | Industrial control device, control method, program, packaging machine, and control device for packaging machine |
-
2019
- 2019-05-24 IT IT102019000007205A patent/IT201900007205A1/en unknown
-
2020
- 2020-05-22 EP EP20742875.6A patent/EP3977222A1/en active Pending
- 2020-05-22 WO PCT/IB2020/054889 patent/WO2020240379A1/en unknown
Also Published As
Publication number | Publication date |
---|---|
WO2020240379A1 (en) | 2020-12-03 |
IT201900007205A1 (en) | 2020-11-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3741690A1 (en) | Packaging machine of products for pourable food preparations | |
CN108733010B (en) | Industrial control device, control method, recording medium, packaging machine, and packaging machine control device | |
US20180345520A1 (en) | Method for processing food products and food processing system with position tracking | |
US20140290180A1 (en) | Method and Production System for Producing and/or Packaging Cigarettes | |
US20140226892A1 (en) | Method and device for visually inspecting objects to be tested during the production and/or packaging of cigarettes | |
US20060231295A1 (en) | Production system | |
WO2020183973A1 (en) | Display system, display method, and display program | |
JP7239588B2 (en) | Method for defect detection in packaging containers | |
US20110019877A1 (en) | Method and Apparatus For Monitoring a Production Line | |
US7117872B2 (en) | Method and a device for managing the supply of wrapping and/or additional and/or auxiliary materials in a system for the manufacture of tobacco products | |
US20170152070A1 (en) | In-line label applicator | |
WO2018078613A1 (en) | System and method for automatic inspection and classification of discrete items | |
CN208580034U (en) | A kind of Multifunctional tobacco package detection device | |
CN104995112B (en) | For the method and apparatus for carrying out indicating fault when article is grouped | |
EP3977222A1 (en) | Method for controlling operation of a packaging machine and related control unit | |
US20230195083A1 (en) | Processing Line with Feed Forward Data Backlog Control and Method | |
WO2020183975A1 (en) | Display system | |
CN108333186A (en) | Go mouldy detecting system and its detection method based on the surfaces Yan Duo | |
JP7476179B2 (en) | A system for controlling the manufacturing process of packaging units | |
WO2020183974A1 (en) | Display system | |
EP3384788B1 (en) | Apparatus and method for rejection of defective rod-like articles from a mass flow of rod-like articles of the tobacco industry | |
KR102528505B1 (en) | Customized Autonomous Artificial Intelligence Platform Using Manufacturing Process Data | |
WO2018012969A1 (en) | A method of evaluating individualized compounds of medicines, a control device, a packaging device and a computer program product | |
CN104176331A (en) | Automatic mark replacement system for cigarette conveying equipment and detection method | |
EP3761132B1 (en) | A method for improving the efficiency of an automatic packaging machine |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20211216 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20240625 |