EP4202765A1 - Verfahren und system zum zählen von backbaren lebensmittelprodukten - Google Patents

Verfahren und system zum zählen von backbaren lebensmittelprodukten Download PDF

Info

Publication number
EP4202765A1
EP4202765A1 EP22215325.6A EP22215325A EP4202765A1 EP 4202765 A1 EP4202765 A1 EP 4202765A1 EP 22215325 A EP22215325 A EP 22215325A EP 4202765 A1 EP4202765 A1 EP 4202765A1
Authority
EP
European Patent Office
Prior art keywords
product
inspection area
food products
processor module
captured image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP22215325.6A
Other languages
English (en)
French (fr)
Inventor
Rafael JUAN FERNÁNDEZ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dulcesa SL
Original Assignee
Dulcesa SL
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dulcesa SL filed Critical Dulcesa SL
Publication of EP4202765A1 publication Critical patent/EP4202765A1/de
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06M—COUNTING MECHANISMS; COUNTING OF OBJECTS NOT OTHERWISE PROVIDED FOR
    • G06M7/00—Counting of objects carried by a conveyor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06M—COUNTING MECHANISMS; COUNTING OF OBJECTS NOT OTHERWISE PROVIDED FOR
    • G06M7/00—Counting of objects carried by a conveyor
    • G06M7/02—Counting of objects carried by a conveyor wherein objects ahead of the sensing element are separated to produce a distinct gap between successive objects
    • G06M7/04—Counting of piece goods, e.g. of boxes

Definitions

  • the present invention relates to the technical field of quality control on food product production lines and, more specifically, to computer vision systems for food products to be baked for the automatic counting and production control thereof.
  • Mass production poses a challenge in terms of process supervision, control and management, among other problems. Although many stages of the production chain are automated, other tasks that are among the simplest are usually still carried out manually, which, in addition to the natural errors introduced by human intervention, entails an interruption in the information chain that feeds the control systems that are increasingly widespread in the industry, based on Big Data technologies and artificial intelligence.
  • mass-produced bakeable food products under very specific manufacturing parameters such as baking time, are continuously introduced by means of conveyor belts into a baking module.
  • This process usually requires control personnel to confirm correct operation, which is done visually (with the accuracy that the operator can guarantee) or, sometimes, it is directly omitted and quality control of the already baked product is postponed. In both cases, the efficiency and ability to respond to errors is low.
  • the present invention describes, in a first aspect, a method for counting bakeable food products that comprises the following steps:
  • applying, by the processor module, an HSV conversion to the captured image is envisaged. More specifically, one embodiment envisages applying, by the processor module, a conversion to black and white of the captured image. More specifically, one embodiment comprises applying, by the processor module, a colour mask on the captured image which eliminates any intermediate value between black and white.
  • the colour transformations allow the background to be removed, which results in a simpler and better-quality image processing.
  • the processing of the images further comprises applying, by the processor module, a blur filter on the captured image. More specifically, one of the embodiments envisages applying, by the processor module, an erosion filter on the captured image. More specifically, one of the embodiments envisages applying, by the processor module, a dilation filter on the captured image.
  • the filters applied to the captured images make it possible to eliminate small defects in the conversion or enhance the shape of the product for better definition.
  • the pre-established minimum threshold of identified pixels to determine the presence of the product is set to at least one third of the width of the captured image.
  • the presence of a product in the inspection area if the number of identified pixels is greater than a pre-established minimum threshold, it must also be verified that the number of identified pixels is greater than said threshold in a certain number of successive readings.
  • the presence of the product is correct and is not due to a false positive caused by some impurity or loose paste stain on the conveyor belt.
  • the present invention relates to a system for counting bakeable food products, which comprises the following elements:
  • an LED lighting module arranged to illuminate the inspection area is envisaged.
  • the system is independent of external lighting conditions and can work under more homogeneous conditions.
  • One specific embodiment of the invention further comprises a conveyor belt for conveying bakeable food products to an oven, wherein the optical sensor is arranged on the conveyor belt along an axis perpendicular to the conveying direction. More specifically, it is envisaged that the optical sensor comprises hinged fastening means that can be coupled to a support structure, to adjust the inspection area along a line perpendicular to the conveying direction.
  • the processor module is a Raspberry Pi computer board and the optical sensor is a Raspberry Pi camera module.
  • Figure 1 shows a perspective view of one of the preferred embodiments of the system of the invention for recognising and counting bakeable food products ( 1 ) arranged on a conveyor belt ( 2 ).
  • Figure 2 shows a perspective view of one of the preferred embodiments of the system of the invention for recognising and counting bakeable food products ( 5 ) arranged on a conveyor belt ( 6 ).
  • a hinged part ( 7 ) fastens the casing ( 3 ) to a support structure ( 8 ), so that the inclination of the casing can be oriented and thereby adjust the image captured by the optical sensor and, therefore, the inspection area.
  • Figure 3 shows a perspective view of one of the preferred embodiments of the system of the invention for recognising and counting bakeable food products arranged on a conveyor belt.
  • the casing also has an LED lighting device ( 9 ).
  • a guide system ( 10 ) can be used to allow longitudinal displacements of the casing along the guide ( 10 ) to vary the inspection area on different points of the conveyor belt.
  • the casing can be fastened by means of screws to a fixed element of the structure.
  • the casing can be coupled to a rod or bar, with a square or round section, as long as they allow the casing to be displaced transversely to the conveying direction of the conveyor belt.
  • the LED lighting module is powered by the GPIO bus at 3.3V or 5V, depending on the needs, or even by an external power supply. With the connections made, the casing is placed in the inspection area for its operation.
  • the operation of the present invention is generally based on the fact that, after performing an HSV conversion of the image by applying colour filters, blurring, eroding, dilation of the pixels and their conversion to black and white, the pixels in a certain area are counted and, after a minimum margin of pixels has been exceeded, the determination is made that there is a product in said area and a product counter is updated.
  • Figure 4 schematically shows the main functional blocks of the invention, according to the main process. Following the flow chart represented, this process begins with the capture stage ( 11 ), in which the optical sensor acquires the images of the products; this is followed by a processing stage ( 12 ) for processing the captured images; after processing, a search stage ( 13 ) for searching for the product in the images occurs; and finally, a last stage of information transmission ( 14 ) is carried out.
  • Figure 5 outlines the steps of the processing stage ( 12 ) according to one of the embodiments wherein, once there is an image capture ( 15 ), an inspection area ( 16 ) is defined in which the product is to be searched. In general, the inspection area should be wide enough to accommodate a product, or a large part of it, preferably defined perpendicular to the product conveying line.
  • a first transformation of the image from RGB to HSV is performed ( 17 )
  • a colour mask is applied ( 18 ) so that any colour that does not fall within the given spectrum tends to 0 (black) and the rest tends to 1 (white), which will give a better contrast between product and background
  • a second transformation is applied to the resulting greyscale image ( 19 ). These colour space transformations will allow the background of the image to be removed.
  • a first blur or distortion filter ( 20 ), which blurs the image and eliminates small gaps in the product, allowing the product's transformation to black and white ( 21 ) to be more homogeneous; a second erosion filter ( 22 ), which eliminates loose pixels that do not correspond to the product image, making it possible to eliminate colour impurities in the background; and a third dilation filter ( 23 ), which increases the size of the white areas, contributing to a better marking of the product and a greater number of white pixels. After this process, only white pixels remain in the processed image, which correspond to the food product.
  • An inspection area that is more adjusted than the initial area is defined ( 24 ) and finally, the white pixels in the image in this adjusted inspection area are counted ( 25 ) to determine whether or not they are greater than a pre-established minimum threshold, which will define in the next stage whether or not the presence of the product in the image is confirmed.
  • a minimum threshold of white pixels can be established to determine the presence of the product close to the width of the image.
  • a value of at least a third or half of the width of the image is typically used to ensure that there will always be a number of pixels that ensures the presence of the product, avoiding failures due to deformation or poor positioning of the product prior to the image capture.
  • Figure 6 outlines the search stage ( 13 ) according to one of the embodiments.
  • the search for the product basically consists of evaluating whether certain requirements are met to determine that there is a product ( 29 ), which in this case are: exceeding a pre-established minimum number of pixels ( 26 ), exceeding a pre-established minimum number of readings ( 27 ) and exceeding a minimum time between product rows ( 28 ).
  • the minimum number of readings is the number of successive times that the white pixel count must be greater than the threshold. It is established to avoid erroneous product detections caused by impurities or loose paste stains on the conveyor belt.
  • the requirement of establishing a minimum time between rows seeks to avoid gaps within one same product and to avoid counting the same product twice. Additionally, one more criterion can be added to verify that, prior to detecting the presence of the current product, a gap has been detected ( 30 ) and, if so, the product counter is updated by one unit ( 31 ) and the search is concluded ( 32 ). To detect gaps, the flow chart takes a path that diverges from product detection.
  • the last stage of information transmission relates to configuring the entire loop of the diagram to determine whether data is to be sent, when it is going to be sent and how it is going to be sent.
  • the area where the camera is located can be defined and it can be coded according to factory parameters. For example, selecting when to send the information can be configured, as long as there is a minimum time gap between products, designed, for example, for discontinuous product cans or if the number of counted units is greater than the minimum entered. It can also be configured to send the histogram (total number of white pixels during the entire product selection process in all readings) for an analysis of results and subsequent adjustment of values and the sending of results by UDP protocol or for execution by a web page.
  • Figure 7 represents, by way of example, one of the images processed in one of the embodiments of the present invention to detect the presence and subsequently count bakeable food products.
  • the bottom ( 36 ) is represented completely in black (although in the figure it has not been coloured to optimise ink in a possible reproduction on paper), offering the maximum possible contrast with the white pixels, which represent the areas of the image occupied by the products ( 37 ).
  • FIG. 8 schematically represents the hardware and software structure of said installation, wherein all the control devices ( 80 ) distributed throughout the factory are located at the base and they communicate through their corresponding communication controllers ( 81 ) with a central server ( 82 ).
  • the central server in communication with the cloud ( 83 ), allows access from different platforms ( 84, 85, 86 ) to use the information collected at the source by the control devices in tasks of supervising, display, data logging, etc.
  • a PLC programmable logic controller
  • a PLC programmable logic controller
  • programmable automaton is an industrial computer that processes all the data of a machine, such as sensors, buttons, timers and any input signal, to subsequently control the actuators (pistons, motors, valves, etc.) and thus be able to control any industrial process automatically.
  • control devices ( 80 ) deployed by a smart factory with a computer vision system such as the one described by the present invention can integrate scales for different types of weighing during the process; metal detectors to alert the user of any metal objects intercepted by the detector's electromagnetic field, which automatically ejects the product from the production line; X-ray systems for detecting dense contaminants such as glass, metal, mineral stone, calcified bone and high-density plastic, regardless of their size, shape or location inside the product, which is automatically ejected in the case of detection following parallel operation to the metal detector; high-precision dynamic weighers to verify the weight of the product on the conveyor belt, unaffected by the vibrations of said belt and reliable so as to properly discard incomplete products at high speed; expiry date printers for continuously printing an expiry date on products; or Raspberry Pi boards with various functions adapted to each situation, such as temperature probes, computer vision or integration of new devices.
  • the communication controllers ( 81 ) are necessary to establish communication between the different control devices and the central server ( 83 ), for example of the OPC Unified Architecture type. Specific communication drivers are used for each piece of equipment to be able to recognise which devices are connected and thus make optimal use of them. To use the data collected by the control devices in the form of variables or registers on different data logging or display and supervising platforms, all the devices send the data via these drivers to the OPC UA server, wherein each driver is given a different configuration to achieve successful communication on one same platform with all the connected devices.
  • the platforms from which a user can access and process the information initially collected by the control devices can comprise, for example, a SCADA system ( 84 ), which is an industrial automation and control tool used in production processes that can control, supervise, collect data, analyse data and generate reports remotely through a computer application. Its main function is to evaluate the data in order to correct possible errors.
  • SCADA system 84
  • an alarm system in which certain variables to be controlled are received, whether they come from a PLC, databases or other integrated systems, where scripts can be created with conditions that, when met, activate an alarm. Selected users can be notified of the alarms via e-mail or telephone, although a sound or light system that can more immediately capture the attention of operators can also be implemented.
  • a smart factory like the one described herein also needs a database ( 86 ) between the platforms.
  • These factories often have an enterprise resource planning (ERP) system to address the business needs from a process standpoint and to integrate company-wide information systems.
  • ERP enterprise resource planning
  • the control devices ( 80 ) directly communicate with the ERP, through TCP, sending data that will be stored in a table of said database ( 86 ).
  • a scale would send the line on which it is working, the ordinal of said scale, the moment of sending, the value of the weight, the established maximum and minimum, and all the information that the user needs to save.
  • Communication with this database ( 86 ) is two-way, such that variables can be created from a query to the database and these variables can be processed, for example, in a SCADA system as described above. Therefore, a log of all the necessary information is obtained to subsequently process it according to the needs.
EP22215325.6A 2021-12-23 2022-12-21 Verfahren und system zum zählen von backbaren lebensmittelprodukten Pending EP4202765A1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
ES202131210A ES2944786A1 (es) 2021-12-23 2021-12-23 Metodo y sistema para contar productos de alimentacion horneables

Publications (1)

Publication Number Publication Date
EP4202765A1 true EP4202765A1 (de) 2023-06-28

Family

ID=84887648

Family Applications (1)

Application Number Title Priority Date Filing Date
EP22215325.6A Pending EP4202765A1 (de) 2021-12-23 2022-12-21 Verfahren und system zum zählen von backbaren lebensmittelprodukten

Country Status (2)

Country Link
EP (1) EP4202765A1 (de)
ES (1) ES2944786A1 (de)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6233966B1 (en) * 1997-03-03 2001-05-22 L'air Liquide, Societe Anonyme Pour L'etude Et Exploitation Des Procedes Georges Claude Freezing tunnel
US20210321820A1 (en) * 2020-04-15 2021-10-21 Air Products And Chemicals, Inc. Sensor Device for Providing Control for a Food Processing System

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2812086B1 (fr) * 2000-07-18 2003-01-24 Air Liquide Procede et dispositif de mesure du taux d'occupation sur un tapis convoyeur notamment d'un tunnel cryogenique de produits transportes par ce convoyeur
FR2812380A1 (fr) * 2000-07-25 2002-02-01 Air Liquide Tunnel cryogenique pour la refrigeration de produits notamment alimentaires, equipe de deflecteurs de gaz de refrigeration
FR2860068A1 (fr) * 2003-09-23 2005-03-25 Air Liquide Procede et installation de determination d'une quantite de produits alimentaires transportes
US20110265492A1 (en) * 2010-04-28 2011-11-03 Newman Michael D Freezer with cryogen injection control system
CN104537671B (zh) * 2015-01-04 2017-12-29 长沙理工大学 一种基于机器视觉的香烟滤棒在线计数和质量检测方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6233966B1 (en) * 1997-03-03 2001-05-22 L'air Liquide, Societe Anonyme Pour L'etude Et Exploitation Des Procedes Georges Claude Freezing tunnel
US20210321820A1 (en) * 2020-04-15 2021-10-21 Air Products And Chemicals, Inc. Sensor Device for Providing Control for a Food Processing System

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KHEKARE GANESH ET AL: "Design of Fruit Segregation and Packaging Machine", 2020 INTERNATIONAL CONFERENCE ON COMPUTATIONAL PERFORMANCE EVALUATION (COMPE), IEEE, 2 July 2020 (2020-07-02), pages 709 - 714, XP033828348, DOI: 10.1109/COMPE49325.2020.9199986 *

Also Published As

Publication number Publication date
ES2944786A1 (es) 2023-06-23

Similar Documents

Publication Publication Date Title
US11722642B2 (en) Machine-vision system and method for remote quality inspection of a product
CN110567974B (zh) 一种基于云端人工智能表面缺陷检测系统
US20230011901A1 (en) Systems and methods for anomaly recognition and detection using lifelong deep neural networks
CN107133725B (zh) 一种电路板的自动化生产管理系统
CN115144399B (zh) 一种基于机器视觉的装配质量检测方法及装置
CN111591715A (zh) 一种皮带纵向撕裂检测方法及装置
CN104931505A (zh) 机器视觉表面检测系统
CN110856437B (zh) 一种smt生产过程控制图模式识别方法
EP4202765A1 (de) Verfahren und system zum zählen von backbaren lebensmittelprodukten
US20220284699A1 (en) System and method of object detection using ai deep learning models
CN113386131A (zh) 一种全自动测试搬运系统及其控制方法
CN115993366B (zh) 基于传感设备的加工件表面检测方法及系统
CN108898576A (zh) 一种智能机器视觉筛选平台
US20230281536A1 (en) Work management device, work management method, and non-transitory computer-readable medium
CN117358615B (zh) 一种自动喷码印刷缺陷检测方法及系统
WO2022158216A1 (ja) 作業管理装置、作業管理方法、および作業管理プログラム
Demčák et al. SMART Identification by Vision System
Fang Application of XueJie vision system in the workpiece sorting of Huibo robot
CN113366528A (zh) 检查系统
JPH0230065B2 (de)
Chung et al. DVQI: A Multi-task, Hardware-integrated Artificial Intelligence System for Automated Visual Inspection in Electronics Manufacturing
Shaloo et al. Flexible automation of quality inspection in parts assembly using CNN-based machine learning
CN116601653A (zh) 作业管理装置、作业管理方法以及作业管理程序
JP2022113099A (ja) 作業管理装置、作業管理方法、および作業管理プログラム
KR20230058743A (ko) 마스킹 기반 딥러닝 이미지 분류 시스템 및 그 방법

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20231030

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR