WO2022231781A1 - Procédé et appareil de détection de défauts pour classement de morceaux de volaille - Google Patents

Procédé et appareil de détection de défauts pour classement de morceaux de volaille Download PDF

Info

Publication number
WO2022231781A1
WO2022231781A1 PCT/US2022/023268 US2022023268W WO2022231781A1 WO 2022231781 A1 WO2022231781 A1 WO 2022231781A1 US 2022023268 W US2022023268 W US 2022023268W WO 2022231781 A1 WO2022231781 A1 WO 2022231781A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
piece
defect
inspection system
generating
Prior art date
Application number
PCT/US2022/023268
Other languages
English (en)
Inventor
Theodore F. Bayer
Randall E. Wilcox
Original Assignee
Syscom Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/699,093 external-priority patent/US11599984B2/en
Application filed by Syscom Inc. filed Critical Syscom Inc.
Publication of WO2022231781A1 publication Critical patent/WO2022231781A1/fr

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/34—Sorting according to other particular properties
    • B07C5/342—Sorting according to other particular properties according to optical properties, e.g. colour
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/36—Sorting apparatus characterised by the means used for distribution
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C2501/00—Sorting according to a characteristic or feature of the articles or material to be sorted
    • B07C2501/0081—Sorting of food items

Definitions

  • the present disclosure relates generally to a method and system for inspecting an item and, more specifically, determining defects in a food item.
  • the primary method of detecting visual defects on a piece of poultry involves poultry processing line inspectors manually inspecting a piece of poultry in a batch on a table or while the piece moves down the processing line. Some defects are visible from the top view of a piece moving down the processing line, but others require picking up the piece and turning it over to view both sides or obtain a much closer view for small defects. With the targeted processing volumes of poultry processing lines, manual visual inspection requires many inspectors that are unable to inspect but only a percentage of pieces moving down the line. Therefore, defects may be missed.
  • An automated inspection system and method for identifying defects on an item such as poultry pieces is set forth.
  • the detecting system uses the combination of a semi-automatic continuously cleaned transparent conveyor belt allowing 360° piece image capture, an analytics pipeline leveraging image processing, deep learning image classification and object localization/detection technologies, and a customizable decision pipeline leveraging the defect features extracted by the analytic pipeline to grade and inform poultry piece human or automated remediation with visual imagery and/or structured information about the defects.
  • a poultry piece is placed on a processing line conveyor belt either manually or from upstream processing.
  • the pieces are spaced and are directed to the center of a belt and then transferred to a transparent belt before entering the analytics enclosure.
  • the transparent belt will move continuously to carry pieces through the analytics enclosure for 360° image capture and then transferred back to a conventional belt exiting the enclosure.
  • a semi-automatic belt cleaning system removes carryback particulates by spraying/rinsing the belt continuously using peracetic acid solution and/or followed by an air knife to reduce bottom image capture visual disturbance from liquid remaining on the belt.
  • Use of a transparent belt in this manner is novel in this industry. Transparent belts are used for back-lighting but viewing through the belt has heretofore not been employed.
  • the fields of view capture images from the top of the belt with a camera/light or electromagnetic radiation (EM) array and from the bottom through the belt with a camera array/light array.
  • EM electromagnetic radiation
  • Each FOV camera/lighting or EM array is triggered using a photo sensor trigger as the piece enters the FOV.
  • a photo sensor will trigger the image capture for each FOV.
  • a system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions.
  • One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.
  • One general aspect includes a method of inspecting a chicken piece.
  • the method also includes generating an image for the chicken piece.
  • the method also includes identifying a defect type, a defect location and an area of each defect on the chicken piece based on the image.
  • the method also includes grading the chicken piece into one of a plurality of grades based on the defect type and the area.
  • Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
  • Implementations may include one or more of the following features.
  • the method where generating the image may include generating the image from within an enclosure. Generating the image may include generating the image when the chicken piece enters a field of view of an image device. Generating the image may include generating a first image for a first side of the chicken piece and a second image for a second side of the chicken piece. Generating the image may include generating a first image for a first side of the chicken piece and generating a second image for a second side of the chicken piece through a transparent conveyor belt. Prior to generating a second image for the second side of the chicken piece through the transparent conveyor belt, cleaning the transparent belt.
  • Generating the image may include generating a first image for a first side of the chicken piece, a second image for a second side of the chicken piece, a third image for the first side of the chicken piece and a fourth image for the second side of the chicken piece.
  • Generating the image may include generating a first image for a first side of the chicken piece with a first image device using a first gain, a second image for a second side of the chicken piece with a second image device using a second gain, a third image for the first side of the chicken piece with a third image device using a third gain and a fourth image for the second side of the chicken piece with a fourth image device using a fourth gain, said first gain different than the third gain and the second gain is different than the fourth gain.
  • Identifying the defect type, the defect location and the area of each defect on the chicken piece may include identifying the defect type, the defect location and the area of each defect on the chicken piece deep learning image classification and deep learning object detection.
  • the method may include sorting the chicken piece in a sorting system based on the grade.
  • the method may include communicating the chicken piece from a sorting system to a remediation system based on the grade.
  • the method may include determining a piece type based on the image.
  • the method may include sorting the piece in a sorting system based on the grade and piece type.
  • One general aspect includes an inspection system for inspecting an item.
  • the inspection system also includes a conveyor belt for moving the item thereon.
  • the system also includes a first image device generating a first image signal of the item from a first field of view.
  • the system also includes a second image device generating a second image signal of the item from a second field of view.
  • the system also includes an electromagnetic source disposed within the enclosure directing electromagnetic radiation to the first field of view and the second field of view.
  • the system also includes a controller coupled to the first image device and the second image device generating a numerical identifier based on the first image signal and the second image signal.
  • the system also includes a display displaying an indicator based on the numerical identifier.
  • Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
  • the inspection system may include an enclosure disposed around the first field of view and the second field of view.
  • the conveyor may include a transparent conveyor belt and where first image device is disposed on a first side of the conveyor belt and the second image device is disposed on a second first side of the conveyor belt.
  • the inspection system may include a cleaning system cleaning the transparent conveyor belt.
  • the first field of view is aligned with the second field of view.
  • the electromagnetic source may include a first portion disposed on a first side of the conveyor belt and a second portion disposed on a second side of the conveyor belt.
  • the cleaning system may include an air knife and a bath.
  • the electromagnetic source may include a visible light system.
  • the inspection system may include a display device coupled to the controller, said display device generating an image of the item and display indicia identifying a surface defect.
  • the first image device may include a first gain and the second image device may include a second gain different that the second gain.
  • the second field of view is spaced apart from the first field of view.
  • the controller generates the numerical identifier based on the first image signal, the second image signal, the third image signal and the fourth image signal.
  • the second image device and the fourth image device are disposed on opposite sides of the conveyor belt as the first image device and the third image device, the first field is aligned with the second field of view and the third field of view is aligned with the fourth field of view.
  • the inspection system may include a sorting system sorting the item based on the numerical identifier.
  • the controller determines the numerical identifier by determining an area of a defect.
  • the controller determines the numerical identifier by determining areas of a plurality of defects and summing the areas to form the numerical identifier.
  • the controller determines the numerical identifier by determining an area of a defect raised to an exponent.
  • the inspection system may include a user interface for changing the exponent.
  • the item may include a poultry piece and where the numerical identifier may include a surface defect.
  • the surface defect may include a filament and cluster of filaments.
  • the surface defect may include at least one of a dermatitis, scabby, and gore.
  • the surface defect may include decolorization.
  • the surface defect may include rods or feathers.
  • the surface defect may include white roots or black roots.
  • the surface defect may include a matter of cut.
  • Fig. 1 is a level block diagrammatic view of the inspection system according to the present disclosure.
  • Fig. 2 is a detailed diagrammatic view of the imaging system of Fig. 1.
  • Fig. 3A is a diagrammatic view of a manual remediation system.
  • Fig. 3B is a diagrammatic view of an automated remediation system.
  • Fig. 4 is a high level block diagrammatic view of an example of the controller.
  • Fig. 5 is a high level flowchart of a method for operating the inspection system.
  • Fig. 6 is a detailed diagrammatic representation of the control modules of the PLC and the I PC.
  • Fig. 7 is a flowchart of a method for operating the batch module.
  • Fig. 8 is a flowchart of a method for operating the presentation module.
  • Fig. 9 is a flowchart for operating the image collection module.
  • Fig. 10 is a flowchart for operating the trigger module.
  • Fig. 11 is a flowchart for operating the piece control module.
  • Fig. 12 is a flowchart of operating the remediation trigger module.
  • Fig. 13 is a flowchart of a method for operating the image capture module.
  • Figs. 14A-14C is a flowchart of a method for operating the analysis module.
  • Figs. 15A-15B is a flowchart for operating the decisioning module.
  • Figs. 16A-16B is a flowchart of a method for operating the single camera grade evaluation of Figs. 15A-15B.
  • Fig. 17A-17B is a flowchart of the method for operating the multi-camera grade evaluation of Figs. 15A-15B.
  • Fig. 18 is a flowchart of the grade bin determination of Fig. 1.
  • Fig. 19 is a representation of a grade threshold matrix.
  • Fig. 20 is a flowchart of a grade bin matrix.
  • Fig. 21 is a flowchart of a method for operating the visualized and robotic piece enablement module.
  • Fig. 22 is an image of a first chicken piece having edge filaments extending therefrom and coordinating data of the filament defect.
  • Fig. 23 is an image of a second chicken piece having a surface defect thereon.
  • module refers to an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and memory that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality.
  • ASIC application specific integrated circuit
  • processor shared, dedicated, or group
  • memory that executes one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality.
  • phrase at least one of A, B, and C should be construed to mean a logical (A or B or C), using a non-exclusive logical OR. Steps within a method may be executed in different order without altering the principles of the present disclosure. The following is described with respect to poultry pieces.
  • the system is constructed of components suitable for the harsh environment of food processing. In animal processing the environment is cold and cleaned often. Waterproof components or enclosures may be used to prevent damage and increase accuracy.
  • the word message is to identify an electronic signal comprising the specific data.
  • Various servers and processors communicate with the electronic signals to perform the various methods.
  • the inspection system 10 has a placement system 12 that is used for placing pieces for inspection.
  • the inspection system 10 may be used for inspecting food such as poultry pieces.
  • other types of pieces, including non-food pieces may be inspected.
  • the placement system 12 is used for placing the pieces onto a conveyor belt system 14.
  • the placement system may be a human system for placing the pieces onto the conveyor belt system in an agreed upon manner.
  • the placement system 12 may also be fully automated. That is, pieces may come in a container. In such a case, a robot or another type of device may position the pieces onto the conveyor belt system 14 from the container.
  • a singulator positions the pieces with a predetermined spacing onto the conveyor belt system 14.
  • the conveyor belt system 14 may be various sizes and operate at various speeds depending upon the desired operating conditions and the types of pieces to be inspected. Regulatory bodies may also dictate line speed for certain types of pieces such as food pieces.
  • the conveyor belt system 14 may be an opaque belt that forms an endless loop to provide the pieces to a transparent conveyor belt system 18 which convey the pieces to an imaging system 20.
  • the imaging system 20 is used for generating images of the pieces. Based upon the images from the imaging system 20, a conveyor belt system 22 receives the pieces and a sorting system 24 sorts the pieces into one or more grade bins 26 that have the pieces sorted therein or one or more remediation systems.
  • the remediation system may be a manual remediation 28A or an automated remediation system 28B that is automatically operated as will be described in further detail below.
  • indicia such as the location of the defects to be remediated may be displayed on a display 30. That is, some of the remediation systems 28 may require manual processing by humans and others may be automated. In either case, the location of the defect provided from the imaging system will allow either a human or an automated system to correct the defect or defects on each piece.
  • a controller 40 is used to control the overall processing and inspecting of the pieces.
  • the controller 40 is illustrated as a single component in Fig. 1. However, multiple controllers for controlling various portions of the system may be used. As will be described in more detail below, one or more programmable logic controllers (PLCs) and industrial personal computer (I PCs) may be used.
  • PLCs programmable logic controllers
  • I PCs industrial personal computer
  • the controller 40 may be microprocessor-based with various logic circuitry programmed to perform inspecting for a particular type of part.
  • the controller 40 may be controlled through a user interface 42.
  • the user interface 42 may be one or more of a switch, a dial, a button, a knob, a keypad, a keyboard, a microphone or a touch screen.
  • the user interface 42 allows a user to configure the inspection system 10, such as configuring the conveyor belt speed, the batch, the type of pieces to expect, the grading system and the like.
  • the user interface 42 also allows requesting numerical identifier data to be displayed such as the number of processed pieces, grading determinations, failures and remediation data.
  • the controller 40 may also be coupled to a display 44.
  • the display 44 may display various control parameters, defect data, processing data and other processing parameters of the inspection system 10.
  • the imaging system 20 includes an enclosure 50 that has an inlet opening 52 that is used for receiving pieces on the transparent conveyor belt system 18.
  • the enclosure 50 also has an exit opening 54 for the transparent conveyor belt system 18 leaving the enclosure 50.
  • the enclosure 50 provides an environment so that external electromagnetic radiation does not interfere with the imaging system 20.
  • the enclosure 50 in some examples, may not be required.
  • the enclosure 50 may be formed of various materials and have access openings to allow access to components therein.
  • the enclosure 50 has a first EM source 56 and a second EM source 58 disposed therein.
  • the first EM source 56 is disposed above the transparent belt and directs electromagnetic radiation on the part to be inspected.
  • the second EM source 58 is disposed below the transparent conveyor belt system 18 and directs the light therethrough toward the piece to be imaged.
  • Various types of electromagnetic radiation may be generated from the EM sources 56, 58 such as but not limited to visible light (from a visible light lighting system), infrared light (both near and far), ultraviolet light, radio waves and X-rays.
  • the wavelength of EM radiation may vary depending on the types of pieces and the types of defects being detected.
  • the EM sources 56, 58 are composed of a plurality of elements.
  • the elements may generate the same band of wavelengths or may generate various bands of wavelengths (equivalently frequencies) that, in combination, are used to illuminate the part to be inspected. For example, separate images at different wavelengths may be used to determine the presence of one or more defects.
  • the use of several light or EM sources used to obtain several images at different frequencies may be referred to as “multispectral”, in the case of more than one, but less than 10 bands of electromagnetic (EM) wavelengths are used, or “hyperspectral” if 10 or more bands of EM wavelengths EM frequency bands generated.
  • the transparent conveyor belt system 18 has a transparent belt 60 through which the electromagnetic (EM) radiation from the second EM source is transmitted to illuminate a piece 61 being inspected.
  • the transparent belt 60 receives the piece 61 from the placement system 12 as mentioned above.
  • the transparent belt 60 is routed using a plurality of rollers 62 and a motor 64.
  • the motor 64 may have an encoder 66 thereon.
  • the encoder 66 allows feedback as to the position of the transparent belt 60.
  • the position of the transparent belt 60 may be used for identifying the piece 61 being conveyed through the imaging system 20. That is, when the piece
  • the position of the encoder is used to identify the piece for remediation and tracking purposes.
  • the transparent belt 60 moves in the direction illustrated by the arrows 68.
  • the movement of the transparent belt 60 positions the pieces 61 to be inspected relative to imaging devices 70A, 70B, 70C and 70D, each of which has a unique identifier.
  • imaging devices 70A, 70B, 70C and 70D each of which has a unique identifier.
  • four imaging devices 70 are provided. However, fewer than four or more than four may be used depending on the complexity and size of the piece to be inspected.
  • the imaging devices 70A-70D are collectively referred to as the imaging device 70.
  • Each imaging device 70 may be formed of a camera that has a sensor therein.
  • the imaging device 70 may be a charged coupled device, a CMOS device or other electro-optical type of sensor used to generate an image signal.
  • the imaging devices 70 receive the wavelengths desired in later analysis.
  • Some imaging devices may receive many wavelengths, referred to as “multispectral imaging” in the case of less than 10 EMF bands, or “hyperspectral imaging”, in the case of more than 10 EMF bands.
  • the information from some of the EMF bands might not be useful in the analysis.
  • only the EMF bands that have been predetermined to be useful for identifying the defect type may be selected for use in the analysis.
  • the information from the EMF bands is used to perform the analysis, referred to as “multispectral analysis” in the example of using 10 or less bands, or “hyperspectral analysis” in the example of more than 10 EMF bands.
  • multiple imaging devices 70 may be used when an imaging device cannot receive all the desired wavelengths.
  • the image signal or signals may have data associated with such as the identifier of the imaging device, a gain setting, a wavelength identifier, and the encoder position of the belt.
  • the imaging devices 70 each have a field of view 72A- 72D, respectively.
  • the field of views 72A and 72B are aligned and capture images of opposite sides of the piece.
  • the fields of view 72C and 72D are aligned and capture images of opposite sides of the piece.
  • the imaging devices 70A and 70C are spaced apart and thus the fields of view 70A and 70C are spaced apart.
  • the imaging devices 70B and 70D are spaced apart and therefore their fields of view 72B and 72D are spaced apart.
  • the number of fields of view correspond to the number of cameras.
  • the gains above the transparent belt A first and third gain were different and a second and fourth gain of the image devices below the transparent belt were different — one high, one low).
  • the first and second image device gains were the same, the gains of the second and fourth image device were the same and the gains of the first and second image devices was different the gains of the third and fourth image devices. That is, one pair was high, and one pair was low.
  • the fields of view 72B and 72D extend through the transparent belt 60 to obtain images of the underside of the piece. Therefore, a clean transparent belt 60 allows the most accurate images to be captured.
  • a bath 74 is used for cleaning the transparent belt 60.
  • the bath 74 in this example, is an acid bath formed using peracetic acid.
  • the transparent belt 60 is routed within the enclosure 50 into the bath 74.
  • a belt cleaning system 76 such an air knife system is used to remove the liquid from the transparent belt 60.
  • the belt cleaning system 76 may also include the acid bath 74. Therefore, the image from the imaging devices 70B and 70D are free from false detections.
  • a photo trigger 78 triggers the imaging devices 70A-70D to generate an image when a piece disposed on the transparent belt 60 enters the respective fields of view 72.
  • the imaging devices 70 form an image of the piece being inspected with each of the field of views.
  • the EM sources 56, 58 may be one of a variety of types of EM sources.
  • various types of electromagnetic radiation may be generated from the EM sources 56, 58.
  • the electronic images and the electronic imaging signal generated by the imaging devices 70A-70D may correspond to the image based upon the type of electromagnetic radiation. For example, visible light, infrared, ultraviolet and x-rays are examples of suitable electromagnetic radiation.
  • Various image signals of a piece may be taken using different types of electromagnetic radiation to detect different types of defects.
  • the EM sources 58 may be flashed for image capture or illuminated constantly.
  • the sorting system 24 sorts the pieces based upon the images obtained from the imaging devices 70.
  • a photo trigger 80 may trigger the display 30 to display an image of the piece.
  • the piece may have indicia such as a location 82 highlighted for remediation of the piece.
  • the photo trigger 80 triggers the image of the piece with the location 82 of the defect highlighted as an overlay on the image.
  • a mask 84 that is described later is illustrated.
  • the mask 84 represents the area outside of which is not considered in the defect determination. The mask 84 may not be displayed.
  • a robot 90 receives coordinates of a detect and an identifier for the piece.
  • the robot 90 may manipulate the piece such as cutting or plucking and replace the piece on the conveyor 22.
  • the piece data may be communicated through an antenna 92 wirelessly to the robot 90.
  • the robot 90 may perform the appropriate remediation. Different types of machines may be used for different types of remediation of different types of defects.
  • the sorting system 24 routes the piece to the appropriate manual remediation system 28A or automated remediation system 28B.
  • Other types of systems may perform different types of remediation. That is, some may use a combination of hand remediation and automated or machine remediation or two different types of automated remediation. Because the coordinates of the defect, the size of the defect, the type of the defect and the type of piece on which the defect occurs is known, the defect may be quickly remediated. Other types of defects or a high severity of a defect may cause the piece to be rejected all together.
  • the controller 40 is illustrated in further detail.
  • the controller 40 is divided into two general functions including a programmable logic controller (PLC) 410 and an industrial personal computer (IPC) 412. While the names programmable logic controller and industrial personal computer are set forth, different types of controllers, different numbers of controllers and locations of controllers may be changed depending upon various system requirements.
  • the programmable logic controller 410 includes a conveyor belt controller 420, a belt cleaning controller 422 and an EM source controller 424.
  • the conveyor belt controller 420 may control the speed and position of the belt actuator motor 64 and therefore the transparent conveyor belt 60.
  • the belt cleaning controller 422 may activate the air knife when the belt is moving.
  • the EM source controller 424 controls the EM sources 56, 58 based upon the trigger 78.
  • the PLC module controls the movement and timing of the conveyor belt, the cleaning system and the electromagnetic source controller.
  • the I PC 412 receives signals from the image devices 70A-70D and communicate them to an image processor 430.
  • the image processor receives signals from the image devices that correspond to the top and bottom signals of a piece that is being inspected.
  • the image devices may have different gains set for the different positions. This may allow different types of defects to be observed.
  • An analyzing module 432 uses a convolutional neural network (CNN) model that allows for continuous improvement of the identification of defects and of the piece types.
  • CNN convolutional neural network
  • step 510 a poultry piece is placed on the conveyor belt system 14 by the placement system 12 with a specified space gap or space from other pieces. This step may be optional if direct placement onto the transparent belt system 18 is performed.
  • step 512 the poultry piece is centered and transferred to the transparent belt system 18 prior to entering the image capture enclosure 50.
  • step 514 once in the image capture enclosure 50, image signals of top and bottom views of the poultry piece are captured with the imaging devices and the EM sources 56, 58. An image signal for each field of view is obtained.
  • the images are transmitted to the IPC 412 and the analyzing module 432 analyzes each image.
  • the analyzing module 432 uses image filters and the convolutional neural network (CNN) models 434, and extracts information from each image related to the piece (poultry-piece features) as well as detailed information of each defect found ( defect-candidate features) on the piece.
  • the type of defect, the area of the defect, the location of the defect, the sum of defects and the like may be determined.
  • the extracted data from the image are passed to the decisioning module 436 where additional computations are performed and piece batch level customizable thresholds are applied to determine a piece status.
  • the piece status may include but is not limited to determining piece status such as a piece grade (0,1 ,2 ... n) or identify the piece as invalid (wrong piece type) or indicate the piece as reevaluate (needs to go through image capture again).
  • step 520 the piece status (grade/invalid/reevaluate pieces) is communicated to the programmable logic controller 410 to direct the appropriate conveyor belt controller 420 to control the dropout sort to occur based on the grade bin specified (targeted sort dropout) and the target encoder position (piece position on the belt).
  • step 522 the piece processing counts (total, by grade, invalid, reevaluate) will be appropriately adjusted based on the piece status and the count updated in the database 440 in the decisioning module 436.
  • step 524 the graded pieces requiring remediation are route the pieces to the appropriate conveyor line to the appropriate remediation system 28.
  • a visual or automated remediation process is invoked once the pieces are positioned in front of either remediator for manual processing or a mechanical device for automated processing. To aid in manual processing, the piece image will be presented to the remediator with defects marked in step 526.
  • the analyzing module 432 described above leverages various convolutional neural network (CNN) models 434.
  • CNN convolutional neural network
  • the CNN models 434 are built by using representative images (training sets) of poultry pieces with the targeted visual defects. For the image classification models those images are labeled with defect types visually seen in the image. For object localization/detection models, the defects are highlighted (annotated) on the images so that the CNN model 434 can learn to identify the location and size of the defect. Once the training set is labeled/annotated the CNN models are built. Each model is installed into the analyzing module 432 and is communicated to the production line industrial personal computer (IPC) 142.
  • IPC industrial personal computer
  • the IPC 142 is signaled at the agreed upon scheduled time to bring the Analytic Pipeline online for use.
  • poultry piece images will be saved with all the poultry-piece and defect-candidate features extracted from the analyzing module 432 and decision module 436.
  • a maintenance process may be run on a regular basis to generate a distribution report of key processed piece features with marginal or low confidence scores as well as a list for manual inspection.
  • the piece images identified in the manual inspection list may be visually inspected and actions taken to improve the confidence score if deemed appropriate. Actions could involve labeling the image (image classification model) or annotating the defects (object localization/detection model) then adding them to the appropriate training set for a future model version build.
  • the image processor 430 establishes a connection to each camera, loads the appropriate profile, and brings each camera online for image acquisition.
  • the analyzing module 432 which includes image processing, deep learning image classification, and object localization/detection tasks for applying the image filters, extracts both poultry- piece features and defect-candidate features from the piece image.
  • the analytics process is performed.
  • the decision module 436 determines the presence of defects. The method processes both the poultry piece features, and defect candidate features collected, resulting in a piece grade determination which is communicated a final event structure database 440.
  • the sorting module 438 may sort the pieces based on the defects.
  • the controller 40 is coupled to the final event structure database 440 that is used to store various data and other numerical identifiers including but not limited to a bin number, the encoder position of the piece that is used as a piece identifier, images from each imagining device, area measurement, coordinates of the defects, perimeter coordinates of the piece, a center X/Y reference point, counts and the like.
  • a batch control module 610 is set forth.
  • the batch control module 610 is a component that provides generates a screen display for the user interface 42 for setup of a batch for processing.
  • the batch control module 610 enables manual starting/resuming and stopping/pausing of the processing system, and reports on batch processing statistics when complete.
  • the PLC 410 also includes a presentation module 612.
  • the presentation module 612 manipulates a group of chicken pieces and places them onto the middle of a moving conveyor belt, with a predetermined separation between the pieces.
  • An image collection module 614 in the PLC 410 communicates the batch number and piece type to an image capture module 622.
  • the batch started/stopped status for camera online/offline processing may also be provided.
  • the image collection module 614 also enables/disables power to the physical cameras and lighting or EM sources.
  • a piece trigger module 616 of the PLC 410 generates a trigger signal from the trigger 78 when the piece is in the field of view (FOV) 72 and ready for capturing of the image.
  • FOV field of view
  • a piece control module 618 controls the transparent conveyor belt system 18 to carry the piece through the light controlled enclosure 50 for image capture and controls the disposition of the piece once the decisioning module 432 has determined the piece grade bin.
  • a belt cleaning control system that controls the belt cleaning system 76 such as the air knife that cleans the belt on a continuous basis.
  • the visualize piece trigger module 620 generates a signal that is communicated to the piece control module 618 when a piece requiring remediation has been routed to the appropriate remediation system 28 and will present the image of the piece with all defect areas visualized on the display 630.
  • the I PC 412 has the image capture module 622 described above.
  • the image capture module 622 manages the imaging devices 70 for the capture of an image signal corresponding to an image of the piece.
  • the image signal is communicated to an analyzing module 432 for defect analysis.
  • the analyzing module 432 and provides the information to the decisioning module 436.
  • the decisioning module 436 implements the decision module 436 and notifies the piece control module 618 to direct the piece to the grade bin assigned.
  • a visualize/mechanical remediation enablement module 628 implements the visualization and remediation method described in further detail below.
  • a flowchart for operating the batch control module 610 is set forth.
  • the piece type is looked up or entered by identifying the batch number in step 710.
  • the user interface 42 may be used to enter the piece type through prompts displayed on the display 44.
  • the piece type and the batch number are provided to the image collection module 614. The system is then ready to start processing.
  • a start signal is communicated to the presentation module 612 from the batch control module 610.
  • a desired speed may be entered or previously provided by the batch number.
  • the start signal is communicated to the start module to start processing at the desired speed.
  • step 716 when a stop signal is requested by the operator by hitting a stop button or communicating a stop signal through the user interface 42, a stop signal is communicated to the other modules in step 718.
  • step 720 when the stop signal is not requested, step 720 is performed.
  • step 722 displays a count such as a piece count, a grade count, a defect count, an incorrect piece count or re-evaluation count.
  • the piece count may correspond to the total pieces processed.
  • the grade count may provide a count of the number of pieces from a batch within each of the grades.
  • a defect count corresponding to the number of pieces for each defect may be provided.
  • An incorrect piece count or a re-evaluation count may also be provided for the number of pieces that were incorrect in a batch or pieces that needed remediation within a batch.
  • step 720 when a count is not requested, step 724 determines whether a pause has been requested. When a pause has been requested, a pause signal is communicated to the presentation in step 726. Pausing may be used to adjust the process or equipment.
  • step 724 when a pause has not been requested, the process repeats again in step 710.
  • step 726 when a pause signal is communicated to the presentation module 612 and the piece control module 618, step 728 is performed.
  • Step 728 is determined whether a resume signal has been requested. When a resume signal has been requested, the resume signal is communicated to the presentation module 612 and the piece control module 618 in step 730. When a resume has not been requested, step 728 is repeated.
  • the presentation module 612 controls the positioning of chicken pieces in the middle of the moving conveyor belt with a predetermined spacing therebetween.
  • a start signal is received.
  • the conveyor systems are provided with a signal to control the movement of the conveyor belts.
  • the placement system 12 of Fig. 1 may be operated to allow the singulator or other placement system to position the pieces on the conveyor belt at a predetermined speed and at a predetermined spacing.
  • step 816 determines whether a stop signal has been received. When a stop signal has been received, step 818 stops the conveyor system and the placement system.
  • the image collection module 614 communicates the batch number in pieces to the image capture module 622 as well as determining whether the batch has started and stopped for the imaging devices.
  • a start signal is received.
  • the start signal may have data such as a start flag, the batch number, the piece type and the processing speed.
  • the start signal is communicated to the imaging devices in step 912.
  • the electromagnetic radiation sources are started.
  • an image device online signal is communicated to the image capture module 622.
  • an imaging device offline signal is generated in step 920.
  • the imaging devices are powered down in response to the imaging device offline signal.
  • the electromagnetic radiation generating devices are powered down.
  • the piece trigger module 616 is module that is activated in response to the trigger within the enclosure 50.
  • step 1010 when an enclosure trigger signal is received, step 1012 communicates an image capture message with the encoder data of the transparent conveyor belt system 18 to the image capture module 622.
  • step 1010 when an enclosure trigger signal is not received, step 1010 repeats to await the next piece.
  • Fig. 11 the operation of the piece control module 618 is described.
  • the piece control module 618 operates the transparent conveyor belt system 18 that is used to carry the pieces through the enclosure 50 for image capture and ultimately the disposition of the piece once the decisioning module 436 determines the piece to be defect free, defective or an incorrect piece.
  • the belt cleaning system 76 such as the air knife is operated in response to the piece control module 618.
  • a start signal is received.
  • the transparent conveyor belt system is started at the desired speed in step 1112.
  • the belt cleaning system 76 is started.
  • the grade bin queue, and the visual inspection bin queues are initialized.
  • the sort piece message may include a batch number, encoder data and the grade bin number.
  • step 1118 When a sort piece message has not been received, step 1118 is again repeated.
  • step 1118 when a sort piece message has been received, step 1120 calculates the bin target encoder position. The bin target encoder position is calculated using the messaging encoder data and the distance constant associated with the specified grade bin number. Step 1122 pushes the encoder data and bin target position into the queue. In step 1124, the batch number and encoder data is pushed onto the visual inspection bin queue.
  • a method may be performed in which the sorting system is operated to route the piece to its desired position. Sorting maybe performed by placing various grade pieces into bins or containers, rejecting failed pieces or pieces of the wrong type.
  • step 1128 when a visualized trigger signal is not received, step 1128 is again performed.
  • step 1128 when a visualized trigger signal is received, the next piece to be visualized is obtained from the queue.
  • a visualized piece message is communicated to the visualize/automated piece module 628. The batch number and the piece number are all communicated in the visualized piece message.
  • the visualized piece trigger module 620 signals the piece control module 618 when a piece require remediation has been routed to a remediation system.
  • step 1210 when a remediation trigger signal has not been received, step 1210 repeats.
  • step 1210 when a remediation signal has been received, step 1212 receives a visual inspection bin number for the remediation device location to obtain the data for the next defective piece.
  • step 1214 a visualize message is communicated to the visualize/automated piece enablement module 628. This allows the visualize module 628 to present an image of the piece to be processed as well as the defect location, type of defect and the like.
  • step 1310 an online message is received. A batch number, piece type and processing speed may all be communicated in the online message.
  • a final evaluation structure (FES) batch record may be provided that includes the batch number, piece type, processing speed and a time stamp for starting the process.
  • FES final evaluation structure
  • step 1314 the imaging device is brought online. Each of the imaging devices may be brought online in response to the online message from step 1310.
  • step 1316 the encoder data is captured from the encoder 66 that moves with the transparent conveyor belt 60. The encoder signal from the encoder 66 may be obtained by the PLC so that the data is coordinated. The PLC may communicate the data to other modules.
  • an image signal is obtained for the piece.
  • An image signal may be generated at each of the imaging devices.
  • the image signal and the imaging device identifier, the batch and the encoder data are all associated.
  • the image is ultimately communicated to the analyzing module 432.
  • the image along with the image signal data, such as the piece image reference, a camera identifier, a batch number and the encoder data, may all be communicated.
  • the analyzing module 432 uses multiple deep learning image classifier modules to determine if a piece is defective or requires upgrade remediation, to identify and grade the defects and provide the information to the decisioning module 436.
  • the image capture signal is received from the image capture module 622.
  • the image capture signal may include a piece image reference, an imaging device identifier and encoder data.
  • the piece image reference may provide an image for each of the imaging devices.
  • an image marking message may have an image node marked thereon.
  • the image node may be formed that includes the image reference, the batch number, the piece type, the camera identifier and the encoder data.
  • the encoder data is used to identify the piece by the position.
  • step 1414 the piece is classified as to piece type which may have a corresponding numerical identifier.
  • Various piece types may be classified such as wings, thighs, drumsticks, breasts and the like.
  • the piece type classifier that classifies the piece type may generate a numerical identifier score for each of the various types.
  • Step 1414 may use a deep learning image classification model that is invoked to extract the poultry-piece: piece-type feature by chicken piece deep learning image classification.
  • the processing batch is typically one-piece type.
  • step 1416 using the poultry-piece: piece-type feature, validation occurs to ensure the correct piece type is being processed based on the batch specified.
  • step 1418 is performed.
  • step 1418 if the incorrect piece type is found or the piece type is unknown, then poultry-piece: decision-status feature is set to “re evaluate” in step 1420 or “invalid” in step 1422 respectively.
  • the analytics module 432 sends a message containing the piece-type features to the decision module 436 in step 1490. No other Analytics Pipeline Method image processing is performed for “invalid” or “reevaluated” identified pieces.
  • poultry-piece perimeter-plot- coordinates
  • poultry-piece piece-area-size features.
  • step 1416 when the piece type with the highest scores equal to the current piece type, step 1430 is performed.
  • step 1430 a piece mask is generated.
  • defects There are many types of defects that may be analyzed for different types of pieces. In this example, a poultry piece is used. Some examples of the types of defects that may be detected in the present system and hair villi (filaments), rods (feathers), white and black root, inflammation, dermatitis, scabby, gore, decoloration such as yellow skin and a matter of cut.
  • the threshold for determining grade may be fixed or adjustable (an adjustable threshold). It may also be controlled by a governmental body.
  • the customer may be allowed to change the threshold at the user interface depending on the requirements of their client.
  • defects to be remedied include hair size verification requirement examples, villi (filaments), single villi > 0.5cm, overall number of villi (5 or more): any size, a noticeable tufts/clusters of villi, rods that cannot exist of any size, white/black root, hair roots cannot exist of any size.
  • a particular defect may be present or when compared to a threshold, is above the threshold.
  • Inflammation may be an example of a defect, that when present, cause the piece to be a candidate for remediation.
  • Another way to determine a defect is in pieces with multiple defects. Each defect score could be weighted, and the overall score compared to a reject threshold to determine if the piece needs remediation.
  • a small defect that alone would not trigger remediation, but may trigger remediation when found together with another small defect in one piece.
  • step 1434 it is determined whether an image is available for edge analysis.
  • Edge analysis is when the edge of the piece rather than the surface of the piece is determined.
  • the edge of the piece may be highlighted with a high gain of the imaging devices. This is suitable for detecting filaments or villi and other types of defects.
  • step 1436 the image is filtered.
  • the filter image steps below and including 1436 the image may be augmented in a variety of way to enhance detection. In this example, an optical density filter may be used. For some type of defects, no filtering may be needed.
  • the filaments or villi on the edge are determined. Step 1438 as well as the subsequent detection steps uses a deep learning object detection/localization model extracts the defect-candidate features found in the image.
  • All poultry-piece and defect-candidate features collected in step 1438 are added to an image marking message in step 1440 and wrapped with an analysis result message in step 1490.
  • the image marking message may have defect specific data/ numerical identifiers for each defect type.
  • the region subnode and the summary subnode may be included for each defect region found.
  • the region subnode may include but is not limited to region area, perimeter, X/Y center coordinates, and plot points.
  • the summary subnode may include max area, a count, area-sum, and an area sum raised to the nth power. All poultry- piece and defect-candidate features are sent to the decision module 436 instance for further processing in step 1492. All the data may be a numerical identifier for the part.
  • the surface defects may be classified in step 1442. That is, other types of defects on the surface of the piece are reviewed.
  • a deep learning image classification model is invoked to classify the defect types found on the piece in step 1442.
  • a list of found defect type(s) will be added to the poultry-piece: defects feature.
  • a specific deep-learning object localization/detection model is invoked.
  • two defect-candidate feature data will be generated: coordinates of the defect region location (defect-candidate: perimeter-plot-coordinates) and the area size (number of pixels) of the defect region (defect-candidate: area-size).
  • defect-candidate max-area feature: created with the area size of the largest defect region found
  • defect-candidate region-count feature that is the count of all the defect regions found
  • defect-candidate area-sum feature, that is the sum of all the defect regions areas found
  • defect-candidate area-size feature, particularly as a cumulative measure, is highly correlated with human (end customer) perception of defect significance, creating a useful method of determining a defect condition.
  • a measure is created that aligns more closely with the human perspective that fewer larger defect has a disproportionately greater negative impact on quality perception than more numerous small defects.
  • step 1443 when other types of defects are not present from step 1442, step 1490 is performed and an analysis result is obtained.
  • step 1443 when other types of defects are present, other defect types are analyzed.
  • step 1444 it is determined whether villi on the surface is present, and the characteristics of the surface villi are determined.
  • the image may be filtered in step 1446. Villi defects are determined in step 1448 using the filtered image.
  • step 1450 an image marking message is generated that has a villi surface as the defect type, the region sub node, such as the region perimeter, the X/Y center of reference and various pilot points.
  • a summary sub-node may be generated that has a count, an area sum, and an area sum raised to the nth power sum may be include in the image marking message data.
  • inflammation at the chicken piece is determined. For inflammation, dermatitis, scabbiness, gore, and yellow skin may be monitored. When inflammation is present (above an inflammation threshold), step 1456 is performed. In step 1456, the image is filtered or augmented as mentioned above. Inflammation defect data are determined in step 1458 using the filtered image with the classification described above. In step 1460, an image marking message having the inflammation defect type having the region sub node data and the summary sub node data for each region may all be determined.
  • step 1464 when the rods or feathers are present (above a rod candidate threshold), step 1466 is performed. Rods are the end of a feather so the two can be used interchangeably.
  • step 1466 the image is filtered or augmented as mentioned above.
  • step 1468 the rod surface and rod edge defect data may be determined.
  • step 1470 an image marking message may be generated with a rod defect type, the region subnode data and the summary subnode data described above for each rod identified.
  • step 1474 is performed.
  • Step 1474 presence of the matter of cut is determined by comparison to a matter of cut threshold.
  • step 1476 is performed in which the image is filter.
  • step 1478 the matter of cut data is detected.
  • an image marking message may be generated that includes the region subnode data and the summary subnode data.
  • step 1482 is performed.
  • step 1490 is performed.
  • step 1484 filters the image to filter out extraneous areas of the image.
  • step 1486 root surface and edge defects are determined for white root and black root.
  • step 1488 generates an image marking message that provides the root or feather defect type, the region subnode data the summary subnode data.
  • step 1490 After step 1440, 1450, 1460, 1470, and 1480, step 1490 generates an analysis result that provides an image marking that is stored within the data base. These results are communicated to the decisioning module in step 1492.
  • the decisioning module 436 evaluates the results from the analyzing module 432 and determines the action that needs to take place such as determining the piece status, such as the grade, whether it is invalid or needs to be reevaluated. This is based on the image marking messages and the data contained therein generated in Figure 14. Ultimately, the piece control module is used to perform the subsequent action.
  • the image marking messages and the data therein are received from the analyzing module 432.
  • the results from the analyzing module 432 may include the image marking messages and an evaluation indicator to initiate evaluation of the data in the image marking messages.
  • step 1512 the image marking message image node variables may be selected.
  • the variables may include the image reference, the batch number, the piece type, the camera identifier and the encoder data.
  • step 1514 it is determined whether the analytics result evaluator indicator is true when the analytics results evaluation indicator is true, evaluation takes place which includes a single grade evaluation that is performed in step 1516. Details of this method will be described in more in Figure 16.
  • step 1518 determines whether all the piece camera views have been processed. If all the piece views have not been processed in step 1518, a wait time is step 1520 is performed.
  • step 1522 a multiple camera grade evaluation is performed. Details of the multi-camera grade evaluation is set forth below in Fig 17.
  • step 1524 a grade bin determination is performed after step 1522. The grade bin evaluation is performed in Figure 18.
  • step 1514 when the analytics results evaluation indicator is not true, steps 1530 to 1536 are performed to bypass the grade evaluations and the grade bin determination.
  • step 1530 the FES piece record variables are set.
  • a piece grade indicator may be set to no grade
  • a piece defect type may be set to an evaluation indicator.
  • a grade bin indicator may be set to the value associated with the grade bin matrix using the piece defect type and the grade indicator.
  • An image reference may be set to the location where the image resides.
  • step 1532 a sort piece notification message may be sent to the piece control module 618.
  • an image record may be created that has the record for the piece including the batch number, the encoder value, the camera identifier, the image reference, the piece type and an evaluation indicator.
  • the image record is stored in the final evaluation structure (FES) image table which is indexed by the batch number and the encoder data.
  • the piece record may also be saved in the FES piece table.
  • Various types of data such as the batch number, the encoder, the defect type, the grade bin, the image references, may all be stored in the piece table which is indexed by the batch number and the encoder data.
  • step 1540 is performed.
  • a sort piece notification message is sent to the piece control module 618.
  • the sort piece notification message may include a batch number, encoder data and the remediation bin.
  • step 1542 may save in the file system the original image marking message received from the analyzing module 432.
  • the image marking message may be indexed by the batch number, the encoder and the image reference number.
  • step 1516 a method for performing the single camera grade evaluation (step 1516) in Fig. 15 is set forth.
  • step 1610 the single grade evaluation is initiated for each analysis node in the imaging marking message of the analysis result.
  • step 1612 it is determined whether subnodes exists. When subnodes do not exist, step 1614 is performed. No subnodes found correlates to no defects being found.
  • step 1614 sets the FES image record variables like max area, Total area, total area raised to the nth power and the region count to 0.
  • step 1616 when subnodes do exist, the 1616 is performed setting FES image record variables. Total area variable may be set to the sum of the region areas, total area to the nth power variable may be set to the sum of the region areas raised to the nth power, and region count variable may be set to the total number of regions found.
  • step 1620 sets the grade equal to grade 1.
  • the thresholds herein may be preset or may be set while running the batch from an input signal from the user interface.
  • Each defect may have different characteristics for grading. For example, for filaments or villi, a physical 0.5cm villi would appear with an approximate length of 36 pixels with a marked area of 365 for a defect score of 136,000 (as shown in Figure 22 below). A villi cluster or group will typically present a marked area larger than that of a 0.5cm villi (see Figure 23). Further, 5 or more villi will present a defect score of 150,000 or greater.
  • a defect may be determined. Different sizes and amounts of villi may be used for the grading thresholds. This is done by the following approach:
  • [0115] Use defect count 5 or more from the analysis to determine if the 5 or more villi criteria has been met. [0116] Use the largest defect region (max region) found from the analysis to determine if the 0.5+cm criteria or villi cluster has been met.
  • step 1618 when the defect type is less than the type two grade threshold in step 1622, the grade indicator is set to grade 2 in step 1624. After step 1622 and grade 2 is not found and when the grade defect is less than the grade three threshold matrix, the grade 3 threshold is set in step 1628. Various numbers of grades may be set and therefore the same logic may be applied to the various grade thresholds.
  • Step 1630 indicates if the defect type is less than the defect type grade n-1 so that the grade is set when the defect type is less than the defect type threshold for the n-1 grade in step 1632.
  • step 1634 sets the grade to grade n. After steps 1620, 1432, 1628, 1632 and 1634, step 1636 creates an image record in the FES system.
  • the record may have the batch number, the encoder data, the grade indicator, the defect type, the max region area, the total area value, the total sum of the region areas raised to the nth power value, the region count value, the camera identifier, an image reference, a piece type, an evaluation indicator as well as the image table index by the batch number and encoder data.
  • Figs. 17A and 17B details of the multiple camera grade evaluation step 1522 in Fig. 15 is set forth.
  • the FES image table is queried.
  • the FES image table is used to select the row where the batch number, the encoder data and the current defect type is filaments or villi edge OR filaments or villi surface in step 1710.
  • the FES grade record variables are set.
  • Step 1712 looks at the villi edge defect type records and selects the record with the largest total of region areas raised to the nth power value.
  • the variable CTAnS is set to the total of region areas raised to the nth power value of the selected record.
  • the image reference 1 variable is set to the image location reference of the selected record.
  • the variable CTAS is set to the total region areas of the selected record.
  • Step 1714 looks at the villi surface defect type records and selects the record with the largest total of region areas raised to the nth power value. The total of region areas raised to the nth power value of the selected record is added to the CTAnS variable.
  • the image reference 2 variable is set to image location reference of the selected record. The total of region areas value of the select record is added to the CTAS variable.
  • the CTAnS value is compared to a grade 1 threshold when the defect type CTAnS is less than the defect grade 1 threshold, step 1718 sets the grade equal to 1.
  • step 1720 the CTAnS is compared to the grade 2 threshold matrix value.
  • step 1722 sets the grade to grade 2.
  • the ⁇ CTAnS is compared to a defect type 3 threshold matrix value.
  • the threshold of grade 3 is set when the defect type CTAnS is less than the grade 3 threshold value.
  • the comparison continues in steps 1728 and 1730 which are performed for n-1 grade. That is, various numbers of grades may be represented in this method.
  • Steps 1728 sets the n-1 grade when the CTAnS is less than the defect type grade n-1.
  • Grade n is set in steps 1732.
  • the grade record is created in the final evaluation structure (FES).
  • the batch number, encoder type, the grade indicator, the defect type, such as villi or a filament, as well as the batch number and encoder data are all saved in the table.
  • Steps 1710 through 1734 are performed for filaments or villi. Other detection may be performed at indicated merely upon their presence or a comparison to a threshold.
  • inflammation is detected.
  • a query may be made to the image table.
  • the FES image table rows where the batch number and encoder values are set to current, and the defect type is set to inflammation.
  • the FES grade record variables are set from values of the record with the largest total of region areas raised to the nth power value.
  • the grade indicator, the total of region areas raised to the nth power, total of region area, and the image reference 1 variables are all set from the associated values of the selected record.
  • a grade record is created in the final evaluation structure.
  • the record is saved with the batch number, the encoder data, the grade indicator, the defect type, total of region areas raised to the nth power, total of region area, and image reference 1 information which is indexed by the batch number and encoder type.
  • the next defect is the presence of a rod.
  • the FES image table is query.
  • the FES image table rows where the batch number and encoder key values are set to the current and the defect type is set to a rod.
  • the FES grade record variables are set from values of the record with the largest total of region areas raised to the nth power value.
  • the grade indicator, the total of region areas raised to the nth power, total of region area, and the image reference 1 variables are all set from the associated values of the selected record.
  • step 1750 the FES grade record is saved with the batch number, the encoder data, the grade indicator, the defect type, total of region areas raised to the nth power, total of region area, and image reference 1 information and the final evaluation structure coordinated by the batch number and encoder data.
  • step 1752 and 1754 the same is performed for a root or feathers type of defect.
  • step 1752 a query is performed for the FES image table. In this example, the rows where the batch number and the encoder key value are set to the current batch number and the defect type is set as the root or feathers.
  • step 1754 the FES grade record variables are set from values of the record with the largest total of region areas raised to the nth power value.
  • the grade indicator, the total of region areas raised to the nth power, total of region area, and the image reference 1 variables are all set from the associated values of the selected record.
  • the FES grade record is saved with the batch number, the encoder data, the grade indicator, the defect type, total of region areas raised to the nth power, total of region area, and image reference 1 information and the final evaluation structure coordinated by the batch number and encoder data.
  • step 1524 of Fig. 15 is set forth in further detail.
  • a query for FES grade table is performed.
  • the final evaluation grade table rows are selected for the batch number and the encoder value being the current value in step 1810. This coordinates to the particular piece being evaluated.
  • step 1812 when all the records for the defects have a grade of one, step 1814 is performed.
  • step 1814 the FES piece records variables are set such as the grade is equal to 1 , no defects are found and the like.
  • step 1820 is performed.
  • step 1820 if the records have a grade value greater than 1 and the defect type is not villi or filament, step 1822 is performed.
  • step 1822 a query is performed at the FES grade table to select the record type with the highest defect type with the highest remediation precedent based on the grade bin matrix.
  • step 1824 sets the FES piece record variables.
  • step 1820 when the record is greater than grade 1 and the defect type is villi, step 1826 a query of the FES grade table is performed.
  • step 1826 the record type of villi is selected. Step 1824 is performed after steps 1822 and 1826.
  • the FES piece records variables are set. The grade is set to the record grade, the defect is set to the defect type and the grade bin is set accordingly.
  • step 1830 creates a piece record in the final evaluation structure.
  • a grade threshold matrix example is set forth.
  • the grade threshold matrix is illustrated and are set with rows corresponding to various types of defects such as the filament edge, the filament surface, inflammation, rods, feathers and roots.
  • Each of the defect types has a row with numbers corresponding to various grades.
  • Figure 20 assess all the defect types associated to a piece and assigns the grade associated with the highest priority defect type based on an established defect type precedence and associates a grade bin to the piece based on the assigned grade using a pre-determined Grade Bin Matrix. Even pieces of defect type of “invalid” or “Reevaluate” are assigned a grade bin number.
  • the grade bin matrix has various defect types that correspond to the rows and several columns including a defect precedents column, a no grade column, a grade 1 column, a grade 2 column, a grade 3 column and a grade 4 column, of course, n-1 and n may be provide in the columns to indicate that various numbers of defects may be determined by the system. Various numbers are provided in the various rows.
  • the visualize/automated piece enablement module 628 is illustrated in further detail.
  • the visualize/mechanical remediation enablement module 628 is used to visualize the piece with the defect on the display 30 of the manual remediation system 28A.
  • Automated remediation may also be provided in which a display is not required and the coordinates area and the like corresponding to the defect are set forth.
  • a visualized piece message is obtained.
  • a query of the FES piece table for the database matching the batch number and encoder data is used to obtain the image desired.
  • the marking message is obtained in step 2114 by batch number, encoder data and the image recording reference.
  • step 2116 the image and the defect position and area are provided to a display which, in step 2116, displays the image with the overlay markings with the defect location identified. This is performed for a manual remediation system.
  • step 2118 provides the coordinates and the data so that the automated remediation system or mechanical means can remediate the defect
  • an image 2210 of a chicken piece within a field of view 2212 is shown.
  • the image of the chicken piece has four areas that have an area outlined thereon.
  • the area outlined corresponds to the severity of the filaments or villus extending from the piece 2210.
  • 16 pixels or 0.22 cm is obtained.
  • An area of 157 is therefore obtained which squares to 2468.
  • the second example provides 36 pixels, which is approximately .5 cm.
  • the area is 366 and the area squared is 135,056.
  • ten pixels corresponds to .14 cm for the area of the filaments.
  • the area is 91 and the area is 8299.
  • the area of 21 pixels corresponds to several villi or filaments in a cluster. Three villi are found in this sample.
  • the area sum of the four areas is 1083 while the area sum to the nth power (2 in this example) is 1 ,172,889. These sums may be added and provided along with the data to show the area sum which is 616 and the area squared sum which is 168,036. Each of these points of data may be used individually or together to determine the grade and defect analysis.
  • a surface defect area 1310 is illustrated on a chicken piece.
  • the defect area may represent surface filaments or villi, inflammation defects (dermatitis, scabby, gore, and yellow skin), decolorization, feather/rods, white root and black roots and a matter of cut.
  • the identified area is about 53 pixels by 98 pixels which corresponds to about 0.8 cm by 1.36 cm.
  • the area corresponds to 4439 which is 19,704,721 .

Landscapes

  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

L'invention concerne un système et un procédé d'inspection d'un morceau de poulet consistant à générer une image du morceau de poulet, identifier un type de défaut, un emplacement de défaut et une zone de chaque défaut sur le morceau de poulet sur la base de l'image, et classer le morceau de poulet dans une classe d'une pluralité de classes sur la base du type de défaut et de la zone.
PCT/US2022/023268 2021-04-29 2022-04-04 Procédé et appareil de détection de défauts pour classement de morceaux de volaille WO2022231781A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202163181914P 2021-04-29 2021-04-29
US63/181,914 2021-04-29
US17/699,093 US11599984B2 (en) 2021-04-29 2022-03-19 Methods and apparatus for detecting defects for poultry piece grading
US17/699,093 2022-03-19

Publications (1)

Publication Number Publication Date
WO2022231781A1 true WO2022231781A1 (fr) 2022-11-03

Family

ID=81448294

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/023268 WO2022231781A1 (fr) 2021-04-29 2022-04-04 Procédé et appareil de détection de défauts pour classement de morceaux de volaille

Country Status (1)

Country Link
WO (1) WO2022231781A1 (fr)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200288731A1 (en) * 2015-03-02 2020-09-17 Valka Ehf Apparatus for processing and grading food articles and related methods

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200288731A1 (en) * 2015-03-02 2020-09-17 Valka Ehf Apparatus for processing and grading food articles and related methods

Similar Documents

Publication Publication Date Title
US10835930B2 (en) Sorting system
CN1515895B (zh) 监控条带材料边缘的装置
EP2029330B1 (fr) Machine pour débiter en tranches ayant unSYSTÈME DE VISION UTILISANT UN ÉCLAIRAGE STROBOSCOPIQUE ainsi que procédé d'inspection d'une face d'un produit
CN107886500A (zh) 一种基于机器视觉和机器学习的产品生产监控方法及系统
US8116554B2 (en) Two colour vision system
CN102639259B (zh) 使用模式识别的检查装置和方法
EP1915240B1 (fr) Systeme de vision bicolore
JP2019149173A (ja) インテリジェントマシンのネットワーク
US20230011901A1 (en) Systems and methods for anomaly recognition and detection using lifelong deep neural networks
US11599984B2 (en) Methods and apparatus for detecting defects for poultry piece grading
JP7088871B2 (ja) 検査装置、検査システム、及びユーザインタフェース
CN115699040A (zh) 用于训练对材料流中的组分进行分类的机器学习模型的方法和系统
CN109903711A (zh) 液晶显示屏缺陷自动检测系统
CN115423785A (zh) 一种缺陷检测系统、方法、装置、电子设备及存储介质
WO2022231781A1 (fr) Procédé et appareil de détection de défauts pour classement de morceaux de volaille
US20220284699A1 (en) System and method of object detection using ai deep learning models
CN113228049B (zh) 用于将奶分类的奶分析仪
CN116840240A (zh) 电源分配器视觉检测系统
CN115672778A (zh) 一种智能视觉识别系统
CN109255805A (zh) 机器学习的工业智能数据收集系统和方法
TW202326114A (zh) 長碼拉鏈的缺陷或瑕疵檢測系統
US20230048485A1 (en) Non-organic contamination removal automation process
Granitto et al. Part Metrology and Defect Detection Using Machine Vision
Szkilnyk Vision-based fault detection in assembly automation
Florez et al. Architecture for fault detection in sandwich panel production using visual analytics

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22720119

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22720119

Country of ref document: EP

Kind code of ref document: A1