WO2023067656A1 - Dispositif de traitement d'objets - Google Patents

Dispositif de traitement d'objets Download PDF

Info

Publication number
WO2023067656A1
WO2023067656A1 PCT/JP2021/038430 JP2021038430W WO2023067656A1 WO 2023067656 A1 WO2023067656 A1 WO 2023067656A1 JP 2021038430 W JP2021038430 W JP 2021038430W WO 2023067656 A1 WO2023067656 A1 WO 2023067656A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
waste
target
transport path
area
Prior art date
Application number
PCT/JP2021/038430
Other languages
English (en)
Japanese (ja)
Inventor
雅信 本江
健 李
圭祐 大島
Original Assignee
株式会社Pfu
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Pfu filed Critical 株式会社Pfu
Priority to JP2023553909A priority Critical patent/JPWO2023067656A1/ja
Priority to PCT/JP2021/038430 priority patent/WO2023067656A1/fr
Publication of WO2023067656A1 publication Critical patent/WO2023067656A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/34—Sorting according to other particular properties
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/36—Sorting apparatus characterised by the means used for distribution
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B09—DISPOSAL OF SOLID WASTE; RECLAMATION OF CONTAMINATED SOIL
    • B09B—DISPOSAL OF SOLID WASTE NOT OTHERWISE PROVIDED FOR
    • B09B5/00—Operations not covered by a single other subclass or by a single other group in this subclass
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G47/00—Article or material-handling devices associated with conveyors; Methods employing such devices
    • B65G47/74—Feeding, transfer, or discharging devices of particular kinds or types
    • B65G47/90—Devices for picking-up and depositing articles or materials

Definitions

  • the technology of the present disclosure relates to an object processing device.
  • An automatic recyclable sorting device that automatically separates recyclable waste from multiple pieces of waste transported along a transport path.
  • the automatic recyclable sorting apparatus uses a learning model created by machine learning to determine whether or not a plurality of pieces of garbage are recyclable pieces based on an image showing a plurality of pieces of garbage.
  • the learning model is updated by subjecting the learning model to additional learning using the newly created teacher data, and the automatic recyclable sorting device uses the updated learning model to classify a plurality of pieces of recyclable waste as recyclable waste. It is possible to improve the accuracy of determining whether or not (Japanese Patent Publication No. 2019-533570, Japanese Patent No. 6854995, International Publication No. 2016/084336).
  • such training data is created using misclassified waste that has been misclassified by an automatic recyclable waste sorting device. For this reason, there is a problem that the burden on workers who handle incorrectly sorted garbage increases.
  • the disclosed technology has been made in view of this point, and is an object processing that facilitates the work of an operator when creating teacher data used for additional learning of a learning model created by machine learning.
  • the purpose is to provide an apparatus.
  • An object processing apparatus includes a first transport unit that transports a plurality of objects along a first transport path, a first imaging unit that captures a first image showing the first transport path, a learning a robot that moves a first object selected from the plurality of objects from the first transport path to a second transport path based on the first image using a model; and a robot that moves the first object to the second transport path. and a second imaging unit that captures a second image showing a second object among the plurality of objects that has been moved from the first transportation path or the second transportation path to an imaging area. , a label information generation unit that generates label information to be assigned to the second object, and a learning unit that updates the learning model based on the second image and the label information.
  • the disclosed object processing device can facilitate an operator's work when creating teacher data used for additional learning of a learning model created by machine learning.
  • FIG. 1 is a plan view showing an automatic recyclable sorting apparatus provided with an object processing apparatus of Embodiment 1.
  • FIG. FIG. 2 is a side view showing the reversing device.
  • FIG. 3 is a block diagram showing an object processing apparatus.
  • FIG. 4 is a plan view showing a worker monitoring a plurality of dusts placed in the first worker dust removal area and the second worker dust removal area.
  • FIG. 5 is a plan view showing the operation of a worker placing non-target waste B in the worker waste supply area.
  • FIG. 6 is a plan view showing the operation of removing the recyclable waste A to be processed from the first transport path by the first worker waste removal area.
  • FIG. 7 is a flow chart showing part of the operation of updating the learning model.
  • FIG. 1 is a plan view showing an automatic recyclable sorting apparatus provided with an object processing apparatus of Embodiment 1.
  • FIG. 2 is a side view showing the reversing device.
  • FIG. 3 is a block diagram showing an object processing apparatus.
  • FIG. 8 is a side view showing the recyclable waste to be treated gripped by the reversing device.
  • FIG. 9 is a side view showing the recyclable waste to be treated reversed by the reversing device.
  • FIG. 10 is a side view showing the recyclable waste to be processed arranged in the erroneously sorted waste image imaging area by the reversing device.
  • FIG. 11 is a plan view showing an object processing apparatus of Example 2.
  • FIG. 12 is a plan view showing an object processing apparatus of Example 3.
  • FIG. 13 is a plan view showing an object processing apparatus of Example 4.
  • FIG. FIG. 14 is a side view showing an incorrectly sorted garbage image pickup area of the object processing apparatus of the fifth embodiment.
  • the object processing apparatus 1 of Embodiment 1 is provided in an automatic recyclable sorting apparatus 10, as shown in FIG.
  • FIG. 1 is a plan view showing an automatic recyclable sorting apparatus 10 provided with an object processing apparatus 1 of Embodiment 1.
  • FIG. The recyclable automatic sorting device 10 includes a first conveying device 2 , a second conveying device 3 and an object processing device 1 .
  • the first transport device 2 is formed by a belt conveyor.
  • a first transport path 5 is formed in the first transport device 2 .
  • the first transport path 5 extends along a horizontal plane, and the straight line along which the first transport path 5 extends is parallel to the transport direction 6 .
  • the first transport path 5 includes a dust supply area 11 , a sorting image capturing area 12 , a robot dust removal area 14 , a first operator dust removal area 15 and a first missorted dust supply area 16 .
  • the separation image capturing area 12 is arranged downstream of the dust supply area 11 in the transport direction 6.
  • the robot dust removal area 14 is arranged downstream of the separation image capturing area 12 in the transport direction 6.
  • the first worker dust removal area 15 is arranged downstream of the robot dust removal area 14 in the transport direction 6 .
  • the first missorted refuse supply area 16 is arranged downstream of the first worker refuse removal area 15 in the conveying direction 6 .
  • the first conveying device 2 conveys the objects arranged on the first conveying path 5 in the conveying direction 6 along the first conveying path 5, and finally disposes the objects arranged on the first conveying path 5 as non-target waste. Place in place.
  • the second conveying device 3 is formed from a belt conveyor.
  • a second transport path 21 is formed in the second transport device 3 .
  • the second transport path 21 extends along a horizontal plane, and the straight line along which the second transport path 21 extends is parallel to the transport direction 6 .
  • the second transport path 21 includes a robot waste supply area 22 , a second worker waste removal area 23 and a second missorted waste supply area 24 .
  • the second worker dust removal area 23 is arranged downstream of the robot dust supply area 22 in the transport direction 6 .
  • the second missorted refuse supply area 24 is arranged downstream of the second worker refuse removal area 23 in the conveying direction 6 .
  • the second transport device 3 transports the objects placed on the second transport path 21 in the transport direction 6 along the second transport path 21, and transports the objects placed on the first transport path 5 to the recyclable waste disposal site. to be placed.
  • the object processing apparatus 1 includes a separation imaging unit 31, a robot 32, a third conveying device 33, a label information detection camera 34, an annotation imaging unit 35, and a control device .
  • the classification imaging unit 31 is arranged in the vicinity of the classification image imaging area 12 .
  • the classification imaging unit 31 captures an image of an object placed in the classification image capturing area 12 under the control of the control device 36 .
  • the robots 32 are positioned proximate the robotic debris removal area 14 and positioned proximate the robotic debris supply area 22 . Under the control of the control device 36 , the robot 32 removes the objects placed in the robot dust removal area 14 from the first transport path 5 and places them in the robot dust supply area 22 of the second transport path 21 .
  • the third conveying device 33 is formed from a belt conveyor.
  • a third transport path 37 is formed in the third transport device 33 .
  • the third transport path 37 extends along a horizontal plane, and the straight line along which the third transport path 37 extends is parallel to the transport direction 6 .
  • the third transport path 37 includes a worker waste supply area 38 and a missorted waste image capturing area 39 .
  • the erroneously sorted waste image pickup area 39 is arranged downstream of the operator waste supply area 38 in the transport direction 6 .
  • the third conveying device 33 conveys the object arranged on the third conveying path 37 in the conveying direction 6 along the third conveying path 37 .
  • the third conveying device 33 is controlled by the control device 36 to finally arrange the objects placed on the third conveying path 37 in the first incorrectly sorted refuse supply area 16 or the second incorrectly sorted refuse supply area 24. do.
  • the label information detection camera 34 is arranged near the first worker dust removal area 15 and near the second worker dust removal area 23 .
  • the label information detection camera 34 is controlled by the control device 36 to capture an image of the first worker dust removal area 15 and the second worker dust removal area 23 .
  • the annotation imaging unit 35 is arranged in the vicinity of the incorrectly sorted garbage image imaging area 39 .
  • the annotation imaging unit 35 is controlled by the control device 36 to capture an image including the incorrectly sorted garbage image imaging region 39 .
  • the object processing apparatus 1 further comprises a reversing device 41, as shown in FIG. FIG. 2 is a side view showing the reversing device 41.
  • the reversing device 41 is formed from an articulated robot.
  • the reversing device 41 is arranged in the vicinity of the erroneously sorted refuse image pickup area 39 of the third conveying path 37 .
  • the reversing device 41 is controlled by the control device 36 to grip an object placed in the incorrectly sorted refuse image imaging area 39, to reverse the gripped object, and to release the gripped object.
  • FIG. 3 is a block diagram showing the object processing device 1.
  • the control device 36 is a computer and includes a storage device 42 and a CPU 43 (Central Processing Unit).
  • the storage device 42 records computer programs installed in the control device 36 and records information used by the CPU 43 .
  • the CPU 43 processes information and controls the storage device 42 by executing a computer program installed in the control device 36 .
  • the CPU 43 further controls the separation imaging section 31 , the robot 32 , the third conveying device 33 , the label information detection camera 34 , the annotation imaging section 35 and the reversing device 41 .
  • the computer programs installed in the control device 36 include multiple computer programs that cause the control device 36 to implement multiple functions.
  • the multiple functions include an object recognition unit 44 , an object selection unit 45 , a teacher data generation unit 46 , an object return unit 47 and a learning unit 48 .
  • the object recognizing unit 44 operates the image pickup unit for sorting so that a garbage sorting image including a plurality of images respectively showing a plurality of objects arranged in the image pickup area 12 for sorting of the first transport path 5 is picked up. 31 is controlled.
  • the object recognition unit 44 performs image processing on the dust sorting image, uses a learning model created by machine learning, associates label information with a plurality of objects appearing in the dust sorting image, and calculates a plurality of sorting data. .
  • the object sorting unit 45 selects an object associated with the recyclable waste to be processed among the plurality of objects arranged in the robot waste removal area 14 as the robot waste.
  • a robot 32 is controlled to be placed in the supply area 22 .
  • the teacher data creation unit 46 controls the label information detection camera 34 so that a monitoring image showing the first worker dust removal area 15 and the second worker dust removal area 23 is captured intermittently.
  • the teacher data creation unit 46 performs image processing on the monitoring image, determines whether or not the resource waste to be processed has been removed from the first worker's dust removal area 15, and determines whether non-target dust has been removed from the second worker's dust removal area 23. Determine whether or not it has been removed.
  • the teacher data creation unit 46 controls the annotation imaging unit 35 so that an annotation image including an object placed in the incorrectly sorted garbage image imaging area 39 is captured.
  • the teaching data creation unit 46 creates teaching data that associates the captured image for annotation with the recyclable waste to be processed. , is recorded in the storage device 42 .
  • the teacher data creation unit 46 creates and stores teacher data that associates the captured annotation image with the non-target dust. Record on device 42 .
  • the object returning unit 47 moves the processing target recyclable waste supplied to the third conveying path 37 to the first missorted waste supply region. 16 to control the third transport device 33 .
  • the object return unit 47 returns the non-target waste supplied to the third conveying path 37 to the second erroneously sorted waste supply area 24. Control the third conveying device 33 so that it is fed.
  • the learning unit 48 additionally learns the learning model used by the object recognition unit 44 using the teacher data created by the teacher data creation unit 46 , and updates the learning model used by the object recognition unit 44 .
  • the operation of the automatic recyclable sorting device 10 includes an operation of sorting a plurality of pieces of refuse and an operation of updating the learning model.
  • the user operates the first transport device 2 to activate the first transport device 2 , operates the second transport device 3 , and activates the second transport device 3 .
  • the user further places a plurality of pieces of dust in the dust supply area 11 of the first transport path 5 .
  • the plurality of garbage includes a plurality of processing target recyclable garbage A and a plurality of non-target garbage B.
  • a plurality of processing target recyclable wastes A are recyclable wastes that need to be removed from the first transport path 5, and are recyclable wastes that need to be moved to a recyclable waste storage place.
  • Bottles made of glass colored in a predetermined color for example, brown
  • the plurality of non-target waste B are different from the plurality of processing target recyclable waste A and need to be placed in the non-target waste storage area.
  • a plurality of wastes placed in the waste supply area 11 are transported along the first transport path 5 in the transport direction 6 by the first transport device and placed in the sorting image capturing area 12 .
  • the plurality of dusts arranged in the sorting image pick-up area 12 are further transported in the transport direction 6 along the first transport path 5 and placed in the robot dust removal area 14 .
  • a plurality of wastes placed in the robotic waste removal area 14 are further transported in the transport direction 6 along the first transport path 5 and placed in non-target waste bins.
  • the control device 36 controls the imaging section 31 for sorting and captures an image for sorting out a plurality of trash arranged in the image capturing area 12 for sorting.
  • the control device 36 uses a learning model created in advance by machine learning to determine the types of a plurality of types of dust, and associates a plurality of pieces of label information with a plurality of types of dust.
  • the label information associated with a piece of garbage out of the plurality of label information indicates the label "recyclable waste A" when the garbage is recyclable waste A to be processed, and the garbage is recyclable waste A to be processed. It shows the label "Untargeted Garbage B" when different.
  • the control device 36 controls the robot 32 to move the processing target recyclable waste A associated with the label “processing target recyclable waste A” out of the plurality of wastes arranged in the robot waste removal area 14 to the first transport path 5 . remove from The control device 36 further controls the robot 32 to arrange the recyclable waste A removed from the first transport path 5 in the robot waste supply area 22 of the second transport path 21 .
  • a plurality of wastes placed in the robot waste supply area 22 are transported along the second transport path 21 in the transport direction 6 by the second transport device 3 and placed in the second worker dust removal area 23 .
  • the waste placed in the second worker waste removal area 23 is further transported in the transport direction 6 along the second transport path 21 and placed in the target recyclable waste storage area.
  • the automatic recyclable sorting device 10 may not remove the recyclable waste A to be processed from the first transport path 5 because the control device 36 erroneously associates the recyclable waste A to be processed with the label "non-target waste B". Further, the automatic recyclable sorting apparatus 10 transfers the untargeted waste B to the robot waste supply area 22 of the second transport path 21 by erroneously associating the untargeted waste B with the label "process target recyclable waste A" by the control device 36. may be placed in For this reason, the worker 51, as shown in FIG. A plurality of dusts placed in the second worker dust removal area 23 are monitored.
  • FIG. 4 is a plan view showing a worker 51 monitoring a plurality of dusts placed in the first worker dust removal area 15 and the second worker dust removal area 23.
  • the worker 51 removes the non-target dust B from the second transport path 21 in the second worker dust removal area 23 when the non-target dust B is placed in the second worker dust removal area 23 .
  • Worker 51 also places the removed non-target waste B in worker waste supply area 38, as shown in FIG.
  • FIG. 5 is a plan view showing the operation of the worker 51 placing the non-target waste B in the worker waste supply area 38. As shown in FIG.
  • FIG. 6 is a plan view showing the operation of the worker 51 removing the recyclable waste A to be treated from the first conveying path 5 in the first worker waste removal area 15.
  • the worker 51 arranges the recyclable waste A removed from the first transport path 5 in the worker waste supply area 38 in the same manner as the non-target waste B removed from the second transport path 21 .
  • the waste placed in the worker waste supply area 38 is transported in the transport direction 6 along the third transport path 37 by the third transport device 33 and placed in the missorted waste image pickup area 39 .
  • FIG. 7 is a flowchart showing part of the operation of updating the learning model.
  • the control device 36 controls the label information detection camera 34 while the operation of sorting a plurality of dusts is being executed, and the first worker dust removal area 15 and the second worker dust removal area 23 are captured.
  • a monitoring image is captured intermittently.
  • the control device 36 processes the monitoring image, determines whether or not the worker 51 has removed the non-target dust B from the first worker dust removal area 15, and determines whether or not the worker 51 has removed the second worker dust removal area 15. 23 is removed.
  • the control device 36 further controls the annotation image pickup unit 35 while the operation of separating a plurality of pieces of garbage is being executed, and intermittently captures the incorrectly sorted garbage image pickup area monitoring image in which the incorrectly sorted garbage image pickup area 39 is captured. I'm taking a picture of it.
  • the control device 36 processes the erroneously sorted waste imaging area monitoring image, It is determined whether or not the recyclable waste A to be processed is placed in the sorted waste image imaging area 39 .
  • the control device 36 controls the annotation imaging unit 35 to display the first processing target recyclable waste in which the processing target recyclable waste A is captured. An image is captured (step S2).
  • the processing target recyclable waste A is transported along the third transport path 37 in the transport direction 6 by the third transport device 33, and transported to the missorted waste image imaging area 39. It is arranged downstream in direction 6 .
  • the control device 36 controls the reversing device 41 so that, as shown in FIG. The arranged recyclable garbage 52 to be treated is grasped.
  • FIG. 8 is a side view showing the recyclable waste 52 gripped by the reversing device 41. As shown in FIG. After the recyclable waste 52 to be processed is gripped, the control device 36 controls the reversing device 41 to reverse the recyclable waste 52 to be processed as shown in FIG. FIG.
  • FIG. 9 is a side view showing the recyclable waste 52 to be treated that has been reversed by the reversing device 41.
  • the control device 36 controls the reversing device 41 to arrange the recyclable waste 52 to be processed in the erroneously sorted waste image capturing area 39 as shown in FIG.
  • FIG. 10 is a side view showing the recyclable waste 52 to be processed arranged in the erroneously sorted waste image imaging area 39 by the reversing device 41 .
  • the control device 36 controls the annotation imaging unit 35 to generate a second processing target recyclable waste image in which the processing target recyclable waste A is captured. is imaged.
  • the second processing target recyclable waste image differs from the first processing target recyclable waste image in that the processing target recyclable waste A is inverted.
  • the control device 36 associates the first processing target recyclable waste image with the label "processing target recyclable waste A” and records it in the storage device 42, and associates the second processing target recyclable waste image with the label "processing target recyclable waste A”. and record it in the storage device 42 (step S3).
  • the control device 36 controls the third conveying device 33 so that the processing target arranged in the erroneously sorted waste image imaging area 39 is captured.
  • the recyclable waste A is placed in the second incorrectly sorted waste supply area 24 (step S4).
  • the recyclable waste A placed in the second incorrectly sorted waste supply area 24 is conveyed in the conveying direction 6 along the second conveying path 21 and placed in the recyclable waste storage area.
  • the control device 36 When it is determined that the non-target dust B has been removed from the second worker dust removal area 23 (step S5, Yes), the control device 36 performs image processing on the erroneously sorted dust imaging area monitoring image and performs erroneous sorting. It is determined whether or not non-target dust B is placed in the dust image imaging area 39 . When it is determined that the non-target waste B has been placed in the erroneously sorted waste image imaging area 39, the control device 36 controls the annotation imaging unit 35 and the reversing device 41 to produce a first non-target waste image in which the non-target waste B is captured. A target dust image and a second non-target dust image are captured (step S6). The first non-target waste image and the second non-target waste image are, like the first process target recyclable waste image and the second process target recyclable waste image, the non-target waste B viewed from two different directions, respectively. ing.
  • the control device 36 stores the first non-target dust image in the storage device 42 in correspondence with the label "non-target dust B", and stores the second non-target dust image in the storage device 42 in correspondence with the label "non-target dust B”. 42 (step S7). After the first non-target waste image and the second non-target waste image are captured, the control device 36 controls the third conveying device 33 so that the non-target waste B disposed in the incorrectly sorted waste image imaging area 39 is captured. is arranged in the first incorrectly sorted refuse supply area 16 of the first transport path 5 (step S8). The non-target waste B placed in the first missorted waste supply area 16 is transported in the transport direction 6 along the first transport path 5 and placed in the non-target waste storage area.
  • the number of first process target recyclable waste images, second process target recyclable waste images, first non-target waste images, and second non-target waste images that are larger than a predetermined number are stored in the storage device 42. will be executed repeatedly until the The control device 36 determines that the first process target recyclable waste image, the second process target recyclable waste image, the first non-target waste image, and the second non-target waste image are recorded in the storage device 42.
  • a plurality of teacher data associates a plurality of images with a plurality of label information. Each of the plurality of images is one of a first processing target recyclable waste image, a second processing target recyclable waste image, a first non-target waste image, and a second non-target waste image.
  • the label information associated with the first processing target recyclable waste image or the second processing target recyclable waste image among the plurality of label information indicates the label “processing target recyclable waste A”.
  • the label information associated with the first non-target dust image or the second non-target dust image among the plurality of label information indicates the label "non-target dust B”.
  • the control device 36 additionally learns a learning model that associates label information with a plurality of pieces of dust appearing in the dust sorting image using a plurality of teacher data, and updates the learning model.
  • the object processing apparatus 1 can improve the accuracy of associating a plurality of objects appearing in the dust sorting image with the label information. For example, the object processing apparatus 1 reduces the frequency of erroneously associating other processing target recyclable waste similar to the processing target recyclable waste erroneously associated with the label “untargeted waste B” with the label “untargeted waste B”. can do.
  • the object processing apparatus 1 further reduces the frequency of erroneously associating other non-target waste similar to the non-target waste erroneously associated with the label 'processing target recyclable waste A' with the label 'processing target recyclable waste A'. can do. Further, when the object processing apparatus 1 creates a plurality of teacher data for additional learning of the learning model, the worker 51 needs to additionally perform the task of correctly sorting the missorted garbage and the separate task. Therefore, the burden on the worker 51 can be reduced.
  • the object processing apparatus 1 of the first embodiment includes a first conveying device 2, a sorting imaging unit 31, a robot 32, a second conveying device 3, an annotation imaging unit 35, a label information detection camera 34, and a learning unit 48.
  • the first conveying device 2 conveys a plurality of pieces of refuse along the first conveying path 5 .
  • the sorting imaging unit 31 captures a dust sorting image in which the first transport path 5 is shown.
  • the robot 32 moves the processing target recyclable waste A sorted from a plurality of wastes based on the waste sorting image from the first transport path 5 to the second transport path 21. .
  • the second conveying device 3 conveys the recyclable waste A to be treated along the second conveying path 21 .
  • the annotation imaging unit 35 captures a first process target recyclable waste image or a first processing target recyclable waste image in which misclassified waste moved from the first conveying path 5 or the second conveying path 21 of the plurality of wastes to the misclassified waste image imaging area 39 is captured. 1 Take a non-target dust image.
  • a label information detection camera 34 captures surveillance images for automatically determining label information assigned to missorted waste.
  • the learning unit 48 updates the learning model based on teacher data that associates the first processing target recyclable waste image or the first non-target waste image with the label information.
  • the worker 51 moves the erroneously sorted waste from the first transport path 5 or the second transport path 21 to the erroneously sorted waste image imaging area 39 .
  • the object processing apparatus 1 of the first embodiment can improve the accuracy of associating a plurality of objects appearing in the dust sorting image with the label information.
  • the object processing apparatus 1 of the first embodiment eliminates the need for the worker 51 to additionally perform a task separate from the task of correctly re-sorting missorted waste when creating training data. The burden on the person 51 can be reduced.
  • control device 36 of the object processing apparatus 1 of the first embodiment determines whether missorted waste has been removed from the first conveying path 5 or not from the second conveying path 21 based on the monitoring image captured by the label information detection camera 34 .
  • Label information is generated by determining whether it has been removed.
  • the object processing apparatus 1 according to the first embodiment does not require the operator 51 to additionally perform the task of specifying the label information associated with the first processing target recyclable waste image or the first non-target waste image, and the teacher The burden on the worker 51 when creating data can be further reduced.
  • the object processing apparatus 1 uses the label information of the first transport path 5 and the second transport path 21 after the first processing target recyclable waste image or the first non-target waste image is captured.
  • a third conveying device 33 is further provided for supplying missorted waste to the conveying path selected based on the above.
  • the object processing apparatus 1 according to the first embodiment transfers misclassified waste to the first transport path 5 or the second transport path 21 after the first process target recyclable waste image or the first non-target waste image is captured.
  • the burden on the worker 51 can be further reduced because the worker 51 does not have to return the device.
  • the object processing apparatus 1 of the first embodiment further includes a reversing device 41 that changes the orientation of the incorrectly sorted refuse in the incorrectly sorted refuse image imaging area 39 .
  • the annotation imaging unit 35 further captures a second process target recyclable waste image or a second non-target waste image in which the missorted waste is captured.
  • the learning unit 48 updates the learning model further based on the second processing target recyclable waste image or the second non-target waste image.
  • the object processing apparatus 1 according to the first embodiment can additionally learn the learning model using other teacher data showing images of the misclassified garbage viewed from other directions, and the plurality of pieces of garbage can be labeled as label information. can be further improved.
  • the object processing apparatus 1 of the first embodiment eliminates the need for the operator 51 to additionally perform work for creating other teaching data showing images of the missorted waste viewed from other directions, and the teaching data can be generated. The burden on the worker 51 when creating can be reduced.
  • FIG. 11 is a plan view showing an object processing apparatus of Example 2.
  • the input device 61 has a target recyclable waste A button 62 and a non-target waste B button 63 .
  • the input device 61 is arranged near the worker waste supply area 38 .
  • the input device 61 outputs to the control device 36 whether or not the processing target recyclable waste A button 62 has been pressed, and outputs to the control device 36 whether or not the non-target waste B button 63 has been pressed.
  • the worker 51 When the non-target waste B is placed on the second transport path 21, the worker 51 removes the non-target waste B from the second transport path 21 and places the non-target waste B in the worker waste supply area 38.
  • the B button 63 is pressed.
  • the worker 51 removes the recyclable waste A to be processed from the first transport path 5, places it in the worker waste supply area 38, and processes it.
  • the target recyclable waste A button 62 is pressed.
  • the control device 36 controls the annotation imaging unit 35 and the reversing device 41 to capture the first non-target dust B viewed from two different directions. A dust image and a second non-target dust image are captured. The control device 36 further records the first non-target dust image and the second non-target dust image in the storage device 42 in association with the label "non-target dust B". After the first non-target waste image and the second non-target waste image are captured, the control device 36 controls the third conveying device 33 so that the non-target waste B disposed in the incorrectly sorted waste image imaging area 39 is captured. is placed in the first missorted refuse supply area 16 of the first transport path 5 . The non-target waste B placed in the first missorted waste supply area 16 is transported in the transport direction 6 along the first transport path 5 and placed in the non-target waste storage area.
  • the control device 36 controls the annotation imaging unit 35 and the reversing device 41, and the first image captures the processing target recyclable waste A viewed from two different directions.
  • a processing target recyclable waste image and a second processing target recyclable waste image are captured.
  • the control device 36 further records the first processing target recyclable waste image and the second processing target recyclable waste image in the storage device 42 in association with the label “processing target recyclable waste A”.
  • the control device 36 controls the third conveying device 33 so that the processing target arranged in the erroneously sorted waste image imaging area 39 is captured.
  • the recyclable waste A is placed in the second missorted waste supply area 24 of the second transport path 21 .
  • the recyclable waste A placed in the second incorrectly sorted waste supply region 24 is transported in the transport direction 6 along the second transport path 21 and placed in the recyclable waste storage area to be processed.
  • the object processing apparatus of the second embodiment creates teacher data, and uses the teacher data to additionally learn and update the learning model. Therefore, the object processing apparatus of the second embodiment can improve the accuracy of associating a plurality of objects appearing in the dust sorting image with the label information in the same manner as the object processing apparatus 1 of the first embodiment. Further, the operator 51 only needs to press the processing target recyclable waste A button 62 or the non-target waste B button 63 when specifying the label information associated with the missorted waste. Therefore, the object processing apparatus according to the second embodiment can reduce the burden on the operator 51 when creating teacher data.
  • FIG. 12 is a plan view showing an object processing apparatus of Example 3.
  • FIG. A fourth transport path 73 is formed in the fourth transport device 71 .
  • the fourth transport path 73 extends along a horizontal plane, and the straight line along which the fourth transport path 73 extends is parallel to the transport direction 6 .
  • the fourth transport path 73 includes a non-target dust supply area 74 and a non-target dust image capturing area 75 .
  • the non-target dust image capturing area 75 is arranged downstream of the non-target dust supply area 74 in the transport direction 6 .
  • the fourth conveying device 71 conveys an object arranged on the fourth conveying path 73 in the conveying direction 6 along the fourth conveying path 73 .
  • the fourth transport device 71 finally places the objects placed on the fourth transport path 73 in the first missorted refuse supply area 16 .
  • a fifth transport path 76 is formed in the fifth transport device 72 .
  • the fifth transport path 76 extends along a horizontal plane, and the straight line along which the fifth transport path 76 extends is parallel to the transport direction 6 .
  • the fifth transport path 76 includes a processing target recyclable waste supply region 77 and a processing target recyclable waste image pickup region 78 .
  • the processing target recyclable waste image pickup area 78 is arranged downstream of the processing target recyclable waste supply region 77 in the transport direction 6 .
  • the fifth conveying device 72 conveys the object arranged on the fifth conveying path 76 in the conveying direction 6 along the fifth conveying path 76 .
  • the fifth transport device 72 finally places the objects placed on the fifth transport path 76 in the second missorted waste supply area 24 .
  • the worker 51 removes the non-target waste B from the second transport path 21 and places it in the non-target waste supply area 74 when the non-target waste B is placed on the second transport path 21 .
  • the fourth conveying device 71 conveys the non-target dust B arranged in the non-target dust supplying area 74 along the fourth conveying path 73 in the conveying direction 6, arranges the non-target dust B in the non-target dust image capturing area 75, It is placed in the missorted waste supply area 16 .
  • the operator 51 removes the target recyclable waste A from the first transport route 5 and places it in the target recyclable waste supply area 77 when the target recyclable waste A is placed on the first transport route 5 .
  • the fifth conveying device 72 conveys the target recyclable waste A placed in the target recyclable waste supply area 77 along the fifth transport path 76 in the transport direction 6 and places it in the target recyclable waste image pickup area 78 . , in the second missorted waste supply area 24 .
  • the control device 36 performs image processing on the erroneously sorted waste imaging area monitoring image, determines whether or not non-target waste B is arranged in the non-target waste image imaging area 75 , and determines whether or not the non-target waste B is placed in the non-target waste image imaging area 75 . It is determined whether or not recyclable garbage A has been placed.
  • the control device 36 views the object from two different directions in the same manner as the object processing apparatus 1 of the first embodiment. A first non-target dust image in which the non-target dust B is captured and a second non-target dust image are captured.
  • the control device 36 selects the first processing target recyclable waste in which the processing target recyclable waste A viewed from two different directions is captured. An image and a second process target recyclable waste image are captured.
  • the object processing apparatus of the third embodiment creates teacher data, and uses the teacher data to additionally learn and update the learning model. Therefore, the object processing apparatus according to the third embodiment can improve the accuracy of associating a plurality of objects appearing in the dust sorting image with the label information, like the object processing apparatus 1 according to the first embodiment. Further, when specifying the label information to be associated with the erroneously sorted garbage, the worker 51 only needs to arrange the erroneously sorted garbage in the non-target garbage image capturing area 75 or the processing target recyclable garbage image capturing area 78. . Therefore, the object processing apparatus according to the third embodiment can reduce the burden on the operator 51 when creating teacher data.
  • the object processing apparatus according to the fourth embodiment is such that the third conveying device 33 of the object processing apparatus according to the second embodiment is replaced with a mounting table 81, and the other parts are the same as those described above. is the same as the object processing apparatus of the second embodiment.
  • FIG. 13 is a plan view showing an object processing apparatus of Example 4.
  • FIG. The annotation imaging unit 35 of the object processing apparatus of the fourth embodiment captures an image of an object placed on the placement table 81 .
  • the worker 51 removes the non-target waste B from the second transport path 21, places the non-target waste B on the mounting table 81, and presses the non-target waste B button. Press 63.
  • the control device 36 controls the annotation imaging unit 35 to capture a non-target dust image including the non-target dust B, and labels the non-target dust image "non-target”.
  • the data is recorded in the storage device 42 in association with "garbage B”.
  • the operator 51 arranges the non-target waste B placed on the placement table 81 on the first transport path 5 after the non-target waste image is captured.
  • the worker 51 removes the recyclable waste A to be processed from the first transport path 5, places the recyclable waste A on the placing table 81, and removes the recyclable waste A to be processed.
  • the trash A button 62 is pressed.
  • the control device 36 controls the annotation imaging unit 35 to capture a processing target recyclable waste image in which the processing target recyclable waste A is captured, and captures the processing target recyclable waste image. It is recorded in the storage device 42 in association with the label "processing target recyclable waste A".
  • the operator 51 arranges the recyclable waste A placed on the placing table 81 on the second transport path 21 after the image of the recyclable waste to be processed is captured.
  • the object processing apparatus of the fourth embodiment creates teacher data, and uses the teacher data to additionally learn and update the learning model. Therefore, the object processing apparatus according to the fourth embodiment can improve the accuracy of associating a plurality of objects appearing in the dust sorting image with the label information, like the object processing apparatus 1 according to the first embodiment. Further, the operator 51 only needs to press the processing target recyclable waste A button 62 or the non-target waste B button 63 when specifying the label information associated with the missorted waste. Therefore, the object processing apparatus according to the fourth embodiment can reduce the burden on the operator 51 when creating teacher data.
  • FIG. 14 is a side view showing the erroneously sorted garbage image capturing area 91 of the object processing apparatus of the fifth embodiment.
  • the region on the downstream side in the transport direction 6 of the erroneously sorted waste image imaging region 91 is lower than the region on the upstream side in the transport direction 6 of the erroneously sorted waste image imaging region 91. , is inclined with respect to the horizontal plane 90. As shown in FIG.
  • the worker 51 removes the non-target waste B from the second transport path 21 and places it in the worker waste supply region 38 when the non-target waste B is placed in the first worker waste removal area 15 .
  • the worker 51 removes the recyclable waste A to be processed from the first transport path 5 and places it in the worker waste supply area 38.
  • Missorted refuse 92 placed in the worker refuse supply area 38 is transported along the third transport path 37 in the transport direction 6 by the third transport device 33 and placed in the missorted refuse image imaging area 91 .
  • the control device 36 controls the annotation imaging unit 35 when it is determined based on the incorrectly sorted refuse imaging area monitoring image that the incorrectly sorted refuse 93 has been placed in the incorrectly sorted refuse image imaging area 91, and the incorrectly sorted refuse image sensing area 91 is detected.
  • a first missorted garbage image in which the garbage 93 is captured is captured.
  • the erroneously sorted refuse 93 arranged in the erroneously sorted refuse image imaging area 91 rolls in the conveying direction 6 due to the inclination of the erroneously sorted refuse image imaging area 91 .
  • Missorted garbage 94 that rolls is captured in the incorrectly sorted garbage 93 in the incorrectly sorted garbage image imaging area 91 at a second timing when a predetermined time has passed since the first timing when the first incorrectly sorted garbage image was captured. is arranged in the area downstream in the transport direction 6 of the area where the was arranged. Furthermore, the portion of the misclassified dust 94 facing the annotation imaging section 35 is different from the portion of the misclassified dust 93 facing the annotation imaging section 35 . At a second timing, the control device 36 controls the annotation imaging section 35 to capture a second incorrectly sorted garbage image including the incorrectly sorted garbage 94 .
  • the control device 36 detects the first error when it is determined based on the monitoring image that the non-target dust B has been removed from the second conveying path 21.
  • the sorted refuse image and the second erroneously sorted refuse image are recorded in the storage device 42 in association with the "untargeted refuse B".
  • the control device 36 further controls the third transport device 33 to classify the erroneously sorted waste into the first erroneously sorted waste. It is placed in the sorted refuse supply area 16 .
  • the control device 36 When it is determined based on the monitoring image that the recyclable waste A to be processed has been removed from the first conveying path 5, the control device 36 "processes" the first erroneously sorted waste image and the second erroneously sorted waste image. It is recorded in the storage device 42 in association with "target recyclable garbage A". When it is determined based on the monitoring image that the recyclable waste A to be processed has been removed from the first conveying path 5, the control device 36 further controls the third conveying device 33 to transfer incorrectly sorted waste to the second conveying route. It is placed in the missorted refuse supply area 24 .
  • the object processing apparatus of the fifth embodiment creates teacher data, and uses the teacher data to additionally learn and update the learning model. Therefore, the object processing apparatus according to the fifth embodiment can improve the accuracy of associating a plurality of objects appearing in the dust sorting image with the label information, like the object processing apparatus 1 according to the first embodiment.
  • the object processing apparatus of the fourth embodiment does not use the reversing device 41, and captures the first incorrectly sorted garbage image and the second incorrectly sorted garbage image, respectively, in which the incorrectly sorted garbage is seen from two different directions. . Therefore, the object processing apparatus according to the fourth embodiment can reduce the load on the control device 36 compared to the object processing apparatus 1 according to the first embodiment.
  • the object processing apparatus of the above-described embodiment two images of the erroneously sorted refuse viewed from two different directions are captured. may be imaged, and only one missorted refuse image may be imaged. Even in such a case, the object processing apparatus of the embodiment can improve the accuracy of associating a plurality of pieces of dust with label information, and can reduce the burden on the operator 51 when creating teacher data.
  • the object processing apparatus of the above-described embodiment separates a plurality of wastes, but may separate a plurality of objects different from wastes.
  • the embodiments have been described above, the embodiments are not limited by the above-described contents.
  • the components described above include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those within the so-called equivalent range.
  • the components described above can be combined as appropriate.
  • at least one of various omissions, replacements, and modifications of components can be made without departing from the gist of the embodiments.
  • Recyclable garbage automatic sorting device 1 Object processing device 2: First conveying device 3: Second conveying device 5: First conveying path 15: First worker garbage removal area 21: Second conveying path 23: Second work Garbage Removal Area 31: Separation Imaging Unit 32: Robot 33: Third Conveying Device 34: Label Information Detecting Camera 35: Annotating Imaging Unit 37: Third Conveying Path 39: Improperly Sorted Garbage Imaging Area 41: Reversing Device 48 : Learning Unit 51: Worker 61: Input Device 71: Fourth Conveying Device 72: Fifth Conveying Device 75: Non-Target Garbage Image Imaging Area 78: Processing Target Recyclable Garbage Image Imaging Area 90: Horizontal Plane 91: Incorrectly Sorted Garbage Image Imaging region

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Environmental & Geological Engineering (AREA)
  • Human Computer Interaction (AREA)
  • Robotics (AREA)
  • Sorting Of Articles (AREA)

Abstract

Ce dispositif de traitement d'objets (1) comprend : une première unité de transport (2) destinée à transporter une pluralité d'objets le long d'un premier trajet de transport (5) ; une première unité d'imagerie (31) destinée à capturer une première image dans laquelle le premier trajet de transport (5) est réfléchi ; un robot (32) destiné à amener un premier objet sélectionné parmi la pluralité d'objets à se déplacer du premier trajet de transport (5) au second trajet de transport (21) sur la base de la première image au moyen d'un modèle entraîné ; une seconde unité de transport (3) destinée à transporter le premier objet le long du second trajet de transport (21) ; une seconde unité d'imagerie (35) destinée à capturer une seconde image dans laquelle un second objet parmi la pluralité d'objets est réfléchi, le second objet ayant été déplacé du premier trajet de transport (5) ou second trajet de transport (21) à une région d'imagerie (39) ; une unité de génération d'informations d'étiquette (34) destinée à générer des informations d'étiquette attribuées au second objet ; et une unité d'apprentissage (48) qui met à jour le modèle entraîné sur la base de la seconde image et des informations d'étiquette.
PCT/JP2021/038430 2021-10-18 2021-10-18 Dispositif de traitement d'objets WO2023067656A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023553909A JPWO2023067656A1 (fr) 2021-10-18 2021-10-18
PCT/JP2021/038430 WO2023067656A1 (fr) 2021-10-18 2021-10-18 Dispositif de traitement d'objets

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/038430 WO2023067656A1 (fr) 2021-10-18 2021-10-18 Dispositif de traitement d'objets

Publications (1)

Publication Number Publication Date
WO2023067656A1 true WO2023067656A1 (fr) 2023-04-27

Family

ID=86058883

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/038430 WO2023067656A1 (fr) 2021-10-18 2021-10-18 Dispositif de traitement d'objets

Country Status (2)

Country Link
JP (1) JPWO2023067656A1 (fr)
WO (1) WO2023067656A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06343946A (ja) * 1993-06-07 1994-12-20 Shinko Electric Co Ltd 廃びん選別装置
JP2019533570A (ja) * 2016-08-04 2019-11-21 ゼンロボティクス オイ 複数の物体から少なくとも１つの物体を分離するための方法、コンピュータ・プログラム、装置およびシステム
JP2021030107A (ja) * 2019-08-16 2021-03-01 株式会社イーアイアイ 物品選別装置、物品選別システムおよび物品選別方法
WO2021039850A1 (fr) * 2019-08-26 2021-03-04 川崎重工業株式会社 Dispositif de traitement d'informations, dispositif de configuration, système de reconnaissance d'image, système de robot, procédé de configuration, dispositif d'apprentissage et procédé de génération de modèle appris

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06343946A (ja) * 1993-06-07 1994-12-20 Shinko Electric Co Ltd 廃びん選別装置
JP2019533570A (ja) * 2016-08-04 2019-11-21 ゼンロボティクス オイ 複数の物体から少なくとも１つの物体を分離するための方法、コンピュータ・プログラム、装置およびシステム
JP2021030107A (ja) * 2019-08-16 2021-03-01 株式会社イーアイアイ 物品選別装置、物品選別システムおよび物品選別方法
WO2021039850A1 (fr) * 2019-08-26 2021-03-04 川崎重工業株式会社 Dispositif de traitement d'informations, dispositif de configuration, système de reconnaissance d'image, système de robot, procédé de configuration, dispositif d'apprentissage et procédé de génération de modèle appris

Also Published As

Publication number Publication date
JPWO2023067656A1 (fr) 2023-04-27

Similar Documents

Publication Publication Date Title
US11682097B2 (en) Method and an apparatus for separating at least one object from a plurality of objects
US11702322B2 (en) Vacuum extraction for material sorting applications
US11433429B2 (en) Logistics sorting system and logistics sorting method
US20230365352A1 (en) Bidirectional air conveyor device for material sorting and other applications
CN110740954A (zh) 包括空间高效的分配站和自动化输出处理的用于处理物体的系统和方法
WO2009148089A1 (fr) Appareil de manutention, dispositif de commande, procédé de commande et programme
JP5686108B2 (ja) 誤作業防止装置および誤作業防止装置が設けられた仕分け設備
CN113731862A (zh) 用于提供各种物体的分拣的分拣系统和方法
JP2001252886A (ja) 物体のハンドリングシステム
JP2012115785A (ja) 廃棄物の選別システム
EP3866988B1 (fr) Système de tri de matériaux provenant de déchets collectés
JP2018122370A (ja) 少なくとも１つのセンサを用いた物品搬送装置
CN114405866B (zh) 视觉引导钢板分拣方法、视觉引导钢板分拣装置和系统
JP6643921B2 (ja) 仕分装置および物品取出方法
WO2023067656A1 (fr) Dispositif de traitement d'objets
CN112791973A (zh) 物流分拣系统
JP2965886B2 (ja) 被識別物の選別装置
CN210753841U (zh) 物流分拣平台与物流分拣系统
Alitappeh et al. Revolutionizing Waste Management: A Smart Materials Recovery Facilility With Robotic and AI Integration
JP6842076B1 (ja) 廃棄物選別システム、及び廃棄物選別方法
KR102717166B1 (ko) 객체 이동속도 추정 기능을 구비한 인공지능 기반의 재활용품 선별 장치
WO2021217923A1 (fr) Système de tri coopératif homme-machine et procédé de tri de multiples types de matériaux utilisant ledit système
WO2023084713A1 (fr) Système de tri d'objets
CN118103880A (zh) 用于在物体处理系统中辅助物体辨识的系统和方法
WO2023119490A1 (fr) Système de tri de récipients

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21961309

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023553909

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21961309

Country of ref document: EP

Kind code of ref document: A1