Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T1/00—General purpose image data processing
  • G06T1/0014—Image feed-back for automatic industrial control, e.g. robot with camera
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B29—WORKING OF PLASTICS; WORKING OF SUBSTANCES IN A PLASTIC STATE IN GENERAL
  • B29D—PRODUCING PARTICULAR ARTICLES FROM PLASTICS OR FROM SUBSTANCES IN A PLASTIC STATE
  • B29D30/00—Producing pneumatic or solid tyres or parts thereof
  • B29D30/06—Pneumatic tyres or parts thereof (e.g. produced by casting, moulding, compression moulding, injection moulding, centrifugal casting)
  • B29D30/0601—Vulcanising tyres; Vulcanising presses for tyres
  • B29D30/0606—Vulcanising moulds not integral with vulcanising presses
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B29—WORKING OF PLASTICS; WORKING OF SUBSTANCES IN A PLASTIC STATE IN GENERAL
  • B29D—PRODUCING PARTICULAR ARTICLES FROM PLASTICS OR FROM SUBSTANCES IN A PLASTIC STATE
  • B29D30/00—Producing pneumatic or solid tyres or parts thereof
  • B29D30/06—Pneumatic tyres or parts thereof (e.g. produced by casting, moulding, compression moulding, injection moulding, centrifugal casting)
  • B29D30/0601—Vulcanising tyres; Vulcanising presses for tyres
  • B29D30/0606—Vulcanising moulds not integral with vulcanising presses
  • B29D30/0629—Vulcanising moulds not integral with vulcanising presses with radially movable sectors
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0004—Industrial image inspection
  • G06T7/001—Industrial image inspection using an image reference approach
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/60—Analysis of geometric attributes
  • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/70—Determining position or orientation of objects or cameras
  • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
  • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/20—Image preprocessing
  • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/40—Extraction of image or video features
  • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
  • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
  • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/61—Control of cameras or camera modules based on recognised objects
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B29—WORKING OF PLASTICS; WORKING OF SUBSTANCES IN A PLASTIC STATE IN GENERAL
  • B29D—PRODUCING PARTICULAR ARTICLES FROM PLASTICS OR FROM SUBSTANCES IN A PLASTIC STATE
  • B29D30/00—Producing pneumatic or solid tyres or parts thereof
  • B29D30/06—Pneumatic tyres or parts thereof (e.g. produced by casting, moulding, compression moulding, injection moulding, centrifugal casting)
  • B29D30/0601—Vulcanising tyres; Vulcanising presses for tyres
  • B29D30/0606—Vulcanising moulds not integral with vulcanising presses
  • B29D2030/0607—Constructional features of the moulds
  • B29D2030/0617—Venting devices, e.g. vent plugs or inserts
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10024—Color image
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10028—Range image; Depth image; 3D point clouds
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20084—Artificial neural networks [ANN]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30108—Industrial image inspection
  • G06T2207/30164—Workpiece; Machine component
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
  • G06V2201/06—Recognition of objects for industrial automation

Definitions

• a system and method for inserting valves into segments of a baking mold for tires is disclosed. More particularly, the invention relates to a system and method for identifying vents of a vulcanization mold for tires whose vents are dispersed to allow the insertion of the corresponding valves therein.
• this type of mold is represented by a mold 10 mainly comprising two shells (not shown) which each mold one of the side walls of a tire P, a plurality of segments 12 which mold the strip rolling Pio of the tire P along the internal surfaces 12a of the segments.
• the segments 12 are movable radially between an open position (shown in Figure 1) and a closed position of the mold 10.
• This type of mold can further comprise at least one clamping ring (not shown) to allow the radial movement of the segments.
• An example of this type of mold is disclosed by the Applicant's patent US10,239,270.
• this type of mold includes a plurality of ventilation holes (or “vents”) to achieve this ventilation during the vulcanization cycles.
• a typical segment mold may include between 4,000 and 12,000 substantially cylindrical vents distributed along each segment of the mold.
• the valve 20 comprises a movable insert 22 which moves up and down in a substantially cylindrical housing 24.
• the movable insert 22 includes a valve stem 26 with a conical section 26a frustrated towards an internal cavity 28 (see Figure 2) and a flat surface 26b towards the surface of the tire.
• the conical section 26a mates with a seating surface 24a of the housing 24 such that, during a vulcanization cycle, the valve is closed by the surface of the approaching tire blank, and, during extraction of the tire, the valve reopens after vulcanization.
• a gasket (not shown) can be placed between the conical section 26a and the seat surface 24a in a manner understood by those skilled in the art.
• valves come in the form of small tubular and rigid mechanical parts (for example, with a diameter of around 2.5mm and a length of 5 to 12mm).
• their installation in the mold consists of force-mounting in vents drilled to a diameter guaranteeing the adjustment and holding of the valves throughout the life of the mold. The installation operation requires:
• valves are placed individually in the mold segments (either by a human operator or a mechanical operator such as a robot). This is usually done using a tweezer-type tool that grips the valve and inserts it precisely into the corresponding vent in the mold. The valve is then hammered into the vent using a hammer and mandrel. This type of insertion requires a lot of effort and takes a lot of time. Each insertion represents several seconds of work, leading to a painful, repetitive task which, for a human operator, is not of great interest. This leads to risks of weariness and forgetting valves, calling into question the proper functioning of the mold. To overcome this problem, there are devices in the prior art for inserting valves into molds.
• German publication DE102010060901 discloses a tool comprising a tubular guide system in which a valve is arranged.
• the tubular system is placed directly above the vent where, by an effort in the axis of the valve, a piston pushes the valve to push it in in a guided and regulated manner.
• a piston pushes the valve to push it in in a guided and regulated manner.
• the piston rises, and a new valve engages in the tubular system.
• Automation therefore lies in the vertical positioning of the vents, but the valves must be positioned precisely so that it finds its bearings.
• Korean patent KR100845093B discloses a valve assembly system incorporating a machine for manufacturing valves in two parts: a body in which the spring is installed and the valve itself.
• the machine can be diverted to serve as the basis for a valve fitting system to insert the valves into the vents.
• it lacks the ability to adapt to any mold shape and also move to position the valves in segments.
• the holes which create the vents are not always made as indicated on the plans, and there are variations due to the manufacturing process causing deviations (for example, vents are added, or molds are modified by hand ). Since the precise knowledge of the position of the vents and/or their axes is not absolutely guaranteed, it is desirable to develop a system which can do without this information, as would a human operator who detects and analyzes himself.
• the disclosed invention employs knowledge of the mold segment to carry out the insertion of the valves in a repetitive manner.
• the insertion of the valves is done with an effort of up to around 70kg, which requires good control of the trajectory of a robot so as not to damage the mold.
• the disclosed invention uses the coordinates of the vents and the detection of their centers and their normals? to give a robot the correct approach and push trajectory to facilitate the installation of valves.
• the invention relates to a system implementing a method for identifying vents of a vulcanization mold for tires comprising one or more segments and an internal surface whose vents are dispersed to allow the insertion of the corresponding valves therein, characterized in that the system comprises: a robot incorporating a detection system with one or more sensors that detect the presence of vent(s) dispersed along the internal surface of the mold segment; a communications network that manages data entering the system from the detection system; and one or more communication servers, each comprising one or more processors operably connected to a memory configured to store an application for analyzing data representative of the imaged molds, and the one or more processors comprising a module for executing the analysis application which carries out image processing, the processor(s) of which are capable of executing programmed instructions stored in the memory to carry out the following steps: a step of detecting the presence of an arrangement of vents in the field of view of the detection system, which triggers to capture at least one image of the internal surface of the mold segment; and a
• the detection system comprises at least one three-dimensional (3D) camera of the RGB-D type attached to the robot which provides 3D images represented in a set of 3D points with coordinates (X , Y, Z)
• the processor(s) are capable of executing programmed instructions stored in the memory to carry out the following steps: a step of annotating the sample positions of the vents, this step comprising a step of creating a coordinate reference of the vents searched in images captured by the detection system of the system; a step of reconstructing the segment comprising a step of constructing an annotated database storing captured images and coordinates of the pixels of the captured images; a step of analyzing the contours of the vents carried out by the system analysis application execution module, this step comprising a step of determining the surface plane by finding the shape closest to a circle which represents the desired vent, this step further comprising a step of determining the vector normal to the surface plane determined to find the axis of insertion
• the execution module of the analysis application stored in the system memory employs annotation software to construct bounding boxes around the vents appearing in the image captured from the mold.
• the processor of the system continuously trains at least one neural network whose output is the classification of the coordinates of the vents, such that the captured images reveal the positions of the vents.
• the at least one neural network is chosen from convolutional neural networks.
• one or more steps employ the use of a neural network of the deformable transform type.
• the robot includes a gripping device supported by a pivotable elongated arm, the gripping device extending from the elongated arm to a free end where a gripper is disposed along 'a common longitudinal axis.
• the gripper includes a pivoting gripper incorporating gripping fingers which extend from a platform where attachment of the gripper to the free end of the gripping device is accomplished, each finger comprising a member with a predetermined length which extends between an actuating end, where movement of the finger is achieved, and an opposing gripping end, where the finger grips the valve.
• the processor(s) are capable of executing programmed instructions stored in the memory to perform a step of setting the robot in motion so that it can place the valve for insertion into a vent identified in a segment of the mold.
• the invention also relates to a method implemented by the disclosed system for identifying vents of a vulcanization mold for tires comprising one or more segments and an internal surface whose vents are dispersed to allow the insertion of the valves. corresponding therein, characterized in that the method comprises the following steps: a step of positioning the mold in a field of view of a detection system of the system, so that the vents defined along the internal surface of at least a segment is visible, during which the detection system flies over the mold; a step of detecting the presence of an arrangement of vents in the field of view of the detection system, which triggers to capture at least one image of the internal surface of the mold segment; and a step of searching, in the image captured by the detection system, for the presence of the vents detected, so that the detection system continues to capture the images if no vent is detected, until the search for the mold is exhausted.
• the method further comprises a control step carried out after the insertion of the valves into the vents of the mold. In certain embodiments of the method of the invention, the method further comprises a final step of positioning the robot directly above an identified vent, in the axis of insertion thereof, during which the robot blows the valve.
• Figure 1 represents a perspective view of one embodiment of a segment type vulcanization mold.
• Figure 2 represents an embodiment of a valve inserted in a vent of the mold of Figure 1.
• Figure 3 represents a schematic view of a system of the invention allowing the insertion valves in a tire vulcanization mold.
• Figure 4 represents an internal surface of a segment of a tire vulcanization mold whose vents are identified by the system of Figure 3.
• Figure 5 shows an example of annotated bounding boxes that are constructed around vents appearing in an image of a mold captured in a process performed by the system of the invention.
• Figure 6 shows an example of the points that define vents appearing on an image of a mold captured in a process performed by the system of the invention.
• Figure 7 represents an example of the points extracted in a process carried out by the system of the invention and forming one or more ellipses.
• Figure 8 represents a normal vector of a vent identified in an image of a mold captured in a method carried out by the system of the invention.
• Figure 9 represents vents identified by contours analyzed during a process carried out by the system of the invention.
• Figure 3 represents a valve insertion system (or “system”) 100 of the invention.
• the system 100 implements a method of the invention allowing the insertion of valves (for example, valves of the type shown in Figure 2) into segments of a vulcanization mold for tires (for example, a mold 10 of the type shown in Figure 1 and having segments 12).
• the disclosed method incorporates a machine learning method which is based on the data corresponding to the images obtained from the mold whose algorithm used analyzes the internal surface of the mold to place and insert the valve into an identified vent.
• a mold 10 is positioned on a work table or on an equivalent support 50 so that the system 100 can process it.
• the support 50 can be configured to move rotatably, reciprocating vertically and/or reciprocating horizontally, thereby allowing the processing of a variety of molds.
• the system includes a robot 102 having a gripping device 104 supported by an elongated pivoting arm 106.
• the gripping device 104 extends from the elongated arm 106 to a free end 104a where a gripper 108 is arranged along a common longitudinal axis.
• Fixing the gripper 108 to the gripping device 104 can be achieved by screwing an adapter to the free end 104a of the gripping device. It is understood that the attachment of the gripper 108 to the gripping device 104 can be accomplished by one or more known attachment means (including, without limitation, welding, gluing and equivalent means).
• the gripper includes a pivotable gripper 108a incorporating gripping fingers (or “fingers") 108b which extend from a platform 108c (where the adapter provides attachment of the gripper 108a to the free end 104a of the gripping device 104).
• Each finger 108b includes a member with a predetermined length that extends between an actuation end (where movement of the finger is accomplished) and an opposite gripping end (where the finger grips a valve 200 held by the gripper during the process).
• Each finger 108b has an internal gripping surface that engages the valve 200 during insertion into an identified vent and an opposing external surface.
• the fingers 108b are arranged so that a predetermined space is defined between the gripping surfaces, allowing movement of the fingers along a common axis during the process implemented by the system 100.
• the robot 102 facilitates gripping of a variety of valves without interruption of the linear movement of the fingers.
• the reciprocating movement of one or more fingers 108b can be carried out by one or more known cylinders which are actuated by a pressurized fluid (for example, compressed air) coming from a conduit (not shown). Consequently, the movement of each finger 108b achieves the corresponding linear movement of the fingers between a waiting position (where the gripping surfaces remain substantially parallel with the space between them) (not shown) and a gripping position (where the gripping surfaces approach to engage the valve 200 and to place it in an insertion position relative to an internal surface 10a of the mold 10) (see Figure 3).
• the cylinder(s) are chosen from commercial cylinders.
• the robot 102 can be set in motion so that the gripper 108 can grasp the valve 200 (as described below). Thanks to the fingers 108b, the gripper 108 grips to hold the valve 200 during movement of the gripper between a gripping position (in which the gripper 108 grips a valve chosen for insertion into a vent corresponding identified) (see Figure 3) and an insertion position (in which the gripper 108 places the valve taken to insert it into the identified vent) (not shown).
• the gripping position of the gripper 108 means that the fingers are in their gripping position of the chosen valve.
• the robot can be configured to have six degrees of freedom allowing it to move on all six axes.
• the robot 102 may be disposed on a support 55 that is configured to move rotatably, alternately vertically, and/or alternately horizontally, thereby allowing processing of a variety of molds.
• the robot 102 is set in motion to place the valve 200 for insertion into a vent identified in a segment 10A of the mold 10.
• the robot 102 may be part of a traveling robot which may be placed in motion either by integrated movement means (for example, integrated motor(s)) or by non-integrated movement means (for example, autonomous mobile trolley(s) or other equivalent mobile means).
• the robot 102 can be attached to a ceiling, a floor, a wall or any support which allows the carrying out of the method implemented by the system 100 (see, for example, support 55 in Figure 3). It is understood that such a robot can be a classic industrial robot or a collaborative robot or even a delta or cable robot.
• the robot 102 includes a sensing system that uses one or more sensors (not shown) to sense information about the physical environment around the robot.
• sensors may be used interchangeably and may refer to one or more devices configured to perform two-dimensional image sensing ( 2D) and/or three-dimensional (3D), 3D depth sensing, and/or other types of sensing of the physical environment around the robot 102.
• the sensors of the robot 100 detection incorporated with the robot 102 can be attached to the elongated arm 106 (for example, at the end 104a) and/or to the gripper 108 of the robot.
• the sensor(s) of the robot detection system 102 detect the presence of one or more vents of a mold.
• an internal surface 10a of a segment 10A of a vulcanization mold 10 is shown in Figure 4 ( Figure 4 represents a photo of the internal surface 10a taken with an RGB type camera).
• a plurality of vents 150 are dispersed along the internal surface 10a of the segment 10A, each vent receiving a corresponding valve 200. It is expected that each vent is substantially cylindrical and that all vents 150 have substantially the same diameters.
• the sensor is triggered when a segment of a mold enters the camera's field of view.
• a snap point may be placed at a known position relative to the sensor (for example, at a known horizontal distance and at a known vertical distance from the sensor position).
• the detection system can determine information about the physical environment around the mold 10 which can be used by a control system of the system 100 (the control system comprising, for example, software for planning movements of the robot 102). .
• the control system could be on robot 102 or it could be in remote communication with the robot.
• one or more 2D or 3D sensors mounted on the robot 102 may be integrated to constitute a digital model of the physical environment (including, where applicable, the side(s), floor and ceiling).
• the control system can cause the movement of the robot 102 to navigate between the tapping positions of the valves during their insertion into the mold 10.
• the detection system comprises at least one camera which provides 3D images represented in a set of 3D points with coordinates (X, Y, Z), and sometimes red, green color values , blue (“RGB” or “RGB-D” format) (called “an RGB-D type camera”).
• an RGB-D type camera is attached to the robot 102 (for example, to the end 104a and/or to the gripper 108). Two or more RGB-D cameras can be oriented to achieve a predetermined overlap between the cameras' fields of view.
• the term "camera” includes one or more cameras.
• RGB-D cameras typically provide depth information using depth maps, being images where each pixel contains the distance between the camera and the corresponding point in space.
• 3D point cloud data from RGB-D type cameras have a much higher measurement rate.
• a point cloud can be constructed from the RGB-D images by computing the real world (e.g., (X, Y, Z) coordinates with intrinsic data from a scanning camera.
• information about the physical environment around the system 100 is obtained from 3D point cloud data obtained from sensing technologies that are capable of capturing the 3D surface geometries of the molds accurately and efficiently. detection could be chosen from commercially available devices (chosen, for example, from cameras sold under the ZIVID® brand from the company Zivid AS, artificial vision systems sold by the company Cognex Corp., and their equivalents) .
• point cloud (singular or plural) is used here to refer to one or more collections of data points in space.
• One or more cameras can collect three-dimensional (3D) data and detect the surfaces of objects (for example, a segment 10A of a mold 10) using a series of coordinates. Storing information as a collection of spatial coordinates can save space because many objects do not fill a large portion of the environment. Even if the information is not visual, interpreting data as a scatterplot helps understand the relationship between multiple variables through classification and segmentation.
• the detection system of the system 100 may include a telemeter means which is used in the working space of the mold 10 to deduce its dimensions.
• the telemeter means comprises a scanner (not shown) for scanning the entire internal surface 10a of the mold 10 in real time in the physical environment around the mold.
• a scanner allows precise generation of the mold.
• the scanner can be provided together with a vision system (not shown) configured to precisely locate vents in a real-time scenario based on the 3D profile generated by the scanner.
• the vision system may receive a CAD file from the mold 10 to match the location of a vent from the CAD file, with the vent identified in real time to accurately locate and determine its coordinates.
• the vision system can receive the CAD file by data transmission methods known to those skilled in the art.
• the vision system may further include at least one camera and at least one sensor (not shown) to determine the location (i.e., coordinates) of the vents based on the data collected in real time and/or the contour profile. generated by the scanner.
• the system 100 comprises a communications network (or “network”) which manages data entering the system from various sources (for example, from at least a robot 102 and the associated detection system).
• the communications network incorporates one or more communications servers (or “servers”) each comprising one or more processors operably connected to a memory.
• the memory is configured to store an application for analyzing data representative of the molds (and segments of the molds) imaged.
• the processor(s) include an analysis application execution module which performs image processing, the processor(s) of which are capable of executing programmed instructions stored in the memory to carry out the steps of the method (as described below). below).
• processor means one or more devices capable of processing and analyzing data and including one or more software for their processing (for example, one or more integrated circuits known to those skilled in the art as being included in a computer, one or more controllers, one or more microcontrollers, one or more microcomputers, one or more programmable logic controllers (or “PLCs”), one or more integrated circuits specific to an application, one or more neural networks, and/or one or more other known equivalent programmable circuits).
• the processor includes software(s) for processing the data captured by the detection system of the system 100 (and the corresponding data obtained) as well as software(s) for identifying and locating variances and identifying their sources to correct them.
• the memory may include both volatile and non-volatile memory devices.
• the non-volatile memory may include solid state memories, such as NAND flash memory, “random access” memory (or “keep-alive memory” or “KAM”) to save various operating variables while the processor is powered off, magnetic and optical storage media, or any other suitable data storage device that retains data when the system 100 is disabled or loses power.
• the volatile memory may include static and dynamic RAM that stores program instructions and data, including a learning application.
• method or “process” may include one or more steps performed by at least one computer system having one or more processors to execute instructions that perform the steps. Unless otherwise indicated, any sequence of steps is given by way of example and does not limit the methods described to any particular sequence.
• the system 100 incorporates a combination of vision and machine learning techniques to correctly and quickly reconstruct the observed scene from scattered three-dimensional (or “3D”) point clouds, resulting from a view of the segment 10A of the mold 10.
• the system 100 therefore achieves continuous improvement in the recognition of vents and their relative distribution along the internal surface 10a of the mold 10.
• the method comprises a step of positioning the mold 10 in the field of view of the robot detection system 102 (for example, positioning the mold on the support 50 as shown in Figure 3).
• the mold 10 is positioned so that the vents 150 defined along the internal surface 10a of at least one segment 10A are visible in the detection field of the sensor (see Figure 4).
• the robot 102 and particularly the integrated detection system flies over the mold 10.
• the method further includes a step of annotating the positions of samples of the vents devoted to learning.
• a reference is created of the coordinates of the vents searched in images captured by the detection system of the robot 102 (for example, an RGB type camera).
• the coordinate reference of the vents which is created during this step includes expected images corresponding to the vents 150 distributed along the internal surface 10a of the mold 10.
• This step can be carried out in advance of other steps of the method of the invention to feed a neural network the true coordinates of the vents and their relative positions with respect to each other.
• at least part of the vent reference can be created by one or more persons skilled in the art.
• a neural network may be trained to recognize the true coordinates of the vents and create bounding boxes (or "boxed regions") around the recognized vents.
• the coordinates of the bounding box of the recognized vent are correlated with the coordinates of the vents sought to calculate displacements between them.
• the framed regions and displacement calculations are transmitted to a neural network (e.g., CNN(s)) to jointly learn the representation of a vent in perspectives different from the images taken by the detection system of the robot 102.
• the method further comprises a step of capturing images of the mold 10 (and more particularly, capturing images of the internal surface 10a of the segment 10A of the mold).
• This step which is carried out by the robot 102 (and particularly by the associated detection system), includes a step of scanning the detection system of the robot 102 above the segment 10A of mold 10.
• Each image captured during this step is composed of a matrix of pixels where each pixel has a different color and a brightness which indicates the position of a vent 150 of the mold 10.
• the images obtained, revealing one or more positions of the vents 150 train at least one neural network to identify all the expected positions of the vents in the imaged mold 10 .
• these image variations serve as input to the neural network whose outputs are the classification of the coordinates of the vents.
• the execution module's algorithm aims to automatically identify and indicate the external profile of the vent as well as the interfaces and perimeters of the vent (for example, its diameter and angle of its cylindrical axis relative to the curvature of the internal surface 10a of the mold 10).
• the execution module therefore uses annotation software making it possible to construct bounding boxes around the vents 150 appearing in the image of mold 10 (see Figure 7).
• the system processor 100 continuously trains the neural network from newly captured data from the mold images obtained by the robot detection system 102.
• the robot 102 takes images (which may include videos) and collects a set of image data from multiple molds of the same type (e.g., of the type represented by mold 10 in Figure 4).
• the image dataset can be annotated based on the data input by the operator to create the ground truth data.
• the entire image data set is annotated. Known variations are manually identified based on the knowledge of mold professionals.
• the server processor can use the ground truth data to train and/or develop one or more neural networks in order to automatically detect the space where the object (for example, the vents 150 of the mold 10).
• ground truth data as described here generally refers to information provided by direct observation of professionals in the field as opposed to information provided by inference. They can have data from multiple sources, including multiple professionals in remote locations, to develop the neural network.
• An image feedback loop Annotated data may be updated with additional ground truth data over time to improve the accuracy of the system 100.
• the method further comprises a step of reconstructing the segment 10A by three-dimensional digitization with a high degree of resolution.
• a 3D camera of the detection system is used in the work space of the mold 10 to immediately reconstruct the volume of the segment 10A and to determine its dimensions and delimit its work area.
• This step includes the construction of an annotated database storing the RGB images, the coordinates (X,Y,Z) of the pixels of the images obtained as well as the coordinates of the bounding boxes.
• the execution module algorithm aims to extract the points which define each vent 150 (see Figure 8).
• the method further comprises a step of analyzing the contours of each vent 150.
• the 3D camera of the robot detection system 102 by the width of its measurement spectrum, gives better precision by calculating the normal on the homogenized plane on the internal surface 10a of the mold 10.
• each of the extracted points (including the center C150) has coordinates corresponding.
• This step includes a step of determining the surface plane by finding the shape closest to the circle represented by the desired vent. Due to the positioning of the vent along the internal surface 10a of the mold 10 (having substantially curved parts), it is understood that the extracted points can form one or more ellipses (see the ellipses given as an example in Figure 9).
• This step also includes a step of determining the vector normal to the surface plane determined during the previous step.
• a normal vector determined passes the center C150 of the vent 150 to find the insertion axis X200 of the valve 200 (oriented, for example, at an angle a relative to the internal surface 10a of the mold 10).
• This step will allow the robot 102 to choose and orient a valve of appropriate diameter (for example, a valve of the type shown in Figure 2) to ensure its correct insertion in the corresponding vent.
• the method further comprises a step of determining the diameter of the vent allowing the drilling of a valve of the corresponding diameter.
• each vent 150 is identified by a contour 150A analyzed in the previous step.
• the corresponding C150 center is identified by a dot.
• the execution module's algorithm aims to recognize the diameter which corresponds closest to the diameters of the known vents (known, for example, in the vent reference created during the annotation step of this embodiment of the process).
• the robot 102 can proceed with the pressing, either through the valve feed head, or by pushing it with a dedicated zone of an effector deposited at end 104a (not shown).
• the use of neural networks provides robustness in the determination of vents and above all speed which eliminates lengthy calculations.
• neural networks and more particularly convolutional neural networks, or "CNN” as a machine learning model, other types machine learning models can be used.
• the CNN(s) can be trained with ground truth data which is generated using sensor data representative of the movement of the robot 102, including the positioning of the gripper 108.
• DETR deformable DETR
• DETR is used for end-to-end object detection, by combining CNN-type neural networks and Transformer-type encoder-decoders.
• DETR first reduces calculations by only looking at a small set of key sample points around a reference (for example, the points form contour 150A around a vent 150 of mold 10) (see Figure 9).
• DETR uses a deformable attention module to aggregate multi-scale features to facilitate detection of small objects. Therefore, DETR can model dependencies between distant objects in the observed scene to achieve the ability to automatically and accurately detect, locate, and classify vents whose corresponding valve insertion is predicted.
• the method may further comprise an optional control step after the insertion of the valves 200 into the vents 150 of the mold 10.
• an operator can carry out a manual control unitary of everything that the robot 102 offers.
• a fully automatic check can be carried out, involving presence detection and/or a probe to validate the presence and correct operation of the valves.
• the system 100 of the invention may include preprogramming of information regarding expected events.
• a setting of the method of the invention may be associated with the parameters of typical physical environments in which the system 100 operates (for example, tire production facilities).
• the system 100 (or another system incorporating the system 100) may receive audio commands (including voice commands) or other audio data representing (for example, a walk or a stopping one or more steps of the process of the invention).
• the request may include a request for the current state of a process in progress (e.g., the number of valves inserted versus the number of vents 150 in mold 10 intended to accommodate a corresponding vent).
• a generated response may be represented audibly, visually, tactilely (e.g., using a haptic interface), and/or virtually and/or augmented. This response, together with the corresponding data, can be recorded in a neural network.
• system 100 may include multiple computing devices that perform various aspects of learning.
• the processor can configure the system 100 on one or more parameters of a vent and its known location.
• one or more means of reinforcement learning could be used.
• a monitoring system could be put in place. At least part of the monitoring or alerting system may be provided in a portable device such as a mobile network device (e.g., a mobile phone, a laptop computer, a portable device(s) connected to the network (including including “augmented reality” and/or “virtual reality” devices, wearable clothing connected to the network and/or all combinations and/or all equivalents). It is conceivable that detection and comparison steps could be carried out iteratively .
• a mobile network device e.g., a mobile phone, a laptop computer, a portable device(s) connected to the network (including including “augmented reality” and/or “virtual reality” devices, wearable clothing connected to the network and/or all combinations and/or all equivalents). It is conceivable that detection and comparison steps could be carried out iteratively .

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Multimedia (AREA)
• Evolutionary Computation (AREA)
• Mechanical Engineering (AREA)
• Artificial Intelligence (AREA)
• Medical Informatics (AREA)
• Software Systems (AREA)
• Computing Systems (AREA)
• Databases & Information Systems (AREA)
• Health & Medical Sciences (AREA)
• General Health & Medical Sciences (AREA)
• Quality & Reliability (AREA)
• Signal Processing (AREA)
• Geometry (AREA)
• Robotics (AREA)
• Moulds For Moulding Plastics Or The Like (AREA)
• Heating, Cooling, Or Curing Plastics Or The Like In General (AREA)
• Image Analysis (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=84331362

Family Applications (1)

Country Status (5)

Families Citing this family (2)

Family Cites Families (5)

• 2022
  • 2022-07-13 FR FR2207219A patent/FR3137867B1/fr active Active
• 2023
  • 2023-07-05 EP EP23738739.4A patent/EP4554784A1/de active Pending
  • 2023-07-05 US US18/993,038 patent/US20260004378A1/en active Pending
  • 2023-07-05 CN CN202380052943.6A patent/CN119546456A/zh active Pending
  • 2023-07-05 WO PCT/EP2023/068606 patent/WO2024012960A1/fr not_active Ceased

Also Published As

Similar Documents

Legal Events