CN118154566A - Method and device for determining chip placement parameters in tray - Google Patents
Method and device for determining chip placement parameters in tray Download PDFInfo
- Publication number
- CN118154566A CN118154566A CN202410341526.XA CN202410341526A CN118154566A CN 118154566 A CN118154566 A CN 118154566A CN 202410341526 A CN202410341526 A CN 202410341526A CN 118154566 A CN118154566 A CN 118154566A
- Authority
- CN
- China
- Prior art keywords
- tray
- point
- chip placement
- size
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000001514 detection method Methods 0.000 claims abstract description 16
- 238000013136 deep learning model Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 16
- 230000007246 mechanism Effects 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000009826 distribution Methods 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 230000009977 dual effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000005452 bending Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000000877 morphologic effect Effects 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- WYTGDNHDOZPMIW-RCBQFDQVSA-N alstonine Natural products C1=CC2=C3C=CC=CC3=NC2=C2N1C[C@H]1[C@H](C)OC=C(C(=O)OC)[C@H]1C2 WYTGDNHDOZPMIW-RCBQFDQVSA-N 0.000 description 1
- 239000011324 bead Substances 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000003754 machining Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30072—Microarray; Biochip, DNA array; Well plate
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a method and a device for determining chip placement parameters in a tray, wherein the method comprises the following steps: acquiring a tray image of a tray to be positioned on the full-automatic burner, performing size detection to obtain a size point, and constructing tray point cloud data by using the tray image; matching the tray point cloud data with a pre-established size form type library to obtain a target size form type; inputting the target size, shape and type and the tray image into a pre-trained intelligent chip placement area identification model to obtain a candidate chip placement area; selecting a chip placement point from the candidate chip placement area to determine the position coordinates of the chip placement point; calculating the clutch degree of the tray between the size point and the chip placement point on the tray image; calculating attitude parameters and rotation direction values of the chip placement points according to the tray clutch degree; and sending the position coordinates, the attitude parameters and the rotation direction values to tray positioning equipment as chip placement parameters. The invention can ensure that the chip to be detected is accurately fed into the chip placement area of the tray.
Description
Technical Field
The invention relates to the field of file management, in particular to a method and a device for determining chip placement parameters in a tray.
Background
In the process of determining the chip placement parameters in the tray, the current industry adopts a manual mode to determine, namely, the tray is fixed by a manual mode, then the Z axis is moved to the upper part of the tray, the height is determined by a manual positioning mode, then the array of the tray is manually arranged, the fixed positions of 2 trays are positioned, the fixed positioning points of 3 trays are also arranged, and the chip placement parameters in the tray are further obtained by a manual mode. Therefore, the positioning of chips in trays in the prior art depends on manual work, resulting in low efficiency, and meanwhile, the chip placement parameters in the trays are affected by subjective factors of the manual work and have deviation.
Disclosure of Invention
Aiming at the technical problems in the prior art, the invention provides a method and a device for determining the placement parameters of chips in a tray, which are used for solving the problem of low efficiency caused by the fact that the positioning of the chips in the tray in the prior art depends on manual work, realizing the combination with automatic positioning equipment of the tray, and accurately providing the placement parameters of the chips in the tray.
The technical scheme for solving the technical problems is as follows:
A method for determining chip placement parameters in a tray comprises the following steps:
acquiring a tray image of a tray to be positioned on a full-automatic burner, performing size detection on the tray image to obtain a size point, and constructing tray point cloud data by utilizing the tray image;
Matching the tray point cloud data with a pre-established size form type library to obtain a target size form type;
Inputting the target size, shape and type and the tray image into a pre-trained intelligent chip placement area identification model to obtain an output candidate chip placement area;
Selecting a chip placement point from the candidate chip placement area, and determining the position coordinates of the chip placement point according to the tray point cloud data;
Calculating the tray clutch degree between the size point and the chip placement point on the tray image;
calculating attitude parameters and rotation direction values of the chip placement points according to the tray clutch degrees, wherein the rotation direction values represent the rotation direction of a rotation mechanism of the tray to be positioned;
and sending the position coordinates, the attitude parameters and the rotation direction values to tray positioning equipment as chip placement parameters.
According to the method for determining the chip placement parameters in the tray provided by the invention, the method for acquiring the tray image of the tray to be positioned on the full-automatic burner, performing size detection on the tray image to obtain the size point, and constructing the tray point cloud data by utilizing the tray image comprises the following steps:
When the laser correlation sensor detects the size of the tray to be positioned, triggering a double-linear-array camera to photograph the size area of the tray to be positioned on the plane where the full-automatic burner is positioned to obtain a left-eye image and a right-eye image, and taking the left-eye image as a tray image;
performing size detection on the left eye image or the right eye image to obtain a size point;
And respectively carrying out distortion correction on the left eye image and the right eye image, respectively extracting angular points from the corrected left eye image and right eye image to serve as key points, reconstructing the key points by utilizing a parallax principle to obtain initial point cloud data, and removing data irrelevant to a tray to be positioned in the initial point cloud data to obtain tray point cloud data.
According to the method for determining the placement parameters of the chips in the tray, which is provided by the invention, the tray clutch degree between the size point and the placement point on the tray image is calculated, and the method comprises the following steps:
determining a linear equation according to the coordinates of the chip placement point and the size point;
extracting center line pixel point coordinates of a tray to be positioned between the size point and the chip placement point from the tray image;
Substituting the linear equation and the coordinates of the central line pixel points into a formula I to obtain the clutch degree of the tray;
wherein epsilon represents the clutch degree of the tray, n represents the number of central line pixel points, a linear equation is expressed as x=ay+b, X g represents the abscissa of the die-placement point, y g represents the ordinate of the die-placement point, x h represents the abscissa of the size point, and y h represents the ordinate of the size point.
According to the method for determining the chip placement parameters in the tray, which is provided by the invention, the attitude parameters and the rotation direction values of the chip placement points are calculated according to the clutch degree of the tray, and the method comprises the following steps:
Comparing the clutch degree of the tray with a preset value;
Calculating a first attitude parameter according to the size point and the chip placement point under the condition that the tray clutch degree does not exceed the preset value, wherein the first attitude parameter is expressed as X g represents the abscissa of the chip placement point, y g represents the ordinate of the chip placement point, x h represents the abscissa of the size point, and y h represents the ordinate of the size point;
Under the condition that the clutch degree of the tray does not exceed the preset value, determining an attitude point according to the chip placement point and the size point, and calculating a second attitude parameter according to the attitude point and the chip placement point, wherein the second attitude parameter is expressed as X g represents the abscissa of the chip placement point, y g represents the ordinate of the chip placement point, x z represents the abscissa of the posture point, and y z represents the ordinate of the posture point;
Calculating a rotation direction value according to a formula II;
Wherein p represents a rotation direction value, p=1 represents a clockwise rotation of the rotation mechanism, p= -1 represents a counterclockwise rotation of the rotation mechanism, y h represents a vertical position of a size point, y g represents a vertical coordinate of a chip placement point, and y z represents a vertical coordinate of a posture point.
According to the method for determining the chip placement parameters in the tray, provided by the invention, the tray point cloud data is matched with a pre-established size form type library to obtain the target size form type, and the method comprises the following steps:
respectively creating a corresponding tray size model for at least one known size form type by using modeling software to obtain a pre-created size form type library;
And respectively matching the tray point cloud data with each tray size model, and taking the size form type corresponding to the matched tray size model as the target size form type.
According to the method for determining the chip placement parameters in the tray, provided by the invention, the pre-trained intelligent chip placement area identification model is used for obtaining the candidate chip placement areas, and the candidate chip placement areas are obtained through training by the following steps:
Constructing a deep learning model, wherein the input of the deep learning model is a tray image and a size, shape and type text, and the output of the deep learning model is a preset shape candidate frame coordinate;
Labeling tray images of known optimal chip placement points based on input and output of the deep learning model to construct a training set, a verification set and a test set;
And training the deep learning model by using the training set, the verification set and the test set to obtain a pre-trained intelligent recognition model of the chip placement area.
According to the method for determining the placement parameters of the chips in the tray, before the plane where the full-automatic burner is located is photographed by using the double-line-array camera, the method further comprises the following steps:
Utilize the calibrator to mark double-line array camera, wherein, the upper cover plate of calibrator includes first ladder, second ladder and third ladder, wherein, first ladder the second ladder the length, the width of third ladder are the same, the height of first ladder is greater than the height of second ladder, the height of second ladder is greater than the height of third ladder, equidistant distribution a plurality of right triangle-shaped light trap on the first ladder, equidistant distribution a plurality of square light trap on the second ladder, equidistant and alternately distributed two rows of circular light trap on the third ladder.
The invention also provides a device for determining the placement parameters of the chips in the tray, which comprises the following steps:
The image acquisition module is used for acquiring a tray image of a tray to be positioned on the full-automatic burner, detecting the size of the tray image to obtain a size point, and constructing tray point cloud data by utilizing the tray image;
the matching module is used for matching the tray point cloud data with a pre-established size form type library to obtain a target size form type;
The chip placement area identification module is used for inputting the target size, shape and type and the tray image into a pre-trained chip placement area intelligent identification model to obtain an output candidate chip placement area;
the chip placement point determining module is used for selecting a chip placement point from the candidate chip placement area and determining the position coordinates of the chip placement point according to the tray point cloud data;
the first calculating module is used for calculating the tray clutch degree between the size point and the chip placement point on the tray image;
The second calculation module is used for calculating the attitude parameter and the rotation direction value of the chip placement point according to the tray clutch degree, wherein the rotation direction value represents the rotation direction of the rotation mechanism of the tray to be positioned;
The chip placement parameter determining module is used for sending the position coordinates, the attitude parameters and the rotation direction values to the tray positioning equipment as chip placement parameters.
The present invention also provides an electronic device including: a memory for storing a computer software program; and the processor is used for reading and executing the computer software program so as to realize the method for determining the chip placement parameters in the tray.
The invention also provides a non-transitory computer readable storage medium, characterized in that the storage medium stores a computer software program which, when executed by a processor, implements a method for determining a chip placement parameter in a tray as described above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements a method of determining a chip placement parameter in a tray as described in any one of the above.
The beneficial effects of the invention are as follows: the method comprises the steps of acquiring tray images of a tray to be positioned on a full-automatic burner, processing the tray images to obtain tray point cloud data of size points and shapes, identifying the target size shape type through the tray point cloud data, inputting the target size shape type and the tray images into a pre-trained intelligent chip placement area identification model to obtain candidate chip placement areas, selecting the chip placement points from the candidate chip placement areas and mapping the chip placement points onto the tray point cloud data of the shapes, so that position coordinates of the chip placement points are obtained, further calculating the tray clutch degree by using the chip placement points and the size points, calculating attitude parameters and rotation direction values of the grabbing points by combining the tray clutch degree, and finally transmitting the position coordinates, the attitude parameters and the rotation direction values to tray positioning equipment, so that accurate chip placement point information is automatically provided for the tray positioning equipment, and the chip to be detected can be accurately fed into the chip placement areas of the tray.
Drawings
FIG. 1 is a schematic flow chart of a method for determining chip placement parameters in a tray according to the present invention;
fig. 2 is a schematic structural view of an image acquisition apparatus provided by the present invention;
FIG. 3 is a schematic view of a stepped calibrator top cover provided by the present invention;
FIG. 4 is a second flow chart of a method for determining placement parameters of chips in a tray according to the present invention;
FIG. 5 is a schematic diagram of a device for determining placement parameters of chips in a tray according to the present invention;
fig. 6 is a schematic diagram of an embodiment of an electronic device according to an embodiment of the present invention;
fig. 7 is a schematic diagram of an embodiment of a computer readable storage medium according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In the description of the present application, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more of the described features. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the description of the present application, the term "for example" is used to mean "serving as an example, instance, or illustration. Any embodiment described as "for example" in this disclosure is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for purposes of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes have not been described in detail so as not to obscure the description of the application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
Fig. 1 is a schematic flow chart of a method for determining a chip placement parameter in a tray according to an embodiment of the present invention, referring to fig. 1, the method for determining a chip placement parameter in a tray is provided, and the method can be implemented through steps S101 to S107, and the following detailed description will be given with reference to each step:
Step S101, collecting a tray image of a tray to be positioned on the full-automatic burner, detecting the size of the tray image to obtain a size point, and constructing tray point cloud data by utilizing the tray image.
In this embodiment, the full-automatic burner is a device for burning chips. Since the size state of the tray affects the chip placement area where chips are fed into the tray, the size of the tray needs to be positioned by the tray positioning device to adjust the posture before the chips enter the chip placement area of the tray. The tray to be positioned refers to a tray which is placed on the full-automatic burner and needs to be adjusted in size state by tray positioning equipment. The tray image may be acquired by any existing image acquisition device.
Step S102, matching the tray point cloud data with a pre-established size form type library to obtain a target size form type.
In this embodiment, the pre-created size and shape type library includes multiple models of known tray size and shape types, and each model in the tray point cloud data and the size and shape type library is matched, for example, the similar or similar size and shape model is found by calculating similarity or characteristic parameters of some characterization forms, and the size and shape type corresponding to the found similar or same model is the target size and shape type, and the target size and shape type is one of multiple known tray size and shape types.
And step S103, inputting the target size, shape and type and the tray image into a pre-trained intelligent chip placement area recognition model to obtain an output candidate chip placement area.
In the embodiment, the intelligent chip placement area identification model can label the chip placement area on the tray image in a specific shape according to the input tray image and the target size and shape type corresponding to the tray image, and the intelligent chip placement area identification model can output the candidate chip placement area in a coordinate form.
And step S104, selecting a chip placement point from the candidate chip placement area, and determining the position coordinates of the chip placement point according to the tray point cloud data.
In this embodiment, the chip placement point is a point selected from the candidate chip placement areas, and the chip placement point may be selected according to the shape of the candidate chip placement area, for example, for a rectangular candidate chip placement area, the intersection point of the diagonal lines may be used as the chip placement point, and for a circular candidate chip placement area, the center of a circle may be selected as the chip placement point.
Step S105, calculating the tray clutch degree between the size point and the chip placement point on the tray image.
In this embodiment, the tray clutch degree refers to the bending degree of the tray section between the dimension point and the chip placement point by adopting numerical representation, and the tray clutch degree can be obtained by adopting any existing curve degree related measurement method.
And S106, calculating attitude parameters and rotation direction values of the chip placement points according to the tray clutch degree, wherein the rotation direction values represent the rotation direction of a rotation mechanism of the tray to be positioned.
Step S107, transmitting the position coordinates, the attitude parameters, and the rotation direction values as chip placement parameters to the tray positioning apparatus.
According to the method for determining the chip placement parameters in the tray, tray images of the tray to be positioned on the full-automatic burner are collected, tray point cloud data of size points and shapes are obtained through processing the tray images, target size form types are identified through the tray point cloud data, then the target size form types and the tray images are input into a pre-trained intelligent chip placement area identification model to obtain candidate chip placement areas, then the chip placement points are selected from the candidate chip placement areas and mapped onto the tray point cloud data of the shapes, so that position coordinates of the chip placement points are obtained, further, tray clutch degrees are calculated by the aid of the chip placement points and the size points, attitude parameters and rotation direction values of grabbing points are calculated by means of combining with the tray clutch degrees, and finally the position coordinates, the attitude parameters and the rotation direction values are sent to tray positioning equipment, so that accurate chip placement point information is automatically provided for the tray positioning equipment, and chips to be detected can be accurately sent into the chip placement areas of the tray.
In some possible implementations, the step S102, collecting a tray image of a tray to be positioned on the full-automatic burner, performing size detection on the tray image to obtain a size point, and constructing tray point cloud data by using the tray image, specifically includes:
When the laser correlation sensor detects the size of the tray to be positioned, triggering the double-linear-array camera to photograph the size area of the tray to be positioned on the plane where the full-automatic burner is positioned to obtain a left-eye image and a right-eye image, and taking the left-eye image as a tray image.
Fig. 2 is a schematic structural diagram of an image capturing apparatus according to the present invention, and referring to fig. 2, the image capturing apparatus includes: the camera is an array camera formed by two linear array cameras, a 450nm narrow-band filter is arranged on a lens, the camera is arranged on a regulator so as to adjust the initial posture, the baseline distance is 300mm, the center of a line of sight is vertical to the full-automatic recorder, and the center of the baseline coincides with the center plane of the full-automatic recorder; the light source is an array light source consisting of 2 blue linear array light sources, and the array light sources are symmetrically arranged on two sides of the camera, so that uniform and higher-power light supplementing can be provided; the calibrator is used for off-line calibration of the internal and external parameters of the array camera; the laser is in a correlation mode, the head and the tail of the tray can be detected, and the laser is arranged at the position of 0.6 m-1.0 m of the outlet of the serpentine vibrator. Along the motion direction of the full-automatic burner, the image acquisition equipment is integrally arranged at a position 0.4m away from the laser and is 3m high. The correlation laser sends a start signal to the camera when detecting the size of the tray, thereby accurately acquiring the tray image.
In the specific implementation process, blue light wavelength is preferable when the double-linear-array camera shoots an image, linearity is good, and the influence of ambient light and heat radiation of a chip to be detected can be well removed by matching with a narrow-band filter, so that a clear tray image is formed, and further, a high-precision tray outline is extracted.
Performing size detection on the left eye image or the right eye image to obtain a size point; the size detection may use either a left-eye image or a right-eye image, and the present invention is not limited to the image used for the size detection.
Illustratively, the size point can perform preprocessing such as mean filtering, histogram equalization and the like on the left-eye image, and perform tray size detection by using morphological characteristics of the size to determine the size point coordinates.
And respectively carrying out distortion correction on the left eye image and the right eye image, respectively extracting angular points from the corrected left eye image and right eye image to serve as key points, reconstructing the key points by utilizing a parallax principle to obtain initial point cloud data, and removing data irrelevant to a tray to be positioned in the initial point cloud data to obtain tray point cloud data. The initial point cloud data may include background data in addition to tray data, where the background may be a full-automatic burner, a sidewall, and the like.
In the specific implementation process, the calibrated internal reference data are used for correcting the left image and the right image respectively, the calibrated external reference is used for enabling the right image to coincide with the left image, key points of the images, namely angular points, are extracted for matching, the key points are reconstructed by utilizing the parallax principle, point cloud data of the whole frame of images are obtained, then the background data irrelevant to the tray to be positioned is removed by utilizing the placing characteristics of the tray, and therefore the tray point cloud data are obtained.
According to the method for determining the chip placement parameters in the tray, accurate chip placement of the size and the shape of the tray is achieved based on the double-linear-array camera, and the two-dimensional image data are converted into tray point cloud data, so that accurate chip placement point positions can be provided for tray positioning equipment.
In some possible implementations, the step S102, the matching the tray point cloud data with the pre-created size form type library to obtain the target size form type specifically includes:
respectively creating a corresponding tray size model for at least one known size form type by using modeling software to obtain a pre-created size form type library;
And respectively matching the tray point cloud data with each tray size model, and taking the size form type corresponding to the matched tray size model as the target size form type.
In the implementation process, based on expert experience of tray morphology types, matching models of different tray sizes, such as a size adherence morphology model, a head body fitting morphology model, a size return-bending morphology model and a size recumbent morphology model, are created by using modeling software, if tray morphology identification is unknown in the implementation process, images and corresponding point cloud data are automatically saved, a model can be quickly created according to the data, and a corresponding new size morphology type is added for the model of the new creation key.
The method for determining the chip placement parameters in the tray can realize the identification of various tray sizes and shapes by using model matching, has high accuracy, and has easy expansion of a size and shape type library and good adaptability.
In some possible implementations, the pre-trained chip placement area intelligent recognition model obtains candidate chip placement areas by training the following steps;
constructing a deep learning model, wherein the input of the deep learning model is a tray image and a size form type text, and the output of the deep learning model is a preset shape candidate frame coordinate;
Labeling the tray image of the known best chip placement point based on the input and output of the deep learning model to construct a training set, a verification set and a test set;
training the deep learning model by using a training set, a verification set and a test set to obtain the intelligent recognition model of the chip placement area trained in advance.
According to the method for determining the chip placement parameters in the tray, the deep learning model is utilized to learn the tray images of the formed type and the chip placement position, so that the deep learning model can predict the range of the chip placement points according to the size type recognition result of the tray form point cloud through model matching and the two-dimensional tray image, the intelligent decision based on the deep learning is realized to quickly and accurately position the chip placement points, and meanwhile, the tray form text information reinforcement learning is utilized.
In some possible implementations, the step S104 selects a chip placement point from the candidate chip placement areas, and determines the position coordinates of the chip placement point according to the tray point cloud data, which specifically includes:
Taking the center point of the candidate chip placement area as the optimal chip placement position to obtain a chip placement point; for example, when the candidate chip placement area is rectangular, the intersection point of the diagonal lines of the rectangular can be selected as the optimal chip placement position, i.e. the chip placement point, and when the candidate chip placement area is circular, the circle center can be selected as the optimal chip placement position, i.e. the chip placement point;
and acquiring the coordinates of the center point, and mapping the coordinates of the center point to the tray point cloud data to obtain the position coordinates corresponding to the chip placement points.
According to the method for determining the chip placement parameters in the tray, the optimal chip placement points are selected from the candidate chip placement areas output by the intelligent chip placement area identification model, and the chip placement points are mapped by using the tray point cloud data, so that accurate chip placement position information is provided for the tray positioning equipment.
In some possible implementations, the step S105 calculates a tray clutch degree between the size point and the chip placement point on the tray image, and specifically includes:
determining a linear equation according to coordinates of the chip placement points and the size points;
extracting center line pixel point coordinates of a tray to be positioned between a size point and a chip placement point from the tray image;
Substituting the linear equation and the coordinates of the central line pixel points into a formula I to obtain the clutch degree of the tray;
wherein epsilon represents the clutch degree of the tray, n represents the number of central line pixel points, a linear equation is expressed as x=ay+b, X g represents the abscissa of the die-placement point, y g represents the ordinate of the die-placement point, x h represents the abscissa of the size point, and y h represents the ordinate of the size point.
In order to judge the bending condition of a section of tray from a chip placement point to a size, the embodiment provides a tray clutch degree quantization calculating method, and the implementation principle is as follows: under an image coordinate system, a linear equation is determined by using coordinates of the chip placement points and the size points, the coordinates of the center line of the tray are extracted, y of each coordinate of the center line is substituted into the linear equation, an absolute difference value of x is calculated, all deviation values are accumulated and divided by the number of the coordinates of the center line, a tray clutch value can be obtained, the obtained tray clutch value can accurately evaluate the bending degree from the chip placement points to the size points of the tray, and preparation is provided for subsequent position coordinate calculation and rotation direction value calculation.
In some possible implementations, the step S106 calculates the attitude parameter and the rotation direction value of the chip placement point according to the tray clutch, which specifically includes:
Comparing the clutch degree of the tray with a preset value;
Calculating a first attitude parameter according to the size point and the chip placement point under the condition that the clutch degree of the tray does not exceed a preset value, wherein the first attitude parameter is expressed as X g represents the abscissa of the chip placement point, y g represents the ordinate of the chip placement point, x h represents the abscissa of the size point, and y h represents the ordinate of the size point;
under the condition that the clutch degree of the tray does not exceed a preset value, determining an attitude point according to the chip placement point and the size point, and calculating a second attitude parameter according to the attitude point and the chip placement point, wherein the second attitude parameter is expressed as X g represents the abscissa of the chip placement point, y g represents the ordinate of the chip placement point, x z represents the abscissa of the posture point, and y z represents the ordinate of the posture point;
In this embodiment, the gesture point is a point that is located at a preset length from the chip placement point, along the centerline, from the dimension direction, and it should be noted that, the preset length may be adjusted according to the actual situation, and the preset length may be set to 20mm as a reference, and then the linear angle of the line connecting the gesture point and the chip placement point is the horizontal rotation gesture value.
Calculating a rotation direction value according to a formula II;
Wherein p represents a rotation direction value, p=1 represents a clockwise rotation of the rotation mechanism, p= -1 represents a counterclockwise rotation of the rotation mechanism, y h represents a vertical position of a size point, y g represents a vertical coordinate of a chip placement point, and y z represents a vertical coordinate of a posture point.
The embodiment takes the preset value as a boundary, defines a corresponding attitude parameter calculation mode in the case that the clutch degree of the tray exceeds the preset value and the case that the clutch degree of the tray does not exceed the preset value, simultaneously provides a value taking strategy of the rotation direction value, realizes providing a generalized pose matrix consisting of seven dimensional generalized parameters consisting of position, attitude and direction for positioning equipment, and realizes accurate guiding of the tray positioning equipment.
In some possible implementations, before photographing the plane of the fully automatic burner using the dual-line camera, the method further includes:
Utilize the calibrator to mark double-line array camera, wherein, the upper cover plate of calibrator includes first ladder, second ladder and third ladder, wherein, the length, the width of first ladder, second ladder, third ladder are all the same, the height of first ladder is greater than the height of second ladder, the height of second ladder is greater than the height of third ladder, equidistant distribution a plurality of right triangle-shaped light trap on the first ladder, equidistant distribution a plurality of square light trap on the second ladder, equidistant and two rows of circular light trap of cross distribution on the third ladder.
In the embodiment, the calibrator is a multi-pattern-based step-type calibrator, and the calibrator consists of a lower cover plate, an upper cover plate, a strip-shaped lamp bead substrate and a light homogenizing sheet, wherein the dimension 1200mm is 60mm, 60mm and the step height is 20mm; referring to fig. 3, the first steps of the upper cover plate are uniformly distributed with right triangle light holes, and the long sides and short sides: 4020mm, center spaced 60mm,9 holes; square light transmission is uniformly distributed on the second step of the upper cover plate, the size is 40mm or 40mm, the distance between middle lines is 65mm, and holes are 6; the third step of the upper cover plate is crossed and distributed with two rows of circular light holes, the radius is 10mm, the vertical distance between the upper round holes and the lower round holes is 30mm, the horizontal distance between the upper round holes and the lower round holes is 20mm, the distance between the round holes in the same row is 40mm, and each row is 9 holes.
The recommended machining precision of the upper cover plate of the calibrator is preferably 0.0025mm. The origin of the coordinate system of the calibrator is the upper left corner point of the light hole at the left end of the second step, the third step is directed along the length direction and is in the positive X-axis direction, the lower left corner point is directed along the length direction and is in the positive Y-axis direction, and the vertical direction and the plane of the calibrator are in the positive Z-axis direction. And calculating the coordinates of each corner point under a calibrator coordinate system. And based on optimization solution, the internal and external parameters can be calibrated by the calibrator by utilizing a coordinate relation model of the world coordinate system point and the pixel point of the double-linear-array camera.
According to the method for determining the chip placement parameters in the tray, through designing the calibrator into a multi-step multi-pattern mode, parameter calibration can be achieved under the condition that the calibrator is fixed, errors introduced by moving the calibrator are avoided, and calibration errors are reduced relative to a single pattern; in addition, the right triangle pattern can intuitively feed back the inclination condition of the calibrator, the square pattern can intuitively feed back the rotation condition of the calibrator, and the round pattern can intuitively feed back the deformation condition of the acquired calibration image.
In yet another embodiment, in order to facilitate understanding of the solution of the present invention, referring to fig. 4, another method for determining a chip placement parameter in a tray is provided in this embodiment, and the specific implementation process is as follows:
Firstly, a double-line array camera for installing image acquisition can be arranged above a flat plate link, and the image acquisition equipment is integrally installed at a position 0.4m away from a laser along the motion direction of a full-automatic burner, wherein the installation height is 3m.
And then, calibrating internal and external parameters by a multi-pattern-based ladder type calibrator based on optimization solution by utilizing the coordinate relation between the world coordinate system point and the pixel point of the double-linear array camera.
Then, a group of laser correlation sensors are used for providing a start signal for the image acquisition equipment, and when the signal is a tray size signal, the acquisition tray image data of the array linear array camera is triggered.
And correcting the left image and the right image respectively by using calibrated internal reference data, overlapping the right image with the left image by using calibrated external reference, extracting key points of the images, namely, angular points, matching, and reconstructing the key points by using a parallax principle to obtain point cloud data of the whole frame of images.
And then, carrying out point cloud rejection by utilizing the placing characteristics of the tray, obtaining the point cloud data of the tray, and matching the point cloud data of the tray with the tray form model to obtain the size form type of the tray.
If the current form category of the tray cannot be identified, the cloud data of the tray is saved, a left-eye image is synchronously taken, preprocessing is carried out, and the chip placement point is detected by combining the detection result of the size coordinates of the tray in the modes of edge detection, morphological processing and the like to obtain the coordinates of the chip placement point, and meanwhile the chip placement point is marked on the tray image to prompt manual secondary judgment.
If the morphology recognition is successful, the left eye image and morphology category information are transmitted into a trained intelligent judgment model of the chip placement points, the deep learning model can return to a chip placement point area, and the center of the area is generally taken as the coordinates of the chip placement points and is set as (x g,yg). And (3) knowing the coordinates corresponding to the positions of the chip placement points, namely the coordinates of the chip placement points, by using the pixel coordinates of the chip placement points and the tray point cloud data, and marking the coordinates as (x 0,y0,z0).
After the tray image is obtained, a sub-thread can be started, preprocessing such as mean value filtering, histogram equalization and the like is carried out on the left eye image in the sub-thread, tray size detection is carried out by utilizing morphological characteristics of the size, size point coordinates are determined and marked as (x h,yh), a skeleton curve from a chip placement point to a size point along the tray is extracted, then the horizontal distance between each point on the curve and a connecting line of each point is calculated, and the average distance is the tray clutch value.
If the tray clutch value is not more than 8, calculating the rotation angle gamma around the Z axis (left-eye camera coordinate system) at the tray chip placement position from the size point and the chip placement point,Since the tray at the chip placement point is perpendicular to the full-automatic writer, the rotation angles around the X and Y axes are both 0, i.e., the attitude parameters are expressed as
If the tray clutch degree value is greater than 8, the tray clutch degree from the chip placement point to the size point is greater, an attitude point needs to be additionally positioned along the size direction according to the designated length, and the attitude point is marked (x z,yz), and then the attitude is marked
Meanwhile, the rotation direction is calculated to obtain a direction representation value p, and the value p is as follows:
The chip placement point coordinates, the chip placement point gesture parameters and the direction representation values jointly form a tray chip placement point generalized gesture, namely a gesture matrix is [ x 0,y0,z0, 0, gamma, p ], the data are sent to tray positioning equipment through a communication system, accurate guiding of the tray positioning equipment is achieved, meanwhile, the system continuously monitors the state of a sensor, and when a trigger signal is detected, the steps are repeated.
The method for determining the placement parameters of the chips in the tray at least has the following beneficial technical effects: the accurate chip placement position and posture can be provided for the tray positioning equipment based on the double-linear-array camera; the identification of various tray forms can be realized by using model matching, and the expansion is easy; the intelligent decision based on deep learning realizes the rapid and accurate positioning of the chip placement points, and the learning effect is enhanced by utilizing the tray morphological text information; the ladder type calibrator based on various patterns can realize the efficient calibration of the double-linear-array camera.
The device for determining the chip placement parameter in the tray provided by the invention is described below, and the device for determining the chip placement parameter in the tray described below and the method for determining the chip placement parameter in the tray described above can be referred to correspondingly.
Optionally, as shown in fig. 5, fig. 5 is a schematic structural diagram of a device for determining a chip placement parameter in a tray according to the present invention, where the device includes an image acquisition module 510, a matching module 520, a chip placement area identification module 530, a chip placement point determination module 540, a first calculation module 550, a second calculation module 560, and a chip placement parameter determination module 570, and the following details of the above modules will be described below:
the image acquisition module 510 is used for acquiring a tray image of a tray to be positioned on the full-automatic burner, detecting the size of the tray image to obtain a size point, and constructing tray point cloud data by utilizing the tray image;
the matching module 520 is configured to match the tray point cloud data with a pre-created size morphology type library to obtain a target size morphology type;
the chip placement area recognition module 530 is configured to input the target size, shape and type and the tray image to a pre-trained intelligent chip placement area recognition model to obtain an output candidate chip placement area;
The chip placement point determining module 540 is configured to select a chip placement point from the candidate chip placement region, and determine a position coordinate of the chip placement point according to the tray point cloud data;
the first calculating module 550 is configured to calculate a tray clutch degree between a size point and a chip placement point on the tray image;
The second calculating module 560 is configured to calculate an attitude parameter and a rotation direction value of the chip placement point according to the tray clutch, where the rotation direction value represents a rotation direction of a rotation mechanism of the tray to be positioned;
the chip placement parameter determining module 570 is configured to send the position coordinates, the gesture parameters, and the rotation direction values as chip placement parameters to the tray positioning device.
According to the device for determining the chip placement parameters in the tray, tray images of the tray to be positioned on the full-automatic burner are collected, tray point cloud data of size points and shapes are obtained through processing the tray images, target size form types are identified through the tray point cloud data, then the target size form types and the tray images are input into a pre-trained intelligent chip placement area identification model to obtain candidate chip placement areas, then the chip placement points are selected from the candidate chip placement areas and mapped onto the tray point cloud data of the shapes, so that position coordinates of the chip placement points are obtained, further, tray clutch degrees are calculated by the aid of the chip placement points and the size points, attitude parameters and rotation direction values of grabbing points are calculated by means of combining with the tray clutch degrees, and finally the position coordinates, the attitude parameters and the rotation direction values are sent to tray positioning equipment, so that accurate chip placement point information is automatically provided for the tray positioning equipment, and chips to be detected can be accurately sent into the chip placement areas of the tray.
Referring to fig. 6, fig. 6 is a schematic diagram of an embodiment of an electronic device according to an embodiment of the invention. As shown in fig. 6, an embodiment of the present invention provides an electronic device 600, including a memory 610, a processor 620, and a computer program 611 stored in the memory 610 and executable on the processor 620, wherein the processor 620 executes the computer program 611 to implement the following steps:
acquiring a tray image of a tray to be positioned on a full-automatic burner, performing size detection on the tray image to obtain a size point, and constructing tray point cloud data by utilizing the tray image;
Matching the tray point cloud data with a pre-established size form type library to obtain a target size form type;
Inputting the target size, shape and type and the tray image into a pre-trained intelligent chip placement area identification model to obtain an output candidate chip placement area;
Selecting a chip placement point from the candidate chip placement area, and determining the position coordinates of the chip placement point according to the tray point cloud data;
Calculating the tray clutch degree between the size point and the chip placement point on the tray image;
calculating attitude parameters and rotation direction values of the chip placement points according to the tray clutch degrees, wherein the rotation direction values represent the rotation direction of a rotation mechanism of the tray to be positioned;
and sending the position coordinates, the attitude parameters and the rotation direction values to tray positioning equipment as chip placement parameters.
Referring to fig. 7, fig. 7 is a schematic diagram of an embodiment of a computer readable storage medium according to an embodiment of the invention. As shown in fig. 7, the present embodiment provides a computer-readable storage medium 700 having stored thereon a computer program 711, which computer program 711, when executed by a processor, performs the steps of:
acquiring a tray image of a tray to be positioned on a full-automatic burner, performing size detection on the tray image to obtain a size point, and constructing tray point cloud data by utilizing the tray image;
Matching the tray point cloud data with a pre-established size form type library to obtain a target size form type;
Inputting the target size, shape and type and the tray image into a pre-trained intelligent chip placement area identification model to obtain an output candidate chip placement area;
Selecting a chip placement point from the candidate chip placement area, and determining the position coordinates of the chip placement point according to the tray point cloud data;
Calculating the tray clutch degree between the size point and the chip placement point on the tray image;
calculating attitude parameters and rotation direction values of the chip placement points according to the tray clutch degrees, wherein the rotation direction values represent the rotation direction of a rotation mechanism of the tray to be positioned;
and sending the position coordinates, the attitude parameters and the rotation direction values to tray positioning equipment as chip placement parameters.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (means) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. A method for determining placement parameters of chips in a tray, comprising:
acquiring a tray image of a tray to be positioned on a full-automatic burner, performing size detection on the tray image to obtain a size point, and constructing tray point cloud data by utilizing the tray image;
Matching the tray point cloud data with a pre-established size form type library to obtain a target size form type;
Inputting the target size, shape and type and the tray image into a pre-trained intelligent chip placement area identification model to obtain an output candidate chip placement area;
Selecting a chip placement point from the candidate chip placement area, and determining the position coordinates of the chip placement point according to the tray point cloud data;
Calculating the tray clutch degree between the size point and the chip placement point on the tray image;
calculating attitude parameters and rotation direction values of the chip placement points according to the tray clutch degrees, wherein the rotation direction values represent the rotation direction of a rotation mechanism of the tray to be positioned;
and sending the position coordinates, the attitude parameters and the rotation direction values to tray positioning equipment as chip placement parameters.
2. The method for determining the placement parameters of the chips in the tray according to claim 1, wherein the collecting the tray image of the tray to be positioned on the full-automatic burner, performing size detection on the tray image to obtain a size point, and constructing tray point cloud data by using the tray image comprises the following steps:
When the laser correlation sensor detects the size of the tray to be positioned, triggering a double-linear-array camera to photograph the size area of the tray to be positioned on the plane where the full-automatic burner is positioned to obtain a left-eye image and a right-eye image, and taking the left-eye image as a tray image;
performing size detection on the left eye image or the right eye image to obtain a size point;
And respectively carrying out distortion correction on the left eye image and the right eye image, respectively extracting angular points from the corrected left eye image and right eye image to serve as key points, reconstructing the key points by utilizing a parallax principle to obtain initial point cloud data, and removing data irrelevant to a tray to be positioned in the initial point cloud data to obtain tray point cloud data.
3. The method of claim 1, wherein calculating a tray clutch between the size point and the chip placement point on the tray image comprises:
determining a linear equation according to the coordinates of the chip placement point and the size point;
extracting center line pixel point coordinates of a tray to be positioned between the size point and the chip placement point from the tray image;
Substituting the linear equation and the coordinates of the central line pixel points into a formula I to obtain the clutch degree of the tray;
wherein epsilon represents the clutch degree of the tray, n represents the number of central line pixel points, a linear equation is expressed as x=ay+b, X g represents the abscissa of the die-placement point, y g represents the ordinate of the die-placement point, x h represents the abscissa of the size point, and y h represents the ordinate of the size point.
4. The method of determining a chip placement parameter in a tray according to claim 1, wherein calculating an attitude parameter and a rotation direction value of the chip placement point according to the tray clutch degree comprises:
Comparing the clutch degree of the tray with a preset value;
Calculating a first attitude parameter according to the size point and the chip placement point under the condition that the tray clutch degree does not exceed the preset value, wherein the first attitude parameter is expressed as X g represents the abscissa of the chip placement point, y g represents the ordinate of the chip placement point, x h represents the abscissa of the size point, and y h represents the ordinate of the size point;
Under the condition that the clutch degree of the tray does not exceed the preset value, determining an attitude point according to the chip placement point and the size point, and calculating a second attitude parameter according to the attitude point and the chip placement point, wherein the second attitude parameter is expressed as X g represents the abscissa of the chip placement point, y g represents the ordinate of the chip placement point, x z represents the abscissa of the posture point, and y z represents the ordinate of the posture point;
Calculating a rotation direction value according to a formula II;
Wherein p represents a rotation direction value, p=1 represents a clockwise rotation of the rotation mechanism, p= -1 represents a counterclockwise rotation of the rotation mechanism, y h represents a vertical position of a size point, y g represents a vertical coordinate of a chip placement point, and y z represents a vertical coordinate of a posture point.
5. The method for determining the placement parameters of the chip in the tray according to claim 1, wherein the matching the tray point cloud data with a pre-created size form type library to obtain the target size form type comprises:
respectively creating a corresponding tray size model for at least one known size form type by using modeling software to obtain a pre-created size form type library;
And respectively matching the tray point cloud data with each tray size model, and taking the size form type corresponding to the matched tray size model as the target size form type.
6. The method for determining chip placement parameters in a tray according to claim 1, wherein the pre-trained chip placement area intelligent recognition model obtains candidate chip placement areas, and is obtained by training the following steps:
Constructing a deep learning model, wherein the input of the deep learning model is a tray image and a size, shape and type text, and the output of the deep learning model is a preset shape candidate frame coordinate;
Labeling tray images of known optimal chip placement points based on input and output of the deep learning model to construct a training set, a verification set and a test set;
And training the deep learning model by using the training set, the verification set and the test set to obtain a pre-trained intelligent recognition model of the chip placement area.
7. The method for determining the placement parameters of the chips in the tray according to claim 2, wherein before photographing the plane of the fully automatic burner using the dual line camera, the method further comprises:
Utilize the calibrator to mark double-line array camera, wherein, the upper cover plate of calibrator includes first ladder, second ladder and third ladder, wherein, first ladder the second ladder the length, the width of third ladder are the same, the height of first ladder is greater than the height of second ladder, the height of second ladder is greater than the height of third ladder, equidistant distribution a plurality of right triangle-shaped light trap on the first ladder, equidistant distribution a plurality of square light trap on the second ladder, equidistant and alternately distributed two rows of circular light trap on the third ladder.
8. A device for determining placement parameters of chips in a tray, comprising:
The image acquisition module is used for acquiring a tray image of a tray to be positioned on the full-automatic burner, detecting the size of the tray image to obtain a size point, and constructing tray point cloud data by utilizing the tray image;
the matching module is used for matching the tray point cloud data with a pre-established size form type library to obtain a target size form type;
The chip placement area identification module is used for inputting the target size, shape and type and the tray image into a pre-trained chip placement area intelligent identification model to obtain an output candidate chip placement area;
the chip placement point determining module is used for selecting a chip placement point from the candidate chip placement area and determining the position coordinates of the chip placement point according to the tray point cloud data;
the first calculating module is used for calculating the tray clutch degree between the size point and the chip placement point on the tray image;
The second calculation module is used for calculating the attitude parameter and the rotation direction value of the chip placement point according to the tray clutch degree, wherein the rotation direction value represents the rotation direction of the rotation mechanism of the tray to be positioned;
The chip placement parameter determining module is used for sending the position coordinates, the attitude parameters and the rotation direction values to the tray positioning equipment as chip placement parameters.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of determining chip placement parameters in a tray as claimed in any one of claims 1 to 7 when the program is executed by the processor.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the method of determining chip placement parameters in a tray according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410341526.XA CN118154566B (en) | 2024-03-25 | 2024-03-25 | Method and device for determining chip placement parameters in tray |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410341526.XA CN118154566B (en) | 2024-03-25 | 2024-03-25 | Method and device for determining chip placement parameters in tray |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118154566A true CN118154566A (en) | 2024-06-07 |
| CN118154566B CN118154566B (en) | 2024-09-06 |
Family
ID=91288422
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410341526.XA Active CN118154566B (en) | 2024-03-25 | 2024-03-25 | Method and device for determining chip placement parameters in tray |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118154566B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050196036A1 (en) * | 2004-03-05 | 2005-09-08 | Leonard Patrick F. | Method and apparatus for determining angular pose of an object |
| CN102519964A (en) * | 2011-12-14 | 2012-06-27 | 浙江大学 | Internal fruit quality information collecting method and device capable of overcoming influences of sizes and postures of fruits |
| CN113222990A (en) * | 2021-06-11 | 2021-08-06 | 青岛高重信息科技有限公司 | Chip counting method based on image data enhancement |
| US20220089419A1 (en) * | 2020-09-23 | 2022-03-24 | Hyundai Motor Company | Pallet Loading Apparatus and Pallet Loading Method |
| CN116309817A (en) * | 2022-12-19 | 2023-06-23 | 东北大学 | Tray detection and positioning method based on RGB-D camera |
-
2024
- 2024-03-25 CN CN202410341526.XA patent/CN118154566B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050196036A1 (en) * | 2004-03-05 | 2005-09-08 | Leonard Patrick F. | Method and apparatus for determining angular pose of an object |
| CN102519964A (en) * | 2011-12-14 | 2012-06-27 | 浙江大学 | Internal fruit quality information collecting method and device capable of overcoming influences of sizes and postures of fruits |
| US20220089419A1 (en) * | 2020-09-23 | 2022-03-24 | Hyundai Motor Company | Pallet Loading Apparatus and Pallet Loading Method |
| CN113222990A (en) * | 2021-06-11 | 2021-08-06 | 青岛高重信息科技有限公司 | Chip counting method based on image data enhancement |
| CN116309817A (en) * | 2022-12-19 | 2023-06-23 | 东北大学 | Tray detection and positioning method based on RGB-D camera |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118154566B (en) | 2024-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108717715B (en) | Automatic calibration method for linear structured light vision system of arc welding robot | |
| CN104266608B (en) | Field calibration device for visual sensor and calibration method | |
| CN105528789B (en) | Robot visual orientation method and device, vision calibration method and device | |
| EP2780849B1 (en) | Automated identification of shoe parts | |
| WO2019188506A1 (en) | Information processing device, object measuring system, object measuring method, and program storing medium | |
| WO2021063128A1 (en) | Method for determining pose of active rigid body in single-camera environment, and related apparatus | |
| CN105091782A (en) | Multilane laser light plane calibration method based on binocular vision | |
| CN114697509B (en) | Split type lens assembling method, device and system | |
| EP2153410B1 (en) | 3d assembly verification from 2d images | |
| CN105865326A (en) | Object size measurement method and image database data acquisition method | |
| CN101314225A (en) | Robot simulation apparatus | |
| EP3049756B1 (en) | Modeling arrangement and method and system for modeling the topography of a three-dimensional surface | |
| JP2014013147A5 (en) | ||
| CN109916308A (en) | A kind of information collecting method and its system of sole | |
| CN106625713A (en) | Method of improving gumming accuracy of gumming industrial robot | |
| KR101974638B1 (en) | Apparatus for processing plant images and method thereof | |
| JP4402458B2 (en) | Method for determining corresponding points in 3D measurement | |
| CN109636857B (en) | Alignment method and calibration system | |
| CN116958146B (en) | Acquisition method and device of 3D point cloud and electronic device | |
| CN111289226A (en) | Line laser flatness detection method based on visual measurement technology | |
| CN116295058A (en) | Train wheel set geometric parameter on-line detection method and system | |
| KR20040010091A (en) | Apparatus and Method for Registering Multiple Three Dimensional Scan Data by using Optical Marker | |
| CN118154566B (en) | Method and device for determining chip placement parameters in tray | |
| CN108682035A (en) | A kind of single step light knife system laser spatial plane equation computational methods | |
| CN117506888A (en) | Visual guiding grabbing method and system based on monocular and structured light sensor fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |