CN112529961A - Object installation method, device, system and storage medium - Google Patents
Object installation method, device, system and storage medium Download PDFInfo
- Publication number
- CN112529961A CN112529961A CN202011541658.5A CN202011541658A CN112529961A CN 112529961 A CN112529961 A CN 112529961A CN 202011541658 A CN202011541658 A CN 202011541658A CN 112529961 A CN112529961 A CN 112529961A
- Authority
- CN
- China
- Prior art keywords
- container
- installation
- pixel position
- position information
- mounting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000009434 installation Methods 0.000 title claims abstract description 325
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000001514 detection method Methods 0.000 claims abstract description 52
- 238000012163 sequencing technique Methods 0.000 claims description 47
- 238000004590 computer program Methods 0.000 claims description 36
- 238000000605 extraction Methods 0.000 claims description 12
- 238000002372 labelling Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 description 12
- 238000004422 calculation algorithm Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 11
- 239000003814 drug Substances 0.000 description 9
- 238000011900 installation process Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 230000007797 corrosion Effects 0.000 description 4
- 238000005260 corrosion Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 238000003708 edge detection Methods 0.000 description 3
- 239000003153 chemical reaction reagent Substances 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 229940079593 drug Drugs 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The application relates to an object installation method, device, system and storage medium. The method comprises the following steps: acquiring an image of an installation container corresponding to an object to be installed; carrying out target detection on the image of the installation container, and determining pixel position information of at least one installation container in the image of the installation container; and installing the object to be installed into the at least one installation container according to the pixel position information of the at least one installation container. By adopting the method, the labor and time for installing the object can be saved.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method, an apparatus, a system, and a storage medium for installing an object.
Background
Object mounting refers to mounting objects into a set container, for example, drug mounting is mounting drugs into a tubular cassette or other type of cassette. After the object is installed, when the installed object is delivered to the user, quantitative delivery can be achieved, and meanwhile the object is convenient for the user to carry. It follows that an object installation process is necessary.
In the related technology, when a medicine is installed, the mouth of a medicine reagent bottle is usually detected through human eyes, the medicine is manually filled into the reagent bottle, then whether the installation is successful or not can be manually detected, and the installed medicine can be packaged when the installation is successful; and when the installation fails, the installation can be carried out again in a manual mode until the installation is successful.
However, the medicine installation technology is time-consuming and labor-consuming.
Disclosure of Invention
In view of the above, it is necessary to provide an object mounting method, apparatus, system, and storage medium capable of saving time and labor in a medicine mounting process.
A method of object installation, the method comprising:
acquiring an image of an installation container corresponding to an object to be installed;
performing target detection on the image of the installation container, and determining pixel position information of at least one installation container in the image of the installation container;
and installing the object to be installed into the at least one installation container according to the pixel position information of the at least one installation container.
In one embodiment, the performing the target detection on the image of the installation container to determine the pixel position information of at least one installation container in the image of the installation container includes:
performing target extraction processing on the image of the installation container, and determining an edge image of the installation container corresponding to the image of the installation container;
and carrying out Hough detection processing on the edge image of the mounting container, and determining pixel position information of at least one mounting container in the image of the mounting container.
In one embodiment, before the mounting the object to be mounted into the at least one mounting container according to the pixel position information of the at least one mounting container, the method further includes:
and sequencing the pixel position information of the at least one mounting container, and determining a pixel position sequencing result of the at least one mounting container.
In one embodiment, the pixel position information of the mounting container includes pixel position information of a nozzle center point of the mounting container, and the pixel position information of the nozzle center point of the mounting container includes X-axis pixel position information of the nozzle center point and Y-axis pixel position information of the nozzle center point.
In one embodiment, the sorting the pixel position information of the at least one installation container and determining a result of sorting the pixel position of the at least one installation container includes:
sequencing the X-axis pixel position information of the central point of the pipe orifice of the at least one mounting container to obtain a sequencing result of the X-axis pixel position of the at least one mounting container;
and sequencing the Y-axis pixel position information of the central point of the pipe orifice of the at least one mounting container based on the X-axis pixel position sequencing result to obtain the pixel position sequencing result of the at least one mounting container.
In one embodiment, the installing the object to be installed into the at least one installation container according to the pixel position information of the at least one installation container includes:
converting the pixel position information of the at least one mounting container from a pixel coordinate system to a world coordinate system, and determining the world coordinate position information of the at least one mounting container;
and installing the object to be installed into the at least one installation container according to the world coordinate position information of the at least one installation container.
In one embodiment, the method further includes:
performing pixel error compensation on the pixel position information of the at least one mounting container according to a preset pixel error threshold range;
and according to a preset world coordinate error threshold range, performing world coordinate error compensation on the world coordinate position information of the at least one installation container.
In one embodiment, the method further includes:
and drawing the pixel position information of the at least one installation container based on the pixel position sequencing result of the at least one installation container to obtain a pixel position information drawing image of the at least one installation container.
In one embodiment, the method further includes:
and grouping and marking the at least one installation container based on the pixel position sorting result of the at least one installation container to obtain at least one group of installation containers.
An object mounting apparatus, the apparatus comprising:
the acquisition module is used for acquiring an image of an installation container corresponding to an object to be installed;
the target detection module is used for carrying out target detection on the image of the installation container and determining pixel position information of at least one installation container in the image of the installation container;
and the mounting module is used for mounting the object to be mounted into the at least one mounting container according to the pixel position information of the at least one mounting container.
An object mounting system comprising a computer device and a robot arm connected to each other, the computer device comprising a memory and a processor, the memory storing a computer program,
the processor is used for acquiring an image of an installation container corresponding to an object to be installed when executing the computer program;
the processor is further configured to perform object detection on the image of the installation container and determine pixel position information of at least one installation container in the image of the installation container when executing the computer program;
the robot arm is configured to mount the object to be mounted in the at least one mounting container according to the pixel position information of the at least one mounting container.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring an image of an installation container corresponding to an object to be installed;
performing target detection on the image of the installation container, and determining pixel position information of at least one installation container in the image of the installation container;
and installing the object to be installed into the at least one installation container according to the pixel position information of the at least one installation container.
According to the object installation method, the device, the computer equipment and the storage medium, the target detection is carried out on the image of the installation container corresponding to the obtained object to be installed, the pixel position information of at least one installation container in the image of the installation container is determined, and the object to be installed is installed in at least one installation container according to the pixel position information of the installation container. In the method, the position information of the installation container can be obtained through the image of the installation container, and the object to be installed is installed in the installation container through the position information of the installation container, and the process is an automatic installation process and does not need manual detection and installation of the object to be installed, so that the labor can be saved; meanwhile, the speed of the automatic installation process is higher than that of the manual installation process, so that the time for installing the object to be installed to the installation container can be saved.
Drawings
FIG. 1 is a diagram illustrating an example of the structure of an object installation system in one embodiment;
FIG. 2 is a flow diagram that illustrates a method for object installation in one embodiment;
FIG. 3 is a flowchart illustrating an object installation step in another embodiment;
FIG. 4 is a flowchart illustrating an object installation method according to another embodiment;
FIG. 5 is a flowchart illustrating an object installation method according to another embodiment;
FIG. 5a is an exemplary diagram of an installation vessel in a simulation phase in another embodiment;
FIG. 6 is an exemplary view of an installation vessel at an actual installation stage in another embodiment;
FIG. 7 is a block diagram showing the structure of an object mounting apparatus according to an embodiment;
FIG. 8 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The object installation method provided by the embodiment of the application can be applied to an object installation system as shown in fig. 1, the object installation system includes a computer device 102 and a robot arm 104 which are connected with each other, wherein the computer device 102 can communicate with the robot arm 104 to realize data transmission, and the communication mode can be a wired or wireless mode. Additionally, the computer device 102 may be a terminal or a server. The robotic arm 104 may include a camera, a processor, and the like.
The execution subject of the embodiment of the present application may be an object mounting system or an object mounting apparatus, and the technical solution of the present application will be described below with the object mounting system as the execution subject.
In one embodiment, an object mounting method is provided, and the embodiment relates to how to obtain position information from an image of a mounting container of an object to be mounted, and a specific process of mounting the object to be mounted by the position information. As shown in fig. 2, the method may include the steps of:
s202, acquiring an image of the installation container corresponding to the object to be installed.
Wherein the object to be installed may be a medicine, a food, or the like. The mounting container may be a tubular vessel, a barreled vessel, a square vessel, etc., and the spout of the mounting container may be circular, square, etc. The number of the installation containers is generally one or more, and if the number of the installation containers is multiple, the multiple installation containers are generally the same installation container, that is, the containers with the same shape, length, width and height.
Specifically, the mechanical arm can be provided with a camera, when an object to be installed needs to be installed, the installation container can be manually or automatically placed in advance, and then the object installation system can control the camera on the mechanical arm to shoot the installation container to obtain an image of the installation container; thereafter, the object mounting system may also control the robot arm to transmit the photographed image to the computer device so that the computer device can obtain an image of the mounting container.
S204, carrying out target detection on the image of the installation container, and determining pixel position information of at least one installation container in the image of the installation container.
Before the target detection is carried out on the image of the installation container, the image of the installation container can be preprocessed, and the preprocessing can comprise graying and/or binarization, image morphology operation and other processing. The image morphology operation herein may include performing erosion operation and dilation operation on the image.
In this step, a target detection related algorithm (i.e., a machine vision related algorithm) may be used to perform target detection on the image of the installation container, and the target detection related algorithm may include an edge detection algorithm, a yolo (young Only Look one) target detection algorithm, an SSD (Single Shot multi box Detector) target detection algorithm, a hough detection algorithm, and so on.
Specifically, after obtaining the image of the installation container, the object installation system may use an object detection related algorithm to detect the installation container in the image of the installation container, and obtain the position information of at least one installation container, where the position information of the installation container refers to the position information of the installation container on the image of the installation container, and therefore is expressed as the pixel position information of the installation container. In addition, at least one of these means the number of mounting containers, and not the number of pixel position information.
In addition, the pixel position information of each mounting container obtained here may include pixel position information of the nozzle of the mounting container, pixel position information of a start point and an end point constituting the length, width, height, radius, diameter, and the like of the mounting container, and the like.
And S206, installing the object to be installed into the at least one installation container according to the pixel position information of the at least one installation container.
In this step, after obtaining the pixel position information of each mounting container, the object mounting system may control the robot arm to mount the object to be mounted into each mounting container directly according to the pixel position information of the mounting container; or the pixel position information of the installation container is subjected to coordinate conversion, and the object to be installed is installed in each installation container according to the converted coordinate information; of course, other methods are also possible, and are not specifically limited herein.
Of course, assuming that there are a plurality of mounting containers, the number or volume of the objects to be mounted, etc. in general, in each mounting container is also the same when mounting the objects to be mounted into the respective mounting containers.
According to the above description, the scheme can automatically realize the installation of the object to be installed, so that the labor and the time can be saved; meanwhile, the machine vision algorithm is added to detect the installation container, so that the accuracy of detecting the installation container can be improved, in addition, the automatic installation process and the machine vision algorithm cannot be influenced by factors such as artificial fatigue and the like, the installation process is stable, the missing detection rate or the false detection rate of the installation container can be reduced, and the detection accuracy in quality control is improved.
In the object installation method, the obtained image of the installation container corresponding to the object to be installed is subjected to target detection, the pixel position information of at least one installation container in the image of the installation container is determined, and the object to be installed is installed in at least one installation container according to the pixel position information of the installation container. In the method, the position information of the installation container can be obtained through the image of the installation container, and the object to be installed is installed in the installation container through the position information of the installation container, and the process is an automatic installation process and does not need manual detection and installation of the object to be installed, so that the labor can be saved; meanwhile, the speed of the automatic installation process is higher than that of the manual installation process, so that the time for installing the object to be installed to the installation container can be saved.
In another embodiment, another object mounting method is provided, and the embodiment relates to a specific process of determining pixel position information of a mounting container by performing object detection on an image of the mounting container. On the basis of the above embodiment, as shown in fig. 3, the above S204 may include the following steps:
s302, performing object extraction processing on the image of the mounting container, and determining an edge image of the mounting container corresponding to the image of the mounting container.
Specifically, after the image of the installation container is obtained, an edge detection algorithm may be used to perform edge detection on each installation container in the image of the installation container, that is, target extraction processing is performed, so that the edge of each installation container may be obtained on the image of the installation container and recorded as an edge image of the installation container.
It should be noted that, since the target extraction is performed on the installation container here, it is a rough inspection process in essence, and therefore the edge of the installation container detected here is a relatively rough edge, then in order to further obtain a finer edge of the installation container, further target detection is performed on the edge image of the installation container as follows.
And S304, carrying out Hough detection processing on the edge image of the mounting container, and determining pixel position information of at least one mounting container in the image of the mounting container.
After the edge images of the installation containers are obtained, the edge of each installation container in the edge images of the installation containers can be traversed, hough detection processing is carried out on the edge pixel of each nozzle according to the detection radius of the nozzle of the installation container set in advance, and image data can be converted from a graphic space to a hough field. In the Hough detection processing process, whether the edge of the pipe orifice of each installation container is circular or not can be judged according to a set threshold condition, and when the edge of the pipe orifice of each installation container is judged to be circular, information such as the midpoint coordinate position and the radius of the pipe orifice of each installation container can be positioned.
The information such as the midpoint coordinate position and the radius of the spout of each mounting container located here is the position information of the mounting container on the edge image, that is, may be referred to as pixel position information of the mounting container.
The object installation method of this embodiment may perform object extraction processing on the image of the installation container, determine an edge image of the installation container, perform hough detection processing on the edge image of the installation container, and determine pixel position information of at least one installation container. In this embodiment, since two-stage object detection processing can be performed on the mounting container by object extraction and hough detection, the finally obtained pixel position information of the mounting container can be more accurate.
In another embodiment, another object installation method is provided, and the embodiment relates to a specific process of how to sort the pixel position information of the installation container. On the basis of the foregoing embodiment, before the foregoing S206, the foregoing method may further include the following step a:
and step A, sequencing the pixel position information of the at least one installation container, and determining the pixel position sequencing result of the at least one installation container.
In this step, generally, when the object to be mounted is mounted, the corresponding mounting containers are a plurality of mounting containers, that is, the object is mounted in batch, and then in order to mount the object quickly, the pixel position information of each mounting container may be sorted before mounting, so as to obtain a sorting result.
Optionally, the pixel position information of the installation container includes pixel position information of a central point of a nozzle of the installation container, and the pixel position information of the central point of the nozzle of the installation container includes X-axis pixel position information of the central point of the nozzle and Y-axis pixel position information of the central point of the nozzle.
After obtaining that each installation container includes pixel position information in two axial directions, the pixel position information of each installation container may be sorted according to the pixel position information in the two axial directions, that is, optionally, as shown in fig. 4, the step a may specifically include the following steps:
s402, sorting the X-axis pixel position information of the central point of the pipe orifice of the at least one mounting container to obtain a sorting result of the X-axis pixel position of the at least one mounting container.
S404, based on the X-axis pixel position sorting result, sorting Y-axis pixel position information of the central point of the pipe orifice of the at least one mounting container to obtain a pixel position sorting result of the at least one mounting container.
In S402-S404, after obtaining the two axial pixel position information of the nozzle center point of each installation container, the X-axis pixel positions of the nozzle center points of the installation containers may be sorted first, the sorting manner may be from small to large or from large to small, and in short, the sorting result of the X-axis pixel positions of the nozzle center points of the installation containers may be obtained after sorting, and is recorded as the X-axis pixel position sorting result of the installation containers. The X-axis pixel positions are adopted for sorting, the sorting belongs to a coarse sorting process, and sorting results of the X-axis pixel positions of all the installation containers can be obtained quickly.
And then, the mounting containers can be further subjected to fine sequencing, namely the Y-axis pixel positions of the nozzle center points of the mounting containers can be sequenced in a small-to-large or large-to-small manner, and in short, the sequencing result of the Y-axis pixel positions of the nozzle center points of the mounting containers, namely the final pixel position sequencing result of the mounting containers can be obtained after sequencing. On the basis of the coarse sorting result, each mounting container is subjected to fine sorting through Y-axis pixel position information, so that the finally obtained pixel position sorting result can be more accurate.
In the object installation method of this embodiment, before the object to be installed is installed, the pixel position information of the at least one installation container may be sorted, and the result of sorting the pixel positions of the at least one installation container is determined. Therefore, the subsequent quick installation of the object can be facilitated, and the installation efficiency of the object to be installed is improved.
In another embodiment, another object installation method is provided, and the embodiment relates to a specific process of installing an object to be installed after coordinate conversion is performed on pixel position information of an installation container. On the basis of the above embodiment, as shown in fig. 5, the above S206 may include the following steps:
s502, converting the pixel position information of the at least one mounting container from the pixel coordinate system to the world coordinate system, and determining the world coordinate position information of the at least one mounting container.
In this step, a conversion relationship between the pixel coordinate system and the world coordinate system, which may include a conversion coefficient and a deflection angle, may be calculated in advance by knowing position information of the point in the pixel coordinate system and position information in the world coordinate system.
Taking the known points P1 and P2 as an example, the conversion coefficient and the deflection angle can be calculated with reference to the following formulas:
in the formula (1), the first and second groups,
refers to the position information of the point P1 on the X-axis of the pixel coordinate system,
refers to the position information of the point P2 on the X-axis of the pixel coordinate system,
refer to the vectors of point P1 and point P2 on the X-axis of the pixel coordinate system; in the same way, the method for preparing the composite material,
refer to the position information of the P1 point on the Y axis of the pixel coordinate system, the position information of the P2 point on the Y axis of the pixel coordinate system, and the vectors of the P1 point and the P2 point on the Y axis of the pixel coordinate system, respectively.
The calculation formula of the conversion coefficient K is as follows (2):
the calculation formula of the deflection angle θ is as follows (3):
θ＝θ1-θ2 (3)
wherein, theta1Is the deflection angle theta in the world coordinate system2Is the deflection angle, theta, in a pixel coordinate system1And theta2The calculation formula (4) is as follows:
where α is a deflection coefficient, which can be set in advance and is a known amount.
Conversion coefficients and deflection angles can be obtained through the formulas (1) to (4), and the conversion relation between the pixel coordinate system and the world coordinate system is obtained.
Then, in this step, after the pixel position information of the mounting container is obtained, the pixel position information of the mounting container can be converted based on the conversion coefficient and the deflection angle calculated above, and the world coordinate position information of the mounting container can be obtained.
And S504, installing the object to be installed into the at least one installation container according to the world coordinate position information of the at least one installation container.
In this step, after the world coordinate position information of each mounting container is obtained, the robot arm may be controlled to sequentially mount the objects to be mounted into each mounting container in accordance with the world coordinate position information of each mounting container.
Furthermore, in the installation object process, due to the fact that Hough detection processing needs to traverse each edge pixel point, when the scheme preprocesses the image of the installation container in the early stage, corrosion and expansion operations to the image exist, the corrosion operations can lead the edge to shrink on the pixel layer, the expansion operations can lead the image edge to expand all around, compared with the actual edge, the corroded or expanded image edge can be expanded or reduced, and the expansion or reduction degree and the corrosion expansion frequency are in positive correlation. In order to reduce the influence of the error to the maximum extent, the scheme adopts the combination of corrosion and expansion operation, so that when the circle of the edge of the pipe orifice of the installation container is positioned in the Hough detection processing process, the error of a few pixels exists in the position of the midpoint of the pipe orifice on a pixel coordinate system.
Therefore, in the preliminary simulation stage of the scheme, error statistics can be carried out according to simulated pipe orifice pixel position information and world coordinate position information of each installation container. When calculating the error, the relative pixel error can be calculated on the X axis and the Y axis respectively according to the pixel position information of the middle points of the nozzles of the two mounting containers in the same row.
Taking a row of installation containers as an example, assuming that the row of installation containers includes installation containers 0, 2, 4, and 6 (serial numbers), the specific calculation manner is as follows:
ξ＝|ΔD1-ΔD2| (6)
wherein, X0、X2、X4、X6X-axis pixel position information, Δ D, representing mounting containers 0, 2, 4, 61A difference value, Δ D, representing X-axis pixel position information of the mounting container 0 and the mounting container 22A difference value indicating X-axis pixel position information of the mounting container 4 and the mounting container 6, and ξ indicates a relative error between these two difference values.
Similarly, the relative error between the Y-axis pixel position information of each mounting container and the relative error of each mounting container in the world coordinate system can be calculated.
In the preliminary simulation stage of the present embodiment, taking 10 installation containers as an example, as shown in fig. 5a, and combining the error calculation manner described above, the relative errors of the 10 installation containers in the pixel coordinate system and the world coordinate system can be calculated, as shown in table 1 below:
TABLE 1
After obtaining the relative errors of the installation container in the pixel coordinate system and in the world coordinate system in the simulation stage, the pixel position information and the world coordinate position information of the installation container in the above S502-S504 may be compensated, and optionally, the following steps B1 and B2 may be used for compensation:
and step B1, carrying out pixel error compensation on the pixel position information of the at least one mounting container according to a preset pixel error threshold range.
The pixel error threshold range preset here includes pixel error threshold ranges in the X-axis direction and the Y-axis direction. The pixel error threshold range may be a data range or a threshold, and here, for example, a threshold is taken as an example, the X axis in table 1 is 1 pixel error, and the Y axis is 2 pixel errors. Then, after obtaining the pixel position information of the mounting containers, the pixel position information in the corresponding axial direction of each mounting container can be compensated according to the pixel error in the X axis and the pixel error in the Y axis, where the compensation can be to add or subtract the pixel error, and finally, the compensated pixel position information of each mounting container can be obtained.
And step B2, carrying out world coordinate error compensation on the world coordinate position information of the at least one mounting container according to a preset world coordinate error threshold range.
For the error compensation of the world coordinate system, the same principle as the error compensation of the pixel coordinate system can be referred to the error compensation of the pixel coordinate system, and details are not repeated here.
The object mounting method of the embodiment may downconvert the pixel position information of the mounting container from the pixel coordinate system to the world coordinate system, and mount the object to be mounted into the mounting container according to the world coordinate position information in the world coordinate system. In the embodiment, the object to be installed can be installed through coordinate system conversion, so that the installation position can be more practical, and the success rate and the accuracy rate of the object installation can be improved.
In another embodiment, another object installation method is provided, and the embodiment relates to a specific process that drawing can be performed after the installation containers are sorted. On the basis of the above embodiment, the above method may further include the following step C:
and C, drawing the pixel position information of the at least one installation container based on the pixel position sequencing result of the at least one installation container to obtain a pixel position information drawing image of the at least one installation container.
In this step, referring to fig. 6, an exemplary diagram of the installation containers in the actual installation stage is provided, after the pixel position sorting result of each installation container is obtained, the edge of the nozzle of each installation container can be drawn on the image of the installation container according to the sorting result, and usually, each nozzle is placed next to each other, so that each drawn nozzle is also next to each other, and thus, if a defect is found in the middle of two or some drawn nozzles through the drawing diagram, the installation container can be found to be missed in time, and thus, the image of the installation container can be rapidly redetected, further, the subsequent missing of the installation object can be avoided, and the accuracy of the installation of the object can be improved.
According to the object installation method, the pixel positions of the installation container can be drawn based on the pixel position sequencing result of the installation container, so that the pixel position drawing image of the installation container is obtained, therefore, when objects are installed subsequently, the objects can be prevented from being installed in a missing mode subsequently through the drawing image, and the accuracy of object installation is improved.
In another embodiment, another object installation method is provided, and the embodiment relates to a specific process of grouping and labeling installation containers according to the pixel position sorting result of the installation containers. On the basis of the above embodiment, the above method may further include the following step D:
and D, grouping and labeling the at least one installation container based on the pixel position sequencing result of the at least one installation container to obtain at least one group of installation containers.
In this step, continuing to refer to the right diagram in fig. 5a and the right diagram in fig. 6, assuming that there are two rows of mounting containers, namely, the first row and the second row, then the first row and the second row may be divided into a group, namely, the first group, the second row and the second row may be divided into a group, namely, the second group, according to the pixel position sorting result of each mounting container, and the following is repeated, and finally, the two rows of mounting containers are grouped into a full good group. Thereafter, each installation container may be numbered according to the grouping situation, starting from 0 or starting from 1, here illustratively from 0, the two installation containers of the first group are respectively labeled as 0 and 1, the two installation containers of the second group are respectively labeled as 2 and 3, and so on.
It should be noted that fig. 5a and fig. 6 are only examples, and do not affect the essence of the embodiments of the present application.
In the object installation method of this embodiment, the installation containers may be grouped and labeled based on the pixel position sorting result of the installation containers, so as to obtain at least one group of installation containers. Through grouping and labeling, the object installation can be conveniently and quickly carried out on the installation container subsequently, and meanwhile, the object can be prevented from being installed in a missing mode to a certain extent.
It should be understood that although the various steps in the flow charts of fig. 2-5 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-5 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 7, there is provided an object mounting apparatus including: an acquisition module 10, an object detection module 11 and a mounting module 12, wherein:
the acquisition module 10 is used for acquiring an image of an installation container corresponding to an object to be installed;
an object detection module 11, configured to perform object detection on the image of the installation container, and determine pixel position information of at least one installation container in the image of the installation container;
and the mounting module 12 is configured to mount the object to be mounted into the at least one mounting container according to the pixel position information of the at least one mounting container.
For specific limitations of the object installation apparatus, reference may be made to the above limitations of the object installation method, which are not described herein again.
In another embodiment, another object installation apparatus is provided, and on the basis of the above embodiment, the object detection module 11 may include an object extraction unit and a hough detection unit, wherein:
an object extraction unit configured to perform an object extraction process on the image of the mounting container and determine an edge image of the mounting container corresponding to the image of the mounting container;
and the Hough detection unit is used for carrying out Hough detection processing on the edge image of the installation container and determining pixel position information of at least one installation container in the image of the installation container.
In another embodiment, another object installation apparatus is provided, and on the basis of the above embodiment, before the installation module 12 installs the object to be installed into the at least one installation container according to the pixel position information of the at least one installation container, the apparatus may further include a sorting module, configured to sort the pixel position information of the at least one installation container, and determine a pixel position sorting result of the at least one installation container.
Optionally, the pixel position information of the installation container includes pixel position information of a central point of a nozzle of the installation container, and the pixel position information of the central point of the nozzle of the installation container includes X-axis pixel position information of the central point of the nozzle and Y-axis pixel position information of the central point of the nozzle.
Optionally, the sorting module may include an X-axis sorting unit and a Y-axis sorting unit, wherein:
the X-axis sequencing unit is used for sequencing the X-axis pixel position information of the central point of the pipe orifice of the at least one mounting container to obtain a sequencing result of the X-axis pixel position of the at least one mounting container;
and the Y-axis sequencing unit is used for sequencing the Y-axis pixel position information of the central point of the pipe orifice of the at least one mounting container based on the X-axis pixel position sequencing result to obtain the pixel position sequencing result of the at least one mounting container.
In another embodiment, another object installation apparatus is provided, and on the basis of the above embodiment, the installation module 12 may include a coordinate conversion unit and an installation unit, wherein:
a coordinate conversion unit for converting the pixel position information of the at least one mounting container from a pixel coordinate system to a world coordinate system, and determining the world coordinate position information of the at least one mounting container;
and the mounting unit is used for mounting the object to be mounted into the at least one mounting container according to the world coordinate position information of the at least one mounting container.
Optionally, the apparatus may further include a pixel error compensation module and a world coordinate error compensation module, wherein:
the pixel error compensation module is used for carrying out pixel error compensation on the pixel position information of the at least one mounting container according to a preset pixel error threshold range;
and the world coordinate error compensation module is used for performing world coordinate error compensation on the world coordinate position information of the at least one installation container according to a preset world coordinate error threshold range.
In another embodiment, another object installation apparatus is provided, and on the basis of the above embodiment, the apparatus may further include a drawing module, configured to draw the pixel position information of the at least one installation container based on a pixel position sorting result of the at least one installation container, so as to obtain a pixel position information drawing image of the at least one installation container.
In another embodiment, another object installation apparatus is provided, and on the basis of the above embodiment, the apparatus may further include a grouping and labeling module, configured to group and label the at least one installation container based on a pixel position sorting result of the at least one installation container, so as to obtain at least one group of installation containers.
For specific limitations of the object installation apparatus, reference may be made to the above limitations of the object installation method, which are not described herein again.
The respective modules in the object installation apparatus described above may be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, an object mounting system is provided, which includes a computer device and a robot arm connected to each other, wherein, taking the computer device as an example of a terminal, an internal structure diagram thereof may be as shown in fig. 8. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement an object installation method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, there is provided an object mounting system comprising a computer device and a robotic arm coupled to each other, the computer device comprising a memory and a processor, the memory storing a computer program,
the processor is used for acquiring an image of an installation container corresponding to an object to be installed when executing the computer program;
the processor is further configured to perform object detection on the image of the installation container and determine pixel position information of at least one installation container in the image of the installation container when executing the computer program;
the robot arm is configured to mount the object to be mounted in the at least one mounting container according to the pixel position information of the at least one mounting container.
In an embodiment, the above processor is specifically adapted to execute the computer program when executing the computer program
Performing target extraction processing on the image of the installation container, and determining an edge image of the installation container corresponding to the image of the installation container; and carrying out Hough detection processing on the edge image of the mounting container, and determining pixel position information of at least one mounting container in the image of the mounting container.
In one embodiment, the processor is further configured to execute the computer program when executing the computer program
And sequencing the pixel position information of the at least one mounting container, and determining a pixel position sequencing result of the at least one mounting container.
In one embodiment, the pixel position information of the mounting container includes pixel position information of a nozzle center point of the mounting container, and the pixel position information of the nozzle center point of the mounting container includes X-axis pixel position information of the nozzle center point and Y-axis pixel position information of the nozzle center point.
In an embodiment, the above processor is specifically adapted to execute the computer program when executing the computer program
Sequencing the X-axis pixel position information of the central point of the pipe orifice of the at least one mounting container to obtain a sequencing result of the X-axis pixel position of the at least one mounting container; and sequencing the Y-axis pixel position information of the central point of the pipe orifice of the at least one mounting container based on the X-axis pixel position sequencing result to obtain the pixel position sequencing result of the at least one mounting container.
In one embodiment, the processor, when executing the computer program, is further configured to convert the pixel position information of the at least one installation container from a pixel coordinate system to a world coordinate system, and determine world coordinate position information of the at least one installation container;
and the mechanical arm is used for installing the object to be installed into the at least one installation container according to the world coordinate position information of the at least one installation container.
In one embodiment, the processor is further configured to execute the computer program when executing the computer program
Performing pixel error compensation on the pixel position information of the at least one mounting container according to a preset pixel error threshold range; and according to a preset world coordinate error threshold range, performing world coordinate error compensation on the world coordinate position information of the at least one installation container.
In one embodiment, the processor is further configured to execute the computer program when executing the computer program
And drawing the pixel position information of the at least one installation container based on the pixel position sequencing result of the at least one installation container to obtain a pixel position information drawing image of the at least one installation container.
In one embodiment, the processor is further configured to execute the computer program when executing the computer program
And grouping and marking the at least one installation container based on the pixel position sorting result of the at least one installation container to obtain at least one group of installation containers.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring an image of an installation container corresponding to an object to be installed; performing target detection on the image of the installation container, and determining pixel position information of at least one installation container in the image of the installation container; and installing the object to be installed into the at least one installation container according to the pixel position information of the at least one installation container.
In one embodiment, the computer program when executed by the processor further performs the steps of:
performing target extraction processing on the image of the installation container, and determining an edge image of the installation container corresponding to the image of the installation container; and carrying out Hough detection processing on the edge image of the mounting container, and determining pixel position information of at least one mounting container in the image of the mounting container.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and sequencing the pixel position information of the at least one mounting container, and determining a pixel position sequencing result of the at least one mounting container.
In one embodiment, the pixel position information of the mounting container includes pixel position information of a nozzle center point of the mounting container, and the pixel position information of the nozzle center point of the mounting container includes X-axis pixel position information of the nozzle center point and Y-axis pixel position information of the nozzle center point.
In one embodiment, the computer program when executed by the processor further performs the steps of:
sequencing the X-axis pixel position information of the central point of the pipe orifice of the at least one mounting container to obtain a sequencing result of the X-axis pixel position of the at least one mounting container; and sequencing the Y-axis pixel position information of the central point of the pipe orifice of the at least one mounting container based on the X-axis pixel position sequencing result to obtain the pixel position sequencing result of the at least one mounting container.
In one embodiment, the computer program when executed by the processor further performs the steps of:
converting the pixel position information of the at least one mounting container from a pixel coordinate system to a world coordinate system, and determining the world coordinate position information of the at least one mounting container; and installing the object to be installed into the at least one installation container according to the world coordinate position information of the at least one installation container.
In one embodiment, the computer program when executed by the processor further performs the steps of:
performing pixel error compensation on the pixel position information of the at least one mounting container according to a preset pixel error threshold range; and according to a preset world coordinate error threshold range, performing world coordinate error compensation on the world coordinate position information of the at least one installation container.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and drawing the pixel position information of the at least one installation container based on the pixel position sequencing result of the at least one installation container to obtain a pixel position information drawing image of the at least one installation container.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and grouping and marking the at least one installation container based on the pixel position sorting result of the at least one installation container to obtain at least one group of installation containers.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of object installation, the method comprising:
acquiring an image of an installation container corresponding to an object to be installed;
carrying out target detection on the image of the installation container, and determining pixel position information of at least one installation container in the image of the installation container;
and installing the object to be installed into the at least one installation container according to the pixel position information of the at least one installation container.
2. The method of claim 1, wherein the performing object detection on the image of the installation container and determining pixel position information of at least one installation container in the image of the installation container comprises:
performing target extraction processing on the image of the installation container, and determining an edge image of the installation container corresponding to the image of the installation container;
and carrying out Hough detection processing on the edge image of the installation container, and determining pixel position information of at least one installation container in the image of the installation container.
3. The method according to claim 2, wherein before the mounting the object to be mounted into the at least one mounting container according to the pixel position information of the at least one mounting container, the method further comprises:
sorting the pixel position information of the at least one installation container, and determining a pixel position sorting result of the at least one installation container; the pixel position information of the mounting container comprises pixel position information of a central point of a pipe orifice of the mounting container, and the pixel position information of the central point of the pipe orifice of the mounting container comprises X-axis pixel position information of the central point of the pipe orifice and Y-axis pixel position information of the central point of the pipe orifice.
4. The method of claim 3, wherein the sorting the pixel position information of the at least one installation container and determining the result of the sorting of the pixel position of the at least one installation container comprises:
sequencing the X-axis pixel position information of the central point of the pipe orifice of the at least one mounting container to obtain a sequencing result of the X-axis pixel position of the at least one mounting container;
and sequencing the Y-axis pixel position information of the central point of the pipe orifice of the at least one mounting container based on the X-axis pixel position sequencing result to obtain a pixel position sequencing result of the at least one mounting container.
5. The method according to any one of claims 1 to 4, wherein the installing the object to be installed into the at least one installation container according to the pixel position information of the at least one installation container comprises:
converting the pixel position information of the at least one mounting container from a pixel coordinate system to a world coordinate system, and determining the world coordinate position information of the at least one mounting container;
and installing the object to be installed into the at least one installation container according to the world coordinate position information of the at least one installation container.
6. The method according to any one of claims 3-4, further comprising:
and drawing the pixel position information of the at least one installation container based on the pixel position sequencing result of the at least one installation container to obtain a pixel position information drawing image of the at least one installation container.
7. The method according to any one of claims 3-4, further comprising:
and grouping and labeling the at least one installation container based on the pixel position sequencing result of the at least one installation container to obtain at least one group of installation containers.
8. An object mounting apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring an image of an installation container corresponding to an object to be installed;
the target detection module is used for carrying out target detection on the image of the installation container and determining pixel position information of at least one installation container in the image of the installation container;
and the mounting module is used for mounting the object to be mounted into the at least one mounting container according to the pixel position information of the at least one mounting container.
9. An object mounting system comprising a computer device and a robot arm connected to each other, the computer device comprising a memory and a processor, the memory storing a computer program, characterized in that,
the processor of the computer device is used for acquiring an image of an installation container corresponding to an object to be installed when executing the computer program;
when the processor of the computer device executes the computer program, the processor is further configured to perform target detection on the image of the installation container, and determine pixel position information of at least one installation container in the image of the installation container;
the mechanical arm is used for installing the object to be installed into the at least one installation container according to the pixel position information of the at least one installation container.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011541658.5A CN112529961A (en) | 2020-12-23 | 2020-12-23 | Object installation method, device, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011541658.5A CN112529961A (en) | 2020-12-23 | 2020-12-23 | Object installation method, device, system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112529961A true CN112529961A (en) | 2021-03-19 |
Family
ID=74975950
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011541658.5A Pending CN112529961A (en) | 2020-12-23 | 2020-12-23 | Object installation method, device, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112529961A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114067102A (en) * | 2021-11-17 | 2022-02-18 | 中国矿业大学 | Liquid level detection control method in mechanical arm liquid pouring task based on target detection |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101359400A (en) * | 2008-09-17 | 2009-02-04 | 湖南大学 | Process for positioning spatial position of pipe mouth based on vision |
| CN102289824A (en) * | 2011-07-06 | 2011-12-21 | 湖南大学 | Method for positioning plane centroid of pipe orifice image of condenser |
-
2020
- 2020-12-23 CN CN202011541658.5A patent/CN112529961A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101359400A (en) * | 2008-09-17 | 2009-02-04 | 湖南大学 | Process for positioning spatial position of pipe mouth based on vision |
| CN102289824A (en) * | 2011-07-06 | 2011-12-21 | 湖南大学 | Method for positioning plane centroid of pipe orifice image of condenser |
Non-Patent Citations (1)
| Title |
|---|
| STEVE CHARLCS 等: "《玻璃体显微手术学》", 31 March 2013, 人民军医出版社 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114067102A (en) * | 2021-11-17 | 2022-02-18 | 中国矿业大学 | Liquid level detection control method in mechanical arm liquid pouring task based on target detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110245662B (en) | Detection model training method and device, computer equipment and storage medium | |
| US20220036587A1 (en) | Calibration method, electronic device, and storage medium | |
| EP3330921A1 (en) | Information processing device, measuring apparatus, system, calculating method, storage medium, and article manufacturing method | |
| JP2019036167A (en) | Image processing apparatus and image processing method | |
| Zhou et al. | Exploring faster RCNN for fabric defect detection | |
| WO2017107865A1 (en) | Image retrieval system, server, database, and related method | |
| CN110648363A (en) | Camera posture determining method and device, storage medium and electronic equipment | |
| CN114494892A (en) | Goods shelf commodity display information identification method, device, equipment and storage medium | |
| CN104517092A (en) | Method and system for detecting QR (quick response) code correction graphics | |
| CN114769021B (en) | Robot spraying system and method based on full-angle template recognition | |
| CN113870189A (en) | Industrial product circular detection method and device | |
| CN112529961A (en) | Object installation method, device, system and storage medium | |
| CN112464785A (en) | Target detection method and device, computer equipment and storage medium | |
| CN111832561A (en) | Character sequence recognition method, device, equipment and medium based on computer vision | |
| CN114693529A (en) | Image splicing method, device, equipment and storage medium | |
| CN113344994B (en) | Image registration method, device, electronic equipment and storage medium | |
| JP3661635B2 (en) | Image processing method and apparatus | |
| CN113269829B (en) | Flow production line target positioning method and device, computer equipment and storage medium | |
| JP4747293B2 (en) | Image processing apparatus, image processing method, and program used therefor | |
| CN113128247B (en) | Image positioning identification verification method and server | |
| US20240169511A1 (en) | Method and apparatus for acquiring images, and storage medium thereof | |
| CN112308061B (en) | License plate character recognition method and device | |
| CN113989793A (en) | Graphite electrode embossed seal character recognition method | |
| CN115063432B (en) | Method and device for identifying alignment mark pattern, computer equipment and storage medium | |
| CN112529829A (en) | Training method and device for burr positioning and burr detection model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |