CN112045681A - Method for acquiring position of mounting hole, method for mounting license plate and corresponding device - Google Patents
Method for acquiring position of mounting hole, method for mounting license plate and corresponding device Download PDFInfo
- Publication number
- CN112045681A CN112045681A CN202010910275.4A CN202010910275A CN112045681A CN 112045681 A CN112045681 A CN 112045681A CN 202010910275 A CN202010910275 A CN 202010910275A CN 112045681 A CN112045681 A CN 112045681A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- license plate
- coordinate system
- mounting hole
- mechanical arm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1687—Assembly, peg and hole, palletising, straight line, weaving pattern movement
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1684—Tracking a line or surface by means of sensors
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for acquiring the position of a mounting hole, a method for mounting a license plate and a corresponding device, wherein the method for acquiring the position of the mounting hole comprises the following steps: through the motion of arm control manipulator, and acquire the two-dimensional image of the tip of car through two-dimensional camera, and through the point cloud model of three-dimensional camera generation tip, the point cloud model includes: three-dimensional coordinates based on a camera coordinate system of points of the outer surface of the tip, and a color; and acquiring the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system based on the two-dimensional image and the point cloud model. Thereby enabling the position information of the mounting hole to be acquired.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a method for acquiring the position of a mounting hole, a method for mounting a license plate and a corresponding device.
Background
Generally, a license plate mounting portion is provided at each of a front end portion and a rear end portion of an automobile, a plurality of mounting holes (for example, four screw holes) are provided at the mounting portion, and correspondingly, a plurality of through holes are provided at the license plate. In the prior art, when the license plate is installed, the through hole on the license plate needs to be aligned with the installation hole, and then the license plate is screwed down by using screws, which is very labor-consuming.
Therefore, it is required to design a license plate installation apparatus capable of automatically installing a license plate, and it can be understood that a core function of the license plate installation apparatus is: how to acquire the position information of the mounting hole.
Disclosure of Invention
In view of the above, the main object of the present invention is to provide a method for obtaining the location of a mounting hole, a method for mounting a license plate, and a corresponding device.
In order to achieve one of the above objects of the invention, an embodiment of the present invention provides a method for obtaining a location of a mounting hole of a license plate installation apparatus, the license plate installation apparatus including: the mechanical arm is provided with a two-dimensional camera and a three-dimensional camera; the method comprises the following steps: through the arm control the manipulator motion, and acquire the two-dimensional image of the tip of car through two-dimensional camera, and generate through three-dimensional camera the point cloud model of tip, the point cloud model includes: three-dimensional coordinates of points of the outer surface of the tip based on a camera coordinate system, and a color; and acquiring the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system based on the two-dimensional image and the point cloud model.
As an improvement of the embodiment of the present invention, the "acquiring a three-dimensional coordinate based on a robot arm coordinate system of a mounting hole based on the two-dimensional image and the point cloud model" specifically includes: acquiring a two-dimensional center coordinate P2d of the mounting hole from the two-dimensional image, acquiring a three-dimensional center coordinate P3d corresponding to the two-dimensional center coordinate P2d from the point cloud model, and generating a homogeneous coordinate Pct corresponding to the three-dimensional center coordinate P3 d; wherein P2d, P3d, and Pct are based on the camera coordinate system; the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system are Thw Tch Pct, wherein Tch is a calibration matrix between the camera coordinate system and the mechanical arm coordinate system, and Thw is a transformation matrix between the mechanical arm coordinate system and the mechanical arm coordinate system.
As an improvement of the embodiment of the present invention, Thw is obtained based on a six-axis vector of the posture of the robot.
The embodiment of the invention provides a device for acquiring the position of a mounting hole of automobile license plate mounting equipment, which comprises: the mechanical arm is provided with a two-dimensional camera and a three-dimensional camera; the system comprises the following modules: the information acquisition module is used for controlling the mechanical arm to move through the mechanical arm, acquiring a two-dimensional image of the end part of the automobile through a two-dimensional camera, and generating a point cloud model of the end part through a three-dimensional camera, wherein the point cloud model comprises: three-dimensional coordinates of points of the outer surface of the tip based on a camera coordinate system, and a color; and the control module is used for acquiring the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system based on the two-dimensional image and the point cloud model.
As an improvement of the embodiment of the present invention, the information obtaining module is further configured to: acquiring a two-dimensional center coordinate P2d of the mounting hole from the two-dimensional image, acquiring a three-dimensional center coordinate P3d corresponding to the two-dimensional center coordinate P2d from the point cloud model, and generating a homogeneous coordinate Pct corresponding to the three-dimensional center coordinate P3 d; wherein P2d, P3d, and Pct are based on the camera coordinate system; the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system are Thw Tch Pct, wherein Tch is a calibration matrix between the camera coordinate system and the mechanical arm coordinate system, and Thw is a transformation matrix between the mechanical arm coordinate system and the mechanical arm coordinate system.
As an improvement of the embodiment of the present invention, Thw is obtained based on a six-axis vector of the posture of the robot.
The embodiment of the invention provides a method for installing license plates for automobile license plate installation equipment, wherein the automobile license plate installation equipment comprises a license plate grabbing device and a screw grabbing device, the license plate grabbing device can grab a license plate, a plurality of through holes are formed in the periphery of the license plate, and the through holes correspond to a plurality of installation holes in the end part of an automobile one by one; the screw grabbing device can grab the screw extending into the through hole and can control the screw to rotate; the method comprises the following steps: based on the method for acquiring the positions of the mounting holes, acquiring three-dimensional coordinates of a plurality of mounting holes in the end part of the automobile based on a mechanical arm coordinate system, and two-dimensional center coordinates of the plurality of mounting holes in the two-dimensional image; wherein the two-dimensional center coordinates are based on a camera coordinate system; obtaining a transformation matrix based on the plurality of two-dimensional center coordinates and the plurality of three-dimensional coordinates; obtaining a license plate installation posture based on the transformation matrix and the length of the screw; and controlling the mechanical arm to move based on the license plate installation posture.
The embodiment of the invention provides a device for installing license plates for automobile license plate installation equipment, which comprises a license plate grabbing device and a screw grabbing device, wherein the license plate grabbing device can grab the license plate, a plurality of through holes are formed in the periphery of the license plate, and the through holes correspond to a plurality of installation holes in the end part of an automobile one by one; the screw grabbing device can grab the screw extending into the through hole and can control the screw to rotate; the system comprises the following modules:
the mounting hole coordinate acquisition module is used for acquiring the position of the mounting hole, acquiring three-dimensional coordinates of a plurality of mounting holes in the end part of the automobile based on a mechanical arm coordinate system and two-dimensional center coordinates of the plurality of mounting holes in the two-dimensional image; wherein the two-dimensional center coordinates are based on a camera coordinate system;
the license plate mounting posture is used for obtaining a transformation matrix based on a plurality of two-dimensional center coordinates and a plurality of three-dimensional coordinates; obtaining a license plate installation posture based on the transformation matrix and the length of the screw;
and the mounting module is used for controlling the movement of the manipulator based on the license plate mounting posture.
Compared with the prior art, the invention has the technical effects that: the embodiment of the invention discloses a method for acquiring the position of a mounting hole, a method for mounting a license plate and a corresponding device, wherein the method for acquiring the position of the mounting hole comprises the following steps: through the motion of arm control manipulator, and acquire the two-dimensional image of the tip of car through two-dimensional camera, and through the point cloud model of three-dimensional camera generation tip, the point cloud model includes: three-dimensional coordinates based on a camera coordinate system of points of the outer surface of the tip, and a color; and acquiring the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system based on the two-dimensional image and the point cloud model. Thereby enabling the position information of the mounting hole to be acquired.
Drawings
FIG. 1 is a schematic diagram of a usage scenario in an embodiment of the invention;
FIG. 2 is a schematic flow chart illustrating a method for obtaining a location of a mounting hole according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating a method of installing a license plate according to an embodiment of the present invention.
Detailed Description
The present invention will be described in detail below with reference to embodiments shown in the drawings. The present invention is not limited to the embodiment, and structural, methodological, or functional changes made by one of ordinary skill in the art according to the embodiment are included in the scope of the present invention.
The following description and the drawings sufficiently illustrate specific embodiments herein to enable those skilled in the art to practice them. Portions and features of some embodiments may be included in or substituted for those of others. The scope of the embodiments herein includes the full ambit of the claims, as well as all available equivalents of the claims. The terms "first," "second," and the like, herein are used solely to distinguish one element from another without requiring or implying any actual such relationship or order between such elements. In practice, a first element can also be referred to as a second element, and vice versa. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a structure, apparatus, or device that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such structure, apparatus, or device. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a structure, device or apparatus that comprises the element. The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like herein, as used herein, are defined as orientations and positional relationships based on the orientation or positional relationship shown in the drawings, and are used for convenience in describing and simplifying the description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be construed as limiting the invention. In the description herein, unless otherwise specified and limited, the terms "mounted," "connected," and "connected" are to be construed broadly, and may include, for example, mechanical or electrical connections, communications between two elements, direct connections, and indirect connections via intermediary media, where the specific meaning of the terms is understood by those skilled in the art as appropriate.
An embodiment of the present invention provides a method for obtaining a location of a mounting hole of license plate mounting equipment, where the license plate mounting equipment includes: the mechanical arm is provided with a two-dimensional camera and a three-dimensional camera; here, in the traffic control department, a special place for installing license plates is generally provided, as shown in fig. 1, in which a parking area and two license plate installation apparatuses are provided, and when a car is parked in the parking area, there are license plate installation apparatuses at both front and rear ends of the car, it is understood that the license plate installation apparatus at the front side portion can perform installation of the license plate at the front side portion, and the license plate installation apparatus at the rear side portion can perform installation of the license plate at the rear side portion.
The mechanical arm is provided with a license plate grabbing device (for example, a suction nozzle is used for sucking the license plate), the license plate grabbing device can grab the license plate, the two-dimensional and three-dimensional cameras can acquire images of the front end part and the rear end part of the automobile, and after the positions of license plate mounting holes in the front end part or the rear end part of the automobile are determined based on the images, the mechanical arm can control the mechanical arm to align the license plate with the mounting holes, and then the license plate can be mounted on the mounting holes; here, it is understood that the two-dimensional camera may acquire two-dimensional images of front and rear end portions of the automobile, and the three-dimensional camera may acquire three-dimensional images of front and rear end portions of the automobile; here, the two-dimensional camera can record all objects within its viewing angle, but the data recorded by the two-dimensional camera does not include the distances of the objects from the camera. And the three-dimensional camera 3D can also accurately know the distance between each point in the image and the camera.
Here, the robot arm is sized long enough to enable the robot arm to touch the front and rear ends of the automobile, respectively.
Here, in practice, a mounting portion for mounting a license plate is provided at both the front end portion and the rear end portion of the automobile, and the mounting portion is generally provided with several mounting holes (generally 4), and therefore, the method of acquiring the positions of the mounting holes may be performed once for each mounting hole, respectively.
The method comprises the following steps:
step 201: through the arm control the manipulator motion, and acquire the two-dimensional image of the tip of car through two-dimensional camera, and generate through three-dimensional camera the point cloud model of tip, the point cloud model includes: three-dimensional coordinates of points of the outer surface of the tip based on a camera coordinate system, and a color;
here, the robot arm needs to control the movement of the robot arm in the left, up, down, left, and right directions so that a two-dimensional and three-dimensional image of the end portion can be captured better. The end portion may be a front end portion or a rear end portion, and the color is in RGB (Red, Green, Blue) format.
In practice, the robot movement can be controlled according to preset steps, such as: presetting a highest scanning position, a lowest scanning position, a leftmost scanning position and a rightmost scanning position of the manipulator, controlling the manipulator to start scanning from the highest scanning position and finish a scanning subtask, and controlling the position of the manipulator to descend by a preset height value, and then controlling the position of the manipulator to descend by the preset height value after finishing the next scanning subtask until the height of the manipulator is lower than the lowest scanning position, wherein the scanning subtask can be as follows: and controlling the manipulator to move from the leftmost scanning position to the rightmost scanning position, and controlling the two-dimensional camera and the three-dimensional camera to respectively complete a camera shooting task.
Here, in practice, the position of the vehicle in the front-rear and left-right directions is generally variable, and the height of the mounting hole is also variable depending on the model of the vehicle, so that it is necessary to first determine the position of the vehicle in the up-down and left-right directions and then control the robot to move up, down, left, and right around the mounting hole. Here, the method may be: firstly, controlling a two-dimensional camera to shoot a picture of the end part at a fixed position, then using an image recognition algorithm to recognize whether the image has a mounting hole, if not, moving the manipulator to the next position, and then continuing shooting until the shot image contains the mounting hole. Then, if the image includes the mounting hole, it is determined whether the mounting hole is located at a middle position (if the distance from the mounting hole to a certain boundary of the image is smaller than a preset threshold value, the mounting hole is not located at the middle position), if not, fine adjustment of the position of the manipulator is required to enable the mounting hole to be located at the middle position, and then the manipulator can be controlled to move up, down, left and right by taking the position as a center to acquire a three-dimensional image.
Here, the method of generating the point cloud model may specifically be: step 1, acquiring a plurality of three-dimensional images of the end part and a depth image corresponding to each three-dimensional image through a three-dimensional camera; step 2, determining a depth value corresponding to a pixel point in each three-dimensional image according to the depth image; and 3, determining the point cloud coordinate of each pixel in each three-dimensional image according to the pixel point coordinate and the depth value in the three-dimensional image and the built-in parameters of the three-dimensional camera to obtain a point cloud model.
Step 202: and acquiring the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system based on the two-dimensional image and the point cloud model.
In this embodiment, the "obtaining a three-dimensional coordinate based on a robot arm coordinate system of a mounting hole based on the two-dimensional image and the point cloud model" specifically includes: acquiring a two-dimensional center coordinate P2d of the mounting hole from the two-dimensional image, acquiring a three-dimensional center coordinate P3d corresponding to the two-dimensional center coordinate P2d from the point cloud model, and generating a homogeneous coordinate Pct corresponding to the three-dimensional center coordinate P3 d; wherein P2d, P3d, and Pct are based on the camera coordinate system; the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system are Thw Tch Pct, wherein Tch is a calibration matrix between the camera coordinate system and the mechanical arm coordinate system, and Thw is a transformation matrix between the mechanical arm coordinate system and the mechanical arm coordinate system.
Here, the "acquiring the two-dimensional center coordinates P2d of the mounting hole from the two-dimensional image" may specifically include: step 1, performing graying processing on a two-dimensional image, and then denoising (for example, removing partial noise points by adopting median filtering); step 2, image binarization processing, for example, binarization processing is carried out by adopting a threshold value of the maximum inter-class variance ratio; step 3, labeling, wherein besides the primary target circular hole in the binarized image, the binarized image also comprises a plurality of other information, and labeling is carried out to label different numbers on different communicated areas so as to prepare for extracting the circular hole; step 4, identifying the round holes, namely firstly calculating the areas of different connected regions of the labeled image, extracting the boundary of each connected region, calculating the perimeter of each connected region, then calculating the circularity of each connected region, and identifying the round holes through the circularity; and 5, acquiring the center of mass of the round hole, and acquiring a two-dimensional coordinate of the center of mass, wherein the two-dimensional coordinate is recorded as a two-dimensional center coordinate P2d of the mounting hole.
Here, the "acquiring the two-dimensional center coordinates P2d of the mounting hole from the two-dimensional image" may specifically include: step 1, detecting a target image; step 2, detecting target edge points; step 3, filling a target area and removing a background through opening and closing operations; and 4, calculating the coordinates of the circle center.
Here, the three-dimensional vector P3d may be represented by one four-dimensional vector Pct, for example, P3d ═ x, y, z, and Pct may be (x, y, z, 1).
Here, a specific example is given:
obtaining a two-dimensional center coordinate P2d of the mounting hole from the two-dimensional image (20,30)tThe two-dimensional center coordinate P2d is the corresponding three-dimensional center coordinate P3d (-0.2,0.4,0.8) in the point cloud model in pixeltThe unit: rice, generating homogeneous coordinate Pct (-0.2,0.4,0.8,1) corresponding to the three-dimensional center coordinate P3dtCalibration matrix
Transformation matrix
The three-dimensional coordinates of the mounting hole based on the robot arm coordinate system are (0,1.24,1)t。
In the present embodiment, Thw is obtained based on a six-axis vector of the posture of the robot.
An embodiment of the present invention provides an apparatus for acquiring a position of a mounting hole of license plate mounting equipment, where the license plate mounting equipment includes: the mechanical arm is provided with a two-dimensional camera and a three-dimensional camera; the system comprises the following modules:
the information acquisition module is used for controlling the mechanical arm to move through the mechanical arm, acquiring a two-dimensional image of the end part of the automobile through a two-dimensional camera, and generating a point cloud model of the end part through a three-dimensional camera, wherein the point cloud model comprises: three-dimensional coordinates of points of the outer surface of the tip based on a camera coordinate system, and a color;
and the control module is used for acquiring the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system based on the two-dimensional image and the point cloud model.
In this embodiment, the information obtaining module is further configured to:
acquiring a two-dimensional center coordinate P2d of the mounting hole from the two-dimensional image, acquiring a three-dimensional center coordinate P3d corresponding to the two-dimensional center coordinate P2d from the point cloud model, and generating a homogeneous coordinate Pct corresponding to the three-dimensional center coordinate P3 d; wherein P2d, P3d, and Pct are based on the camera coordinate system;
the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system are Thw Tch Pct, wherein Tch is a calibration matrix between the camera coordinate system and the mechanical arm coordinate system, and Thw is a transformation matrix between the mechanical arm coordinate system and the mechanical arm coordinate system.
In the present embodiment, Thw is obtained based on a six-axis vector of the posture of the robot.
The third embodiment of the invention provides a method for installing license plates for license plate installation equipment, wherein the license plate installation equipment comprises a license plate grabbing device and a screw grabbing device, the license plate grabbing device can grab the license plate, a plurality of through holes are formed in the periphery of the license plate, and the through holes correspond to a plurality of installation holes in the end part of an automobile one by one; the screw grabbing device can grab the screw extending into the through hole and can control the screw to rotate; as shown in fig. 3, the method comprises the following steps:
step 301: acquiring three-dimensional coordinates of a plurality of mounting holes in the end part of the automobile based on a mechanical arm coordinate system and two-dimensional center coordinates of the plurality of mounting holes in the two-dimensional image based on the method in the first embodiment; wherein the two-dimensional center coordinates are based on a camera coordinate system;
step 302: obtaining a transformation matrix based on the plurality of two-dimensional center coordinates and the plurality of three-dimensional coordinates; obtaining a license plate installation posture based on the transformation matrix and the length of the screw;
here, in many automobiles, the plane on which the license plate is mounted is an inclined plane that inclines forward or backward, and therefore, the license plate mounting posture needs to be acquired, that is, the license plate needs to be inclined accordingly.
Here, assume that there are a total of four mounting holes, which are respectively: a (-0.2,0.8,1.2), B (0.4,0.8,1.2), C (0.4,1,1.2) and D (-0.2,0.8,1.2), and the two-dimensional center coordinates of the four mounting holes in the two-dimensional image are respectively: a (-0,0,0), B (0.6,0,0), C (0.6,0.2,0), and D (0,0.2,0), and the transformation matrix from the camera coordinate system to the robot arm coordinate system is calculated as:
based on a transformation matrix TtwThe mounting posture of the license plate is obtained to be (-0.2,0.8,1.2,0,0, 0).
Step 303: and controlling the mechanical arm to move based on the license plate installation posture.
For example, the length of the screw is corrected to (-0.2,0.8,1.19,0,0,0), and then the robot is controlled to move to align the through hole of the license plate with the mounting hole, and then the license plate is screwed down by the screw.
It should be understood that although the present description refers to embodiments, not every embodiment contains only a single technical solution, and such description is for clarity only, and those skilled in the art should make the description as a whole, and the technical solutions in the embodiments can also be combined appropriately to form other embodiments understood by those skilled in the art.
The above-listed detailed description is only a specific description of a possible embodiment of the present invention, and they are not intended to limit the scope of the present invention, and equivalent embodiments or modifications made without departing from the technical spirit of the present invention should be included in the scope of the present invention.
Claims (8)
1. A method for obtaining a mounting hole position for an automotive license plate installation apparatus, the automotive license plate installation apparatus comprising: the mechanical arm is provided with a two-dimensional camera and a three-dimensional camera; the method is characterized by comprising the following steps:
through the arm control the manipulator motion, and acquire the two-dimensional image of the tip of car through two-dimensional camera, and generate through three-dimensional camera the point cloud model of tip, the point cloud model includes: three-dimensional coordinates of points of the outer surface of the tip based on a camera coordinate system, and a color;
and acquiring the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system based on the two-dimensional image and the point cloud model.
2. The method for obtaining the location of the mounting hole according to claim 1, wherein the "obtaining the three-dimensional coordinates of the mounting hole based on the robot arm coordinate system based on the two-dimensional image and the point cloud model" specifically comprises:
acquiring a two-dimensional center coordinate P2d of the mounting hole from the two-dimensional image, acquiring a three-dimensional center coordinate P3d corresponding to the two-dimensional center coordinate P2d from the point cloud model, and generating a homogeneous coordinate Pct corresponding to the three-dimensional center coordinate P3 d; wherein P2d, P3d, and Pct are based on the camera coordinate system;
the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system are Thw Tch Pct, wherein Tch is a calibration matrix between the camera coordinate system and the mechanical arm coordinate system, and Thw is a transformation matrix between the mechanical arm coordinate system and the mechanical arm coordinate system.
3. The method of obtaining a mounting hole location of claim 2, wherein:
the Thw is obtained based on a six-axis vector of the posture of the robot.
4. An apparatus for obtaining a mounting hole position for a license plate mounting device, the license plate mounting device comprising: the mechanical arm is provided with a two-dimensional camera and a three-dimensional camera; the system is characterized by comprising the following modules:
the information acquisition module is used for controlling the mechanical arm to move through the mechanical arm, acquiring a two-dimensional image of the end part of the automobile through a two-dimensional camera, and generating a point cloud model of the end part through a three-dimensional camera, wherein the point cloud model comprises: three-dimensional coordinates of points of the outer surface of the tip based on a camera coordinate system, and a color;
and the control module is used for acquiring the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system based on the two-dimensional image and the point cloud model.
5. The apparatus for obtaining the location of the installation hole according to claim 4, wherein the information obtaining module is further configured to:
acquiring a two-dimensional center coordinate P2d of the mounting hole from the two-dimensional image, acquiring a three-dimensional center coordinate P3d corresponding to the two-dimensional center coordinate P2d from the point cloud model, and generating a homogeneous coordinate Pct corresponding to the three-dimensional center coordinate P3 d; wherein P2d, P3d, and Pct are based on the camera coordinate system;
the three-dimensional coordinates of the mounting hole based on the mechanical arm coordinate system are Thw Tch Pct, wherein Tch is a calibration matrix between the camera coordinate system and the mechanical arm coordinate system, and Thw is a transformation matrix between the mechanical arm coordinate system and the mechanical arm coordinate system.
6. The device for obtaining the position of the mounting hole according to claim 5, wherein:
the Thw is obtained based on a six-axis vector of the posture of the robot.
7. The method for installing the license plate for the automobile license plate installation equipment comprises a license plate grabbing device and a screw grabbing device, wherein the license plate grabbing device can grab the license plate, a plurality of through holes are formed in the periphery of the license plate, and the through holes correspond to a plurality of installation holes in the end part of an automobile one by one; the screw grabbing device can grab the screw extending into the through hole and can control the screw to rotate; the method is characterized by comprising the following steps:
acquiring the three-dimensional coordinates of a plurality of mounting holes at the end part of the automobile based on the mechanical arm coordinate system and the two-dimensional center coordinates of the plurality of mounting holes in the two-dimensional image based on the method for acquiring the positions of the mounting holes according to any one of claims 1 to 3; wherein the two-dimensional center coordinates are based on a camera coordinate system;
obtaining a transformation matrix based on the plurality of two-dimensional center coordinates and the plurality of three-dimensional coordinates; obtaining a license plate installation posture based on the transformation matrix and the length of the screw;
and controlling the mechanical arm to move based on the license plate installation posture.
8. A device for installing license plates for automobile license plate installation equipment comprises a license plate grabbing device and a screw grabbing device, wherein the license plate grabbing device can grab license plates, a plurality of through holes are formed in the periphery of each license plate, and the through holes correspond to a plurality of installation holes in the end portion of an automobile one by one; the screw grabbing device can grab the screw extending into the through hole and can control the screw to rotate; the system is characterized by comprising the following modules:
a mounting hole coordinate acquisition module, which is used for acquiring the position of the mounting hole based on the method for acquiring the position of the mounting hole, acquiring the three-dimensional coordinates of a plurality of mounting holes at the end part of the automobile based on a mechanical arm coordinate system and the two-dimensional center coordinates of the plurality of mounting holes in the two-dimensional image according to any one of claims 1 to 3; wherein the two-dimensional center coordinates are based on a camera coordinate system;
the license plate mounting posture is used for obtaining a transformation matrix based on a plurality of two-dimensional center coordinates and a plurality of three-dimensional coordinates; obtaining a license plate installation posture based on the transformation matrix and the length of the screw;
and the mounting module is used for controlling the movement of the manipulator based on the license plate mounting posture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010910275.4A CN112045681B (en) | 2020-09-02 | 2020-09-02 | Method for acquiring position of mounting hole, method for mounting license plate and corresponding device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010910275.4A CN112045681B (en) | 2020-09-02 | 2020-09-02 | Method for acquiring position of mounting hole, method for mounting license plate and corresponding device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112045681A true CN112045681A (en) | 2020-12-08 |
| CN112045681B CN112045681B (en) | 2022-05-27 |
Family
ID=73606630
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010910275.4A Active CN112045681B (en) | 2020-09-02 | 2020-09-02 | Method for acquiring position of mounting hole, method for mounting license plate and corresponding device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112045681B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040001149A1 (en) * | 2002-06-28 | 2004-01-01 | Smith Steven Winn | Dual-mode surveillance system |
| CN106182004A (en) * | 2016-08-01 | 2016-12-07 | 上海交通大学 | The method of the industrial robot automatic pin hole assembling that view-based access control model guides |
| CN106530276A (en) * | 2016-10-13 | 2017-03-22 | 中科金睛视觉科技（北京）有限公司 | Manipulator positioning method and system for grabbing of non-standard component |
| CN107767423A (en) * | 2017-10-10 | 2018-03-06 | 大连理工大学 | A kind of mechanical arm target positioning grasping means based on binocular vision |
| CN107984201A (en) * | 2017-11-30 | 2018-05-04 | 中国地质大学（武汉） | A kind of screw hole positioning of view-based access control model servo and lock unload screw method |
| CN109035200A (en) * | 2018-06-21 | 2018-12-18 | 北京工业大学 | A kind of bolt positioning and position and posture detection method based on the collaboration of single binocular vision |
| CN109176521A (en) * | 2018-09-19 | 2019-01-11 | 北京因时机器人科技有限公司 | A kind of mechanical arm and its crawl control method and system |
| CN109940626A (en) * | 2019-01-23 | 2019-06-28 | 浙江大学城市学院 | A kind of thrush robot system and its control method based on robot vision |
| CN110378376A (en) * | 2019-06-12 | 2019-10-25 | 西安交通大学 | A kind of oil filler object recognition and detection method based on machine vision |
| CN110488261A (en) * | 2019-08-23 | 2019-11-22 | 南京理工大学 | A kind of shelf locating hole checking device and method based on LiDAR |
| CN111112981A (en) * | 2020-01-13 | 2020-05-08 | 苏州崮麓智能设备有限公司 | Equipment and method for automatically installing license plate by robot |
-
2020
- 2020-09-02 CN CN202010910275.4A patent/CN112045681B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040001149A1 (en) * | 2002-06-28 | 2004-01-01 | Smith Steven Winn | Dual-mode surveillance system |
| CN106182004A (en) * | 2016-08-01 | 2016-12-07 | 上海交通大学 | The method of the industrial robot automatic pin hole assembling that view-based access control model guides |
| CN106530276A (en) * | 2016-10-13 | 2017-03-22 | 中科金睛视觉科技（北京）有限公司 | Manipulator positioning method and system for grabbing of non-standard component |
| CN107767423A (en) * | 2017-10-10 | 2018-03-06 | 大连理工大学 | A kind of mechanical arm target positioning grasping means based on binocular vision |
| CN107984201A (en) * | 2017-11-30 | 2018-05-04 | 中国地质大学（武汉） | A kind of screw hole positioning of view-based access control model servo and lock unload screw method |
| CN109035200A (en) * | 2018-06-21 | 2018-12-18 | 北京工业大学 | A kind of bolt positioning and position and posture detection method based on the collaboration of single binocular vision |
| CN109176521A (en) * | 2018-09-19 | 2019-01-11 | 北京因时机器人科技有限公司 | A kind of mechanical arm and its crawl control method and system |
| CN109940626A (en) * | 2019-01-23 | 2019-06-28 | 浙江大学城市学院 | A kind of thrush robot system and its control method based on robot vision |
| CN110378376A (en) * | 2019-06-12 | 2019-10-25 | 西安交通大学 | A kind of oil filler object recognition and detection method based on machine vision |
| CN110488261A (en) * | 2019-08-23 | 2019-11-22 | 南京理工大学 | A kind of shelf locating hole checking device and method based on LiDAR |
| CN111112981A (en) * | 2020-01-13 | 2020-05-08 | 苏州崮麓智能设备有限公司 | Equipment and method for automatically installing license plate by robot |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112045681B (en) | 2022-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109690623B (en) | System and method for recognizing pose of camera in scene | |
| JP4612635B2 (en) | Moving object detection using computer vision adaptable to low illumination depth | |
| CN108141570B (en) | Calibration device, calibration method, and calibration program storage medium | |
| KR101811157B1 (en) | Bowl-shaped imaging system | |
| JP3868876B2 (en) | Obstacle detection apparatus and method | |
| WO2021046716A1 (en) | Method, system and device for detecting target object and storage medium | |
| JP2022514912A (en) | Sensor calibration methods, devices, systems, vehicles, equipment and storage media | |
| WO2016117200A1 (en) | Outside environment recognition device for vehicles and vehicle behavior control device using same | |
| CN106651963B (en) | A kind of installation parameter scaling method of the vehicle-mounted camera for driving assistance system | |
| WO2016125014A1 (en) | Vehicle speed detection | |
| CN108197523B (en) | Night vehicle detection method and system based on image conversion and contour neighborhood difference | |
| JP2012239157A (en) | Surround view camera automatic calibration with extrinsic parameters only | |
| US11887336B2 (en) | Method for estimating a relative position of an object in the surroundings of a vehicle and electronic control unit for a vehicle and vehicle | |
| JP5146330B2 (en) | Vehicle road sign recognition device | |
| CN109703465B (en) | Control method and device for vehicle-mounted image sensor | |
| CN110717445B (en) | Front vehicle distance tracking system and method for automatic driving | |
| JP6151535B2 (en) | Parameter acquisition apparatus, parameter acquisition method and program | |
| JP2022514429A (en) | Calibration method for image acquisition equipment, equipment, systems, equipment and storage media | |
| WO2018235300A1 (en) | Object detection device, object detection method, and computer-readable recording medium | |
| CN112172797B (en) | Parking control method, device, equipment and storage medium | |
| JP5959311B2 (en) | Data deriving apparatus and data deriving method | |
| JP4344860B2 (en) | Road plan area and obstacle detection method using stereo image | |
| JP2015184929A (en) | Three-dimensional object detection apparatus, three-dimensional object detection method and three-dimensional object detection program | |
| KR20170026743A (en) | Automatic calibration method based on the simplified pattern for the vehicle image registration and the method thereof | |
| JPWO2020090897A1 (en) | Position detection device, position detection system, remote control device, remote control system, position detection method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |