CN111273704B - Method and device for automatically inserting external equipment hole - Google Patents
Method and device for automatically inserting external equipment hole Download PDFInfo
- Publication number
- CN111273704B CN111273704B CN202010072741.6A CN202010072741A CN111273704B CN 111273704 B CN111273704 B CN 111273704B CN 202010072741 A CN202010072741 A CN 202010072741A CN 111273704 B CN111273704 B CN 111273704B
- Authority
- CN
- China
- Prior art keywords
- external equipment
- hole
- coordinate position
- dimensional
- external
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D3/00—Control of position or direction
- G05D3/12—Control of position or direction using feedback
- G05D3/20—Control of position or direction using feedback using a digital comparing device
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/30—Interpretation of pictures by triangulation
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Multimedia (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention aims to provide a method and a device for automatically inserting an external equipment hole, wherein a three-dimensional coordinate position of the external equipment hole in a photo is identified from the photo by taking a photo of the side surface of the external equipment hole on a local equipment, and the three-dimensional coordinate position of the external equipment hole is converted into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula; the external equipment plug is controlled to move to the three-dimensional target coordinate position, so that the external equipment plug is inserted into the external equipment hole, the position of the external equipment hole of the equipment is accurately positioned, and the external equipment plug of the detection equipment can be reliably controlled to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
Description
Technical Field
The invention relates to the field of computers, in particular to a method and a device for automatically inserting an external device hole.
Background
The existing mode for detecting the performance of intelligent equipment such as mobile phones needs to manually insert an external equipment plug of the detection equipment into an external equipment hole of the intelligent equipment such as the mobile phones, and is time-consuming and labor-consuming, and high in labor cost.
Disclosure of Invention
An object of the present invention is to provide a method and apparatus for automatically inserting an external device hole.
According to one aspect of the present invention, there is provided a method of automatically inserting an external device hole, the method comprising:
taking a picture of the side face of an external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the external equipment plug to move to the three-dimensional target coordinate position so as to insert the external equipment plug into the external equipment hole.
Further, in the above method, taking a photograph of a side surface of the local device where the external device hole is located, identifying a three-dimensional coordinate position of the external device hole in the photograph from the photograph, and converting the three-dimensional coordinate position of the external device hole into a three-dimensional target coordinate position of an external device plug to be inserted into the external device hole, including:
scanning the side surface of the external equipment hole of the local equipment by using line laser;
Shooting each imaging line of the line laser on the side surface where the external equipment hole is located in the scanning process;
splicing all imaging lines to form a three-dimensional point cloud contour of the side surface of the equipment, where the external equipment hole is located;
obtaining a three-dimensional coordinate position in the photo of the external equipment hole based on a three-dimensional point cloud contour of the side surface where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z;
converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
controlling the external device plug to move to the three-dimensional target coordinate position so as to insert the external device plug into the external device hole, including:
and controlling the plug of the external equipment to be inserted into the hole of the external equipment based on the three-dimensional target coordinate position (x ', y ', z ') through a PLC control module.
Further, in the above method, obtaining the three-dimensional coordinate position in the photograph of the external device hole based on the three-dimensional point cloud contour of the side surface where the external device hole is located, including a plane coordinate position (x, y) and a depth coordinate position z, includes:
Acquiring a three-dimensional point cloud profile ps1 of the side surface where the external equipment hole is located;
carrying out normal vector filtering on the three-dimensional point cloud contour ps1, discarding a certain point in the three-dimensional point cloud contour ps1 if the included angle between the normal vector of the certain point and the vertical direction of the three-dimensional point cloud contour ps1 exceeds a preset threshold value a1, otherwise, reserving the certain point;
finally obtaining a filtered point cloud ps2 from all reserved points;
performing depth map projection on the point cloud ps2 to obtain a depth image d;
extracting each cavity area contour from the depth image d, and taking the cavity area contour extracted from the depth image d to be the largest as a contour c1 of an external equipment hole;
traversing the coordinates of each point in the contour c1, and acquiring corresponding three-dimensional coordinate values (x, y, z) of each point in the point cloud ps2 according to the coordinates of the point in the contour c1;
and adding the three-dimensional coordinate values (x, y, z) of all points in the outline c1 to obtain an average value so as to obtain the central position of the outline c1 of the external equipment hole.
Further, in the above method, converting the plane coordinate position (x, y) and the depth coordinate position z of the external device hole into the three-dimensional target coordinate position (x ', y ', z ') of the external device plug based on a preset space coordinate conversion formula includes:
Based on a preset space coordinate conversion formula, converting the central position of the outline c1 of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug.
Further, in the above method, controlling, by the PLC control module, the external device plug to be inserted into the external device hole based on the three-dimensional target coordinate position (x ', y ', z '), includes:
and controlling the first moving device to drive the external equipment plug to move towards the three-dimensional target coordinate position (x ', y', z ') based on the three-dimensional target coordinate position (x', y ', z') through the PLC control module so as to enable the external equipment plug to be inserted into the external equipment hole.
Further, in the above method, shooting each imaging line of the line laser on the side surface where the external device hole is located in the scanning process includes:
and shooting all imaging lines of the line laser on the side surface of the external equipment hole, which is parallel to the short side of the side surface of the external equipment hole in the scanning process.
Further, in the above method, the side surface of the external device hole of the line laser scanning device includes:
the side face of the external equipment hole of the equipment is scanned by blue line laser.
Further, in the above method, converting the three-dimensional coordinate position of the external device hole into the three-dimensional target coordinate position of the external device plug to be inserted into the external device hole based on a preset space coordinate conversion formula includes:
identifying the shape of the external equipment hole from the photo;
determining a corresponding preset space coordinate conversion formula based on the shape;
and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of the external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
According to another aspect of the present invention, there is also provided an apparatus for automatically inserting an external device hole, the apparatus comprising:
the device comprises an identification module, a display module and a display module, wherein the identification module is used for shooting a picture of the side face of an external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and the moving module is used for controlling the external equipment plug to move to the three-dimensional target coordinate position so as to insert the external equipment plug into the external equipment hole.
Further, in the above device, the identification module is configured to scan a side surface of the external device hole of the local device with line laser; shooting each imaging line of the line laser on the side surface where the external equipment hole is located in the scanning process; splicing all imaging lines to form a three-dimensional point cloud contour of the side surface of the equipment, where the external equipment hole is located; obtaining a three-dimensional coordinate position in the photo of the external equipment hole based on a three-dimensional point cloud contour of the side surface where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z; converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
the mobile module is used for controlling the plug of the external equipment to be inserted into the hole of the external equipment based on the three-dimensional target coordinate position (x ', y ', z ') through the PLC control module.
Further, in the above apparatus, the identification module is configured to obtain a three-dimensional point cloud profile ps1 of a side surface where the external device hole is located; carrying out normal vector filtering on the three-dimensional point cloud contour ps1, discarding a certain point in the three-dimensional point cloud contour ps1 if the included angle between the normal vector of the certain point and the vertical direction of the three-dimensional point cloud contour ps1 exceeds a preset threshold value a1, otherwise, reserving the point, and finally obtaining filtered point cloud ps2 from all reserved points; performing depth map projection on the point cloud ps2 to obtain a depth image d; extracting each cavity area contour from the depth image d, and taking the cavity area contour extracted from the depth image d to be the largest as a contour c1 of an external equipment hole; traversing the coordinates of each point in the contour c1, and acquiring corresponding three-dimensional coordinate values (x, y, z) of each point in the point cloud ps2 according to the coordinates of the point in the contour c1; and adding the three-dimensional coordinate values (x, y, z) of all points in the outline c1 to obtain an average value so as to obtain the central position of the outline c1 of the external equipment hole.
Further, in the above device, the identification module is configured to convert the center position of the outline c1 of the external device hole into a three-dimensional target coordinate position (x ', y ', z ') of the external device plug based on a preset spatial coordinate conversion formula.
Further, in the above device, the moving module is configured to control, by using the PLC control module, the first moving device to drive the external device plug to move toward the three-dimensional target coordinate position (x ', y', z ') based on the three-dimensional target coordinate position (x', y ', z'), so that the external device plug is inserted into the external device hole.
Further, in the above device, the identification module is configured to capture each imaging line of the line laser on the side surface where the external device hole is located and parallel to the short side of the side surface where the external device hole is located in the scanning process.
Further, in the above apparatus, the identification module is configured to use a side surface of the external device hole of the blue line laser scanning device.
Further, in the above apparatus, the identifying module is configured to identify a shape of the external device hole from the photograph; determining a corresponding preset space coordinate conversion formula based on the shape; and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of the external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
According to another aspect of the present invention, there is also provided a computing-based apparatus, including:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
taking a picture of the side face of an external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the external equipment plug to move to the three-dimensional target coordinate position so as to insert the external equipment plug into the external equipment hole.
According to another aspect of the present invention, there is also provided a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
taking a picture of the side face of an external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
And controlling the external equipment plug to move to the three-dimensional target coordinate position so as to insert the external equipment plug into the external equipment hole.
Compared with the prior art, the method and the device have the advantages that the three-dimensional coordinate position of the external equipment hole in the photo is identified from the photo by taking the photo of the side face of the external equipment hole on the local equipment, and the three-dimensional coordinate position of the external equipment hole is converted into the three-dimensional target coordinate position of the external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula; the external equipment plug is controlled to move to the three-dimensional target coordinate position, so that the external equipment plug is inserted into the external equipment hole, the position of the external equipment hole of the equipment is accurately positioned, and the external equipment plug of the detection equipment can be reliably controlled to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the accompanying drawings in which:
FIG. 1 is a flow chart of a method of dynamically inserting an external device aperture according to an embodiment of the present invention;
Fig. 2 shows a schematic diagram of an imaged line of an embodiment of the present application.
The same or similar reference numbers in the drawings refer to the same or similar parts.
Detailed Description
The application is described in further detail below with reference to the accompanying drawings.
In one exemplary configuration of the application, the terminal, the device of the service network, and the trusted party each include one or more processors (CPUs), input/output ports, network ports, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer readable media, as defined herein, does not include non-transitory computer readable media (transmission media), such as modulated data signals and carrier waves.
As shown in fig. 1, the present invention provides a method for automatically inserting an external device hole, the method comprising:
step S1, taking a picture of the side face of an external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
here, the local device may be an intelligent terminal such as a mobile phone, a PAD, etc.;
the external equipment hole can be a USB hole or an earphone hole on an intelligent terminal such as a mobile phone, a PAD and the like;
and S2, controlling the external equipment plug to move to the three-dimensional target coordinate position so as to insert the external equipment plug into the external equipment hole.
The invention can accurately position the position of the external equipment hole of the equipment, and further can reliably control the external equipment plug of the detection equipment to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
In an embodiment of the method for automatically inserting an external device hole, step S1, taking a photo of a side surface of a local device where the external device hole is located, identifying a three-dimensional coordinate position of the external device hole in the photo from the photo, and converting the three-dimensional coordinate position of the external device hole into a three-dimensional target coordinate position of an external device plug to be inserted into the external device hole based on a preset space coordinate conversion formula, including:
Step S11, scanning the side surface of the external equipment hole of the local equipment by using line laser;
the local device can be various devices such as a mobile phone, a computer, a PAD and the like with external device holes, and the local device generally comprises 6 surfaces, wherein the top surface is one surface with a display screen, the bottom surface is the back surface of the display screen, the other four surfaces are four side surfaces respectively connected with the top surface and the bottom surface, and the external device holes are generally arranged on one side surface of the local device;
step S12, shooting all imaging lines of the line laser on the side surface where the external equipment hole is located in the scanning process;
the line laser scans the side surface where the external equipment hole is located to form an imaging line reflecting the uneven profile of the side surface where the external equipment hole is located;
step S13, splicing all imaging lines to form a three-dimensional point cloud outline of the side surface of the equipment where the external equipment hole is located;
step S14, obtaining a three-dimensional coordinate position in the photo of the external equipment hole based on the three-dimensional point cloud outline of the side surface where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z;
step S15, converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
Step S2, controlling the external device plug to move to the three-dimensional target coordinate position, so as to insert the external device plug into the external device hole, including:
and S21, controlling the plug of the external equipment to be inserted into the hole of the external equipment by a PLC control module based on the three-dimensional target coordinate position (x ', y ', z ').
The invention can accurately position the position of the external equipment hole of the equipment, and further can reliably control the external equipment plug of the detection equipment to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
In an embodiment of the method for automatically inserting an external device hole of the present invention, step S14, obtaining a three-dimensional coordinate position in the photograph of the external device hole based on a three-dimensional point cloud contour of a side surface where the external device hole is located, including a plane coordinate position (x, y) and a depth coordinate position z, includes:
step S141, obtaining a three-dimensional point cloud profile ps1 of the side surface where the external equipment hole is located;
step S142, performing normal vector filtering on the three-dimensional point cloud contour ps1, if the included angle between the normal vector of a certain point in the three-dimensional point cloud contour ps1 and the vertical direction of the three-dimensional point cloud contour ps1 exceeds a preset threshold a1, discarding the certain point, otherwise, reserving the certain point, and finally obtaining a filtered point cloud ps2 from all reserved points;
Here, the smooth transition area between the inner wall of the external device hole or the bottom and the front of the mobile phone may also generate point clouds, but this part of point clouds is not needed, and especially, the point inside the external device hole may make the USB center extraction inaccurate. Because the points inside the external equipment hole are concave inwards, unnecessary points are filtered by judging that the included angle exceeds a preset threshold value a 1;
step S143, carrying out depth map projection on the point cloud ps2 to obtain a depth image d;
step S144, extracting each cavity area contour from the depth image d, and taking the cavity area contour extracted from the depth image d to be the largest as a contour c1 of an external equipment hole;
step S145, traversing the coordinates of each point in the contour c1, and obtaining the three-dimensional coordinate values (x, y, z) corresponding to each point in the point cloud ps2 according to the coordinates of the point in the contour c1;
step S146, adding the three-dimensional coordinate values (x, y, z) of all points in the contour c1 to obtain an average value, so as to obtain the center position of the contour c1 of the external device hole.
Here, the embodiment can efficiently and accurately obtain the center position of the outline c1 of the external equipment hole so as to accurately position the position of the external equipment hole.
In an embodiment of the method for automatically inserting an external device hole of the present invention, step S15, based on a preset space coordinate transformation formula, converts a plane coordinate position (x, y) and a depth coordinate position z of the external device hole into a three-dimensional target coordinate position (x ', y ', z ') of an external device plug, including:
based on a preset space coordinate conversion formula, converting the central position of the outline c1 of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug.
The central position of the outline c1 of the external equipment hole is converted into the three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug, so that the external equipment plug can be conveniently and accurately controlled to be accurately inserted into the external equipment hole.
In an embodiment of the method for automatically inserting an external device hole of the present invention, step S16, controlling, by a PLC control module, the external device plug to be inserted into the external device hole based on the three-dimensional target coordinate position (x ', y ', z '), includes:
and controlling the first moving device to drive the external equipment plug to move towards the three-dimensional target coordinate position (x ', y', z ') based on the three-dimensional target coordinate position (x', y ', z') through the PLC control module so as to enable the external equipment plug to be inserted into the external equipment hole.
In an embodiment of the method for automatically inserting an external device hole of the present invention, step S12, shooting each imaging line of the line laser on the side surface where the external device hole is located in the scanning process, includes:
and shooting all imaging lines of the line laser on the side surface of the external equipment hole, which is parallel to the short side of the side surface of the external equipment hole in the scanning process.
Here, as shown in fig. 2, the side surface where the external device hole is located is a rectangular frame surrounded by a long side and a short side, and each imaging line 3 that is parallel to the short side 1 of the side surface where the external device hole 4 is located, but not parallel to the long side 2, is formed by the line laser in the shooting and scanning process, so that each obtained imaging line is as short as possible, and the accuracy of the three-dimensional point cloud contour of the side surface where the external device hole of the device is formed by splicing each imaging line is ensured.
In an embodiment of the method for automatically inserting an external device hole of the present invention, step S11, scanning a side surface of the device hole with a line laser, includes:
the side face of the external equipment hole of the equipment is scanned by blue line laser.
The blue line laser has the characteristics of short wavelength and strong energy, and each imaging line of the line laser on the equipment can be obtained efficiently and reliably by using the blue line laser.
In an embodiment of the method for automatically inserting an external device hole of the present invention, in the step S1, based on a preset space coordinate conversion formula, the converting the three-dimensional coordinate position of the external device hole into the three-dimensional target coordinate position of the external device plug to be inserted into the external device hole includes:
step S101, recognizing the shape of the external equipment hole from the photo;
step S102, determining a corresponding preset space coordinate conversion formula based on the shape;
step S103, converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of the external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
Here, for example, when the external device hole is a USB hole, the USB hole may include different shapes such as an apple USB hole, a type ec, a MICRO, and the like, and the three-dimensional coordinate position of the external device hole may be more reliably and accurately converted into the three-dimensional target coordinate position of the external device plug to be inserted into the external device hole by respectively corresponding to different preset space coordinate conversion formulas of different shapes.
The invention provides a device for automatically inserting and inserting an external device hole, which comprises:
The device comprises an identification module, a display module and a display module, wherein the identification module is used for shooting a picture of the side face of an external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and the moving module is used for controlling the external equipment plug to move to the three-dimensional target coordinate position so as to insert the external equipment plug into the external equipment hole.
The invention can accurately position the position of the external equipment hole of the equipment, and further can reliably control the external equipment plug of the detection equipment to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
In an embodiment of the device for automatically inserting an external device hole of the present invention, the identification module is configured to use a side surface of the line laser scanning device where the external device hole is located;
the device can be various devices such as a mobile phone, a computer, a PAD and the like with external device holes, and the device generally comprises 6 surfaces, wherein the top surface is one surface with a display screen, the bottom surface is the back surface of the display screen, the other four surfaces are four side surfaces respectively connected with the top surface and the bottom surface, and the external device holes are generally arranged on one side surface of the device;
The identification module is used for shooting all imaging lines of the line laser on the side surface where the external equipment hole is located in the scanning process;
scanning the side surface of the external equipment hole of the line laser equipment to form an imaging line reflecting the uneven profile of the side surface of the external equipment hole;
the identification module is used for splicing all imaging lines to form a three-dimensional point cloud outline of the side surface of the equipment where the external equipment hole is located;
the identification module is used for obtaining a three-dimensional coordinate position in the photo of the external equipment hole based on a three-dimensional point cloud contour of the side surface where the external equipment hole is located, and the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z;
the identification module is used for converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (value x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
the mobile module is used for controlling the plug of the external equipment to be inserted into the hole of the external equipment based on the three-dimensional target coordinate position (x ', y ', z ') through the PLC control module.
The invention can accurately position the position of the external equipment hole of the equipment, and further can reliably control the external equipment plug of the detection equipment to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
In an embodiment of the device for automatically inserting the external device hole, the identification module is used for acquiring a three-dimensional point cloud profile ps1 of the side surface where the external device hole is located; carrying out normal vector filtering on the three-dimensional point cloud contour ps1, discarding a certain point in the three-dimensional point cloud contour ps1 if the included angle between the normal vector of the certain point and the vertical direction of the three-dimensional point cloud contour ps1 exceeds a preset threshold value a1, otherwise, reserving the point, and finally obtaining filtered point cloud ps2 from all reserved points; performing depth map projection on the point cloud ps2 to obtain a depth image d; extracting each cavity area contour from the depth image d, and taking the cavity area contour extracted from the depth image d to be the largest as a contour c1 of an external equipment hole; traversing the coordinates of each point in the contour c1, and acquiring corresponding three-dimensional coordinate values (x, y, z) of each point in the point cloud ps2 according to the coordinates of the point in the contour c1; and adding the three-dimensional coordinate values (x, y, z) of all points in the outline c1 to obtain an average value so as to obtain the central position of the outline c1 of the external equipment hole.
Here, the smooth transition area between the inner wall of the external device hole or the bottom and the front of the mobile phone may also generate point clouds, but this part of point clouds is not needed, and especially, the point inside the external device hole may make the USB center extraction inaccurate. Because the points inside the external equipment hole are concave inwards, unnecessary points are filtered by judging that the included angle exceeds a preset threshold value a 1.
Here, the embodiment can efficiently and accurately obtain the center position of the outline c1 of the external equipment hole so as to accurately position the position of the external equipment hole.
In an embodiment of the device for automatically inserting an external device hole of the present invention, the identification module is configured to convert a center position of a contour c1 of the external device hole into a three-dimensional target coordinate position (x ', y ', z ') of the external device plug based on a preset spatial coordinate conversion formula.
The central position of the outline c1 of the external equipment hole is converted into the target coordinate value (x ', y ', z ') of the PLC control module, so that the external equipment plug can be conveniently and accurately controlled to be inserted into the external equipment hole.
In an embodiment of the device for automatically inserting the external device hole, the moving module is configured to control, by using the PLC control module, the first moving device to drive the external device plug to move toward the three-dimensional target coordinate position (x ', y', z ') based on the three-dimensional target coordinate position (x', y ', z'), so that the external device plug is moved to a position corresponding to the external device hole, and the external device plug is inserted into the external device hole.
In an embodiment of the device for automatically inserting the external device hole of the present invention, the identification module is configured to capture each imaging line of the line laser on the side surface where the external device hole is located and parallel to the short side of the side surface where the external device hole is located in the scanning process.
Here, as shown in fig. 2, the side surface where the external device hole is located is a rectangular frame surrounded by a long side and a short side, and each imaging line 3 that is parallel to the short side 1 of the side surface where the external device hole 4 is located, but not parallel to the long side 2, is formed by the line laser in the shooting and scanning process, so that each obtained imaging line is as short as possible, and the accuracy of the three-dimensional point cloud contour of the side surface where the external device hole of the device is formed by splicing each imaging line is ensured.
In an embodiment of the device for automatically inserting an external device hole of the present invention, the scanning module is configured to use blue line laser to scan a side surface of the device hole.
The blue line laser has the characteristics of short wavelength and strong energy, and each imaging line of the line laser on the equipment can be obtained efficiently and reliably by using the blue line laser.
In an embodiment of the device for automatically inserting an external device hole of the present invention, the identification module is configured to identify a shape of the external device hole from the photograph; determining a corresponding preset space coordinate conversion formula based on the shape; and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of the external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
Here, for example, when the external device hole is a USB hole, the USB hole may include different shapes such as an apple USB hole, a type ec, a MICRO, and the like, and the three-dimensional coordinate position of the external device hole may be more reliably and accurately converted into the three-dimensional target coordinate position of the external device plug to be inserted into the external device hole by respectively corresponding to different preset space coordinate conversion formulas of different shapes.
According to another aspect of the present invention, there is also provided a computing-based apparatus comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
taking a picture of the side face of an external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the external equipment plug to move to the three-dimensional target coordinate position so as to insert the external equipment plug into the external equipment hole.
According to another aspect of the present application, there is also provided a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
taking a picture of the side face of an external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the external equipment plug to move to the three-dimensional target coordinate position so as to insert the external equipment plug into the external equipment hole.
Details of each device and storage medium embodiment of the present application may refer to corresponding parts of each method embodiment, and are not described herein.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
It should be noted that the present invention may be implemented in software and/or a combination of software and hardware, e.g., using Application Specific Integrated Circuits (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to perform the steps or functions described above. Likewise, the software programs of the present invention (including associated data structures) may be stored on a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. In addition, some steps or functions of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
Furthermore, portions of the present invention may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or techniques in accordance with the present invention by way of operation of the computer. Program instructions for invoking the inventive methods may be stored in fixed or removable recording media and/or transmitted via a data stream in a broadcast or other signal bearing medium and/or stored within a working memory of a computer device operating according to the program instructions. An embodiment according to the invention comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to operate a method and/or a solution according to the embodiments of the invention as described above.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the apparatus claims can also be implemented by means of one unit or means in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.
Claims (16)
1. A method of automatically inserting an external device aperture, wherein the method comprises:
taking a picture of the side face of an external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
Controlling the external equipment plug to move to the three-dimensional target coordinate position so as to insert the external equipment plug into the external equipment hole;
taking a picture of the side face of the external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of the external equipment plug to be inserted into the external equipment hole, wherein the picture comprises the following steps:
scanning the side surface of the external equipment hole of the local equipment by using line laser;
shooting each imaging line of the line laser on the side surface where the external equipment hole is located in the scanning process;
splicing all imaging lines to form a three-dimensional point cloud contour of the side surface of the equipment, where the external equipment hole is located;
obtaining a three-dimensional coordinate position in the photo of the external equipment hole based on a three-dimensional point cloud contour of the side surface where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z;
converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
Controlling the external device plug to move to the three-dimensional target coordinate position so as to insert the external device plug into the external device hole, including:
and controlling the plug of the external equipment to be inserted into the hole of the external equipment based on the three-dimensional target coordinate position (x ', y ', z ') through a PLC control module.
2. The method of claim 1, wherein deriving three-dimensional coordinate locations in the photograph of the external device aperture based on a three-dimensional point cloud profile of a side of the external device aperture, including a planar coordinate location (x, y) and a depth coordinate location z, comprises:
acquiring a three-dimensional point cloud profile ps1 of the side surface where the external equipment hole is located;
carrying out normal vector filtering on the three-dimensional point cloud contour ps1, discarding a certain point in the three-dimensional point cloud contour ps1 if the included angle between the normal vector of the certain point and the vertical direction of the three-dimensional point cloud contour ps1 exceeds a preset threshold value a1, otherwise, reserving the certain point;
finally obtaining a filtered point cloud ps2 from all reserved points;
performing depth map projection on the point cloud ps2 to obtain a depth image d;
extracting each cavity area contour from the depth image d, and taking the cavity area contour extracted from the depth image d to be the largest as a contour c1 of an external equipment hole;
Traversing the coordinates of each point in the contour c1, and acquiring corresponding three-dimensional coordinate values (x, y, z) of each point in the point cloud ps2 according to the coordinates of the point in the contour c 1;
and adding the three-dimensional coordinate values (x, y, z) of all points in the outline c1 to obtain an average value so as to obtain the central position of the outline c1 of the external equipment hole.
3. The method of claim 2, wherein converting the planar coordinate position (x, y) and the depth coordinate position z of the external device aperture into the three-dimensional target coordinate position (x ', y ', z ') of the external device plug based on a preset spatial coordinate conversion formula, comprises:
based on a preset space coordinate conversion formula, converting the central position of the outline c1 of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug.
4. The method of claim 3, wherein controlling, by a PLC control module, insertion of the external device plug into the external device aperture based on the three-dimensional target coordinate location (x ', y ', z '), comprises:
and controlling the first moving device to drive the external equipment plug to move towards the three-dimensional target coordinate position (x ', y', z ') based on the three-dimensional target coordinate position (x', y ', z') through the PLC control module so as to enable the external equipment plug to be inserted into the external equipment hole.
5. The method of claim 1, wherein capturing each imaged line of the line laser on the side of the external device aperture during scanning comprises:
and shooting all imaging lines of the line laser on the side surface of the external equipment hole, which is parallel to the short side of the side surface of the external equipment hole in the scanning process.
6. The method of claim 2, wherein scanning the side of the device on which the device aperture is located with the line laser comprises:
the side face of the external equipment hole of the equipment is scanned by blue line laser.
7. The method of claim 1, wherein converting the three-dimensional coordinate position of the external device hole to the three-dimensional target coordinate position of an external device plug to be inserted into the external device hole based on a preset spatial coordinate conversion formula comprises:
identifying the shape of the external equipment hole from the photo;
determining a corresponding preset space coordinate conversion formula based on the shape;
and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of the external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
8. An apparatus for automatically inserting an external device aperture, wherein the apparatus comprises:
the device comprises an identification module, a display module and a display module, wherein the identification module is used for shooting a picture of the side face of an external equipment hole on the local equipment, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
the moving module is used for controlling the external equipment plug to move to the three-dimensional target coordinate position so as to insert the external equipment plug into the external equipment hole;
the identification module is used for scanning the side face of the external equipment hole of the local equipment by using line laser; shooting each imaging line of the line laser on the side surface where the external equipment hole is located in the scanning process; splicing all imaging lines to form a three-dimensional point cloud contour of the side surface of the equipment, where the external equipment hole is located; obtaining a three-dimensional coordinate position in the photo of the external equipment hole based on a three-dimensional point cloud contour of the side surface where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z; converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
The mobile module is used for controlling the plug of the external equipment to be inserted into the hole of the external equipment based on the three-dimensional target coordinate position (x ', y ', z ') through the PLC control module.
9. The device of claim 8, wherein the identification module is configured to obtain a three-dimensional point cloud profile ps1 of a side surface where the external device hole is located; carrying out normal vector filtering on the three-dimensional point cloud contour ps1, discarding a certain point in the three-dimensional point cloud contour ps1 if the included angle between the normal vector of the certain point and the vertical direction of the three-dimensional point cloud contour ps1 exceeds a preset threshold value a1, otherwise, reserving the point, and finally obtaining filtered point cloud ps2 from all reserved points; performing depth map projection on the point cloud ps2 to obtain a depth image d; extracting each cavity area contour from the depth image d, and taking the cavity area contour extracted from the depth image d to be the largest as a contour c1 of an external equipment hole; traversing the coordinates of each point in the contour c1, and acquiring corresponding three-dimensional coordinate values (x, y, z) of each point in the point cloud ps2 according to the coordinates of the point in the contour c1; and adding the three-dimensional coordinate values (x, y, z) of all points in the outline c1 to obtain an average value so as to obtain the central position of the outline c1 of the external equipment hole.
10. The apparatus of claim 9, wherein the identification module is configured to convert a center position of the outline c1 of the external device hole into a three-dimensional target coordinate position (x ', y ', z ') of the external device plug based on a preset spatial coordinate conversion formula.
11. The device according to claim 9, wherein the moving module is configured to control, by the PLC control module, the first moving device to move the external device plug to the three-dimensional target coordinate position (x ', y', z ') based on the three-dimensional target coordinate position (x', y ', z'), so that the external device plug is inserted into the external device hole.
12. The device of claim 8, wherein the identification module is configured to capture each imaging line of the line laser on the side where the external device hole is located and parallel to a short side of the side where the external device hole is located during scanning.
13. The apparatus of claim 8, wherein the identification module is configured to use a side of the blue line laser scanning device where the external device aperture is located.
14. The apparatus of claim 8, wherein the identification module is configured to identify a shape of the external device aperture from the photograph; determining a corresponding preset space coordinate conversion formula based on the shape; and converting the three-dimensional coordinate position of the external equipment hole into the three-dimensional target coordinate position of the external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
15. A computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method of any of claims 1 to 7.
16. A computer readable storage medium having stored thereon computer executable instructions, wherein the computer executable instructions when executed by a processor cause the processor to perform the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010072741.6A CN111273704B (en) | 2020-01-21 | 2020-01-21 | Method and device for automatically inserting external equipment hole |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010072741.6A CN111273704B (en) | 2020-01-21 | 2020-01-21 | Method and device for automatically inserting external equipment hole |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111273704A CN111273704A (en) | 2020-06-12 |
| CN111273704B true CN111273704B (en) | 2023-09-15 |
Family
ID=70999099
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010072741.6A Active CN111273704B (en) | 2020-01-21 | 2020-01-21 | Method and device for automatically inserting external equipment hole |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111273704B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA3129917A1 (en) | 2019-02-12 | 2020-08-20 | Ecoatm, Llc | Connector carrier for electronic device kiosk |
| KR20210127199A (en) | 2019-02-18 | 2021-10-21 | 에코에이티엠, 엘엘씨 | Neural network-based physical state evaluation of electronic devices, and related systems and methods |
| US11922467B2 (en) | 2020-08-17 | 2024-03-05 | ecoATM, Inc. | Evaluating an electronic device using optical character recognition |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105069420A (en) * | 2015-07-27 | 2015-11-18 | 上海振华重工(集团)股份有限公司 | Automatic container lockhole identifying-positioning method and system |
| WO2016185637A1 (en) * | 2015-05-20 | 2016-11-24 | 三菱電機株式会社 | Point-cloud-image generation device and display system |
| CN107578464A (en) * | 2017-06-30 | 2018-01-12 | 长沙湘计海盾科技有限公司 | A kind of conveyor belt workpieces measuring three-dimensional profile method based on line laser structured light |
| CN107834614A (en) * | 2017-07-28 | 2018-03-23 | 张涛 | A kind of charging method for adapting to different model mobile phone automatic charging |
| CN108981604A (en) * | 2018-07-11 | 2018-12-11 | 天津工业大学 | A kind of precision component three-dimensional overall picture measurement method based on line laser |
| CN109977954A (en) * | 2019-04-01 | 2019-07-05 | 上海电气集团股份有限公司 | The identification of electric vehicle charge interface and localization method and system |
-
2020
- 2020-01-21 CN CN202010072741.6A patent/CN111273704B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016185637A1 (en) * | 2015-05-20 | 2016-11-24 | 三菱電機株式会社 | Point-cloud-image generation device and display system |
| CN105069420A (en) * | 2015-07-27 | 2015-11-18 | 上海振华重工(集团)股份有限公司 | Automatic container lockhole identifying-positioning method and system |
| CN107578464A (en) * | 2017-06-30 | 2018-01-12 | 长沙湘计海盾科技有限公司 | A kind of conveyor belt workpieces measuring three-dimensional profile method based on line laser structured light |
| CN107834614A (en) * | 2017-07-28 | 2018-03-23 | 张涛 | A kind of charging method for adapting to different model mobile phone automatic charging |
| CN108981604A (en) * | 2018-07-11 | 2018-12-11 | 天津工业大学 | A kind of precision component three-dimensional overall picture measurement method based on line laser |
| CN109977954A (en) * | 2019-04-01 | 2019-07-05 | 上海电气集团股份有限公司 | The identification of electric vehicle charge interface and localization method and system |
Non-Patent Citations (1)
| Title |
|---|
| 基于线激光扫描和图像处理的基准孔检测技术研究;谭小群;唐婧仪;于薇薇;李希;孙智金;;现代制造工程(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111273704A (en) | 2020-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111273704B (en) | Method and device for automatically inserting external equipment hole | |
| CN111311749B (en) | Method and device for automatically inserting USB hole of equipment | |
| CN109190539A (en) | Face identification method and device | |
| JP2022539910A (en) | Electronic device screen area defect inspection method and apparatus | |
| TW201941110A (en) | Qr code identifying and reading method and apparatus, and device | |
| CN111291661B (en) | Method and equipment for identifying text content of icon in screen | |
| US11361508B2 (en) | Object scanning using planar segmentation | |
| CN105578023A (en) | Image quick photographing method and device | |
| CN111210473A (en) | Mobile phone contour positioning method and equipment | |
| CN114529566B (en) | Image processing method, device, equipment and storage medium | |
| CN111325715A (en) | Camera color spot detection method and device | |
| US11238563B2 (en) | Noise processing method and apparatus | |
| CN111272067A (en) | Mobile phone detection method and equipment | |
| CN110827248A (en) | Method and device for identifying screen area of electronic device | |
| CN110855979A (en) | Camera module detection method and device | |
| JP5857712B2 (en) | Stereo image generation apparatus, stereo image generation method, and computer program for stereo image generation | |
| CN111272388A (en) | Method and device for detecting camera flash lamp | |
| CN108573510B (en) | Grid map vectorization method and device | |
| CN114690144A (en) | Point cloud data labeling method, device and system | |
| CN116311135A (en) | Data dimension reduction method, data dimension reduction system and controller for semantic information | |
| CN110555874B (en) | Image processing method and device | |
| CN111311678B (en) | USB position positioning method and device | |
| CN110880023A (en) | Method and device for detecting certificate picture | |
| CN111292374B (en) | Method and equipment for automatically plugging and unplugging USB interface | |
| CN115205364A (en) | Object height determination method and device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 1101-1103, No. 433, Songhu Road, Yangpu District, Shanghai Applicant after: Shanghai wanwansheng Environmental Protection Technology Group Co.,Ltd. Address before: Room 1101-1103, No. 433, Songhu Road, Yangpu District, Shanghai Applicant before: SHANGHAI YUEYI NETWORK INFORMATION TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |